Skip to content
Iñigo Goikoetxea Fanega · LLM / AI engineerdemo api
PROJECT GenSIE-26
THESIS Structured Information Extraction from Texts in Spanish Using LLM Systems
STATUS published · IberLEF 2026

Extraction
you can audit.

Arbitrary JSON schemas, filled from Spanish text by small open LLMs. Grammar-constrained decoding guarantees the shape; a hallucination filter keeps out what the text cannot support.

Open the demoPaper PDFCase study
trace · e231 · RiverProfile
El río Arga es uno de los principales ríos de Navarra. Nace en el área de Quinto Real, en los Pirineos navarros, y atraviesa Pamplona y Puente la Reina antes de desembocar en el río Aragón. Tiene una longitud de 145 km.
01clean source text219 chars
02build prompt769 tok · 0 examples
03constrained decoding85 tok · 408 ms
04hallucination filter0 removed
{
  "name": "Arga",
  "tributary_of": "Aragón",
  "length_km": 145,
  "source_range": "Pirineos navarros",
  "passes_through": ["Pamplona", "Puente la Reina"],
  "avg_flow_m3s": null  // not in the text
}
e231 · 854 tok · 0.4 s · openai/gpt-oss-20brecorded run · 2026-10-28

Recognition

Published paper · CEUR Workshop Proceedings, Vol. 4273

Inigo at IberLEF 2026 Task GenSIE: Schema-Guided Zero-Shot Information Extraction in Spanish with Small Language Models. Iñigo Goikoetxea Fanega, Mikel Sesma Sara and Javier Lasheras Navas (Universidad Pública de Navarra and Tracasa Instrumental).

It describes the zero-shot submission, since the task prohibited fine-tuning; the fine-tuning work is the thesis extension and is not part of the paper. Read the paper (PDF)

Award · Cátedra Tracasa

The thesis received one of three awards for the best final projects of the first cohort of the Master's in Machine Learning at the Universidad Pública de Navarra, sponsored by the Cátedra Tracasa de Ciencias de la Computación e Inteligencia Artificial and granted without ranking at the Machine Learning TFM Showcase. Read the news article

F1 · dev
0.80
best zero-shot · Qwen3-14B · E226 · 149 tasks
Micro-F1 · test
0.74
published · 145 instances · self-evaluated
Experiments
280
IDs in the registry · 452 runs
Synthetic pairs
3,948
Spanish · GuideX-style pipeline

Dev and test figures are not comparable with each other. Every number on this site names its source file in the page source.

What I built

01Zero-shot pipelineFew-shot retrieval, grammar-constrained decoding and a hallucination filter, submitted to GenSIE 2026.02Fine-tuning on a supercomputerLoRA and full fine-tuning of three open models on MareNostrum 5 H100 GPUs, tracked in MLflow.03Synthetic Spanish dataA five-stage generator of schema–instance pairs that feeds both branches.

Stack

Pythonlanguage
PyTorchtraining
Hugging FaceTransformers · PEFT · TRL
vLLMmodel serving
Outlines · xgrammarconstrained decoding
Pydanticschemas · validation
scikit-learnretrieval · evaluation
MLflowexperiment tracking
Langfuseinference tracing
Dockerself-hosted tracking
NVIDIA H100GPUs on MareNostrum 5
SLURMHPC job scheduling
Groqdemo inference
CloudflarePages · Functions · KV
GitHub ActionsCI · tests · deploy

Only tools used in the project repositories.

The finding

Training appears to help less as the inference pipeline improves. On the strongest model the best zero-shot pipeline beat both fine-tuning regimes; training mattered more for restraint than for raw F1.

On the strongest model, the zero-shot pipeline beat fine-tuningDev-set F1 per model for the untuned baseline, the best zero-shot pipeline and the best fine-tuned model. Qwen3-14B: zero-shot 0.7999 beats fine-tuning 0.7054. Llama-3.2-3B: 0.7459 vs 0.7481. Salamandra-7B: 0.6276 vs 0.6406.On the strongest model, the zero-shot pipeline beat fine-tuninguntuned baselinebest zero-shot pipelinebest fine-tuned0.50.60.70.8F1 on the GenSIE development set (149 tasks)fine-tuned − zero-shotQwen3-14Buntuned baseline: 0.6577best fine-tuned (full FT): 0.7054best zero-shot pipeline: 0.7999−0.09450.79990.7054Llama-3.2-3Buntuned baseline: 0.6703best fine-tuned (full FT): 0.7481best zero-shot pipeline: 0.7459+0.0022Salamandra-7Buntuned baseline: 0.4843best fine-tuned (LoRA): 0.6406best zero-shot pipeline: 0.6276+0.0130Source: gensie-solution docs/tuned_vs_untuned.md · single-seed runs

Under the hood

During development every training run was logged to MLflow, and every model call and evaluated task to Langfuse. See one real record of each.

How it is traced →

Building with LLMs in production?
github ↗