NER Travel Order

Problem

Travel order processing required manual extraction of key entities (departure location, arrival location, dates, transportation type) from unstructured French text. This was time-consuming, error-prone, and didn't scale for batch processing.

Solution

Trained and compared two NER models on a French corpus with BIO tagging, then shipped the one that made sense in production:

• Fine-tuned CamemBERT on a 12,000-sentence annotated French corpus
• Two models, benchmarked head to head: a custom spaCy v3 pipeline (~10 ms per sentence measured on CPU, 26 MB) against a fine-tuned CamemBERT (440 MB), which is better on ambiguous sentences on the same corpus
• spaCy chosen for production: 5 F1 points traded for a 10× latency reduction and a 29× smaller model, the right call for CPU-only serving
• Token classification with BIO tagging (Begin, Inside, Outside) for 6 entity types: DEP, ARR, TEMP, TRANSPORT, PER, MISC
• Evaluation-first approach: entity-level F1, precision, recall metrics (not token-level)
• Production pipeline: inference optimized for batch and real-time processing
• Bilingual support: French text input with structured JSON output

Results & Metrics

Technology Stack

CamemBERT Hugging Face PyTorch spaCy Python seqeval JSON Flask

Key Learnings

• French NLP matters: CamemBERT outperforms multilingual models on French-specific text
• Entity-level evaluation is crucial: Token-level accuracy hides extraction failures; entity F1 is the real metric
• Annotation quality drives performance: Small, high-quality dataset beats large, noisy one
• BIO tagging disambiguation: Careful label scheme design prevents entity boundary errors
• Production readiness: Inference optimization + versioning + monitoring separate research from deployment