Execution plane / Tier 2 / Phase 3
ML & Decision Intelligence
When is a classical model the better tool?
Keep discriminative and optimisation models in the architecture as first-class tools and routing signals — often cheaper, faster and more accurate than asking a language model.
- Classifiers for routing, triage, intent and safety pre-screening
- Rerankers and scorers supporting retrieval
- Anomaly detection and forecasting feeding agent decisions
- Optimisation and constraint solving where the answer is a plan, not prose
- Feature store, training pipelines and drift monitoring for these models
training and feature data
exposed to agents as tools
routing signals for model tier selection
classifiers behind guardrails
rerankers in the retrieval path
drift and performance monitoring
held to the same evaluation discipline
Data Science / Machine Learning
- Head of Data Science — accountable owner
- Data scientist — model development and validation
- MLOps engineer — deployment, monitoring, retraining
- Decision scientist — where optimisation beats generation
- Model risk reviewer in regulated sectors
Architectural recommendation, not a research finding.
A scale-phase optimisation: introduce when volume makes a measured LLM path too slow, too costly or too imprecise for a repeated decision. Do not create a parallel ML organisation — reuse the same registry, evaluation and observability harnesses.
Signals to watch
- Accuracy or AUC by model against its baseline
- Feature and prediction drift
- Share of decisions served by classical models versus the LLM path
- Cost and latency delta against the LLM alternative
- A language model used where a small classifier is faster, cheaper and more accurate
- Drift unmonitored because these models sit outside the agent telemetry
- Two disconnected lifecycles: one for models, one for agents
- Existing model-risk governance not applied to models now invoked by agents
Tool and standards snapshot · last reviewed approximately mid-2026
Open source
- scikit-learn
- XGBoost or LightGBM
- PyTorch
- MLflow
- Feast
- Evidently for drift
- River for online learning
- OR-Tools
- statsforecast or Prophet
- BentoML or Seldon Core
Managed
- Managed ML platforms and feature stores
- AutoML services for tabular baselines
Standards
- Model cards and datasheets
- Existing model-risk management standards where the sector requires them
This harness is easy to forget in an agent-centric design and quick to pay for itself. The most reliable cost reduction available is usually not a cheaper model tier — it is not calling a language model at all.