Retrieval-augmented generation has become the default way to ground large language models in enterprise knowledge. Over the past three years we have built conversational assistants across industries (insurance, healthcare, retail, and more), on all major cloud platforms and with most of the popular toolkits. We open the session with a selection of these reference cases and what they had in common.

One question was central to every one of them: how do you measure quality? Open-source frameworks such as RAGAS or DeepEval give a first answer, but not at a level of detail that survives contact with a real project. Scores move without explaining why, practitioners lose trust in the measurements, and evaluation quietly gets abandoned — precisely when it is needed most. So we built our own RAG evaluation suite: an extensive set of quality indicators, measured reliably and attributable to the pipeline stage that caused the failure. We walk through the framework, its components, and how it works in practice.

Reliable measurement changes what is possible. Once quality can be trusted, alternative architectures and configurations stop being a matter of opinion and become hypotheses that can be tested at scale. The next logical question follows: can this be systematized into a workflow that yields optimal RAG performance for a given use case — and across use cases?
This is not just a hypothesis. We present a study that treats RAG optimization as a prediction problem. Surrogate models trained on hundreds of fully evaluated configurations learn the dependencies between design choices and performance metrics (R² ≥ 0.84 across all metrics and domains), reveal consistent parameter interaction patterns across domains with very different corpora, transfer to unseen domains with little or no target-domain data, and recover top-performing configurations from a search space of over 61,000 candidates in seconds rather than days.

We close with where this is heading: enterprises rolling out RAG systems that deliver optimal performance on day one, instead of after months of manual tuning (or never).

What you'll take away:

• Why generic RAG evaluation frameworks break down in enterprise projects, and what a trustworthy evaluation suite needs to cover
• A practical workflow for turning evaluation results into architecture and configuration decisions
• Evidence that RAG design spaces are structured, transferable, and efficiently optimizable