Grappling with LLM Evaluation: A Student’s Field Notes

One of the biggest shifts for me moving from traditional NLP tasks to working with large language models has been evaluation. In my earlier coursework, evaluating a model was often straightforward—calculate the F1-score, check the accuracy, and you had a pretty clear idea of how well you were doing. But with generative models, how do you grade an essay, a poem, or a snippet of code? It’s a question my peers and I debate constantly.

The Old Guard: BLEU, ROUGE, and Their Limits

When you first start, everyone points you to the classic n-gram-based metrics: BLEU, ROUGE, and METEOR. They’re the old guard of text generation evaluation, born from the machine translation era. They work by comparing the overlap of words and phrases (n-grams) between the model’s output and a set of reference texts.

They’re useful for a quick “sanity check,” but we quickly learn their limitations. They can penalize creativity and semantic equivalence. If a model generates a perfectly valid and fluent sentence that just happens to use different words than the reference text, its score plummets. It’s a classic case of “hitting the target but missing the point.”

Moving Towards Meaning with Semantic Metrics

This is where more modern metrics like BERTScore have been a huge step forward. Instead of just matching words, BERTScore uses contextual embeddings to see if the generated text and the reference text mean the same thing. This feels much more aligned with what we actually care about. When I’m fine-tuning a model for summarization, I care more that it captures the essence of the source text than whether it uses the exact same phrasing.

The Gold Standard: Human Evaluation

No matter how sophisticated our automated metrics get, the conversation in our lab always comes back to human evaluation. Is the output fluent? Is it relevant to the prompt? And crucially, is it faithful to the source material without hallucinating facts?

This is the gold standard, but it’s also a huge bottleneck. As a student, running large-scale human evaluations is often prohibitively expensive and time-consuming. Plus, human judgment is subjective. Getting consistent scores from different evaluators (good inter-annotator agreement) is a challenge in itself.

The Bigger Picture: Is the Model Safe and Reliable?

Lately, our program has put a heavy emphasis on responsible AI, so our evaluations are expanding. We’re now looking at metrics for robustness and bias. How does the model perform when you give it a slightly weird or even adversarial input? Does it amplify harmful stereotypes? Tools like StereoSet and CrowS-Pairs provide benchmarks for measuring these social biases, and it’s becoming a standard part of our evaluation checklist. It’s a reminder that a “good” model isn’t just accurate; it’s also fair and safe.

For me, learning to evaluate LLMs has been a lesson in itself. There’s no single number that tells the whole story. A truly comprehensive evaluation is a mosaic, combining automated metrics for a quick read, human feedback for the ground truth, and rigorous testing for safety and bias. It’s one of the most challenging, but also one of the most important, parts of building models responsibly.