LLM Prompt Optimization for Legal-Clause Classification

Terms-of-Service documents are dense with clauses that carry very different legal weight, and classifying them accurately is the foundation for any tool that helps people understand what they are agreeing to. Off-the-shelf LLM prompts handled the task unevenly - strong on some clause types, weak on others - which limited how far the results could be trusted.

The research question was practical: how much can careful prompt design alone improve classification quality, without retraining or fine-tuning the underlying model?

At the Technical University of Munich I optimised LLM prompts to improve accuracy, precision, recall and F1 for classifying legal clauses in Terms-of-Service documents. Rather than changing the model, I focused on prompt-strategy optimisation and iterative feedback loops.

I ran the work as a measured, feedback-driven cycle: test a prompting strategy against the metrics, analyse where it failed, refine the prompt, and repeat. I evaluated across both binary and multi-label classification setups to make sure improvements generalised rather than overfitting one framing of the task.

The iterative approach delivered up to a 20% improvement in the core metrics across both binary and multi-label tasks, showing how much headroom disciplined prompt engineering can unlock before any model changes are needed.