EU Researchers Chart a Domain-Specific Path for LLMs in the Social Sciences and Humanities

A new evaluation framework aimed at making large language models trustworthy research assistants for the social sciences and humanities is taking shape under the European project LLMs4EU. Detailed in a paper titled "Integrating Knowledge Graphs and Multilingual Scholarly Corpora for Domain-Adaptive LLMs in SSH," the work was submitted to arXiv on July 7, 2026, and is positioned as a candidate presentation for the LLMs4SSH workshop at the LREC 2026 conference.

The project sits within ALT-EDIC, a European digital infrastructure consortium, and tackles a problem familiar to anyone who has tried to use general-purpose LLMs for serious academic work: these models are rarely built with the linguistic diversity, citation practices, or epistemic rigor that humanities and social science research demands. Literature reviews in SSH fields routinely span multiple languages, draw on decades of scholarship, and require precise sourcing, conditions under which off-the-shelf chatbots tend to falter or fabricate references outright.

Combining Knowledge Graphs with Multilingual Text

The researchers' approach centers on pairing structured knowledge graphs with multilingual scholarly corpora to adapt LLMs specifically for domain use. Rather than treating literature synthesis and bibliographic discovery as generic text-generation tasks, the framework grounds model outputs in curated, structured knowledge that can be traced back to source material. That grounding matters most in disciplines where a misattributed claim or an invented citation can undermine an entire argument.

An Evaluation Framework Built for Rigor

What distinguishes this use case from typical LLM benchmarking efforts is the breadth of its assessment criteria. The team evaluates models across four quantitative dimensions: retrieval accuracy, summarization quality, traceability of generated claims back to source documents, and hallucination detection. That last category addresses one of the most persistent weaknesses of current-generation LLMs, their tendency to produce plausible-sounding but false statements, a flaw with outsized consequences in scholarly contexts.

Crucially, the framework doesn't stop at automated metrics. It incorporates qualitative review from Digital Humanities experts, adding a human-judgment layer that automated benchmarks alone often miss. This hybrid structure reflects a growing recognition across the research community that quantitative scores don't fully capture whether a model's output is actually useful, or trustworthy, for domain specialists.

Why This Matters Beyond One Workshop Paper

The initiative reflects a broader push within European AI policy and research circles to ensure that LLM development doesn't sideline the humanities in favor of STEM-centric benchmarks and use cases. By building infrastructure specifically for multilingual, citation-heavy scholarly work, LLMs4EU and ALT-EDIC are signaling that domain adaptation, not just scale, will determine whether LLMs become genuinely useful tools for researchers outside computer science.

If accepted at LLMs4SSH, the work would offer the conference a concrete case study in how to evaluate LLMs for tasks where factual precision and source traceability aren't optional features but core requirements. For institutions weighing whether to deploy AI tools in scholarly workflows, that kind of rigorous, discipline-specific benchmarking may prove more instructive than another round of general-purpose leaderboard results.