Document Type
Honors Thesis
Abstract
Large Language Models (LLMs) struggle on factual, relation-heavy questions without structured external knowledge. This thesis implements a standards-based pipeline that connects LLMs to Resource Description Framework (RDF) knowledge graphs through the Model Context Protocol (MCP). The system comprises an MCP tool that executes SPARQL against approved endpoints, an Natural Language (NL) to SPARQL step in which the LLM (ChatGPT) generates the SPARQL query using prompt-based templates, followed by a generation stage conditioned on the retrieved graph facts. Final experiments focus on the Rhea knowledge graph. We evaluated the same base model with and without MCP on a 50-question benchmark spanning nine task types, with no follow-up interactions. iv For each question we recorded a binary judgment (MCP better, baseline better, or tie) with a short rationale, and we analyzed proxy indicators such as identifier yield and equation presence. Using Rhea, a curated knowledgebase of biochemical reactions, MCP typically produced richer, better-grounded responses, enumerating more relevant Rhea reactions and primary identifiers (≈10.9 vs. ≈3.1 Rhea IDs per answer), with especially large gains on equation-oriented queries. Failures occurred when natural-language-to-SPARQL generation mis-specified entities or qualifiers, or when endpoints errored—cases where baseline answers could be stronger. Latency was not an evaluation metric; qualitatively, MCP was slower due to tool calls, while baseline could also be slow for heavier reasoning. Allowing brief clarifications is expected to further improve MCP accuracy and coverage.
Disciplines
Computer Engineering | Engineering | Other Computer Engineering
Publication Date
Fall 2025
Language
English
Faculty Mentor of Honors Project
Ashraf Aboulnaga
License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Recommended Citation
Tahmid, Talha, "IMPROVED CONTEXT FOR LLM QUERIES ON KNOWLEDGE GRAPHS USING THE MODEL CONTEXT PROTOCOL" (2025). 2025 Fall Honors Capstones Projects. 23.
https://mavmatrix.uta.edu/honors_fall2025/23