TopicNodeParser¶

MedGraphRAG aims to improve the capabilities of LLMs in the medical domain by generating evidence-based results through a novel graph-based Retrieval-Augmented Generation framework, improving safety and reliability in handling private medical data.

TopicNodeParser implements an approximate version of the chunking technique described in the paper.

Here is the technique as outlined in the paper:

Large medical documents often contain multiple themes or diverse content. To process these effectively, we first segment the document into data chunks that conform to the context limitations of Large Language Models (LLMs). Traditional methods such as chunking based on token size or fixed characters typically fail to detect subtle shifts in topics accurately. Consequently, these chunks may not fully capture the intended context, leading to a loss in the richness of meaning.

To enhance accuracy, we adopt a mixed method of character separation coupled with topic-based segmentation. Specifically, we utilize static characters (line break symbols) to isolate individual paragraphs within the document. Following this, we apply a derived form of the text for semantic chunking. Our approach includes the use of proposition transfer, which extracts standalone statements from a raw text Chen et al. (2023). Through proposition transfer, each paragraph is transformed into self-sustaining statements. We then conduct a sequential analysis of the document to assess each proposition, deciding whether it should merge with an existing chunk or initiate a new one. This decision is made via a zero-shot approach by an LLM. To reduce noise generated by sequential processing, we implement a sliding window technique, managing five paragraphs at a time. We continuously adjust the window by removing the first paragraph and adding the next, maintaining focus on topic consistency. We set a hard threshold that the longest chunk cannot excess the context length limitation of LLM. After chunking the document, we construct graph on each individual of the data chunk.

In [ ]:

%pip install llama-index llama-index-node-parser-topic

%pip install llama-index llama-index-node-parser-topic

Setup Data¶

Here we consider a sample text.

Note: The propositions were created by an LLM, which might lead to longer processing times when creating nodes. Exercise caution while experimenting.

In [ ]:

text = """In this paper, we introduce a novel graph RAG method for applying LLMs to the medical domain, which we refer to as Medical Graph RAG (MedRAG). This technique improves LLM performance in the medical domain by response queries with grounded source citations and clear interpretations of medical terminology, boosting the transparency and interpretability of the results. This approach involves a three-tier hierarchical graph construction method. Initially, we use documents provided by users as our top-level source to extract entities. These entities are then linked to a second level consisting of more basic entities previously abstracted from credible medical books and papers. Subsequently, these entities are connected to a third level—the fundamental medical dictionary graph—that provides detailed explanations of each medical term and their semantic relationships. We then construct a comprehensive graph at the highest level by linking entities based on their content and hierarchical connections. This method ensures that the knowledge can be traced back to its sources and the results are factually accurate.

To respond to user queries, we implement a U-retrieve strategy that combines top-down retrieval with bottom-up response generation. The process begins by structuring the query using predefined medical tags and indexing them through the graphs in a top-down manner. The system then generates responses based on these queries, pulling from meta-graphs—nodes retrieved along with their TopK related nodes and relationships—and summarizing the information into a detailed response. This technique maintains a balance between global context awareness and the contextual limitations inherent in LLMs.

Our medical graph RAG provides Intrinsic source citation can enhance LLM transparency, interpretability, and verifiability. The results provides the provenance, or source grounding information, as it generates each response, and demonstrates that an answer is grounded in the dataset. Having the cited source for each assertion readily available also enables a human user to quickly and accurately audit the LLM’s output directly against the original source material. It is super useful in the field of medicine that security is very important, and each of the reasoning should be evidence-based. By using such a method, we construct an evidence-based Medical LLM that the clinician could easiely check the source of the reasoning and calibrate the model response to ensure the safty usage of llm in the clinical senarios.

To evaluate our medical graph RAG, we implemented the method on several popular open and closed-source LLMs, including ChatGPT OpenAI (2023a) and LLaMA Touvron et al. (2023), testing them across mainstream medical Q&A benchmarks such as PubMedQA Jin et al. (2019), MedMCQA Pal et al. (2022), and USMLE Kung et al. (2023). For the RAG process, we supplied a comprehensive medical dictionary as the foundational knowledge layer, the UMLS medical knowledge graph Lindberg et al. (1993) as the foundamental layer detailing semantic relationships, and a curated MedC-K dataset Wu et al. (2023) —comprising the latest medical papers and books—as the intermediate level of data to simulate user-provided private data. Our experiments demonstrate that our model significantly enhances the performance of general-purpose LLMs on medical questions. Remarkably, it even surpasses many fine-tuned or specially trained LLMs on medical corpora, solely using the RAG approach without additional training.
"""

from llama_index.core import Document

documents = [Document(text=text)]

text = """In this paper, we introduce a novel graph RAG method for applying LLMs to the medical domain, which we refer to as Medical Graph RAG (MedRAG). This technique improves LLM performance in the medical domain by response queries with grounded source citations and clear interpretations of medical terminology, boosting the transparency and interpretability of the results. This approach involves a three-tier hierarchical graph construction method. Initially, we use documents provided by users as our top-level source to extract entities. These entities are then linked to a second level consisting of more basic entities previously abstracted from credible medical books and papers. Subsequently, these entities are connected to a third level—the fundamental medical dictionary graph—that provides detailed explanations of each medical term and their semantic relationships. We then construct a comprehensive graph at the highest level by linking entities based on their content and hierarchical connections. This method ensures that the knowledge can be traced back to its sources and the results are factually accurate. To respond to user queries, we implement a U-retrieve strategy that combines top-down retrieval with bottom-up response generation. The process begins by structuring the query using predefined medical tags and indexing them through the graphs in a top-down manner. The system then generates responses based on these queries, pulling from meta-graphs—nodes retrieved along with their TopK related nodes and relationships—and summarizing the information into a detailed response. This technique maintains a balance between global context awareness and the contextual limitations inherent in LLMs. Our medical graph RAG provides Intrinsic source citation can enhance LLM transparency, interpretability, and verifiability. The results provides the provenance, or source grounding information, as it generates each response, and demonstrates that an answer is grounded in the dataset. Having the cited source for each assertion readily available also enables a human user to quickly and accurately audit the LLM’s output directly against the original source material. It is super useful in the field of medicine that security is very important, and each of the reasoning should be evidence-based. By using such a method, we construct an evidence-based Medical LLM that the clinician could easiely check the source of the reasoning and calibrate the model response to ensure the safty usage of llm in the clinical senarios. To evaluate our medical graph RAG, we implemented the method on several popular open and closed-source LLMs, including ChatGPT OpenAI (2023a) and LLaMA Touvron et al. (2023), testing them across mainstream medical Q&A benchmarks such as PubMedQA Jin et al. (2019), MedMCQA Pal et al. (2022), and USMLE Kung et al. (2023). For the RAG process, we supplied a comprehensive medical dictionary as the foundational knowledge layer, the UMLS medical knowledge graph Lindberg et al. (1993) as the foundamental layer detailing semantic relationships, and a curated MedC-K dataset Wu et al. (2023) —comprising the latest medical papers and books—as the intermediate level of data to simulate user-provided private data. Our experiments demonstrate that our model significantly enhances the performance of general-purpose LLMs on medical questions. Remarkably, it even surpasses many fine-tuned or specially trained LLMs on medical corpora, solely using the RAG approach without additional training. """ from llama_index.core import Document documents = [Document(text=text)]

In [ ]:

print(documents[0].get_content())

print(documents[0].get_content())

In this paper, we introduce a novel graph RAG method for applying LLMs to the medical domain, which we refer to as Medical Graph RAG (MedRAG). This technique improves LLM performance in the medical domain by response queries with grounded source citations and clear interpretations of medical terminology, boosting the transparency and interpretability of the results. This approach involves a three-tier hierarchical graph construction method. Initially, we use documents provided by users as our top-level source to extract entities. These entities are then linked to a second level consisting of more basic entities previously abstracted from credible medical books and papers. Subsequently, these entities are connected to a third level—the fundamental medical dictionary graph—that provides detailed explanations of each medical term and their semantic relationships. We then construct a comprehensive graph at the highest level by linking entities based on their content and hierarchical connections. This method ensures that the knowledge can be traced back to its sources and the results are factually accurate.

To respond to user queries, we implement a U-retrieve strategy that combines top-down retrieval with bottom-up response generation. The process begins by structuring the query using predefined medical tags and indexing them through the graphs in a top-down manner. The system then generates responses based on these queries, pulling from meta-graphs—nodes retrieved along with their TopK related nodes and relationships—and summarizing the information into a detailed response. This technique maintains a balance between global context awareness and the contextual limitations inherent in LLMs.

Our medical graph RAG provides Intrinsic source citation can enhance LLM transparency, interpretability, and verifiability. The results provides the provenance, or source grounding information, as it generates each response, and demonstrates that an answer is grounded in the dataset. Having the cited source for each assertion readily available also enables a human user to quickly and accurately audit the LLM’s output directly against the original source material. It is super useful in the field of medicine that security is very important, and each of the reasoning should be evidence-based. By using such a method, we construct an evidence-based Medical LLM that the clinician could easiely check the source of the reasoning and calibrate the model response to ensure the safty usage of llm in the clinical senarios.

To evaluate our medical graph RAG, we implemented the method on several popular open and closed-source LLMs, including ChatGPT OpenAI (2023a) and LLaMA Touvron et al. (2023), testing them across mainstream medical Q&A benchmarks such as PubMedQA Jin et al. (2019), MedMCQA Pal et al. (2022), and USMLE Kung et al. (2023). For the RAG process, we supplied a comprehensive medical dictionary as the foundational knowledge layer, the UMLS medical knowledge graph Lindberg et al. (1993) as the foundamental layer detailing semantic relationships, and a curated MedC-K dataset Wu et al. (2023) —comprising the latest medical papers and books—as the intermediate level of data to simulate user-provided private data. Our experiments demonstrate that our model significantly enhances the performance of general-purpose LLMs on medical questions. Remarkably, it even surpasses many fine-tuned or specially trained LLMs on medical corpora, solely using the RAG approach without additional training.

Setup LLM And Embedding Model¶

In [ ]:

import os

os.environ["OPENAI_API_KEY"] = "sk-..."  # Replace with your OpenAI API key

import os os.environ["OPENAI_API_KEY"] = "sk-..." # Replace with your OpenAI API key

In [ ]:

from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI

embed_model = OpenAIEmbedding()
llm = OpenAI(model="gpt-4o-mini")

from llama_index.embeddings.openai import OpenAIEmbedding from llama_index.llms.openai import OpenAI embed_model = OpenAIEmbedding() llm = OpenAI(model="gpt-4o-mini")

Define TopicNodeParser¶

In [ ]:

from llama_index.node_parser.topic import TopicNodeParser

from llama_index.node_parser.topic import TopicNodeParser

LLM based topic similarity.¶

In [ ]:

node_parser = TopicNodeParser.from_defaults(
    llm=llm,
    max_chunk_size=1000,
    similarity_method="llm",  # can be "llm" or "embedding"
    window_size=2,  # paper suggests window_size=5
)

node_parser = TopicNodeParser.from_defaults( llm=llm, max_chunk_size=1000, similarity_method="llm", # can be "llm" or "embedding" window_size=2, # paper suggests window_size=5 )

In [ ]:

nodes = node_parser.get_nodes_from_documents(documents, show_progress=True)

nodes = node_parser.get_nodes_from_documents(documents, show_progress=True)

Parsing nodes:   0%|          | 0/1 [00:00<?, ?it/s]

Let's inspect chunks.¶

In [ ]:

print(nodes[0].get_content())

print(nodes[0].get_content())

This paper introduces a novel graph RAG method for applying LLMs to the medical domain. The novel graph RAG method is referred to as Medical Graph RAG (MedRAG). The Medical Graph RAG technique improves LLM performance in the medical domain. The Medical Graph RAG technique responds to queries with grounded source citations. The Medical Graph RAG technique provides clear interpretations of medical terminology. The Medical Graph RAG technique boosts the transparency of the results. The Medical Graph RAG technique boosts the interpretability of the results. The Medical Graph RAG approach involves a three-tier hierarchical graph construction method.

In [ ]:

print(nodes[1].get_content())

print(nodes[1].get_content())

Documents provided by users are used as the top-level source to extract entities. The extracted entities are linked to a second level consisting of more basic entities. The more basic entities are previously abstracted from credible medical books and papers. The extracted entities are connected to a third level, which is the fundamental medical dictionary graph. The fundamental medical dictionary graph provides detailed explanations of each medical term. The fundamental medical dictionary graph provides the semantic relationships of medical terms.

In [ ]:

print(nodes[2].get_content())

print(nodes[2].get_content())

A comprehensive graph is constructed at the highest level by linking entities based on their content. A comprehensive graph is constructed at the highest level by linking entities based on their hierarchical connections.

Embedding based topic similarity.¶

In [ ]:

node_parser = TopicNodeParser.from_defaults(
    embed_model=embed_model,
    llm=llm,
    max_chunk_size=1000,
    similarity_method="embedding",  # can be "llm" or "embedding"
    similarity_threshold=0.8,
    window_size=2,  # paper suggests window_size=5
)

node_parser = TopicNodeParser.from_defaults( embed_model=embed_model, llm=llm, max_chunk_size=1000, similarity_method="embedding", # can be "llm" or "embedding" similarity_threshold=0.8, window_size=2, # paper suggests window_size=5 )

In [ ]:

nodes = node_parser.get_nodes_from_documents(documents, show_progress=True)

nodes = node_parser.get_nodes_from_documents(documents, show_progress=True)

Parsing nodes:   0%|          | 0/1 [00:00<?, ?it/s]

Let's inspect chunks.¶

In [ ]:

print(nodes[0].get_content())

print(nodes[0].get_content())

This paper introduces a novel graph RAG method for applying LLMs to the medical domain. The novel graph RAG method is referred to as Medical Graph RAG (MedRAG). The Medical Graph RAG technique improves LLM performance in the medical domain. The Medical Graph RAG technique responds to queries with grounded source citations. The Medical Graph RAG technique provides clear interpretations of medical terminology. The Medical Graph RAG technique boosts the transparency of the results. The Medical Graph RAG technique boosts the interpretability of the results. The Medical Graph RAG approach involves a three-tier hierarchical graph construction method.

In [ ]:

print(nodes[1].get_content())

print(nodes[1].get_content())

Documents provided by users are used as the top-level source to extract entities. The extracted entities are linked to a second level consisting of more basic entities. The more basic entities are previously abstracted from credible medical books and papers. The extracted entities are connected to a third level, which is the fundamental medical dictionary graph. The fundamental medical dictionary graph provides detailed explanations of each medical term. The fundamental medical dictionary graph provides semantic relationships between medical terms. A comprehensive graph is constructed at the highest level by linking entities based on their content. A comprehensive graph is constructed at the highest level by linking entities based on their hierarchical connections. The Medical Graph RAG method ensures that the knowledge can be traced back to its sources. The Medical Graph RAG method ensures that the results are factually accurate.

In [ ]:

print(nodes[2].get_content())

print(nodes[2].get_content())

The U-retrieve strategy is implemented to respond to user queries. The U-retrieve strategy combines top-down retrieval with bottom-up response generation.