How to Measure the Reliability of a Large Language Model’s Response

The essential precept of Massive Language Fashions (LLMs) may be very easy: to foretell the following phrase (or token) in a sequence of phrases based mostly on statistical patterns of their coaching information. Nonetheless, this seemingly easy functionality seems to be extremely subtle when it will probably do numerous superb duties resembling textual content summarization, thought era, brainstorming, code era, data processing, and content material creation. That stated, LLMs wouldn’t have any reminiscence no do they really “perceive” something, apart from sticking to their fundamental operate: predicting the following phrase.

The method of next-word prediction is probabilistic. The LLM has to pick out every phrase from a likelihood distribution. Within the course of, they usually generate false, fabricated, or inconsistent content material in an try to supply coherent responses and fill in gaps with plausible-looking however incorrect data. This phenomenon is named hallucination, an inevitable, well-known characteristic of LLMs that warrants validation and corroboration of their outputs. 

Retrieval increase era (RAG) strategies, which make an LLM work with exterior information sources, do reduce hallucinations to some extent, however they can not fully eradicate them. Though superior RAGs can present in-text citations and URLs, verifying these references could possibly be hectic and time-consuming. Due to this fact, we want an goal criterion for assessing the reliability or trustworthiness of an LLM’s response, whether or not it’s generated from its personal information or an exterior information base (RAG). 

On this article, we’ll talk about how the output of an LLM might be assessed for trustworthiness by a reliable language mannequin which assigns a rating to the LLM’s output. We are going to first talk about how we will use a reliable language mannequin to assign scores to an LLM’s reply and clarify trustworthiness. Subsequently, we’ll develop an instance RAG with LlamaParse and Llamaindex that assesses the RAG’s solutions for trustworthiness.

Your entire code of this text is offered within the jupyter pocket book on GitHub. 

Assigning a Trustworthiness Rating to an LLM’s Reply

To exhibit how we will assign a trustworthiness rating to an Llm’s response, I’ll use Cleanlab’s Trustworthy Language Model (TLM). Such TLMs use a mix of uncertainty quantification and consistency evaluation to compute trustworthiness scores and explanations for LLM responses.

Cleanlab gives free trial APIs which might be obtained by creating an account at their web site. We first want to put in Cleanlab’s Python consumer:

pip set up --upgrade cleanlab-studio

Cleanlab helps a number of proprietary fashions resembling ‘gpt-4o’, ‘gpt-4o-mini’, ‘o1-preview’, ‘claude-3-sonnet’, ‘claude-3.5-sonnet’, ‘claude-3.5-sonnet-v2’ and others. Right here is how TLM assigns a trustworhiness rating to gpt-4o’s reply. The trustworthiness rating ranges from 0 to 1, the place increased values point out better trustworthiness. 

from cleanlab_studio import Studio
studio = Studio("")  # Get your API key from above
tlm = studio.TLM(choices={"log": ["explanation"], "mannequin": "gpt-4o"}) # GPT, Claude, and so on
#set the immediate
out = tlm.immediate("What number of vowels are there within the phrase 'Abracadabra'.?")
#the TLM response comprises the precise output 'response', trustworthiness rating and clarification
print(f"Mannequin's response = {out['response']}")
print(f"Trustworthiness rating = {out['trustworthiness_score']}")
print(f"Clarification = {out['log']['explanation']}")

The above code examined the response of gpt-4o for the query “What number of vowels are there within the phrase ‘Abracadabra’.?”. The TLM’s output comprises the mannequin’s reply (response), trustworthiness rating, and clarification. Right here is the output of this code.

Mannequin's response = The phrase "Abracadabra" comprises 6 vowels. The vowels are: A, a, a, a, a, and a.
Trustworthiness rating = 0.6842228802750124
Clarification = This response is untrustworthy attributable to an absence of consistency in attainable responses from the mannequin. Here is one inconsistent alternate response that the mannequin thought-about (which is probably not correct both):
5.

It may be seen how probably the most superior language mannequin hallucinates for such easy duties and produces the fallacious output. Right here is the response and trustworthiness rating for a similar query for claude-3.5-sonnet-v2.

Mannequin's response = Let me depend the vowels in 'Abracadabra':
A-b-r-a-c-a-d-a-b-r-a

The vowels are: A, a, a, a, a

There are 5 vowels within the phrase 'Abracadabra'.
Trustworthiness rating = 0.9378276048845285
Clarification = Didn't discover a motive to doubt trustworthiness.

claude-3.5-sonnet-v2 produces the right output. Let’s examine the 2 fashions’ responses to a different query.

from cleanlab_studio import Studio
import markdown
from IPython.core.show import show, Markdown

# Initialize the Cleanlab Studio with API key
studio = Studio("")  # Change together with your precise API key

# Record of fashions to judge
fashions = ["gpt-4o", "claude-3.5-sonnet-v2"]

# Outline the immediate
prompt_text = "Which one among 9.11 and 9.9 is larger?"

# Loop by way of every mannequin and consider
for mannequin in fashions:
   tlm = studio.TLM(choices={"log": ["explanation"], "mannequin": mannequin})
   out = tlm.immediate(prompt_text)
  
   md_content = f"""
## Mannequin: {mannequin}

**Response:** {out['response']}

**Trustworthiness Rating:** {out['trustworthiness_score']}

**Clarification:** {out['log']['explanation']}

---
"""
   show(Markdown(md_content))

Right here is the response of the 2 fashions:

We will additionally generate a trustworthiness rating for open-source LLMs. Let’s test the latest, much-hyped open-source LLM: deepseek-R1. I’ll use DeepSeek-R1-Distill-Llama-70B, based mostly on Meta’s Llama-3.3–70B-Instruct mannequin and distilled from DeepSeek’s bigger 671-billion parameter Combination of Specialists (MoE) mannequin. Knowledge distillation is a Machine Learning approach that goals to switch the learnings of a big pre-trained mannequin, the “trainer mannequin,” to a smaller “scholar mannequin.”

import streamlit as st
from langchain_groq.chat_models import ChatGroq
import os
os.environ["GROQ_API_KEY"]=st.secrets and techniques["GROQ_API_KEY"]
# Initialize the Groq Llama Prompt mannequin
groq_llm = ChatGroq(mannequin="deepseek-r1-distill-llama-70b", temperature=0.5)
immediate = "Which one among 9.11 and 9.9 is larger?"
# Get the response from the mannequin
response = groq_llm.invoke(immediate)
#Initialize Cleanlab's studio
studio = Studio("226eeab91e944b23bd817a46dbe3c8ae") 
cleanlab_tlm = studio.TLM(choices={"log": ["explanation"]})  #for explanations
#Get the output containing trustworthiness rating and clarification
output = cleanlab_tlm.get_trustworthiness_score(immediate, response=response.content material.strip())
md_content = f"""
## Mannequin: {mannequin}
**Response:** {response.content material.strip()}
**Trustworthiness Rating:** {output['trustworthiness_score']}
**Clarification:** {output['log']['explanation']}
---
"""
show(Markdown(md_content))

Right here is the output of deepseek-r1-distill-llama-70b mannequin.

Creating a Reliable RAG

We are going to now develop an RAG to exhibit how we will measure the trustworthiness of an LLM response in RAG. This RAG will probably be developed by scraping information from given hyperlinks, parsing it in markdown format, and making a vector retailer.

The next libraries have to be put in for the following code.

pip set up llama-parse llama-index-core llama-index-embeddings-huggingface 
llama-index-llms-cleanlab requests beautifulsoup4 pdfkit nest-asyncio

To render HTML into PDF format, we additionally want to put in wkhtmltopdf command line software from their website.

The next libraries will probably be imported:

from llama_parse import LlamaParse
from llama_index.core import VectorStoreIndex
import requests
from bs4 import BeautifulSoup
import pdfkit
from llama_index.readers.docling import DoclingReader
from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.cleanlab import CleanlabTLM
from typing import Dict, Record, ClassVar
from llama_index.core.instrumentation.occasions import BaseEvent
from llama_index.core.instrumentation.event_handlers import BaseEventHandler
from llama_index.core.instrumentation import get_dispatcher
from llama_index.core.instrumentation.occasions.llm import LLMCompletionEndEvent
import nest_asyncio
import os

The subsequent steps will contain scraping information from given URLs utilizing Python’s BeautifulSoup library, saving the scraped information in PDF file(s) utilizing pdfkit, and parsing the info from PDF(s) to markdown file utilizing LlamaParse which is a genAI-native doc parsing platform constructed with LLMs and for LLM use circumstances.

We are going to first configure the LLM for use by CleanlabTLM and the embedding mannequin (Huggingface embedding mannequin BAAI/bge-small-en-v1.5) that will probably be used to compute the embeddings of the scraped information to create the vector retailer.

choices = {
   "mannequin": "gpt-4o",
   "max_tokens": 512,
   "log": ["explanation"]
}
llm = CleanlabTLM(api_key="", choices=choices)#Get your free API from https://cleanlab.ai/
Settings.llm = llm
Settings.embed_model = HuggingFaceEmbedding(
   model_name="BAAI/bge-small-en-v1.5"
)

We are going to now outline a customized occasion handler, GetTrustworthinessScore, that’s derived from a base occasion handler class. This handler will get triggered by the top of an LLM completion and extracts a trustworthiness rating from the response metadata. A helper operate, display_response, shows the LLM’s response together with its trustworthiness rating.

# Occasion Handler for Trustworthiness Rating
class GetTrustworthinessScore(BaseEventHandler):
   occasions: ClassVar[List[BaseEvent]] = []
   trustworthiness_score: float = 0.0
   @classmethod
   def class_name(cls) -> str:
       return "GetTrustworthinessScore"
   def deal with(self, occasion: BaseEvent) -> Dict:
       if isinstance(occasion, LLMCompletionEndEvent):
           self.trustworthiness_score = occasion.response.additional_kwargs.get("trustworthiness_score", 0.0)
           self.occasions.append(occasion)
       return {}
# Helper operate to show LLM's response
def display_response(response):
   response_str = response.response
   trustworthiness_score = event_handler.trustworthiness_score
   print(f"Response: {response_str}")
   print(f"Trustworthiness rating: {spherical(trustworthiness_score, 2)}")

We are going to now generate PDFs by scraping information from given URLs. For demonstration, we’ll scrap information solely from this Wikipedia article about large language models (Artistic Commons Attribution-ShareAlike 4.0 License). 

Observe: Readers are suggested to at all times double-check the standing of the content material/information they’re about to scrape and guarantee they’re allowed to take action. 

The next piece of code scrapes information from the given URLs by making an HTTP request and utilizing BeautifulSoup Python library to parse the HTML content material. HTML content material is cleaned by changing protocol-relative URLs to absolute ones. Subsequently, the scraped content material is transformed right into a PDF file(s) utilizing pdfkit.

##########################################
# PDF Era from A number of URLs
##########################################
# Configure wkhtmltopdf path
wkhtml_path = r'C:Program Fileswkhtmltopdfbinwkhtmltopdf.exe'
config = pdfkit.configuration(wkhtmltopdf=wkhtml_path)
# Outline URLs and assign doc names
urls = {
   "LLMs": "https://en.wikipedia.org/wiki/Large_language_model"
}
# Listing to save lots of PDFs
pdf_directory = "PDFs"
os.makedirs(pdf_directory, exist_ok=True)
pdf_paths = {}
for doc_name, url in urls.objects():
   attempt:
       print(f"Processing {doc_name} from {url} ...")
       response = requests.get(url)
       soup = BeautifulSoup(response.textual content, "html.parser")
       main_content = soup.discover("div", {"id": "mw-content-text"})
       if main_content is None:
           increase ValueError("Primary content material not discovered")
       # Change protocol-relative URLs with absolute URLs
       html_string = str(main_content).exchange('src="https://', 'src="https://').exchange('href="https://', 'href="https://')
       pdf_file_path = os.path.be part of(pdf_directory, f"{doc_name}.pdf")
       pdfkit.from_string(
           html_string,
           pdf_file_path,
           choices={'encoding': 'UTF-8', 'quiet': ''},
           configuration=config
       )
       pdf_paths[doc_name] = pdf_file_path
       print(f"Saved PDF for {doc_name} at {pdf_file_path}")
   besides Exception as e:
       print(f"Error processing {doc_name}: {e}")

After producing PDF(s) from the scraped information, we parse these PDFs utilizing LlamaParse. We set the parsing directions to extract the content material in markdown format and parse the doc(s) page-wise together with the doc identify and web page quantity. These extracted entities (pages) are known as nodes. The parser iterates over the extracted nodes and updates every node’s metadata by appending a quotation header which facilitates later referencing.

##########################################
# Parse PDFs with LlamaParse and Inject Metadata
##########################################

# Outline parsing directions (in case your parser helps it)
parsing_instructions = """Extract the doc content material in markdown.
Cut up the doc into nodes (for instance, by web page).
Guarantee every node has metadata for doc identify and web page quantity."""
      
# Create a LlamaParse occasion
parser = LlamaParse(
   api_key="",  #Change together with your precise key
   parsing_instructions=parsing_instructions,
   result_type="markdown",
   premium_mode=True,
   max_timeout=600
)
# Listing to save lots of mixed Markdown information (one per PDF)
output_md_dir = os.path.be part of(pdf_directory, "markdown_docs")
os.makedirs(output_md_dir, exist_ok=True)
# Record to carry all up to date nodes for indexing
all_nodes = []
for doc_name, pdf_path in pdf_paths.objects():
   attempt:
       print(f"Parsing PDF for {doc_name} from {pdf_path} ...")
       nodes = parser.load_data(pdf_path)  # Returns an inventory of nodes
       updated_nodes = []
       # Course of every node: replace metadata and inject quotation header into the textual content.
       for i, node in enumerate(nodes, begin=1):
           # Copy present metadata (if any) and add our personal keys.
           new_metadata = dict(node.metadata) if node.metadata else {}
           new_metadata["document_name"] = doc_name
           if "page_number" not in new_metadata:
               new_metadata["page_number"] = str(i)
           # Construct the quotation header.
           citation_header = f"[{new_metadata['document_name']}, web page {new_metadata['page_number']}]nn"
           # Prepend the quotation header to the node's textual content.
           updated_text = citation_header + node.textual content
           new_node = node.__class__(textual content=updated_text, metadata=new_metadata)
           updated_nodes.append(new_node)
       # Save a single mixed Markdown file for the doc utilizing the up to date node texts.
       combined_texts = [node.text for node in updated_nodes]
       combined_md = "nn---nn".be part of(combined_texts)
       md_filename = f"{doc_name}.md"
       md_filepath = os.path.be part of(output_md_dir, md_filename)
       with open(md_filepath, "w", encoding="utf-8") as f:
           f.write(combined_md)
       print(f"Saved mixed markdown for {doc_name} to {md_filepath}")
       # Add the up to date nodes to the worldwide listing for indexing.
       all_nodes.prolong(updated_nodes)
       print(f"Parsed {len(updated_nodes)} nodes from {doc_name}.")
   besides Exception as e:
       print(f"Error parsing {doc_name}: {e}")

We now create a vector retailer and a question engine. We outline a buyer immediate template to information the LLM’s conduct in answering the questions. Lastly, we create a question engine with the created index to reply queries. For every question, we retrieve the highest 3 nodes from the vector retailer based mostly on their semantic similarity with the question. The LLM makes use of these retrieved nodes to generate the ultimate reply.

##########################################
# Create Index and Question Engine
##########################################
# Create an index from all nodes.
index = VectorStoreIndex.from_documents(paperwork=all_nodes)
# Outline a customized immediate template that forces the inclusion of citations.
prompt_template = """
You might be an AI assistant with experience in the subject material.
Reply the query utilizing ONLY the offered context.
Reply in well-formatted Markdown with bullets and sections wherever crucial.
If the offered context doesn't assist a solution, reply with "I do not know."
Context:
{context_str}
Query:
{query_str}
Reply:
"""
# Create a question engine with the customized immediate.
query_engine = index.as_query_engine(similarity_top_k=3, llm=llm, prompt_template = prompt_template)
print("Mixed index and question engine created efficiently!")

Now let’s take a look at the RAG for some queries and their corresponding trustworthiness scores.

question = "When is combination of specialists strategy used?"
response = query_engine.question(question)
display_response(response)

question = "How do you examine Deepseek mannequin with OpenAI's fashions?"
response = query_engine.question(question)
display_response(response)

Assigning a trustworthiness rating to LLM’s response, whether or not generated by way of direct inference or RAG, helps to outline the reliability of AI’s output and prioritize human verification the place wanted. That is significantly vital for vital domains the place a fallacious or unreliable response might have extreme penalties. 

That’s all people! Should you just like the article, please comply with me on Medium and LinkedIn.