I clarify easy methods to construct an app that generates a number of selection questions (MCQs) on any user-defined topic. The app is extracting Wikipedia articles which can be associated to the consumer’s request and makes use of RAG to question a chat mannequin to generate the questions.
I’ll exhibit how the app works, clarify how Wikipedia articles are retrieved, and present how these are used to invoke a chat mannequin. Subsequent, I clarify the important thing parts of this app in additional element. The code of the app is on the market here.
App Demo
The gif above exhibits the consumer getting into the educational context, the generated MCQ and the suggestions after the consumer submitted a solution.
On the first display screen the consumer describes the context of the MCQs that needs to be generated. After urgent “Submit Context” the app searches for Wikipedia articles which content material matches the consumer question.
The app splits every Wikipedia web page into sections and scores them based mostly on how intently they match the consumer question. These scores are used to pattern the context of the following query which is displayed within the subsequent display screen with 4 decisions to reply. The consumer can choose a selection and submit it by “Submit Reply”. It’s also attainable to skip this query by way of “Subsequent Query”. On this case it’s thought-about that the query didn’t meet the consumer’s expectation. Will probably be averted to make use of the context of this query for the era of following questions. To finish the session the consumer can select “Finish MCQ”.
The following display screen after the consumer submitted a solution exhibits if the reply was right and offers a further clarification. Following, the consumer can both get a brand new query by way of “Subsequent Query” or finish the session with “Finish MCQ”.
The tip session display screen exhibits what number of questions have been appropriately and wrongly answered. Moreover, it additionally incorporates the variety of questions the consumer rejected by way of “Subsequent Query”. If the consumer selects “Begin New Session” the beginning display screen will likely be displayed the place a brand new context for the following session will be supplied.
Idea
The goal of this app is to supply top quality and up-to-date questions on any user-defined matter. Thereby consumer suggestions is taken into account to make sure that the generated questions are assembly the consumer’s expectations.
To retrieve high-quality and up-to-date context, Wikipedia articles are chosen with respect to the consumer’s question. Every article is break up into sections whereas each part is scored based mostly on its similarity with the consumer question. If the consumer rejects a query the respective part rating will likely be downgraded to scale back the chance of sampling this part once more.
This course of will be separated into two workflows:
- Context Retrieval
- Query Technology
That are described under.
Context Retrieval
The workflow how the context of the MCQs is derived from Wikipedia based mostly on the consumer question is proven under.
The consumer inserts the question that describes the context of the MCQs in the beginning display screen. An instance of the consumer question may very well be: “Ask me something about stars and planets”.
To effectively seek for Wikipedia articles this question is transformed into key phrases. The key phrases of the question above are: “Stars”, “Planets”, “Astronomy”, “Photo voltaic System”, and “Galaxy”.
For every key phrase a Wikipedia search is executed of which the highest three pages are chosen. Not every of those 15 pages are an excellent match to the question supplied by the consumer. To take away irrelevant pages on the earliest attainable stage the vector similarity of the embedded consumer question and web page excerpt is calculated. Pages which similarity is under a threshold are filtered out. In our instance 3 of 15 pages have been eliminated.
The remaining pages are learn and divided into sections. As not the complete web page content material could also be associated to the consumer question, splitting the pages into sections permits to pick out components of the web page that match particularly properly to the consumer question. Therefore, for every part the vector similarity in opposition to the consumer question is calculated and sections with low similarity are filtered out. The remaining 12 pages contained 305 sections of which 244 have been saved after filtering.
The final step of the retrieval workflow is to assign a rating to every part with respect to the vector similarity. This rating will later be used to pattern sections for the query era.
Query Technology
The workflow to generate a brand new MCQ is proven under:
Step one is to pattern one part with respect to the part scores. The textual content of this part is inserted along with the consumer question right into a immediate to invoke a chat mannequin. The chat mannequin returns a json formatted response that incorporates the query, reply decisions, and an evidence of the right reply. In case the context supplied just isn’t appropriate to generate a MCQ that addresses the consumer question the chat mannequin is instructed to return a key phrase to determine that the query era was not profitable.
If the query era was profitable, the questions and the reply decisions are exhibited to the consumer. As soon as the consumer submits a solution it’s evaluated if the reply was right, and the reason of the right reply is proven. To generate a brand new query the identical workflow is repeated.
In case the query era was not profitable, or the consumer rejected the query by clicking on “Subsequent Query” the rating of the part that was chosen to generate the immediate is downgraded. Therefore, it’s much less possible that this part will likely be chosen once more.
Key Parts
Subsequent, I’ll clarify some key parts of the workflows in additional element.
Extracting Wiki Articles
Wikipedia articles are extracted in two steps: First a search is run to seek out appropriate pages. After filtering the search outcomes, the pages separated by sections are learn.
Search requests are despatched to this URL. Moreover, a header containing the requestor’s contact data and a parameter dictionary with the search question and the variety of pages to be returned. The output is in json format that may be transformed to a dictionary. The code under exhibits easy methods to run the request:
headers = {'Person-Agent': os.getenv('WIKI_USER_AGENT')}
parameters = {'q': search_query, 'restrict': number_of_results}
response = requests.get(WIKI_SEARCH_URL, headers=headers, params=parameters)
page_info = response.json()['pages']After filtering the search outcomes based mostly on the pages’ excerpts the textual content of the remaining pages is imported utilizing wikipediaapi:
import wikipediaapi
def get_wiki_page_sections_as_dict(page_title, sections_exclude=SECTIONS_EXCLUDE):
wiki_wiki = wikipediaapi.Wikipedia(user_agent=os.getenv('WIKI_USER_AGENT'), language='en')
web page = wiki_wiki.web page(page_title)
if not web page.exists():
return None
def sections_to_dict(sections, parent_titles=[]):
consequence = {'Abstract': web page.abstract}
for part in sections:
if part.title in sections_exclude: proceed
section_title = ": ".be part of(parent_titles + [section.title])
if part.textual content:
consequence[section_title] = part.textual content
consequence.replace(sections_to_dict(part.sections, parent_titles + [section.title]))
return consequence
return sections_to_dict(web page.sections)To entry Wikipedia articles, the app makes use of wikipediaapi.Wikipedia, which requires a user-agent string for identification. It returns a WikipediaPage object which incorporates a abstract of the web page, web page sections with the title and the textual content of every part. Sections are hierarchically organized which means every part is one other WikipediaPage object with one other listing of sections which can be the subsections of the respective part. The operate above reads all sections of a web page and returns a dictionary that maps a concatenation of all part and subsection titles to the respective textual content.
Context Scoring
Sections that match higher to the consumer question ought to get the next chance of being chosen. That is achieved by assigning a rating to every part which is used as weight for sampling the sections. This rating is calculated as follows:
[s_{section}=w_{rejection}s_{rejection}+(1-w_{rejection})s_{sim}]
Every part receives a rating based mostly on two elements: how typically it has been rejected, and the way intently its content material matches the consumer question. These scores are mixed right into a weighted sum. The part rejection rating consists of two parts: the variety of how typically the part’s web page has been rejected over the very best variety of web page rejections and the variety of this part’s rejections over the very best variety of part rejections:
[s_{rejection}=1-frac{1}{2}left( frac{n_{page(s)}}{max_{page}n_{page}} + frac{n_s}{max_{s}n_s} right)]
Immediate Engineering
Immediate engineering is an important facet of the Studying App’s performance. This app is utilizing two prompts to:
- Get key phrases for the wikipedia web page search
- Generate MCQs for sampled context
The template of the key phrase era immediate is proven under:
KEYWORDS_TEMPLATE = """
You are an assistant to generate key phrases to seek for Wikipedia articles that include content material the consumer needs to study.
For a given consumer question return at most {n_keywords} key phrases. Make sure that each key phrase is an effective match to the consumer question.
Somewhat present fewer key phrases than key phrases which can be much less related.
Directions:
- Return the key phrases separated by commas
- Don't return the rest
"""This method message is concatenated with a human message containing the consumer question to invoke the Llm mannequin.
The parameter n_keywords set the utmost variety of key phrases to be generated. The directions make sure that the response will be simply transformed to a listing of key phrases. Regardless of these directions, the LLM typically returns the utmost variety of key phrases, together with some much less related ones.
The MCQ immediate incorporates the sampled part and invokes the chat mannequin to reply with a query, reply decisions, and an evidence of the right reply in a machine-readable format.
MCQ_TEMPLATE = """
You're a studying app that generates multiple-choice questions based mostly on instructional content material. The consumer supplied the
following request to outline the educational content material:
"{user_query}"
Based mostly on the consumer request, following context was retrieved:
"{context}"
Generate a multiple-choice query instantly based mostly on the supplied context. The right reply have to be explicitly said
within the context and may all the time be the primary choice within the decisions listing. Moreover, present an evidence for why
the right reply is right.
Variety of reply decisions: {n_choices}
{previous_questions}{rejected_questions}
The JSON output ought to comply with this construction (for variety of decisions = 4):
{{"query": "Your generated query based mostly on the context", "decisions": ["Correct answer (this must be the first choice)","Distractor 1","Distractor 2","Distractor 3"], "clarification": "A quick clarification of why the right reply is right."}}
Directions:
- Generate one multiple-choice query strictly based mostly on the context.
- Present precisely {n_choices} reply decisions, guaranteeing the primary one is the right reply.
- Embrace a concise clarification of why the right reply is right.
- Don't return the rest than the json output.
- The supplied clarification mustn't assume the consumer is conscious of the context. Keep away from formulations like "As said within the textual content...".
- The response have to be machine readable and never include line breaks.
- Examine whether it is attainable to generate a query based mostly on the supplied context that's aligned with the consumer request. If it's not attainable set the generated query to "{fail_keyword}".
"""
The inserted parameters are:
user_query: textual content of consumer questioncontext: textual content of sampled partn_choices: variety of reply decisionsprevious_questions: instruction to not repeat earlier questions with listing of all earlier questionsrejected_questions: instruction to keep away from questions of comparable nature or context with listing of rejected questionsfail_keyword: key phrase that signifies that query couldn’t be generated
Together with earlier questions reduces the prospect that the chat mannequin repeats questions. Moreover, by offering rejected questions, the consumer’s suggestions is taken into account when producing new questions. The instance ought to make sure that the generated output is within the right format in order that it may be simply transformed to a dictionary. Setting the right reply as the primary selection avoids requiring a further output that signifies the right reply. When exhibiting the alternatives to the consumer the order of decisions is shuffled. The final instruction defines what output needs to be supplied in case it’s not attainable to generate a query matching the consumer question. Utilizing a standardized key phrase makes it straightforward to determine when the query era has failed.
Streamlit App
The app is constructed utilizing Streamlit, an open-source app framework in Python. Streamlit has many capabilities that permit so as to add web page components with just one line of code. Like for instance the aspect wherein the consumer can write the question is created by way of:
context_text = st.text_area("Enter the context for MCQ questions:")the place context_text incorporates the string, the consumer has written. Buttons are created with st.button or st.radio the place the returned variable incorporates the knowledge if the button has been pressed or what worth has been chosen.
The web page is generated top-down by a script that defines every aspect sequentially. Each time the consumer is interacting with the web page, e.g. by clicking on a button the script will be re-run with st.rerun(). When re-running the script, it is very important carry over data from the earlier run. That is performed by st.session_state which might include any objects. For instance, the MCQ generator occasion is assigned to session states as:
st.session_state.mcq_generator = MCQGenerator()in order that when the context retrieval workflow has been executed, the discovered context is on the market to generate a MCQ on the subsequent web page.
Enhancements
There are a lot of choices to boost this app. Past Wikipedia, customers might additionally add their very own PDFs to generate questions from customized supplies—comparable to lecture slides or textbooks. This could allow the consumer to generate questions on any context, for instance it may very well be used to arrange for exams by importing course supplies.
One other facet that may very well be improved is to optimize the context choice to reduce the variety of rejected questions by the consumer. As an alternative of updating scores, additionally a ML mannequin may very well be skilled to foretell how possible it’s {that a} query will likely be rejected with respect to options like similarity to accepted and rejected questions. Each time one other query is rejected this mannequin may very well be retrained.
Additionally, the generated query may very well be saved in order that when a consumer needs to repeat the educational train these questions may very well be used once more. An algorithm may very well be utilized to pick out beforehand wrongly answered questions extra steadily to deal with enhancing the learner’s weaknesses.
Abstract
This text showcases how retrieval-augmented era (RAG) can be utilized to construct an interactive studying app that generates high-quality, context-specific multiple-choice questions from Wikipedia articles. By combining keyword-based search, semantic filtering, immediate engineering, and a feedback-driven scoring system, the app dynamically adapts to consumer preferences and studying targets. Leveraging instruments like Streamlit permits fast prototyping and deployment, making this an accessible framework for educators, college students, and builders alike. With additional enhancements—comparable to customized doc uploads, adaptive query sequencing, and machine learning-based rejection prediction—the app holds robust potential as a flexible platform for customized studying and self-assessment.
Additional Studying
To study extra about RAGs I can advocate these articles from Shaw Talebi and Avishek Biswas. Harrison Hoffman wrote two wonderful tutorials on embeddings and vector databases and building an LLM RAG Chatbot. How you can handle states in streamlit will be present in Baertschi’s article.
If not said in any other case, all photos have been created by the writer.
