Using an LLM to Learn (and Prove You've Learned) a New Topic
Today's exercise is a two-part process. You will act as both an "Expert" to create a test and a "Learner" to take one.
It's easy to read an AI's explanation and think you understand it. This is passive learning and is often shallow. To ensure active learning, we will use an "Expert-in-the-Loop" method.
Your first task is to act as an expert. You will select a paper or topic you know well and use an LLM as an *assistant* to create a high-quality test for it.
Choose a paper or a niche research area that you know very well. This should be something you could comfortably create an exam for.
Use an LLM to help you generate a *draft* of your test. You are the expert, so you will guide it.
Act as an expert professor in [Your Field]. I am also an expert, and I need your help creating a test for my students. The test will be on the attached paper: "[Paper Title]".
Please help me generate a 10-question test (a mix of 5 high-level conceptual questions and 5 low-level detail-oriented questions) that a student who *fully understands* this paper should be able to answer.
The test document MUST start with a clear title: "Test on: [Paper Title / Topic]".
Do NOT provide the answers. Just list the 10 questions.
This is the most important step in this phase. The AI's draft is just a start. You, as the human expert, must now review, edit, and finalize these questions.
Edit the Google Doc until you are satisfied that it is a high-quality, rigorous test.
Once your expert-vetted test is finalized in your Google Doc (with a clear title!), please save it to the class "Test Pool" folder so your classmates can use it.
Open Test Pool Folder →Now you will switch roles. Your goal is to use an AI tutor to learn the material on one of the expert-vetted tests well enough to pass it.
Go to the class "Test Pool" folder (link is in Part 1) and select one Google Doc. Read the title and choose a test on a topic you are unfamiliar with. Read the 10 questions to see what you will be tested on.
Begin your study session. Use prompts to understand the context, importance, and main ideas of the paper or topic mentioned in your chosen test.
[Attach the paper for your test]
"Explain this paper to me like I'm a graduate student in a completely different field. What is the core problem it's trying to solve?"
"Why is this work important? What specific work does this paper build on or respond to?"
"What is the main 'contribution' or 'finding' of this paper?"
This is where you tackle the parts you find confusing, likely guided by the specific questions on your test.
"My test has a question about 'Equation (2)'. Can you explain that equation in simpler terms? What does each variable represent?"
"I need to understand the 'baseline model' they are comparing against for one of the test questions. Can you explain what it is and why they chose it?"
"Generate pseudo-code for the algorithm described in Figure 4."
You have finished your study session. Before you look at the test questions again, you must perform a self-assessment.
On a scale of 1-10, how well do you feel you understand the material right now? 1 means "not at all," 10 means "I could explain this to a professor."
Make a mental note of this number. This is your "perceived understanding."
A growing body of evidence suggests that the fluency of LLM explanations can be deceptive. Because the answer is so clear and confident, it can make you *feel* like you've learned the material, even if you haven't truly internalized it.
This "illusion of understanding" is exactly what we are testing for. Be honest with your self-assessment. We will compare this number to your actual test score.
This is the moment of truth. You will now test what you *actually* learned.
Open the Google Doc test you selected from the class pool. Close any browser tabs with the AI tutor.
Without asking the LLM for help, write out your answers to all 10 questions from memory (you can do this in a new Google Doc).
Start a new, fresh LLM session (to be safe). Provide the AI with the original paper, the expert-generated questions, and your answers.
Act as a strict but fair grader. I have attached the original research paper. I also have a list of 10 expert-generated questions about this paper, along with my answers.
Your task is to grade my answers. For each question, tell me if my answer is:
- Correct
- Partially Correct
- Incorrect
After each grade, you MUST provide a 1-2 sentence justification for your grade, explaining *why* my answer was right or wrong.
[Attach the original paper]
---
Here are the expert questions and my answers:
[Paste the 10 Questions + Your 10 Answers here]
Read the AI's grades and justifications. Think about the following questions: