The conversation isn’t stored in my account. The Evidence Checker simply generates a prompt and opens Gemini (https://gemini.google.com) in a new browser tab.
Before anything else, check whether your previous session might still be open in another browser tab or window. Sometimes those sessions remain open even if you’ve navigated away from them. Check each tab -- you might find it there.
Thanks for getting back to me, Jim. Damn...It's lost. Important to let folks know that if they use your app to access an AI it's running in your google ai studio, not their own AI account.
How about the licensing body gives teachers access to research databases and journals so they can look for themselves? BCBAs and other professionals get such access as part of their credentialing. How about we stoop pushing fads and gimmicks at teachers and give the actual research so they can use the actual research to teach in a research-based way.
The research to practice pipeline breakdown is so real! Teachers don't have time, academics live in the ivory tower, and to your point--there just isn't a reliable translator to provide a trustworthy and detailed enough summary that is actually relevant. Looking forward to using your tool! This is the stuff we should be using AI for.
I appreciate the way you're trying to scaffold educators in their use of AI to seek more clarity around educational research. The flattening at many layers is a problem, and the best thing we could do is arm teachers with critical literacies for evaluating evidence. Can you build prompts for curriculum publishers, too?😆
I read your generator, and I think a component that might be missing is related to the population studied in whatever evidence the AI draws from.
Many studies remain locked behind paywalls with human verification systems — which means AI is typically limited to scraping abstracts, and abstracts generalize in their own way. But even with full-text access, there's another problem: we apply findings to populations they were never designed for. We've been generalizing medical studies conducted on males to females for centuries. The same pattern holds in education. What produces statistically significant effects in a sample of secondary students could be wildly inappropriate practice for early childhood or elementary populations — and sample size and type matters enormously in how much weight any single study can bear.
Adding a population and sample specificity layer to the prompt generator would go a long way. Though that adjustment itself reveals more tension: prompts designed to privilege experimental and quasi-experimental designs can disadvantage qualitative work — which can be precisely the methodology you need to consider when studying the messy work of educating young children, capturing classroom dynamics, or asking questions of equity and access. Some of the most foundational work in education lives in qualitative studies that reveal implications for quantitative research designs.
These are hard problems to solve with a single prompt, but they're worth naming as you continue developing the tool.
This is such a great tool to make AI use more efficient, productive, and informative. You make a great point about the breadth and depth of research out there that it is hard on a daily basis to incorporate that research into practice, this tool helps both in giving clear next steps for tomorrow but also clearing away noise for further study.
Nice! I just tested the prompt with a question about sight words ("Should we teach "sight words"? If so how should we support students to learn sight words"? (a topic for which there is a lot of bad information about there) and Claude sent back a good, evidence-backed response. (Haven't tested it out on the others).
I think I'd want to alter that prompt only slightly: "(...) At this point, I don’t want you to behave like a motivational speaker, advocate, consultant, or influencer. Instead, act like a careful educational researcher who is trying to accurately represent empirical evidence—which you will need to source—but also to detect any grey zones. Respond clearly, avoid but avoid pedagogical jargon (even if it may be cited in the sources)."
Thanks for the prompter page "Evidence" Checker. May I suggest a slightly altered passage:
"At this point, the response shouldn't sound stylistically like a motivational speaker, advocate, consultant, or influencer. Instead, act like a careful educational researcher who's trying to accurately represent empirical evidence—which you'll need to source. Respond clearly and as concisely as possible, but avoid pedagogical jargon (even if it may be cited in the sources)."
This is a fascinating post, and it resonates strongly with my experience in schools. Teachers are constantly navigating conflicting educational claims, and I’ve seen how difficult it can be to separate trend from evidence. A tool like The Evidence Checker could shift the dynamic by putting high-quality research directly in teachers’ hands, allowing them to make informed decisions rather than relying on simplified or ideologically driven guidance. In practice, the challenge is connecting evidence to context—how it actually plays out in classrooms with diverse learners. From my perspective, the real power of this approach is in enabling teachers to interrogate claims, ask follow-up questions, and adapt strategies thoughtfully.
This is really great idea and thank you for your work on it. As someone observing and with a stake in our school district's move from Lucy Calkins to stuctured literacy, and now working on math, I can see how teachers and administrators badly need a tool like this. It was, after all, not looking at research that led our district to choose TCRWP in the first place in the aughts. It was curriculum choice by vibes.
One thing I'm not quite clear on: the AI is trained on the internet writ large? And we presume that a subset of the data are these peer reviewed educational research studies?
Or, is this a download of an AI to a local server, and fed only the peer reviewed studies?
Actually, we simply leave it up to the user to choose their favourite AI system: ChatGPT, Perplexity, Gemini, or Claude. All we do is provide the prompt. So the AI's are all using their default training sets to answer questions. I suspect the training set for all of these platforms includes the entire web, including all of the recent academic literature. We considered building the tool so that it would use OpenAI API to send the prompt to ChatGPT automatically. We may still do that... it would make the tool easier to use. However, we do like the transparency of letting the user choose for themselves which GenAI tool to use, and without having our code sitting in the middle. It makes it clear to everyone that we're not interfering in some way with the responses that users get.
I have a number of concerns about this approach, and I’m perhaps grossly misinformed about it. First of all, with regard to transparency, if we have identified a need for such a tool for teachers to be able to access research in the field of education, their wheelhouse, how can we expect teachers to understand the differences among any of these proprietary AIs? It seems at minimum we should rely on open source models. Secondly if all of these are trained on the web in general then any claim made by anyone- a homeschooler on a Reddit sub, essays presented as data by an educational publisher, or educators who make claims without any evidence, can show up in LLMs, correct? Even within the peer reviewed literature there have been issues with replicating research results. How do you know from LLMs you’re getting relevant, high quality results? Isn’t this the very problem we’re trying to solve?
A response to "The Evidence Checker" — and a proposal for what comes next
There's a tool making rounds in education circles called The Evidence Checker. The idea is simple and genuinely useful: teachers type a question about pedagogy, copy a structured prompt, paste it into an AI, and get a more rigorous, evidence-informed answer than they'd find on social media or in a typical professional development session.
I think it's a good tool. I also think it solves the wrong problem — or at least, only half of it.
Here's the thing the article gets exactly right: teachers are surrounded by confident educational claims with no reliable way to evaluate them. Inquiry learning. Explicit instruction. Learning styles. Retrieval practice. Every workshop comes with certainty. The underlying evidence is rarely examined. The Evidence Checker helps teachers push back on that.
But there's a deeper structural problem the article doesn't address. And no teacher-facing evidence tool can solve it.
Even a perfectly evidence-informed teacher delivers instruction to a distribution of learners. The teacher-first approach optimizes for an imagined average student. No student is average.
The student sitting in the third row who scored 72% on the last test may have a fundamental misconception, a working memory overload on that particular day, an anxiety response to timed assessments, or a motivation deficit rooted in a belief that she's just "bad at this." These require entirely different responses. No synthesis of research literature tells a teacher which one it is — certainly not for that specific student, on that specific day.
The evidence that matters most for any individual student is not in a journal article. It's in that student's individual learning record: IF we choose to build one.
==========================================
What a student-first approach actually looks like
The foundational reframe is this: the student is not the subject of measurement. The student is the scientist experimenting.
Evidence about how they learn is not collected by a system and delivered to a teacher. It's generated by the student, interpreted by the student, and owned by the student. The system doesn't tell the student that re-reading is less effective than retrieval practice. The student discovers this through their own data.
That distinction is not cosmetic. When students generate their own evidence, the resulting behavioral change is far more durable than when they receive a teacher's recommendation. Self-generated evidence is experienced as discovery, not compliance.
A student who graduates with a rich, evidence-based understanding of how they specifically learn has something more valuable than any particular body of knowledge: the ability to learn anything efficiently for the rest of their life.
What does this look like in practice? Two measurement layers are running simultaneously.
The first layer is course-specific. It captures the signals that matter for this material: prior knowledge, acquisition rate, which specific misconceptions appear (not just whether answers are wrong), how much is retained after 48 hours versus one week, and whether the student's effort is producing learning or just producing fatigue.
The second layer is the one that compounds. It's a cross-course meta-learning profile — a growing record of how this particular person learns. Which study methods produce durable retention for them? At what point in a session does their comprehension degrade? Whether they learn new concepts better from examples before theory or theory before examples. How do their performances change under time pressure relative to their actual knowledge? This layer follows the student across every course they take. It gets richer with each one.
========================================
The self-discovery loop
The framework isn't a measurement system that happens to students. It's a structured experimentation cycle that students conduct — supported by the system and the teacher.
A student forms a hypothesis: "I think I learn better by re-reading than by self-testing." They log it, run the experiment across a few sessions, compare their predicted score to their actual score on a delayed retention check, and update their self-model accordingly. The system facilitates this. The teacher reads the signal, not the raw data.
This is how the goal structure changes, too. In the current model, goals are set by the curriculum: coverage, assessment performance, and grade distributions. Student goals are often different. They want to genuinely understand something. They want to apply it in a context that matters to them. They want to know whether they're actually learning, NOT just performing on the next test.
A student-first framework asks three layers of questions before and during each course:
- What do you want to be able to do that you can't do now?
- What learning behavior do you want to test?
(and the one almost no educational system ever asks)
- What do you want to discover about yourself as a learner?
What this does to the teacher's role
This framework doesn't diminish the teacher. It gives the teacher better information to do what ONLY a human can do: have a meaningful, specific conversation with a specific student about their specific learning journey.
The teacher's dashboard in this model doesn't show class averages. It surfaces anomalies: "This student's predicted versus actual scores show systematic overconfidence on conceptual questions, but accurate calibration on procedural ones." "This student's retention drops sharply after 48 hours — they may need a different spacing schedule." "This student has discovered retrieval practice works for them but hasn't yet applied it to the current unit."
The system might suggest: "Based on this student's error signature, their confusion appears to be conceptual, not procedural. A Socratic conversation about the underlying principle may be more effective than additional practice problems." The teacher decides whether and how to act on this — informed by richer individual data than any general research synthesis can provide.
The Evidence Checker asks: What does research say teachers should do?
The student-first framework asks: What does this student's learning signal say they need next?
Both questions matter. Only one is currently being asked systematically!
The article's closing line is more right than it knows
The Evidence Checker piece ends with this: "When access to evidence becomes more widely distributed, the professional conversation may begin to shift."
I agree. And I'd push it one step further.
The most transformative shift is not distributing research evidence to teachers more efficiently. It's distributing learning evidence to students — about themselves — in real time. Students deserve what teachers now have access to through tools like The Evidence Checker: the ability to question claims, compare approaches, and make evidence-informed judgments. The claims students most need to evaluate aren't claims about pedagogy. They're claims about themselves. Their own patterns. Their own most effective path to understanding.
Teaching is too important to be driven by slogans and trends. The article is right about that.
Learning is too important to be driven by system goals, institutional incentives, and population-level research averages.
I'm not proposing to replace the teacher with an algorithm. Rather, I am proposing to give the teacher better information about specific students — and to give students something no educational system has ever systematically offered them: the evidence to understand how they, specifically, learn best.
Hi Meri,
Thanks for your message.
The conversation isn’t stored in my account. The Evidence Checker simply generates a prompt and opens Gemini (https://gemini.google.com) in a new browser tab.
Before anything else, check whether your previous session might still be open in another browser tab or window. Sometimes those sessions remain open even if you’ve navigated away from them. Check each tab -- you might find it there.
You could also try:
1. Going directly to Gemini: https://gemini.google.com
2. Checking which Google account is active in the top-right corner
3. Looking through your Gemini chat history on the left-hand side
4. Switching between any personal, institutional, or alternate Google accounts you may have been signed into
I don’t have access to users’ Gemini conversations from my side, so unfortunately I can’t retrieve the session directly.
Let me know what you find.
Jim
Thanks for getting back to me, Jim. Damn...It's lost. Important to let folks know that if they use your app to access an AI it's running in your google ai studio, not their own AI account.
How about the licensing body gives teachers access to research databases and journals so they can look for themselves? BCBAs and other professionals get such access as part of their credentialing. How about we stoop pushing fads and gimmicks at teachers and give the actual research so they can use the actual research to teach in a research-based way.
The research to practice pipeline breakdown is so real! Teachers don't have time, academics live in the ivory tower, and to your point--there just isn't a reliable translator to provide a trustworthy and detailed enough summary that is actually relevant. Looking forward to using your tool! This is the stuff we should be using AI for.
I appreciate the way you're trying to scaffold educators in their use of AI to seek more clarity around educational research. The flattening at many layers is a problem, and the best thing we could do is arm teachers with critical literacies for evaluating evidence. Can you build prompts for curriculum publishers, too?😆
I read your generator, and I think a component that might be missing is related to the population studied in whatever evidence the AI draws from.
Many studies remain locked behind paywalls with human verification systems — which means AI is typically limited to scraping abstracts, and abstracts generalize in their own way. But even with full-text access, there's another problem: we apply findings to populations they were never designed for. We've been generalizing medical studies conducted on males to females for centuries. The same pattern holds in education. What produces statistically significant effects in a sample of secondary students could be wildly inappropriate practice for early childhood or elementary populations — and sample size and type matters enormously in how much weight any single study can bear.
Adding a population and sample specificity layer to the prompt generator would go a long way. Though that adjustment itself reveals more tension: prompts designed to privilege experimental and quasi-experimental designs can disadvantage qualitative work — which can be precisely the methodology you need to consider when studying the messy work of educating young children, capturing classroom dynamics, or asking questions of equity and access. Some of the most foundational work in education lives in qualitative studies that reveal implications for quantitative research designs.
These are hard problems to solve with a single prompt, but they're worth naming as you continue developing the tool.
I really like the idea of this, thank you for sharing all about it.
Excellent. I tried the prompt in Claude (AnthropicAI) looking at interleaving.
This is such a great tool to make AI use more efficient, productive, and informative. You make a great point about the breadth and depth of research out there that it is hard on a daily basis to incorporate that research into practice, this tool helps both in giving clear next steps for tomorrow but also clearing away noise for further study.
Nice! I just tested the prompt with a question about sight words ("Should we teach "sight words"? If so how should we support students to learn sight words"? (a topic for which there is a lot of bad information about there) and Claude sent back a good, evidence-backed response. (Haven't tested it out on the others).
I like https://consensus.app/ for an overview of the literature on a given question.
I think I'd want to alter that prompt only slightly: "(...) At this point, I don’t want you to behave like a motivational speaker, advocate, consultant, or influencer. Instead, act like a careful educational researcher who is trying to accurately represent empirical evidence—which you will need to source—but also to detect any grey zones. Respond clearly, avoid but avoid pedagogical jargon (even if it may be cited in the sources)."
Thanks for the prompter page "Evidence" Checker. May I suggest a slightly altered passage:
"At this point, the response shouldn't sound stylistically like a motivational speaker, advocate, consultant, or influencer. Instead, act like a careful educational researcher who's trying to accurately represent empirical evidence—which you'll need to source. Respond clearly and as concisely as possible, but avoid pedagogical jargon (even if it may be cited in the sources)."
This is a fascinating post, and it resonates strongly with my experience in schools. Teachers are constantly navigating conflicting educational claims, and I’ve seen how difficult it can be to separate trend from evidence. A tool like The Evidence Checker could shift the dynamic by putting high-quality research directly in teachers’ hands, allowing them to make informed decisions rather than relying on simplified or ideologically driven guidance. In practice, the challenge is connecting evidence to context—how it actually plays out in classrooms with diverse learners. From my perspective, the real power of this approach is in enabling teachers to interrogate claims, ask follow-up questions, and adapt strategies thoughtfully.
This is the perfect response to provide teachers with the information they need to make informed data-driven decisions.
This is really great idea and thank you for your work on it. As someone observing and with a stake in our school district's move from Lucy Calkins to stuctured literacy, and now working on math, I can see how teachers and administrators badly need a tool like this. It was, after all, not looking at research that led our district to choose TCRWP in the first place in the aughts. It was curriculum choice by vibes.
One thing I'm not quite clear on: the AI is trained on the internet writ large? And we presume that a subset of the data are these peer reviewed educational research studies?
Or, is this a download of an AI to a local server, and fed only the peer reviewed studies?
Actually, we simply leave it up to the user to choose their favourite AI system: ChatGPT, Perplexity, Gemini, or Claude. All we do is provide the prompt. So the AI's are all using their default training sets to answer questions. I suspect the training set for all of these platforms includes the entire web, including all of the recent academic literature. We considered building the tool so that it would use OpenAI API to send the prompt to ChatGPT automatically. We may still do that... it would make the tool easier to use. However, we do like the transparency of letting the user choose for themselves which GenAI tool to use, and without having our code sitting in the middle. It makes it clear to everyone that we're not interfering in some way with the responses that users get.
I have a number of concerns about this approach, and I’m perhaps grossly misinformed about it. First of all, with regard to transparency, if we have identified a need for such a tool for teachers to be able to access research in the field of education, their wheelhouse, how can we expect teachers to understand the differences among any of these proprietary AIs? It seems at minimum we should rely on open source models. Secondly if all of these are trained on the web in general then any claim made by anyone- a homeschooler on a Reddit sub, essays presented as data by an educational publisher, or educators who make claims without any evidence, can show up in LLMs, correct? Even within the peer reviewed literature there have been issues with replicating research results. How do you know from LLMs you’re getting relevant, high quality results? Isn’t this the very problem we’re trying to solve?
A response to "The Evidence Checker" — and a proposal for what comes next
There's a tool making rounds in education circles called The Evidence Checker. The idea is simple and genuinely useful: teachers type a question about pedagogy, copy a structured prompt, paste it into an AI, and get a more rigorous, evidence-informed answer than they'd find on social media or in a typical professional development session.
I think it's a good tool. I also think it solves the wrong problem — or at least, only half of it.
Here's the thing the article gets exactly right: teachers are surrounded by confident educational claims with no reliable way to evaluate them. Inquiry learning. Explicit instruction. Learning styles. Retrieval practice. Every workshop comes with certainty. The underlying evidence is rarely examined. The Evidence Checker helps teachers push back on that.
But there's a deeper structural problem the article doesn't address. And no teacher-facing evidence tool can solve it.
Even a perfectly evidence-informed teacher delivers instruction to a distribution of learners. The teacher-first approach optimizes for an imagined average student. No student is average.
The student sitting in the third row who scored 72% on the last test may have a fundamental misconception, a working memory overload on that particular day, an anxiety response to timed assessments, or a motivation deficit rooted in a belief that she's just "bad at this." These require entirely different responses. No synthesis of research literature tells a teacher which one it is — certainly not for that specific student, on that specific day.
The evidence that matters most for any individual student is not in a journal article. It's in that student's individual learning record: IF we choose to build one.
==========================================
What a student-first approach actually looks like
The foundational reframe is this: the student is not the subject of measurement. The student is the scientist experimenting.
Evidence about how they learn is not collected by a system and delivered to a teacher. It's generated by the student, interpreted by the student, and owned by the student. The system doesn't tell the student that re-reading is less effective than retrieval practice. The student discovers this through their own data.
That distinction is not cosmetic. When students generate their own evidence, the resulting behavioral change is far more durable than when they receive a teacher's recommendation. Self-generated evidence is experienced as discovery, not compliance.
A student who graduates with a rich, evidence-based understanding of how they specifically learn has something more valuable than any particular body of knowledge: the ability to learn anything efficiently for the rest of their life.
What does this look like in practice? Two measurement layers are running simultaneously.
The first layer is course-specific. It captures the signals that matter for this material: prior knowledge, acquisition rate, which specific misconceptions appear (not just whether answers are wrong), how much is retained after 48 hours versus one week, and whether the student's effort is producing learning or just producing fatigue.
The second layer is the one that compounds. It's a cross-course meta-learning profile — a growing record of how this particular person learns. Which study methods produce durable retention for them? At what point in a session does their comprehension degrade? Whether they learn new concepts better from examples before theory or theory before examples. How do their performances change under time pressure relative to their actual knowledge? This layer follows the student across every course they take. It gets richer with each one.
========================================
The self-discovery loop
The framework isn't a measurement system that happens to students. It's a structured experimentation cycle that students conduct — supported by the system and the teacher.
A student forms a hypothesis: "I think I learn better by re-reading than by self-testing." They log it, run the experiment across a few sessions, compare their predicted score to their actual score on a delayed retention check, and update their self-model accordingly. The system facilitates this. The teacher reads the signal, not the raw data.
This is how the goal structure changes, too. In the current model, goals are set by the curriculum: coverage, assessment performance, and grade distributions. Student goals are often different. They want to genuinely understand something. They want to apply it in a context that matters to them. They want to know whether they're actually learning, NOT just performing on the next test.
A student-first framework asks three layers of questions before and during each course:
- What do you want to be able to do that you can't do now?
- What learning behavior do you want to test?
(and the one almost no educational system ever asks)
- What do you want to discover about yourself as a learner?
What this does to the teacher's role
This framework doesn't diminish the teacher. It gives the teacher better information to do what ONLY a human can do: have a meaningful, specific conversation with a specific student about their specific learning journey.
The teacher's dashboard in this model doesn't show class averages. It surfaces anomalies: "This student's predicted versus actual scores show systematic overconfidence on conceptual questions, but accurate calibration on procedural ones." "This student's retention drops sharply after 48 hours — they may need a different spacing schedule." "This student has discovered retrieval practice works for them but hasn't yet applied it to the current unit."
The system might suggest: "Based on this student's error signature, their confusion appears to be conceptual, not procedural. A Socratic conversation about the underlying principle may be more effective than additional practice problems." The teacher decides whether and how to act on this — informed by richer individual data than any general research synthesis can provide.
The Evidence Checker asks: What does research say teachers should do?
The student-first framework asks: What does this student's learning signal say they need next?
Both questions matter. Only one is currently being asked systematically!
The article's closing line is more right than it knows
The Evidence Checker piece ends with this: "When access to evidence becomes more widely distributed, the professional conversation may begin to shift."
I agree. And I'd push it one step further.
The most transformative shift is not distributing research evidence to teachers more efficiently. It's distributing learning evidence to students — about themselves — in real time. Students deserve what teachers now have access to through tools like The Evidence Checker: the ability to question claims, compare approaches, and make evidence-informed judgments. The claims students most need to evaluate aren't claims about pedagogy. They're claims about themselves. Their own patterns. Their own most effective path to understanding.
Teaching is too important to be driven by slogans and trends. The article is right about that.
Learning is too important to be driven by system goals, institutional incentives, and population-level research averages.
I'm not proposing to replace the teacher with an algorithm. Rather, I am proposing to give the teacher better information about specific students — and to give students something no educational system has ever systematically offered them: the evidence to understand how they, specifically, learn best.
God bless you!!!