NUUBI
Back to Blog

How AI and Humans Get Better Outcomes Together

· Nuubi Team

Two complaints about AI are both true. The first is that it can do the thinking for you: ask, paste, submit, and nobody had to understand anything. The second is that it can quietly add work, as you re-explain your context to a chatbot, copy text between apps, and check answers you cannot trace. If a tool does either one, a fast demo can hide a worse result for the people using it. The more useful question is which parts of the work belong to the person, which belong to the tool, and whether the tool respects that split.

What Research Shows About AI and Cognitive Offloading

Three recent studies are worth reading together. In a mixed-methods study of 666 participants, Michael Gerlich found that frequent AI tool use was associated with lower critical-thinking scores, with cognitive offloading, the habit of handing the mental work to the tool, mediating the link (https://www.mdpi.com/2075-4698/15/1/6). A Microsoft Research survey of 319 knowledge workers found a related pattern: people who reported higher confidence in the AI's answers also reported putting less critical-thinking effort into the task, with their role shifting toward checking and oversight (https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf). And in an MIT Media Lab preprint, 54 participants wrote essays with an LLM, a search engine, or no tool while wearing EEG caps. The LLM group showed the weakest brain connectivity, and the authors describe an accumulating cognitive debt that showed up when the tool was taken away (https://www.media.mit.edu/publications/your-brain-on-chatgpt/).

These studies have limits. Gerlich's study and the Microsoft survey are survey-based, so they show associations, not cause. The MIT study is a small preprint: 54 people took part in the first three sessions and only 18 completed the last one, its authors say peer review is only beginning, and a published commentary argues some of its results could be interpreted more conservatively (https://arxiv.org/abs/2601.00856). Read together, they suggest the risk grows when the tool takes over the moment where judgment forms. They do not show that AI is bad for thinking. The MIT study also holds a hopeful result: people who wrote on their own for three sessions and then switched to the AI showed higher memory recall and stronger activation in prefrontal and occipito-parietal areas, while people who went the other way showed reduced connectivity that the authors read as under-engagement. That is the think-first order, and the next section is built on it.

A Division of Labor That Keeps the Thinking Human

One pattern avoids both failure modes: think first without AI, then with AI, then compare. Write your own answer, outline, or conclusion before you ask the tool anything. Then ask, and push back on what comes back. Then compare the two and explain, at least to yourself, where you agree, where you disagree, and why.

The order matters. Going first means your judgment forms before the machine's answer anchors it. Comparing afterward means the AI's output is material to react to, not a verdict to accept. The person keeps the framing, the values, and the final call. The AI contributes speed, patient follow-up questions, and a second perspective at 11 pm. Each side does what it is good at, and neither adds a second job for the other.

Two experiments point the same way. In a study of 199 people, Buçinca and colleagues found that designs forcing people to engage analytically before accepting an AI suggestion significantly reduced overreliance on it, though people rated those designs least favorably, so expect some grumbling about think-first (https://arxiv.org/abs/2102.09692). In a field experiment with nearly 1,000 high school math students, Bastani and colleagues found that students with a standard ChatGPT-style tutor did worse than students who never had access once the tool was removed, while a tutor with learning safeguards largely avoided that harm (https://www.pnas.org/doi/10.1073/pnas.2422633122).

The Busywork Test for AI Tools

The second failure mode gets less attention. A tool that makes you re-type your course context into a chat window, assemble documents one prompt at a time, or verify every answer against sources you cannot see has not saved work. It has moved the work somewhere less visible.

So before adopting anything, ask where the tool's answers come from and how much setup it expects. A tool grounded in materials the instructor already controls, fitting the systems a course already uses, removes steps. A tool that needs constant feeding adds them.

What Human-in-the-Loop Feedback Looks Like in Practice

The clearest example I know is feedback on student drafts. In Nuubi, a student writes a draft first and uploads it privately. Drafts stay private, and they are never used to train models. The student goes back and forth with the AI before anything is submitted. Because the student's own attempt comes first, the think-first order is built into the workflow. The feedback cites the course materials and rubric the instructor chose, so the student revises against the actual course rather than generic advice. The student can accept the feedback, push back on it, or revise and ask again.

The instructor can see the whole exchange, and that visibility changes what the work means. A single polished submission says very little. A visible back-and-forth shows which feedback the student accepted, which they questioned, and how the draft changed. Faculty can read the exchanges quickly, and summaries are provided when a thread gets long. If the feedback misreads the rubric, the instructor can correct it in their own feedback on review. The instructor, not the AI, keeps the final say on every grade and every judgment call.

Questions to Ask an AI Tool About Human Judgment

If you are evaluating a tool, a few questions get at the division of labor quickly. Who makes the final call, and can the tool override a person, or only advise? Can the instructor see the process, or only the output? Does the tool ask for the student's own attempt first, or does it answer before the student has thought? Does it remove steps from the course, or quietly add new ones? A tool with good answers to these questions still needs an engaged instructor and an engaged student, and good design assumes that.

Building for the Human in the Loop

I build Nuubi, and these questions are why its Draft Feedback, Course Q&A, and AI-fluency coaching all keep the instructor in the loop instead of routing around them. If you are working through where AI fits in your courses, I would be glad to compare notes. What part of the work do you most want students to keep for themselves?

Sources

Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1), 6. https://www.mdpi.com/2075-4698/15/1/6

Lee, H. P. et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. CHI 2025. https://www.microsoft.com/en-us/research/wp-content/uploads/2025/01/lee_2025_ai_critical_thinking_survey.pdf

Kosmyna, N. et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. MIT Media Lab, arXiv preprint. https://www.media.mit.edu/publications/your-brain-on-chatgpt/

Stankovic, M., Hirche, E., Kollatzsch, S., & Doetsch, J. N. (2025). Comment on: Your Brain on ChatGPT. arXiv:2601.00856. https://arxiv.org/abs/2601.00856

Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. Proc. ACM Hum.-Comput. Interact., 5(CSCW1). https://arxiv.org/abs/2102.09692

Bastani, H., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS, 122(26). https://www.pnas.org/doi/10.1073/pnas.2422633122