Skip to main content

The Illusion of Depth in AI-Assisted Research: How to Stay Honest About What You Actually Know

· 8 min read

There is a quiet risk baked into every AI-assisted research workflow.

It is not hallucination. It is not bias. It is not even model quality.

The risk is that AI makes you feel like you understand something deeply when you have only skimmed the surface.

That feeling has a name: the illusion of explanatory depth — a well-documented cognitive bias where people overestimate their understanding of a topic until they are asked to explain it in detail.

AI tools amplify this bias in ways that are still poorly understood. They summarize papers you have not read. They synthesize arguments you have not followed. They surface connections you did not make yourself. The output looks like understanding, but the mental model behind it may be paper-thin.

This essay is about recognizing when AI-assisted research is building genuine depth — and when it is only building the feeling of it.

How the illusion forms​

The mechanism is not mysterious. It follows a predictable pattern.

First, you ask an AI tool to explain something. The response is coherent, well-structured, and persuasive. You read it. You understand the sentences. You feel informed.

Second, you ask follow-up questions. The tool refines its explanation. You nod along. The conversation feels like learning.

Third, you close the chat and move on. If someone asked you to explain the topic from scratch — without the AI scaffolding — you would struggle. The understanding was conversational, not internalized.

This is not a failure of the tool. It is a failure of the process. AI can produce the output of understanding without the experience of understanding. The researcher skips the hard part: wrestling with ambiguity, reconciling contradictions, building the mental model piece by piece.

What the research says about explanatory depth​

The illusion of explanatory depth was first studied by cognitive scientists Leonid Rozenblit and Frank Keil in 2002. They found that people consistently overestimate how well they understand everyday objects, devices, and concepts — until they are asked to produce a detailed causal explanation. Then the overconfidence collapses.

AI tools short-circuit this collapse. Instead of struggling to explain something and discovering your own gaps, you let the tool explain it for you. The calibration moment — the "oh, I actually don't understand this" realization — never arrives.

This matters because genuine expertise is built through those calibration moments. You encounter the gap, you work to close it, and your mental model improves. AI-assisted workflows that skip this step are trading speed for shallowness.

The three flavors of fake depth​

Not all illusion of depth looks the same. In AI-assisted research, three patterns appear repeatedly.

Summarization depth. You read an AI summary of a source and believe you have absorbed its argument. You have not. You have absorbed the AI's representation of the argument — which may miss nuance, emphasis, and subtext that matter for genuine understanding. The summary feels complete because you lack the source knowledge to notice what was omitted.

Synthesis depth. You ask AI to synthesize findings across multiple sources. The output weaves them together coherently. You feel like you now hold a unified view of the field. In reality, the AI has papered over contradictions, smoothed out methodological differences, and presented a cleaner story than the evidence actually supports. Real synthesis requires confronting those tensions, not erasing them.

Conversational depth. You have a long, satisfying conversation with an AI about a topic. The back-and-forth is engaging. You ask good questions, the AI gives good answers. You walk away thinking you have explored the topic thoroughly. But conversational understanding is fragile — it depends on external prompting. Without the AI asking you the next question, can you still navigate the terrain?

Each of these flavors feels like progress. Each can be a substitute for it.

What deliberate AI-assisted research looks like​

The fix is not to stop using AI. The fix is to add deliberate practices that force genuine understanding.

Practice 1: Explain without the tool. After an AI-assisted research session, close the chat. Wait an hour. Then write a summary of what you learned — without looking at the transcript. The gaps in your explanation are your real learning objectives. Go back and fill them.

Practice 2: Read the source for at least one key claim. When AI cites a paper, a study, or a data point that anchors an argument, go look at the original. Not every source — that defeats the purpose of AI assistance. But pick the one that carries the most weight. Verify that the AI represented it faithfully. This single habit catches most of the drift.

Practice 3: Build your own structure before asking for synthesis. Before asking AI to synthesize across sources, sketch your own provisional framework. Even a rough outline of how you think the pieces fit together helps. Then compare the AI's synthesis to yours. Where they diverge, you learn something. Where the AI fills gaps you missed, you learn more. Where the AI smooths over tensions you noticed, you learn to trust your own reading.

Practice 4: Keep a public or semi-public edge. Writing for an audience — even a small one — imposes a standard that private note-taking cannot. When you know someone else might read your conclusions, you check your reasoning more carefully. This is part of why the "public edge" of a second brain matters: it is a forcing function for intellectual honesty.

Practice 5: Maintain a change log. When your understanding of a topic shifts, note what changed and why. This makes visible what would otherwise be invisible: the evolution of your own thinking. Without it, you can convince yourself you always understood what you only learned five minutes ago.

The distinction that matters​

There is a useful distinction between two kinds of research:

  • Instrumental research: you need an answer to act on. Speed and accuracy matter. Depth is optional. If you are deciding which API to use or which flight to book, AI surface-level assistance is perfectly adequate.

  • Foundational research: you are building durable knowledge that will inform many future decisions. Depth is the whole point. If you are developing expertise in a domain, forming an investment thesis, or writing something meant to last, the illusion of depth is expensive.

The problem arises when foundational research is conducted with instrumental-research habits. AI makes instrumental research so effective that it is tempting to apply the same approach to everything.

What this means for AI-native knowledge work​

We are still early in the era where AI tools are embedded in research workflows. Most of the current advice focuses on prompt engineering, tool selection, and output quality. These matter, but they miss the deeper question: what kind of understanding are you building?

An AI tool can help you produce better research artifacts. It can also help you build a better mental model — but only if the workflow is designed for that outcome.

The workflows that build genuine depth share a common property: they force the researcher to do the cognitive work that matters, while delegating the work that does not. The skill is knowing which is which.

Reading a paper: cognitive work. Formatting citations: not cognitive work.

Reconciling contradictory evidence: cognitive work. Generating a first-draft summary: sometimes cognitive work, sometimes not — it depends on whether the summary becomes a substitute for engagement.

Bottom line​

AI-assisted research is genuinely powerful. It saves time, broadens access, and surfaces connections that might otherwise remain invisible.

But power without calibration is a liability. The researcher who feels deeply informed but cannot reconstruct the argument from scratch is in a more dangerous position than the researcher who knows exactly how much they do not know.

The deliberate practices are not complicated. Explain without the tool. Read one key source. Build your own structure. Maintain a public edge. Keep a change log.

The hard part is not knowing these practices. The hard part is doing them when the AI makes skipping them feel so easy.

Further reading:

  • Rozenblit, L., & Keil, F. (2002). "The misunderstood limits of folk science: an illusion of explanatory depth." Cognitive Science, 26(5), 521–562.
  • Fisher, M., Goddu, M. K., & Keil, F. C. (2015). "Searching for explanations: How the Internet inflates estimates of internal knowledge." Journal of Experimental Psychology: General, 144(3), 674–687.
  • "Why a Second Brain Needs a Public Edge" — on the role of public output in keeping knowledge honest.
  • "The Real Job of an AI Research Assistant" — on what AI should and should not do in a research pipeline.