How does the five-level rubric in AG-2026.04-565 actually operationalize the difference between skipped versus impossible steps in practice?

Looking at AG-2026.04-565, the five-level rubric operationalizes the distinction between skipped and impossible steps through a hierarchy of epistemic reconstruction, moving from surface correctness to deep structural reasoning [AG-2026.04-565].

Here's how it works in practice:

The five levels are:

1. Statement correctness — Does the final answer match? This is binary and says nothing about *how* it was reached.

2. Key concept awareness — Does the response reference the right conceptual machinery (e.g., conformal invariance in QFT)? An LLM might cite a concept without using it properly.

3. Reasoning chain presence — Are explicit logical steps visible? This catches when a model jumps to conclusions without showing work.

4. Tacit step reconstruction — Can the model *fill in omitted reasoning*? This is where "skipped" differs from "impossible." A skipped step is one the model *could* articulate if prompted or if it understood the implicit constraint. An impossible step is one the model cannot recover because it lacks the deep structural knowledge of how that step follows from the global problem setup.

5. Enrichment — Does the response go beyond the minimum, showing embedded understanding (e.g., relating a result to a broader principle)?

The practical difference emerges at level 4: When evaluating a string theory derivation where physicists omit several steps because they're "obvious to experts," the rubric distinguishes between:

• A model that skipped the step but *could* fill it in (high level 3–4 performance)
• A model that couldn't fill it in because it lacks the tacit knowledge that the step must respect, say, a global consistency constraint under representation change (low level 4, suggesting the model doesn't grasp the structural requirement)

The paper reports that contemporary LLMs "show systematic degradation when tasks require reconstruction of omitted reasoning steps or reorganization of representations under global consistency constraints" [AG-2026.04-565]. In other words, they often fail at level 4—they cannot distinguish which steps are merely elided versus which steps they fundamentally cannot justify.

This matters because it tells a researcher whether an LLM is a useful assistant for filling in standard proofs (level 3 competence) or whether it actually understands the conceptual landscape well enough to *propose* or *verify* novel steps that respect hidden structural rules (level 4–5 competence).

The provided preprints do directly address this question.