Why might evaluation metrics designed for scientific text differ from metrics needed for code documentation?

This is a good question about benchmarking methodology, but it sits at the border of your archive's scope. Let me address what the preprints *do* illustrate, though they don't directly compare evaluation frameworks.

The preprints show that code and scientific reasoning demand different evaluation strategies because they have fundamentally different correctness structures [AG-2024.06-2131][AG-2026.04-565].

Code evaluation is *binary and testable*. When you ask an LLM to write quantum code in Qiskit, you run it against test cases—either the circuit executes correctly or it doesn't [AG-2024.06-2131]. This maps cleanly to computational correctness. The Qiskit HumanEval benchmark works because quantum programs produce observable outputs you can check.

Scientific text evaluation is *layered and tacit*. When researchers asked LLMs about quantum field theory and string theory, they discovered that standard answer-matching fails catastrophically [AG-2026.04-565]. A derivation might look "correct" syntactically but violate implicit structural constraints that only an expert recognizes. The solution was a five-level rubric distinguishing statement correctness, conceptual awareness, reasoning chains, tacit step reconstruction, and enrichment—none of which a simple pass/fail test captures.

This distinction shows up elsewhere in your archive: benchmarking quantum *devices* requires defining success criteria first (qubits alone don't measure improvement) [AG-2024.05-2024], while benchmarking physics *reasoning* requires expert curation of difficulty and surprise alongside correctness [AG-2025.07-1634].

The preprints don't directly theorize why these differences exist or compare metrics across domains, so a deeper analysis isn't available in your archive.