Could degrading data systematically expose which assumptions are causing neural-network brittleness?

This is a fascinating question about whether we can reverse-engineer neural network brittleness by deliberately corrupting inputs and watching how predictions fail.

The archive doesn't directly address this degradation-based diagnostic approach. However, the papers do illuminate *why* networks are brittle in ways that suggest what such an experiment might reveal.

The core insight comes from a quantum-mechanical analogy: [AG-2024.02-1732] shows that adversarial vulnerabilities in neural networks mirror the uncertainty principle from physics. In quantum mechanics, you cannot simultaneously know a particle's position and momentum precisely—there's a fundamental trade-off. Similarly, neural networks exhibit a kind of "conjugate" relationship between their inputs and the gradients that drive learning. Small perturbations in input space (which we could think of as degradation) couple tightly to the loss function in ways that expose the network's decision boundaries. This suggests that *systematic* input degradation might indeed reveal phase transitions or instability thresholds—the points where the network's assumptions break down.

More concretely, [AG-2024.06-754] demonstrates that we can diagnose trainability problems by measuring information loss as data flows through layers. Using "reconstruction entropy"—how much of the original input can be reconstructed from hidden activations—they predicted whether a network would train successfully. This is methodologically close to your idea: degrade the input deliberately, then measure what information survives in each layer. The entropy signal was sensitive to the network's "phase behaviour," meaning it could detect qualitative shifts in how the network processes data.

The quantum-machine-learning papers also sketch a pattern: [AG-2024.03-1539] and [AG-2024.07-1685] show that different circuit architectures (the "assumptions" about how to structure computation) have measurably different robustness profiles. This opens the door to systematic testing: if you degrade data in specific ways—say, blurring, noise, occlusion—and track which architectural assumptions fail first, you'd be mapping the network's hidden dependency structure.

Unfortunately, none of these papers actually execute the experiment you're describing: a systematic degradation protocol designed to expose which assumptions are doing the work.