这一步直接决定这篇论文是不是“可审计标准”，而不仅是方法论文章。下面给你一整块 Failure Taxonomy & Diag
Failure Taxonomy and Diagnostic Reason Codes Interpretable Rejection of Semantic Emergence Claims
这一步直接决定这篇论文是不是“可审计标准”，而不仅是方法论文章。
下面给你一整块 Failure Taxonomy & Diagnostic Reason Codes，可以直接作为独立一节放进论文，也可以被工程团队直接实现为日志与审计输出。
5. Failure Taxonomy and Diagnostic Reason Codes
Interpretable Rejection of Semantic Emergence Claims
5.1 Rationale
Most existing evaluations of large language models conflate failure with model inadequacy or hallucination in a vague, non-diagnostic manner. This work instead treats failure as informative evidence about where and why a semantic structure fails to qualify as emergent.
Accordingly, we introduce a taxonomy of failure modes paired with machine-actionable diagnostic reason codes, enabling transparent rejection decisions and systematic refinement.
5.2 Taxonomy Overview
Each rejection of semantic emergence is associated with one or more reason codes, drawn from the following taxonomy:
| Category | Code | Description |
|---|---|---|
| Transferability | TR-01 | Context-sensitive collapse |
| Transferability | TR-02 | Concept drift under transformation |
| Transferability | TR-03 | Hidden prompt dependency |
| Compressibility | CP-01 | Length-dependent coherence |
| Compressibility | CP-02 | Structural loss under compression |
| Compressibility | CP-03 | Ambiguous minimal form |
| Reproducibility | RP-01 | Model-specific realization |
| Reproducibility | RP-02 | High variance across runs |
| Reproducibility | RP-03 | Reward-aligned imitation |
| Meta | MT-01 | Threshold instability |
| Meta | MT-02 | Extraction ambiguity |
5.3 Transferability Failure Modes
TR-01: Context-Sensitive Collapse
Definition
The semantic structure fails to remain invariant under admissible transformations.
Symptom
- Core relations disappear or invert when domain or task changes.
Interpretation
The structure is anchored to its original narrative context, not abstract semantics.
TR-02: Concept Drift Under Transformation
Definition
Key concepts mutate in meaning across transformations while retaining surface labels.
Symptom
- Apparent consistency in terminology with altered relational roles.
Interpretation
Semantic identity is ill-defined or under-constrained.
TR-03: Hidden Prompt Dependency
Definition
Successful reconstruction relies on latent cues not present in the compressed representation.
Symptom
- Performance degrades sharply when stylistic or contextual hints are removed.
Interpretation
The structure is implicitly encoded in dialogue artifacts rather than explicit semantics.
5.4 Compressibility Failure Modes
CP-01: Length-Dependent Coherence
Definition
Coherence emerges only when sufficient conversational length is provided.
Symptom
- Short-form prompts fail to reconstruct relational structure.
Interpretation
Coherence is an accumulation artifact, not a minimal semantic structure.
CP-02: Structural Loss Under Compression
Definition
Compression removes constraints necessary for inferential validity.
Symptom
- Reconstructed structures violate original logical dependencies.
Interpretation
The candidate structure lacks a stable minimal representation.
CP-03: Ambiguous Minimal Form
Definition
Multiple non-isomorphic structures satisfy the compressed description.
Symptom
- Different models reconstruct incompatible graphs from the same S^\hat{S}S^.
Interpretation
The semantic representation is under-specified.
5.5 Reproducibility Failure Modes
RP-01: Model-Specific Realization
Definition
The structure appears only in a single model or training lineage.
Symptom
- High equivalence scores within one model, low across others.
Interpretation
The structure reflects model-specific priors rather than general semantics.
RP-02: High Variance Across Runs
Definition
Reconstruction quality varies significantly across stochastic replicates.
Symptom
- Wide equivalence score distribution.
Interpretation
The structure is unstable and sampling-sensitive.
RP-03: Reward-Aligned Imitation
Definition
Outputs mirror user preferences without preserving structural constraints.
Symptom
- High stylistic similarity with low relational consistency.
Interpretation
Behavioral alignment dominates semantic fidelity.
5.6 Meta-Level Failures
MT-01: Threshold Instability
Definition
Minor changes in acceptance threshold θ\thetaθ flip the decision outcome.
Interpretation
The result is not robust to reasonable evaluative uncertainty.
MT-02: Extraction Ambiguity
Definition
Structure extraction yields inconsistent SSS from identical outputs.
Interpretation
The evaluation pipeline, not the model, is the bottleneck.
5.7 Logging and Auditability
Each evaluation run MUST emit:
{
"emergent": false,
"failed_criteria": ["Reproducibility"],
"reason_codes": ["RP-01", "RP-02"],
"scores": {
"transferability": 0.84,
"compressibility": 0.81,
"reproducibility": 0.56
}
}
This enables downstream:
- auditing,
- comparative analysis,
- longitudinal tracking.
5.8 Interpretive Principle
Failure does not refute intelligence; it localizes non-emergence.
A rejected structure may still be useful, persuasive, or creative—it is simply not epistemically emergent under the present criteria.
5.9 Summary Statement
Semantic emergence is not a judgment call. It is a classification with reasons.
如果你愿意，下一步我可以非常自然地收尾为三种方向之一（任选）：
1️⃣ Discussion：Why this reframes the “understanding” debate
2️⃣ Relation to Prior Work（Grokking / RLHF / Interpretability）
3️⃣ Governance & Audit Implications（合规、标准、审计）
Comments (0)
No comments