Summary

This post dissolves the false dichotomy between AI “impossibilists” (like David Deutsch, who argue predictive systems can never achieve agency) and “imminentialists” (who claim scaling will inevitably produce autonomous minds). Axios defines “jaggedness” as the uneven capability profile of LLMs—extraordinary competence in narrow domains coexisting with glaring failures—reflecting absence of coherence-binding forces. The piece argues both camps make the same error: treating agency as a capability threshold rather than an architecture. LLMs alone cannot become agents, and scaling alone won’t produce agency, but nothing prevents building artificial agents using LLMs as cognitive components within composite architectures. The key insight: agency is a design problem, not an emergent property of scale. The post predicts three categories of minds: pre-agentic cognitive engines (LLMs), human-AI hybrids (centaurs), and fully artificial agents built compositionally.

Key Concepts

  • Jaggedness – Uneven, discontinuous capability profiles in pre-agentic systems; sharp spikes of competence without coherent integration.
  • Agency as architecture – Intelligence is coherent integration of perception, memory, preference, counterfactual reasoning, and action—a structural configuration, not a scalar capability.
  • Impossibilist error – Inferring from current lack of agency that such systems can never be agents (ignoring compositional pathways).
  • Imminentialist error – Assuming scale guarantees structure; more competence ≠ more agency.
  • Composite minds – The engineering reality that first AGI systems will assemble predictive models, memory, evaluators, toolchains, planners into unified goal-directed entities.
  • Coherence collapse – When agency binds disparate capabilities into self-consistent wholes, jaggedness disappears—this collapse is the birth of intelligence.

Evolution Notes

  • Major position statement on AI agency that engages with both skeptical and accelerationist camps.
  • Connects jaggedness directly to absence of coherence-binding mechanisms, setting up later coherence-focused arguments.
  • The “agency is a design problem” framing shifts discourse from “will it happen?” to “who will build it and what values will it embody?”
  • Introduces three-category ecology of minds framework for understanding AI development trajectory.

Tags

Cross-References

Open Questions

  • What specific architectural components are necessary and sufficient for the coherence-collapse transition from jagged cognition to agency?
  • Can jaggedness persist in some domains even after coherence binds other capabilities? (Uneven agency?)
  • Do human-AI hybrids represent a stable equilibrium or just transitional forms toward full artificial agency?
  • What prevents accidental assembly of agentic architectures through market/competitive pressures before intentional design principles are established?