WHAT WOULD ACTUALLY COUNT AS ARTIFICIAL SUPERINTELLIGENCE?

Imagine a system that proposes a scientific theory, designs experiments and repeatedly achieves results beyond leading specialists. Would that establish artificial superintelligence? Only after examining what it did, what assistance it received and whether its advantage survives independent scrutiny.

Artificial superintelligence is an extraordinary capability claim. What evidence would distinguish broad superiority from specialized excellence, memorization or a carefully selected success?

This article proposes an operational approach: sustained, substantial superiority over strong human expertise across a broad and varied range of cognitive tasks, including unfamiliar problems. That is a working standard for investigation, not a universally accepted definition or a prediction that a particular system will arrive on a particular date.

1. FIRST DEFINE WHAT IS SUPPOSED TO BE SUPERIOR

We infer intelligence through performance, learning and adaptation. Definitions emphasize different combinations. An ASI claim must therefore state its criteria before presenting results.

Morris and colleagues’ Levels of AGI framework separates performance, generality and autonomy. Narrow excellence, broad competence and independent operation are different achievements.

Breadth and depth are necessary: advantages across different intellectual activities and transfer beyond rehearsed tasks. Excellence at every conceivable activity would be impossible to test; selecting only favorable tasks would make the standard meaningless.

2. SUPERHUMAN SPECIALIZATION IS NOT GENERAL SUPERINTELLIGENCE

Outperforming people at one task does not establish broad reasoning across unfamiliar situations. This distinction identifies what specialized breakthroughs establish without diminishing them.

Jumper and colleagues’ 2021 AlphaFold paper reported highly accurate protein-structure prediction, including CASP14 results. This major achievement did not establish superiority across economics, engineering, social reasoning and experimental research.

An exceptional mathematical or software result can contribute to an ASI case without establishing the whole claim. A collection of tools may form a broadly capable system; assess that complete system rather than attributing everything to one component.

3. THE HUMAN COMPARISON MUST BE STRONG AND EXPLICIT

“Better than humans” is incomplete. Better than an average adult, a trained professional, a leading specialist or a coordinated research team? A result can change dramatically depending on the comparison group and available resources.

Use qualified people with appropriate tools, information and time. Comparing a tool-enabled AI with an unaided human under an artificial deadline may show speed advantages, not general superiority.

Equal time tests throughput; equal expenditure examines efficiency; generous resources probe achievable quality. Report these separately. A system can be faster but less accurate, more capable but more expensive, or dependent on human assistance.

For scientific or economic impact, compare with human teams assisted by existing software. Defeating an artificially weakened opponent cannot establish the broader claim.

4. UNFAMILIAR PROBLEMS MUST REMAIN UNFAMILIAR

Prior exposure can imitate flexible reasoning. Training on benchmarks, solutions or related templates weakens evidence of generalization. Withholding exact wording may not make the underlying task unfamiliar.

François Chollet’s On the Measure of Intelligence distinguishes accumulated skill from skill-acquisition efficiency. Its formalization is not universal consensus, but prior experience can conceal weaknesses in learning something new.

Combine protected tasks, new combinations and unfamiliar environments whose rules must be discovered. Limit tuning against the final assessment. Acknowledge that proving the absence of related training material is difficult.

Test changed assumptions, representations and feedback. Learning unfamiliar notation or adapting to a broken rule provides different evidence from reproducing a celebrated answer.

5. A HIGH AVERAGE MUST NOT HIDE DEEP GAPS

A system might dominate coding yet fail unfamiliar causal reasoning, or write scientific explanations without choosing experiments that distinguish competing hypotheses.

Report a capability profile showing domains, difficulty, reliability and resource use. Otherwise, easy successes can conceal important areas where experts remain substantially stronger.

Breadth cannot be established by counting many versions of the same task. Writing code, explaining code and answering programming quizzes share substantial structure. A stronger portfolio includes scientific inference, formal reasoning, planning under uncertainty, design with competing constraints and learning new task rules.

Human expertise is uneven too. Are weaknesses limited exceptions within robust superiority, or evidence that apparent generality reflects favorable task selection? Declare the criteria for that judgment in advance.

6. SCIENTIFIC CREATIVITY NEEDS EXTERNAL CONSEQUENCES

Novel text is not necessarily a novel discovery. A convincing theory can be false; a surprising design can violate a constraint; a proposed experiment can fail to distinguish the explanations it claims to test.

Consider a hypothetical materials-research challenge. A candidate system receives incomplete measurements, limited experimental capacity and a requirement to improve a material under specified conditions. It must identify uncertainties, propose informative tests, update after disappointing results and produce a candidate that survives independent measurement.

Require reproducible improvement, predictions made before measurements and disclosure of external assistance. A proposal establishes proposal-writing ability; validated results establish more, but only within the scope tested.

Comparable challenges across unrelated areas strengthen generality. Reward validated discoveries even when their reasoning differs from human approaches. Incomprehension neither proves genius nor invalidates independently verified performance.

7. EXTENDED WORK REQUIRES MORE THAN ONE GOOD ANSWER

Many valuable tasks involve a chain of decisions: inspect evidence, choose an approach, build something, detect mistakes and revise. A system that solves isolated exercises may still fail when earlier decisions shape later information.

METR fits success probability against measured or estimated human-expert task duration. Its 50% horizon is where that fitted relationship reaches 50% success on the evaluated task distribution—not a guarantee for each task of that length. It also does not mean that the AI itself works for that many hours.

Software tasks cover only part of intellectual work; 50% success may be inadequate for consequential deployment. Test higher reliability, varied domains and sensitivity to evaluation conditions.

Test whether objectives persist, contradictory evidence is noticed and setbacks are overcome. Operating time is not dependable progress. Report repeated completion rates and supervision or recovery costs.

8. COUNT THE WHOLE SYSTEM AND ITS HIDDEN HELP

AI may depend on search, memory, code execution and specialist models. These components are legitimate: human experts also use instruments and accumulated knowledge.

Identify model versions, tools, prompts, compute and human interventions. If specialists repeatedly correct errors, report an assisted outcome rather than autonomous achievement.

Coordinated models could demonstrate broad superiority without a central “mind.” Calling them an agent society proves nothing about coordination. Measure combined capability and dependencies.

If a hundred trials produce one exceptional solution, disclose the trials, selection and cost. A best-of-many result differs from dependable first-attempt success. Potential and reliability are distinct claims.

9. HUMAN LIMITATIONS DO NOT MAKE VERIFICATION IMPOSSIBLE

People might struggle to judge superior intelligence. But evaluation does not always require reproducing how an answer was generated.

A mathematical result may admit a checkable proof. A scientific prediction may survive independent experiments. A design may outperform alternatives under controlled measurements. Generating a solution can be much harder than checking some of its properties.

Verification has limits: proofs depend on specifications, simulations can omit failure conditions, and experiments may validate narrow results without establishing long-term consequences.

Use independent assessors, complementary measurements and replication. State what remains unchecked. An achievement that cannot yet be convincingly evaluated remains unresolved; difficulty verifying it does not make it true.

10. CAPABILITY, CONSCIOUSNESS AND AUTHORITY ARE DIFFERENT QUESTIONS

Extraordinary capability need not establish subjective experience; humanlike conversation need not establish superiority. Capability assessment does not settle consciousness or moral status.

Autonomy is another separate dimension. A powerful system could be restricted to answering research questions, while a less capable one could be permitted to change accounts or execute transactions. Permission to act is a deployment choice, not a direct measurement of intelligence.

The same separation applies to safety. Competence does not establish benevolence, reliable obedience or alignment with every affected person’s interests. A system can optimize an objective impressively while the objective itself omits important constraints.

Phuong and colleagues’ dangerous-capability evaluations examine specific risks rather than infer safety from general performance. Dangerous behavior does not define ASI. Capability, risk and acceptable use require separate arguments.

11. SELF-IMPROVEMENT IS A CLAIM TO TEST, NOT A MAGIC ACCELERATOR

Improving AI research would be significant, but optimizing one component does not establish unlimited recursive improvement. Gains may depend on human direction, compute, data or favorable evaluation choices.

Test what changed, what improved and whether the improvement generalizes. A model that raises a benchmark score by exploiting its evaluation has not necessarily become more broadly capable. An improvement should survive independent testing on tasks that did not guide the optimization.

Experiments, hardware, energy, data and verification can constrain progress. Rapid improvement is not thereby impossible, but its speed and extent cannot be inferred from one successful iteration.

Recursive improvement is therefore neither required as the sole definition of ASI nor sufficient evidence of it. A broadly superior system could exist without independently modifying itself. A self-modifying system could remain narrow or repeatedly make itself worse.

12. WHAT A CREDIBLE ASI EVIDENCE PACKAGE WOULD CONTAIN

Begin with an explicit claim: which cognitive activities, which human comparators, what margin of superiority and under which resource conditions. Register the evaluation rules before seeing the results, including how failures and exceptions will affect the conclusion.

Use protected tasks, strong baselines and independent teams. Repeat across conditions. Disclose methods sufficiently for scrutiny while protecting sensitive material. Report uncertainty, assistance, resources and failed attempts.

Then require converging evidence: unfamiliar-task learning, sustained problem solving, validated discoveries and broad expert-level comparisons. No single benchmark should carry the entire inference. Tests should be refreshed without moving the goalposts simply because a system meets a previously stated criterion.

A further complication is that observed performance need not reveal maximum capability. Research on AI sandbagging has shown experimentally induced selective underperformance in evaluated language models. That finding does not establish that all poor performance is deception. It motivates varied elicitation methods and caution about treating one failed test as definitive incapacity.

Successful outcomes under controlled conditions provide evidence of capability there; unsuccessful outcomes require diagnosis. Neither should be stretched into a claim about every future setting. An ASI assessment should remain open to revision as stronger evidence arrives.

Broad claims need evidence that extends beyond any single test or domain.

13. WHAT WOULD JUSTIFY THE NAME?

Under the working standard proposed here, the strongest case would be a system that repeatedly exceeds strong human expertise across varied cognitive domains, learns unfamiliar tasks effectively and produces independently validated results under documented conditions.

Superiority would survive changed problems, expert baselines and repeated testing. Resources, assistance and failure boundaries would remain visible. Accomplishment would matter more than personality or profound-sounding explanations.

This would still not prove omniscience, unlimited power or a right to govern human decisions. Intellectual capability does not eliminate incomplete information, physical limits, conflicting values or the need for accountable institutions.

The label should summarize evidence. ASI deserves recognition when broad, durable superiority best explains reproducible performance, and the claim remains precise enough to challenge.

AI DISCLAIMER

This article is AI-generated and intended mainly for educational and entertainment purposes. It does not constitute technical, financial, consulting, legal, commercial, investment or other professional advice. Conduct your own research and seek appropriately qualified advice where required. Future scenarios are hypothetical and do not establish that artificial superintelligence exists or predict its arrival. © 2026 Datasuperiority. All rights reserved.