The LLM Consolidated Testing Framework: 36 Signals, 37,000 Prompts, and a Methodology Built on Evidence

By Alex | AI Visibility | 10 min read

Introducing the LLM Consolidated Testing Framework: 36 identified signals tested across 37,000 prompt executions spanning multiple industries and LLM providers. Learn how the framework is building an evidence-based map of what influences AI visibility across ChatGPT, Perplexity, and Google AI Overview.

SEO Has 200 Signals. The AI Visibility Equivalent Is Being Built Now.

The SEO industry did not arrive at its understanding of Google's ranking factors through a formal announcement. It built that knowledge through obsessive testing, empirical observation, and a lot of well-organised disagreement. In 2005, Moz published its first Ranking Factors survey. Practitioners voted on what they believed was moving the needle. It was rough, crowdsourced, and still more rigorous than anything that existed before it.

From there, signal-finding became a discipline of its own. SEOs ran controlled experiments, published correlation studies, and debated methodology with genuine ferocity. Companies like Moz, Searchmetrics, and Ahrefs published annual ranking factor studies. Individual practitioners shared test results publicly. A rough consensus formed over time: content quality, backlink authority, and technical health were the core levers, with everything else orbiting around them.

That process took the best part of two decades. The question for AI visibility is whether it can be compressed.

The Problem With Waiting

The parallel is already playing out. Brands are losing traffic to AI-generated summaries. Marketers are asking why their brand appears in some AI responses and not others. The answers exist somewhere in the gap between how LLMs process training data, how retrieval-augmented systems choose sources, and how users phrase their queries.

Unlike Google, which at least acknowledges the existence of ranking signals, the major AI providers publish almost nothing. OpenAI does not have a Search Central blog. Perplexity does not run Webmaster conferences. Google AI Overview behaviour is tangled up with conventional search documentation in ways that make the two hard to separate.

So the signal-finding has to happen through direct testing, controlled experiments, and careful reading of the published research that does exist. That is the purpose of the LLM Consolidated Testing Framework.

What the LLM Consolidated Testing Framework Is

The LLM Consolidated Testing Framework is a structured programme for identifying, testing, and documenting the signals that influence how brands and content appear in AI-generated responses. It runs tests across multiple industries, multiple business sizes, and multiple LLM providers simultaneously. The goal is not to generate proprietary intelligence for a single brand or agency. It is to build a shared, evidence-based understanding of what influences AI visibility across the board.

The framework currently covers 36 identified signals across six categories: entity authority, content structure, source credibility, prompt alignment, cross-model consistency, and temporal relevance. These signals did not emerge from assumption. They came from three sources: published academic research, citation pattern analysis across 37,000 prompt executions, and practitioner observation across dozens of industries and business sizes.

Where the 36 Signals Came From

Published Academic Research

The academic literature on LLM citation behaviour has grown significantly in the last two years. Research from teams at MIT, Google DeepMind, and independent AI research groups has examined which types of content are more likely to be synthesised into model outputs. Veeramachaneni et al. (2023) confirmed that entity salience correlates directly with LLM recall probability. Work on retrieval-augmented generation systems has shown the importance of semantic depth and source authority. These studies do not translate cleanly into practitioner advice, but they represent the closest thing available to an empirical foundation.

What the Testing Programme Looks Like in Practice

Having identified 36 signals is a starting point, not an answer. The signal hypotheses are directionally credible, but the SEO industry taught a clear lesson: directional credibility is not enough. Correlation studies move a field forward, but controlled tests build the playbook practitioners can actually use.

Each test in the programme targets a single signal, controls for as many variables as possible, and runs across multiple brands, multiple industries, and multiple query types simultaneously. Results are reported in plain language that marketing teams can act on without a technical background. When a test is inconclusive, that finding is published. When a signal behaves differently across models than the hypothesis predicted, the framework is updated accordingly.

The first signal under structured testing is Google Entity Salience. It is one of the most well-evidenced signals in the framework, and it has a concrete measurement mechanism: Google's Natural Language API scores entity prominence on a 0-to-1 scale using 12 weighted factors including title presence, first-paragraph clarity, mention frequency, and schema markup coverage. That baseline makes it possible to test whether improvements to entity salience scores produce measurable changes in AI response mention rates, and whether those effects differ across ChatGPT, Perplexity, and Google AI Overview.

Who the Framework Is Built For

The LLM Consolidated Testing Framework is designed to be useful across the full range of people working on AI visibility: in-house marketing teams trying to understand why their brand appears in some AI responses and not others, agencies building AI visibility services for clients, and researchers who want a structured starting point for more rigorous investigation.

The testing data spans businesses across the size and sector spectrum, from early-stage companies with thin digital footprints to established brands with significant existing authority. The patterns that emerge from that breadth are more useful than patterns observed in a single industry or a single competitive environment.

What Comes Next

Results from the Entity Salience test series will be published here as they emerge. The methodology is shared openly. Each instalment will follow the same structure: state the signal, describe the hypothesis, explain the test design, report the results, and translate those results into practical steps.

SEOs spent 25 years building one of the most data-driven disciplines in digital marketing. That body of knowledge exists because practitioners refused to accept uncertainty as a permanent answer and invested in the testing infrastructure to find out what actually worked. The AI visibility space is at the same inflection point now.

The framework currently covers 36 signals across six categories. If you are working on AI visibility and believe there are signals missing from that list, reach out. The signal list is a living document, and the testing programme is designed to grow with the evidence. The more inputs the framework receives from practitioners working across different industries and LLM environments, the more useful the resulting playbook becomes for everyone.