The math behind BrandPulse

1What we scored

We scored 2,710 company websites in 35 categories, 2 to 200 in each, between 2026-09-30 and 2026-10-11.

Left out: 7 listed sites that block automated visitors (nothing we saw was the real site), 61 not scored yet, and 9 retired assessments (an older method or a test). 12 sites could not be reached when we checked: it scores 0, ranks last, and is kept out of the scale-setting.

2How a score is built

A score is a plain average twice over: each dimension averages its measures, and the overall averages the eight dimensions, each worth exactly one eighth, before the category's overall scale is applied.

1. Measures. About thirty measures are read, tested or looked up, each on a 0–100 scale, then put on the 1–5 scale: 1 Poor, 2 Weak, 3 Fair, 4 Strong, 5 Excellent.
2. Eight dimensions. Each dimension is the plain average of its measures.
3. The overall. The eight dimensions count equally: 12.5% each. Nothing is weighted by hand.
4. The overall scale. The average is put on the category's overall scale (see The overall scale): the order of sites never changes.

A worked example: Spencer Trask & Co.

Spencer Trask & Co. scores 4.0 (Strong) among Venture / PE, other industries sites. Start from a typical site of the category (3.44, the average of the category's medians), add each dimension's difference from its median at one eighth of the weight, then the effect of the category's scale (+0.14) and rounding (-0.02), which gives the average of the dimensions (3.7); the category's overall scale then adds +0.30.

3Fair comparison within a category

Each category sets its own scale so its best tenth score about 4.5; the scale keeps the order of sites close to intact (rank correlation 0.89 or higher between raw and scaled scores).

Averaging thirty measures pulls every site towards 3, which hides real differences. So each category's scores are stretched against the category's own sites: its 90th percentile lands on 4.5 and its 10th near 2.0. That is also why a 3.8 in one category is not the same as a 3.8 in another.

The order is not kept perfectly because each dimension is clamped to 1–5 and a dimension whose spread is too narrow, or that would put more than 10% of sites at 1.0 or 5.0, is stretched less. A ρ of 0.89 still means the scaled ranking broadly follows the raw one.

4The overall scale

Each category's overall is put on its own scale too: its median stays where it is and its best tenth score about 4.5. it rescales scores, it does not make any site better.

Averaging the eight dimensions pulls every overall score towards the middle: the dimensions are only loosely related (see the next sections), so a site that is Excellent on some is rarely Excellent on all, and before this step almost no site scored Excellent overall although hundreds did on single dimensions. So the average is put on the category's own overall scale, the same way the dimensions are: the typical site keeps its score, the best tenth land about 4.5, and the very best near 5.0.

It is a rescaling, nothing more. Every site keeps its order in its category (ties stay ties), sites at or below the median keep their score, and nothing about a site is measured differently. A higher number after this step is a clearer reading of where a site stands among its peers, not a better site.

Overall calibrated per category since 2026-10-08. The method version stays the same: the measures and how they are scored did not change.

5How precise the benchmark is

Each category's median is known to within ±0.45 points and its top-quarter line to within ±0.65, with 95% confidence.

More sites make a tighter interval. Re-drawing smaller samples from each category shows how much the median would wobble with fewer sites:

The eight dimensions are less precise than the overall: a dimension's median is known to within ±1.58 at worst, because its scores sit on fewer steps.

6How sure each site's score is

A typical site's likely range is 0.9 points wide, so its band holds across the whole range for 16% of sites; ranks are approximate, not exact.

Every report shows a likely range: the score's 80% range if the site had been judged on a different but similar set of measures. It is a deliberately demanding test, and the honest reading is this: a site's level (well above, near or below its category's middle) is well supported; its band holds for 16% of sites and can tip to the next one for the rest; and its exact place in a ranking of more than a hundred is not well supported.

A typical site's likely range covers about 0–75 places of its category's ranking. Read a rank as “around here”, and compare sites a band apart rather than a place apart.

7Do the eight dimensions hold together?

They measure related but distinct things: two dimensions correlate 0.17 on average, and Cronbach's α is 0.37–0.82.

If the eight dimensions were one thing measured eight times, they would move together and α would be near 1, and eight of them would be pointless. If they were unrelated noise, α would be near 0. BrandPulse sits in between, by design: a site can be strong on Audience appeal and weak on Reach. By the usual rule of thumb for a single-trait test (α of 0.7 or more), the overall would count as a broad index rather than one trait, and that is what it is: a balanced summary of eight separate qualities.

The closest pair is Audience appeal and Message & positioning (ρ 0.50); the least related is Audience appeal and Reach (ρ 0.03).

8Can we trust the AI judges?

Jev and a regular AI land within one level of each other on 94% of 73,640 questions, and when Jev says it is 90–100% sure it is right, or rightly says it can't tell, 97% of the time.

Two AIs that work differently answer the same questions about every site. Where they agree, the score is on firm ground; where they split, the report says so. Jev is also tested on questions whose answer we already know, so its stated confidence can be checked against how often it is right.

Agreement within one level is high, but some of it would happen by chance (65%), so κ corrects for that. An unweighted κ of 0.33 counts any difference as a miss and reads as fair agreement; the weighted κ of 0.72, which counts a near miss as nearly right, reads as substantial (Landis and Koch's scale). The two AIs rarely disagree by much, but they often differ by one step (50% of answers), which is why the score uses Jev, the judge whose confidence is tested.

On the whole known-answer test, Jev was right 51% of the time, said it couldn't tell 19% of the time and was wrong 30% of the time, over 11,439 questions (the regular AI: 50%, 18% and 32%). The test is hard by design: it asks for facts some pages never state. Any answer Jev gives below 60% confidence is marked for a person to check.

9Robustness checks

Swapping in the regular AI's answer wherever the two AIs disagreed moves a typical score by 0.0 and changes the band of 199 of 2,704 sites (7.4%); leaving out any one dimension keeps the ranking (ρ 0.83 or higher).

A sound score should not hinge on one judge or one dimension. Two checks test that.

Sensitivity: the median move is 0.00 points, 90% of sites move 0.20 or less, and the largest move is 0.80.

10Are the findings real?

100 of 229 tests on this page stay significant after correcting for running many at once.

Each research finding is tested in each category, so a pattern that only appears in the pooled numbers can't pass for a general rule. The effect size says how big a difference is; the p-value how surprising it would be if there were no difference at all; the adjusted q-value corrects for the number of tests on this page.

Two honest notes. The top quarter is picked by its overall score, which includes every dimension, so some gap on every dimension is built in; what the finding adds is which dimension separates them most. And “Technical health is where the top quarter's lead is smallest” is about size, not absence: pooled across the categories the difference is small (rank-biserial r 0.29) but statistically detectable.

Explore's scatterplots, tested the same way. Pooled across the categories, 3 of 5 are weaker than ρ 0.1; the score and a site's estimated traffic barely move together (ρ +0.07). A link that shows in one category only is reported for that category, not as a rule.

Correlations are patterns, not causes: two things that rise together may both follow something else.

11Limitations

The numbers are sound for what they claim, a careful benchmark of well-known company websites, and no more than that.

• A curated sample. Well-known companies with public websites in each category (2,710 in all), not a random sample of every company.
• Bot-blocking sites are left out (7 so far), so the benchmark cannot speak for them.
• Mostly one assessment per site. 2,699 of 2,710 sites have been assessed once, so a score is a snapshot of the site on that day.
• AI judges can share blind spots. Two judges that work differently, audit tools that measure rather than judge, and the known-answer test reduce this; they cannot rule it out.
• Patterns, not causes. A habit common in the top quarter does not prove it lifts a score.
• Traffic is estimated from a public popularity rank (Tranco), never measured.
• Each category has its own scale. Compare a site with its own category, and compare shapes, not numbers, across categories.
• Ranks are approximate. As section 5 shows, a site's exact place can move many places within its likely range; its band is the steadier reading.

12The Creative score

1,831 benchmark sites have a Creative score. Its typical 80% range is 0.40 points wide, and it moves with the BrandPulse score at ρ +0.52: related, not the same measure.

On the creative measures, Jev and the regular AI give the same level on 44% of 10,980 answers and are within one level on 98%.