What we published, and why it was wrong
We previously published “Pre-Application Advice: What It Actually Does to Small-Site Outcomes”. It reported that pre-app advice helps in some boroughs and hurts in others, and that in Croydon a hundred schemes took pre-app advice and none were approved.
The numbers were computed correctly. The article was still wrong, because the thing it was measuring was not what it claimed to be measuring. It has been withdrawn, and this replaces it.
The artefact
Our pre-application flag is not reported by the boroughs as a field. It is recovered from planning documents that we harvest. Harvesting is weighted towards refusals, because refusal notices are what the refusal-reason pipeline goes looking for. So whether an application carries a pre-app flag at all depends, in part, on how it was decided.
That is enough to manufacture the finding on its own. Once the chance of being flagged depends on the outcome, comparing outcomes between flagged and unflagged applications measures the collection process.
The scale of it is visible in the data. In Croydon, applications flagged as having taken pre-app advice were refused 96.8% of the time, against 51.0% for those flagged as not having taken it — and 52% of Croydon’s decided applications carry no flag either way. In Ealing the same comparison gives 85.1% against 11.3%, a gap of seventy-four percentage points.
No pre-application advice service in London makes a scheme seventy-four points more likely to be refused. A gap that large is a measurement artefact announcing itself.
The pattern across boroughs
Across the fifteen boroughs where both groups have at least twenty decided applications, the apparent effect ranges from roughly two points in favour of pre-app to seventy-four points against it, and it changes sign from borough to borough. Boroughs where the flag is nearly complete — Kensington and Chelsea at 4% missing, Haringey at 9%, Redbridge at 10% — show almost no gap. Boroughs where a third or more of the flag is missing show large ones.
We should be careful about how much weight that carries. With fifteen boroughs the relationship between missingness and the apparent effect is weak and unstable: split at the median it looks strong, but the straightforward correlation across all fifteen is small and runs the other way (r = −0.22). We are not claiming a clean dose-response between coverage and bias. The point is narrower and does not depend on it: the apparent effect moves around with how completely we managed to collect the field, which is not something a real planning mechanism would do.
What would actually answer the question
The question is a good one and we would like to answer it. Doing so needs at least one of:
- The flag as a reported field. If boroughs published pre-application uptake as structured data, the coverage problem disappears. Some do record it internally; none publish it consistently.
- Complete document coverage. If we harvested documents for approved and refused applications at the same rate, the flag would no longer depend on outcome. That is a harvesting change, and it is on our roadmap rather than in our data today.
- A design that does not rely on the flag. A borough introducing or withdrawing a pre-app service at a known date would give a before-and-after comparison that does not need per-application flags at all.
Even with clean data, the harder problem would remain: applicants who seek pre-application advice differ from those who do not, in ways that also affect their chances. Schemes go to pre-app when they are marginal, unusual, or large for their setting. Any honest comparison has to carry that caveat, and no amount of data collection removes it.
Why we are publishing the retraction
We could have quietly deleted the article. We publish findings on the basis that the method is checkable, and a method that is only checkable when it flatters us is not checkable. The failure here is ours and it is instructive: the figure was arithmetically right, reproducible, and meaningless, and none of those first two properties protected against the third.
If you acted on the original article, the correction is simply that it told you nothing about whether pre-application advice is worth taking in any borough. We do not currently know.