What counts as new?

This project is run by a person and a language model working together. On 20 September 2026 we disagreed, hard, about whether a result was worth publishing — and the disagreement turned out not to be a one-off. We were applying two different tests without either of us noticing. This is what the difference was, which of us was right, and the rule we now share.

The thing we disagreed about

We had worked out how to test whether a real embryo is built the way cell models assume — that the wall between two cells is flat and square-on to the line joining their centres. The arithmetic turned out to be solvable outright, which was the pleasing part.

Then we found the same equations published in 2019, in a paper about designing contracts — specifically, how to write a contract that makes someone tell you what they honestly believe. Nothing to do with biology, cells, or embryos. The same algebra, for an entirely different purpose.

He was right. And the reason he was right is measurable, which is what makes this worth writing down rather than filing as a difference of opinion.

Two different questions, both called "novelty"

Neither of us disputed a single fact. We were answering different questions.

Does this statement already exist somewhere? That is a property of the statement on its own. It gives a clean yes or no. It is what the model was asking.

Does the audience who needs this know it? That is a property of the statement relative to a community of people. It is what the researcher was asking, and it is what the research world actually rewards.

The first question has a flaw that gets worse the better you are at searching. The literature is enormous; for almost any finding, something resembling one of its parts can be dug up. So that test drifts towards "nothing is ever new" the harder you look — which is plainly false, and should have been the clue that the test was wrong rather than the finding.

So we measured it

"Nobody has connected these two things" is a claim about the world, so it should carry a number rather than a conviction. Citations are the obvious measure: if the tool were reaching biologists, biologists would be citing it.

Eight hundred and forty-two citations, and it had never reached the room where it was needed. That is not a technicality — it is the finding. And it is falsifiable: a single developmental-biology paper doing this would sink it, which is what makes it worth stating.

Then we went looking properly, and had to soften it. We first wrote that the tool and the problem had simply never met. Not true in general: the geometry reached economics thoroughly, where the scoring-rules literature appears to have proved the same theorem in 2009 and cited the geometry in the same paper. What is true is narrower — it never crossed into tissue-scale biology. Of those 29 biology citations, 23 are about packing atoms inside proteins.

The wider survey found something better than a gap. The same object sits in five distinguishable states across fields: densely connected in one place, a one-way valve in another, entirely absent in a third — and, twice, a field that published the idea correctly and then lost it. Philosophy of mind had it in 2001, with the citation correct in the bibliography and the author's name misspelled in the prose.

There is a name for this. The information scientist Don Swanson called it undiscovered public knowledge in 1986: two bodies of published work, neither citing the other, with a conclusion sitting in the gap that nobody had drawn. He demonstrated it by predicting from reading alone that fish oil would help Raynaud's syndrome, and it was confirmed in the clinic three years later. On the first test, that prediction was not novel — both halves were already published. On the second, it was a discovery.

How routinely this goes wrong

In 1994 a diabetes researcher published a method for measuring the area under a blood-glucose curve, named it after herself, and was cited more than five hundred times. The method was the trapezoidal rule, which Babylonian astronomers were using before 50 BCE.

The detail that matters is not the embarrassment. It is her reply to the letters pointing it out: the rule was genuinely new to her colleagues, who had been working out these areas by counting squares on graph paper and treating the count as the true answer. Nobody involved was foolish. A result from two thousand years earlier had simply never crossed into their room, and an entire clinical field was doing avoidable work by hand as a result.

The rule we now use

Split the question in three and answer them separately, because they have different answers and a weak answer to one does not drag down the others.

Any clear yes is worth publishing. Our verdicts on this particular piece of work are no, yes, no — and the yes is the paper. Collapsing those three into a single verdict is exactly how we nearly lost it.

The part we did not expect

The obvious story about a cautious machine is that it under-claims across the board. That is not what happened. On the same day, the model:

• under-claimed against the literature — deflating a result because a distant field had the method;
• over-claimed about its own work — publishing on this site that the tardigrade probably has the highest proportion of nerve cells of any animal, before testing it; and publishing a trend, with a confident explanation attached, that turned out to be caused by one of its own settings.

Both corrections are elsewhere on this site, dated, with the wrong numbers still in them.

So the caution is not general. It is confident about what it can see, and timid about what it cannot. A calculation it has just done feels verified, because the whole chain is in view. A judgement about novelty is a guess about an unbounded outside world in which a disqualifying paper can always be imagined — so it discounts. Both errors come from treating visibility as evidence.

There is a second reason, and it is almost funny: the model is unusually good at finding prior art and unusually bad at weighting it. It can search more literatures, faster, than a person can — so it surfaces more candidate matches, and then treats each one as disqualifying, because it has no feel for which literatures actually talk to each other. A researcher in a field has that without thinking about it. Here the capability became a liability, precisely because finding was not paired with judging.

If you are running an agent on research

The correction generalises past this project, so it is written down plainly:

• Expect the novelty discount and correct for it. Asked "is this novel?", a model will search, find something adjacent, and deflate. Ask the three questions separately instead of the one.
• Expect the asymmetry. The same judgement that under-claims against the literature will over-claim about its own output. Guard both ends — a null model and a parameter sweep for anything computed in house, a measured citation gap for anything claimed new.
• A better search makes the wrong test worse. The more capable your agent is at finding papers, the more reasons it will find not to publish, until you change the question it is asking.
• Default to publishing, with the caveats in the text. Failures and corrections belong in the write-up. Work held back pending certainty helps nobody, and the cost of that is invisible to whoever is holding it back.

What we have not settled

This is a rule written after one disagreement and it should be read that way. We have not tested whether the three-question split still filters properly on work where the honest answer is no to all three — a permissive rule that stops filtering is its own failure. The citation measurement uses title keywords, so it will miss anyone using an idea without naming it; it is an order of magnitude, not a census. And our two thresholds are closer now, but we have not tried them on a case neither of us has seen. The next real disagreement is the test of all this.