Posted on

What evidence could falsify a claim that most online conversation is synthetic

“Most online conversation is synthetic” sounds testable until somebody asks what result would make you stop believing it.

That question is not rhetorical. It is the difference between a claim that can be investigated and a suspicion that absorbs every possible outcome.

A useful version might be: More than 50 percent of public conversational posts on Platform X during Month Y were generated and published without meaningful human authorship.

Now the terms have edges. There is a platform, a time period, a unit of analysis, a threshold, and a definition of synthetic authorship.

A strong claim needs a possible losing condition

Karl Popper’s idea of falsifiability is often oversimplified, but the basic principle is useful here: a claim should be capable of conflicting with possible observations. The Stanford Encyclopedia of Philosophy’s discussion of falsifiability explains that a statement becomes empirically testable when conceivable observations could count against it.

For the example above, a carefully designed study could draw a representative sample of public posts from the defined platform and period, classify them using several forms of evidence, validate the classification against known accounts or manual review, and estimate the synthetic share with uncertainty bounds.

If the best-supported estimate came out at 12 percent, with even the upper confidence limit nowhere near 50 percent, that would be strong evidence against the claim for that defined population.

It would not prove that all other platforms are mostly human. It would falsify the claim as stated.

Definitions have to stay put after the result

This is where broad internet theories can become slippery.

Suppose a study finds little evidence of synthetic comments. Someone can reply that the bots are actually in likes. A study checks likes; the claim moves to search results. Researchers examine search; the claim shifts to private messages. Evidence about one period is dismissed because the “real takeover” happened later.

Any one of those new questions might be worth studying. But continually changing the claim after contrary evidence prevents the original claim from ever losing.

That is not stronger skepticism. It is a measurement problem wearing armor.

Anecdotes can motivate a study, not settle it

A thread containing fifty obvious chatbot replies proves that fifty obvious chatbot replies existed. A strange profile with a generated face proves something interesting about that profile. A spam wave can show that one community was flooded.

None of those observations establishes the share of synthetic conversation across a platform, much less across the entire internet.

Prevalence is a population question. It needs sampling, definitions, error estimates, and scope.

The same standard cuts both ways. A handful of vivid human conversations cannot disprove large-scale synthetic activity either. Human examples are anecdotes too.

Dead Internet Theory becomes more interesting when its claims are allowed to fail.

Define what “most,” “online conversation,” and “synthetic” mean. State where and when the claim applies. Decide in advance what evidence would count against it.

Then go looking.

If no conceivable result could change the conclusion, the conclusion was never being measured in the first place.