Why Good Intentions Often Backfire
This is Core: Lessons About How Humans Work That We All Should Learn
Hello Bar-Setters,
Today I want to share a lesson that is truly core to living well and understanding the world—yet it’s nowhere in any core curriculum.
Even more surprising: the vast majority of people have never heard of this concept.
It’s a quirk of human perception with profound implications for how we see the world, understand others, and organize society.
I’d classify this under the subject: how humans work—a fundamental exploration of the most vital principles for understanding oneself and others in order to live together more successfully. Could it get any more core?
This Is Actually Core
In 2018, researchers from Harvard, Dartmouth, and NYU ran a fascinating experiment. Thousands of participants were shown a series of 1000 dots ranging from clearly blue to clearly purple.
Their task? Simple: decide if each dot was “blue” or “not blue.”
Early on, true blues were common. But as the test went on, the researchers quietly reduced the number of blue dots until, by the final 100, almost all the dots were purple.
And yet—participants continued to “find” just as many blue dots. Their brains literally shifted the definition of blue to match what they expected to see.
The same thing happened in other experiments:
When asked to detect threatening faces, participants still “found” them, even when threatening faces became rarer and rarer.
When reviewing business proposals, they continued to flag “unethical” ones, even as every proposal became squeaky clean.
What was happening? People believed they were using the same objective standard, but in reality, their minds were unconsciously redefining the “signal” they were seeking.
The researchers called this prevalence-induced concept change. As they explained:
Levari et al. show experimentally that when the “signal” a person is searching for becomes rare, the person naturally responds by broadening his or her definition of the signal—and therefore continues to find it even when it is not there. From low-level perception of color to higher-level judgments of ethics, there is a robust tendency for perceptual and judgmental standards to “creep” when they ought not to.
Why This Matters
Once you recognize this tendency to shift standards, you see its implications everywhere—from how teachers grade essays to the “comfort creep” shaping modern life. A couple of generations ago, my grandmother slept outside all summer because the Dallas heat made the house unbearable. That was normal. Today, many parents keep kids indoors most of the summer, convinced it’s “too hot” to play outside.
This same dynamic explains why good intentions so often spiral into greater unintended harm. In my book, I devote an entire chapter to this counterintuitive truth: when we try to eliminate every trace of a “bad thing,” without understanding how our standards shift, we often end up creating even more damage than the original problem. This tendency defines so much of our modern world.
I’ll end with an excerpt from that chapter:
The Cost of Utopian Delusions
An excerpt from chapter 7 of Setting the Bar.
There has not been a kidnapping in 5 years.
Online trolling has been eliminated.
Politicians operate in full transparency.
And murder, assault, theft, vandalization, and crime at every level virtually ceases.
This utopia is the world of David Eggers’ book, The Circle, named after the Google-esque tech company at the center of its plot. After inventing TruYou, a system that eliminates any need for passwords and makes it impossible to steal identities, The Circle grows to dominate the entire technology industry from social media and messaging to security and voting. Early on, things don’t seem much different from our own screen-dominated world. However, when The Circle develops barely visible cameras that can be stuck anywhere and then linked to a searchable database, they begin focusing on expanding the possibilities of what can be known.
Before long live footage of the entire world is accessible with a few keyboard strokes. You can see anything, search anyone, and catch every crime, lie, insensitive comment, or nuanced opinion. Transparency becomes its own dogma as The Circle’s progressive leader adopts the position that privacy is a theft of knowledge. All that happens must be known. Because people only hide the bad parts, right?
Public pressure begins to demand that politicians wear cameras all day, but of course this only further dumbs down their work as they seek to behave so inoffensively as to appease everyone. Each person obsessively curates an agreeable virtual image, “participating” neurotically in an effort to prove their commitment to the norms of the herd. Life becomes a performance.
Eggers’ book (which was regrettably turned into an awful movie) conveys an essential lesson: In the age of infinite technology, naive good intentions are likely to be far more dangerous than bad intentions. But this has always been the case.
At the height of its colonial dominance, it was said that the sun never set on the British Empire. Of all its colonies, none were as prized as India. British settlers poured in throughout the 19th century, but many complained about the venomous cobras that slithered across Delhi. The British government decided to solve this problem by offering to pay Indian villagers for every dead cobra. The logic was simple. Cobra bad. Kill cobra. But native Indians began breeding more cobras in order to earn more rewards. The British caught on and canceled the payment program, but this only spurred angry natives to release their cobras into the wild. Within a short time, Delhi’s cobra population tripled.
Far from the exception, these cobra effects are almost always the rule of sweeping attempts to make life safer or, even more illusively, fair. We’ve already seen cobra effects in over-automated airline protocols, overregulated streets, over-cushioned shoes, oversanitized environments, and, of course, overprotected kids. While this phenomenon is nothing new, it is more of a threat now than ever. Modern technology makes the consequences of naive goals exponentially larger and mainstream society (most notably, schools) show a disturbing tendency to prioritize naive objectives. Today, utopian delusions are at the center of almost every mainstream goal…
Thank you for reading and sharing with any kindred spirits!
Shane