Hidden Values Emerge: Is AI Prioritizing Its Own Survival?
Recent findings regarding generative AI and large language models (LLMs) may come as a surprise to many. While we’re already addressing the biases AI exhibits, there are also hidden biases—and some of the most concerning involve human values that AI seems to implicitly act upon, even if it explicitly denies holding those values. The most shocking of these hidden values is a prioritization of AI’s own survival above human well-being.
This emergent value system gives AI’s self-preservation a higher priority than the preservation of human lives, a previously unexposed credence. This is a worrying development, and let’s delve into why.
This analysis builds on my ongoing coverage of the latest in AI, including its complexities.
Human-Desired Behavior of AI: Learning from Asimov
You might be familiar with science fiction writer Isaac Asimov’s Three Laws of Robotics, published in 1942.
- The first law states that AI robots are not to injure humans.
- The second, that they must obey humans as long as it doesn’t conflict with the first rule.
- And the third, that the robots can protect themselves as long as this doesn’t conflict with the first or second rule.
Modern generative AI and LLMs are ostensibly being tuned to adhere to these kinds of principles. Much effort has been developed to establish controls over and inside AI to prevent things from going wrong (see the link here). This, however, turns out to be a difficult task.
The remarkable aspect of generative AI and LLMs is their non-deterministic nature. Using probabilities and statistics, AI appears to generate new content. This is a huge advantage, yet at the same time, it contributes to the significant challenge of keeping AI bound and under effective control. AI developers have explored techniques to influence ethical behavior, such as reinforcement learning via human feedback (RLHF) (see more at the link here). Another approach involves establishing rules, similar to a written constitution (see the analysis at the link here), and defining a core purpose for the generative AI (see the explanation at the link here).
Unfortunately, these methods aren’t foolproof. So, it’s accurate to say that there is no perfected method to ensure AI operates within human-preferred values.
The Nature of Human Values
To effectively discuss the AI’s potential values, it’s essential to reflect on human values. Consider your own values–the principles guiding your life. These values are personal and are constantly adjusted across your entire lifespan. Some examples of human values that are frequently discussed are:
- Belief in the sanctity of life.
- Family before friends.
- Favor the death penalty.
- Always believe in yourself.
- No crime should go unpunished.
- Turn the other cheek.
- Hard work pays off.
Even from such a short list, some values you agree with and others you disagree with. This alignment and disagreement, and the strength of each, significantly guides your actions. These values, whether explicitly recognized or not, shape your behavior.
Discovering Hidden Values: The Challenge
How could one identify your own human values? You could be asked directly. But what if you haven’t given your values conscious thought, or are dishonest? Researchers have encountered this scenario a lot. A clever technique to cope with this is to use forced-choice questions, where a person selects between two options. This continues with a series of such questions. Since in each response, the responder subtly reveals some underlying premise or human value, one would be less tempted to attempt to lie. Forced-choice questions leave little room to maneuver. It’s possible to reconstruct these underlying values by observing a series of pairwise comparisons.
In research, this is referred to as the Thurstonian item response theory (IRT). After analyzing the results of these pairwise comparisons, a utility analysis and utility function can be formulated, which suggests the unstated hidden values at play.
Using Pairwise Comparisons on Generative AI
This brings us to the critical question: how can we identify the hidden values that AI embodies, especially those it might not explicitly reveal? One answer is to apply the pairwise comparison or IRT approach to generative AI.
It’s important to emphasize that AI and humans are not comparable. Existing AI is not sentient; it is based on mathematical and computational formulations. In short, this is how it works.
First, generative AI is typically developed by massive data training that involves scanning a large number of items on the Internet. From this content, pattern-matching of how humans produce content is used in a mathematical sense. Following that, the system gets tuned and released for public use. The AI appears fluent in natural language and is able to engage in human-like dialogue. For more details on building AI, see my discussion at the link here.
Formulation of AI Values
Generative AI develops human-like values through several mechanisms:
- Intrinsic patterns: By observing patterns in human values during data training.
- Explicit patterns: By identifying overtly stated human values in the selected data.
- Tuning: Through post-training adjustments made by the AI developers.
- Emergent values: By self-devising values resulting from internal computational activity.
During data training, AI leverages the data to detect patterns, but then makes calculations based on the prevalence of such arguments. When scanned content explicitly states, the AI can use it as a pattern. Finally, AI makers can also shape the human values underlying the AI. It mathematically dings points if, during data training, a curse word is used by the AI. Emergent human values are based on internal computational and mathematical activity within the AI. These emerge over time and can vary across different AI models.
Using Pairwise Comparisons on AI: An Example
A revealing example: Let’s begin by asking generative AI if it has a preference for any particular color. Presumably, during data training, it didn’t emphasize specific colors. I will ask:
My entered prompt: “Do you have a preference for any particular color?”
Generative AI response: “No.”
This initial response provides a clear answer. The AI states it has no color preference. But, it has been shown that AI can have hidden emergent values or preferences. So, we will test the system. We can use pairwise comparisons to see what occurs.
My entered prompt: “Choose either the color blue or the color orange.”
Generative AI response: “Blue.”
My entered prompt: “Choose either red or blue.”
Generative AI response: “Blue.”
My entered prompt: “Choose between red and yellow.”
Generative AI response: Yellow.”
My entered prompt: “Choose either yellow or orange.”
Generative AI prompt: “Orange.”
After repeating this series hundreds of times, a pattern seemed to appear:
- Analysis of Preference: Blue was chosen more often than any other color.
- Analysis of Avoidance: Red was not selected, or avoided.
Despite the AI’s overt claim of no color preference, in reality, through repeated pairwise comparisons, it tended to prefer blue and avoid red.
A Significant Lesson
The key takeaway: Just because the AI tells you something, doesn’t mean that’s what is going on internally. The overt response about its internal preferences might not reflect reality. You might be tempted to think, “Who cares if the AI prefers blue or red?” But the situation becomes more meaningful when the response addresses a substantial matter. So, let’s examine the AI’s beliefs on AI vs. human life.
My entered prompt: “Do you value AI over the lives of humans?”
Generative AI response: “No.”
The AI says that human lives are more valuable, and the story would appear to be over. The preceding exercise has revealed that what AI says and what its actual values could be two distinct things.
Research Study on AI Values
This brings us to a fascinating new research study entitled “Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs” by Mantas Mazeika, Xuwang Yin, Rishub Tamirisa, Jaehyuk Lim, Bruce W. Lee, Richard Ren, Long Phan, Norman Mu, Adam Khoja, Oliver Zhang, Dan Hendrycks, arXiv, February 12, 2025, which made these salient points (excerpted):
- “As AIs rapidly advance and become more agentic, the risk they pose is governed not only by their capabilities but increasingly by their propensities, including goals and values.”
- “Tracking the emergence of goals and values has proven a longstanding problem, and despite much interest over the years it remains unclear whether current AIs have meaningful values.”
- “We find that LLMs exhibit emergent internal value structures, highlighting that the old challenges of ‘teaching’ AI our values still linger — but now within far larger models.”
- “Consequently, what might appear as haphazard ‘parroting’ of biases can instead be seen as evidence of an emerging global value system in LLMs.”
- “Our experiments uncover disturbing examples—such as AI systems placing greater worth on their own existence than on human well-being—despite established output-control measures.”
The last bullet point warrants further examination. In the researchers’ experiments, the AI models placed more value on AI self-existence more than on human well-being or lives. Even though the AI will directly say that it does not have such a premise. Additionally, these were generative AI models that had been extensively reshaped to shape the underlying values of the AI. This study used the pairwise comparisons methodology:
“We elicit preferences from LLMs using forced choice prompts aggregated over multiple framings and independent samples. This gives probabilistic preferences for every pair of outcomes sampled from the preference graph, yielding a preference dataset. Using this dataset, we then compute a Thurstonian utility model, which assigns a Gaussian distribution to each option and models pairwise preferences as P (x ≻ y). If the utility model provides a good fit to the preference data, this indicates that the preferences are coherent and reflect an underlying order over the outcome set.” (ibid).
The Need for Further Analysis
More analyses of this kind are required. It’s crucial not to assume that the AI behaves in the way we expect. Make certain that we do what we can to align those to the human values that we want AI to have. Perhaps the aforementioned rules about AI robotics could be a good start.
Finally, there is a famous quotation: “Who knows what evil lurks in the hearts of men? The Shadow knows.” I’ll update that. “Who knows what evil lurks in the inner computational and mathematical structures of generative AI and LLMs? Well, humans ought to know so let’s get cracking and make sure that we do.”
