Artificial Intelligence is codified metis
but it still does not enable perfect control or generate true novelty yet.
One of the major themes in James Scott’s book ‘Seeing Like A State’ is how metis and intuitive knowledge are often superior to rational knowledge, i.e. “knowledge embedded in local experience” is superior to “the more general, abstract knowledge deployed by the state and its technical agencies”. It is the complexity of the environment that makes metis superior.
Mētis resists simplification into deductive principles which can successfully be transmitted through book learning, because the environments in which it is exercised are so complex and nonrepeatable that formal procedures of rational decision making are impossible to apply.
Does this argument really hold in the age of machine learning and artificial intelligence? Clearly, some of the reasons why the high-modernist interventions of the 20th century failed are not relevant anymore. Some high-modernist schemes failed because they were too simple and did not take into account the nuances of the environment. For example, the first European settlers in North America were told by their Native American neighbours to plant corn “when the oak leaves were the size of a squirrel’s ear”. This is a great example of a heuristic that subsumes within it the impact of “many partly redundant signals” and enables the farmer to avoid planting the crop before a frost in a more sophisticated manner than just planting according to the time of year or the weather.
However, as Scott himself admits, this can be translated into codified, legible terms.
Practical knowledge like [this] can, of course, be translated into more universalistic scientific terms. A botanist might observe that the first growth of oak leaves is made possible by rising ground and ambient temperatures, which also assure that maize will grow and that the probability of a killing frost is negligible. The mean soil temperature at a given depth might do just as well.
Metis of this nature can be replicated by better data and more complex algorithms. And to a large extent, this is already the case with modern agriculture. Scott also criticises the high-modernist playbook for its essentially static solutions that do not adapt to changes in the environment.
The modernist visual aesthetic that animated planned villages has a curiously static quality to it. It is rather like a completed picture that cannot be improved upon. Its design is the result of scientific and technical laws, and the implicit assumption is that, once built, the task then becomes one of maintaining its form.
This, too, is a criticism that does not hold today. In fact, one of Scott’s examples (quoted below) is simply an argument for using a more sophisticated machine learning approach and incorporating all the variables that are relevant to the problem:
the planners also operated with a standardized model of the cultivators themselves, assuming that all peasants would desire roughly the same crop mix, techniques, and yields. Such an assumption completely ignored key variables, such as family size and composition, sideline occupations, gender divisions of labor, and culturally conditioned needs and tastes. The fact was that each family had its own particular mix of resources and goals that would affect its agricultural strategy year by year in ways that the overall plan did not provide for.
Again both these problems simply need better algorithmic control with more fine-grained approaches that take more variables into account and adapt over time, e.g. machine learning.
Machine learning is codified metis
But what about the intuitive expertise of an experienced practitioner that cannot be communicated in a codified manner, knowledge that is “so implicit and automatic that its bearer is at a loss to explain it”? In fact, much of recent machine learning and AI functions like a codified version of this sort of metis and intuitive knowledge. This is not a novel argument, and many others have made the same point:
Machine learning is mētis for computers. It is knowledge gained through experience with a large group of similar situations, rather than knowledge which can be systematically described. How the program works is usually completely illegible to even the person who wrote the code. Once trained, the program functions as a ‘black box’, which converts the input (e.g. a picture) into an output (e.g. whether or not it contains an apple), without us knowing what happens inside.
Unlike traditional codified, algorithmic approaches, “a trained neural network is still essentially a metis solution, turning a body of experience (training data) into a specialized knowledge of a problem domain”.
Machine learning and AI allow us to control and automate illegible domains without having to convert the domain into a legible representation/model that we humans can understand. This is a significant step forward, and the recent explosion of large language models is another huge step in this direction (as I briefly touched upon in my previous essay).
So does this mean that Scott’s critique of the high-modernist approach is no longer valid? Does machine learning and AI enable us to repeat the Soviet experiment, except with better data and algorithms, and construct the utopia of our dreams? The answer, unfortunately, is that we cannot. As I’ve described in a previous essay, more complex control still falls victim to Goodhart’s Law. As a result, control and optimisation inevitably become more complex, expensive, and less effective. Even private firms are not immune to this lifecycle.
Too much climbing, not enough wandering
But there is another fundamental problem that machine learning and AI have not yet managed to solve. Every one of these approaches is still focused on exploitation and climbing up hills rather than exploration and wandering into the unknown. Even generative AI simply recombines and remixes existing components. Every complex adaptive system must include components and processes within it that search for novelty for novelty’s sake (in small doses). In the modern world, we climb too much and wander too little and machine learning and AI only make this problem worse.
Although the Soviet Union was extremely inefficient by the time of its collapse, the fundamental source of its downfall was its complete inability to innovate. The post-Soviet capitalist world is not much better in this regard. We can see the consequences of excessive hill-climbing and optimisation all around us. Our sports, our music and our movies have suffered from the homogenising consequences of the ‘Moneyball-For-Everything’ approach that has taken over all competitive domains today.
This is not an argument that our systems are getting stuck on local optima. In fact, machine learning is surprisingly good at not getting stuck at local optima1. This is also not an argument that we do not wander enough in the known domain. If anything, large language models like GPT-3 and ChatGPT enable this to an unprecedented extent. The argument is that we do not explore beyond the known domain and seek novelty for its own sake. One of the fundamental reasons for this 'exploration deficit' is our refusal to engage in 'goalless' action that does not have a clearly defined objective. One of my favourite books, 'Why Greatness Cannot be Planned', explains why even just having a clearly defined objective can thwart creativity and innovation. That will be the subject of a future essay.
Our intuition about 2/3D landscapes does not carry over at all into higher dimensions. High dimensional spaces suffer from saddle points more than local minima e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7720171/