Never retrofit an old system
The Soviet experience applied to the NHS
The Soviet Union collapsed because Gorbachev tried to reform the system. Much like a legacy codebase that is impossible to replace, the Soviet system had become too rigid and fragile to reform without triggering a systemic collapse. But the Soviet economy had been in a state of terminal decline since the 1960s, and Gorbachev was correct in his assessment that the status quo could not continue. Gorbachev knew that change was needed but did not know how to deliver change effectively.
One of the reasons for this decline was an astonishing reduction in the productivity of capital invested in the economy. Gorbachev himself experienced this when increased expenditure would be swallowed up by the system with barely any improvement in output, let alone productivity. Many systems in the West have reached this stage. The British healthcare system is an excellent example. As the tweet below suggests, we’ve reached the stage where increased resources seem to be unable to improve system performance.
Retrofitting and Soviet Decline
So why have we reached this stage where throwing more resources at our failing systems makes so little difference to their performance? Because we are repeating the same fundamental mistake that the Soviets made from the 1960s to the 1980s, when they chose to pump resources into retro-fitting and reconstructing existing industrial projects rather than focusing solely on green-field investment. In his book ‘Farm to Factory’, Robert Allen explains the damage that this caused and the discrepancy in productivity between the new green-field projects and older units where capacity was increased.
At first glance, it seems obvious that we should first add capacity to existing units before we build green-field projects. The Soviets believed that “they could save on the cost of structures by adding new equipment to established factories”, but nothing could be further from the truth, and Allen explains why:
While reconstruction expenditure sounds like an efficient way to invest in industry, it proved highly wasteful. The aim of investment is either to increase output or to reduce costs, and replacement investment did neither well. Retrofitting new equipment was a much more expensive way to increase capacity than “green field” investment. The problems are familiar to anyone who has renovated a bathroom: new equipment is intended for new installation and does not conform to the connections, power requirements, or placement of the old models. Space is often an important constraint. New equipment may operate at a higher volume of production, thus requiring a greater flow of raw materials and finished product. These flows cannot be economically handled in the cramped confines of old facilities. For the same reason, the economies attainable from the integration of successive stages of production cannot be achieved when equipment is retrofitted.
In fact, Gosplan themselves found that “ it cost 55 percent more to increase capacity in old works than in green field projects”. The Soviet steel industry was the best example of this dynamic. Green-field steel plants that were started in the 1960s enjoyed economies of scale and contributed to increased productivity. However, older plants swallowed up both capital and human resources for little increase in output.
Retrofitting software is even more expensive than hardware
One may argue that retrofitting Soviet steel plants has little in common with overhauling modern service industries such as healthcare. But the cost and time difference between building new software from scratch and overhauling legacy software is even more pronounced than the same difference in the world of industrial hardware.
Building new systems in UK healthcare: An outline
So what does it mean to enable greenfield projects and build new systems in the context of a modern service organisation? Let me take the concrete example of British healthcare to illustrate a few key principles.
Grow a new small organisation responsible for part of the remit of the old organisation. In the case of the NHS, an obvious way to do this would be a completely new organisation for a region or a sub-domain (e.g. cancer). This does not mean that the existing NHS organisation is relieved of responsibility in that region/domain. The public has the option to choose either organisation. This is analogous to how software companies typically replace their old monolithic systems and databases - progressively replacing separate functions of the monolith with services, and keeping the monolith alive until the last such function has been replaced.
Embrace duplication. The most common argument against the green-field approach is the argument the Soviets assumed was correct - surely it is less expensive to patch up the old system? As the Soviets found out later, nothing could be further from the truth. The level of duplication required to grow and build new systems to replace the incumbent system is temporary and cheaper not only in the long run but in the short run as well.
New organisations must have a separate chain of command all the way to the top. The CEO of any new organisation cannot report to anyone with a vested interest in the old organisation. This is also true of control functions. Regulatory monitoring of the new organisation too must be performed by a new control function.
The new organisation must not be restricted in its remit from taking on any adjacent activity that it deems necessary to deliver its mandate. For example, it is likely that the lack of social care is the bottleneck that currently prevents the NHS from releasing beds when patients farewell enough to be discharged. If the new organisation needs to deliver social care to solve a bottleneck, it should be able to. As software systems grow, progress often slows due to “dependency hell” as every new initiative requires a team to get work done by many other teams that it has no control over. Over time, organisations too suffer from a version of this problem due to the specialisation in roles and boundaries and the splitting of the organisation into separate silos. In most “legacy” organisations, private or government, it becomes impossible to implement any new initiatives as any meaningful initiative depends on coordination and the integrated delivery of services across these boundaries.
In a previous essay and this one, I’ve taken the example of the British healthcare system. But no one should be under the impression that these problems or principles only apply to government organisations. In the next essay, I will focus on a canonical example of a large corporation that faced the same problems and solved these problems using the same principles described above.