Jon Stephens, Group Chief Product Officer at Nortal, July 8, 2022
The writings of Charles Darwin, Joseph Schumpeter and Clayton Christensen span almost 200 years and they are rarely compared, however, there is a common theme that runs through all of their work that holds the key to developing better products and services.
These thinkers all produced famous analyses of different systems. Darwin of course focused on biological systems, Schumpeter focused on economic systems and Christensen focused on market / organisational systems. There is a common theme across all of their analyses – that there is an underlying process of experimentation which selects the winners from the losers.
I like winning and rather dislike loosing. When I develop products and services for clients, I want them to win, to be successful. And I believe that the process these great thinkers all analysed, experimentation, can also be used to select winning strategies, products, services, features and even user experience details.
The systems that Darwin, Schumpeter and Christensen analysed can be thought of as being on different ‘system levels’, see Figure 1 below. Darwin’s biological system is at the top level because the experimentation has the broadest scope – every living thing. Schumpeter’s economic system is the next level down and Christensen’s market and organisational system level is next.
I believe there are at least two additional levels of experimentation within the organisation, product level innovation and feature level optimisation.
When we design innovation, product and feature experiments we need to use different types of experiment and different tools depending on what system level we are focusing on. An experiment to determine the best call to action on a product page should look different to an experiment to determine whether customers would buy a new product.
However, no matter what system level we are focusing on, we can develop better experiments by understanding what makes the forces of experimentation work on these other levels. The lessons learned in one level can often be applied at the other levels.
Figure 1: Applying experimentation to different system levels
Darwin’s theory of natural selection (originally published jointly with Alfred Russel Wallace) is driven by key principles of variation and selection:
But if variations useful to any organic being do occur, assuredly individuals thus characterised will have the best chance of being preserved in the struggle for life; and from the strong principle of inheritance they will tend to produce offspring similarly characterised. This principle of preservation, I have called, for the sake of brevity, Natural Selection. – Charles Darwin, On the Origin of Species, 1859
In biology variation is driven by random mutations and selection pressure is driven by competition for food, shelter and suitable reproductive partners. As I develop innovation strategies, products and services I try think about the different ways that I can replicate these two conditions.
In the absence of mutations in the world of business variations have to be engineered. Too much randomness could cause wasted effort, but equally opportunities will be missed if these variations are too narrow, only based on linear thinking. Bringing creativity into the process of developing variations (or you could say concepts or prototypes) is crucial – ideation methods such as SCAMPER and worst possible idea are some of my favourites that help with this.
Selection pressure usually comes from finite budgets in organisations, but we need to clearly define the level of evidence we need to decide which innovations, strategies, products or features we deem to be successful. In healthcare for example there is a tendency to assume that controlled, randomised academic studies are needed before the adoption of even quite small healthcare pathway changes. In reality this slows down innovation and performance improvement. Changes that don’t require clinical guideline changes could be adopted and evaluated much more quickly if less academic rigour is required. Having a clear framework for evaluation at the start can save a lot of subjective arguments, and time, later.
The economic system of capitalism is really a giant machine for running experiments. Joseph Schumpeter developed the theory of creative destruction which says that successful businesses and industries are developed by a continual cycle of creation of new more efficient things, which destroy and replace the old, less efficient things. The success of an economic system depends on the intensity of the creative destruction process. The theory has three main principles:
When developing experiments in individual organisations (level 3 system) we can use these three principles as a useful reminder to:
The need for organisations to innovate in the face of disruption is most famously described by Harvard professor Clayton Christensen, originally in 1995 and updated more recently:
“Disruption” describes a process whereby a smaller company with fewer resources is able to successfully challenge established incumbent businesses. Specifically, as incumbents focus on improving their products and services for their most demanding (and usually most profitable) customers, they exceed the needs of some segments and ignore the needs of others. Entrants that prove disruptive begin by successfully targeting those overlooked segments, gaining a foothold by delivering more-suitable functionality—frequently at a lower price.”- Clayton Christensen
When we are designing experiments, we should involve not just the most profitable customer segments but also to those which are less profitable and currently underserved. For incumbent organisations this can be a defensive strategy against disruption. For new entrants this can be a route to getting a foothold in a market, from where feature experimentation can then be targeted at moving upstream to serve more profitable customers.
Christensen also stresses that the disruptive process takes time and that not all disruptions are successful. When designing experiments we need to focus not just on the experiments themselves but on the processes we use to connect together and repeat experiments. For a previous client of mine I developed a system for experimentation that tested very specific hypotheses in small ‘sprints’ and then combined the more successful changes together into ‘pilots’, the impact of which were then evaluated over a longer time period.
We also need to ensure that we have appetite for failed experiments – at Microsoft for example only a third of experiments are successful, roughly a third fail and the other third are inconclusive. If you don’t manage expectations about failure rates you can find an experimentation programme closed down before it has had a chance to demonstrate value.
At these levels experiments can look very different depending on whether you are operating at level 4 or 5. At level 4, a product level, we should be experimenting with completely different products, features and business models. These experiments could be using paper prototypes, fake landing pages and commercial models to reduce costs and get to ‘product-market fit’ as quickly and cheaply as possible.
At a feature level it’s more about experimenting with specific changes in UI or UX, using A/B or multivariate testing to generate evidence that a variation should be selected over the existing experience. There’s now a myriad of platforms designed to help us run experiments like this at scale. Experimentation at this level is more about improving what exists already rather than inventing completely new things. That’s not to say this is not valuable, small changes can have huge impacts, as Airbnb found when changing the icon for shortlisting a property from a star to a heart increased engagement by 30%.
Experimentation at both the product and feature level can be improved by understanding the work Darwin, Schumpeter and Christensen, even though their work is focused on systems of experimentation that operate at different levels. Their work can prompt us to think about important experimentation questions such as:
Looking at systems beyond the one we are currently experimenting in can help us design better product and service experiments, although to fully answer the questions above we need to pair this with expert knowledge of the product or service system in which we operate.
Source of the hero picture: The circular nature of time by harisla: https://wewastetime.wordpress.com/2012/11/14/the-circular-nature-of-time-ii/