Twenty years ago, when teaching at the IMD business school in Switzerland, one of my popular assertions was: “Instead of seeking the customer in each individual, we should seek the individual in each customer.” Today, with technologies such as the internet of things and 3D printing, and with consumers taking to social media, companies have an incredible range of opportunities to customise their products and services for the individual. This enables marketers to offer much more variety to consumers.

In certain industries, customisation does not have a cost penalty. It is relatively easy to customise the coffee at Starbucks or the home page of the Amazon website for each user. But even when customisation is cost-effective, it has to be smart. It should not simply overwhelm consumers by offering them infinite options to select from.

There is the presumption that increasing the choice for consumers – which would allow them to select exactly what is most appropriate – is preferable. I mean, who could be against more choice?

From an economics perspective, more choice leads to utility maximisation through better preference matching. If sellers are evenly distributed along the circumference of a circular product space, then removing a subset of these choices will increase the average travel cost for consumers.

Psychology demonstrates that people prefer making their own choices rather than having them externally determined. This leads to feelings of control over one’s fate, instead of the hopelessness and helplessness you feel when someone else chooses for you.

Paralysed by choice

Yet there is a stream of research, pursued over the past decade by my ex-colleagues from London Business School, Simona Botti and Sheena Iyengar, that demonstrates why more choice is not always positive. In fact, while consumers may express a preference for greater choice, several experiments reveal that when complexity is high or decisions need to be made under time pressure, too much choice can be paralysing.

Consider, for example, an experiment conducted in a supermarket where researchers set up two different tasting booths. One had 24 flavours of jam whilst the other had just six. About 60% of shoppers stopped at the booth with 24 flavours to get a taste; only 40% of the shoppers stopped by the booth that had six flavours.

So far, so good for the benefits of greater choice. But the purchase behaviour paints a different picture. Only 3%of those who stopped by the 24-flavour booth bought a jam. In contrast, 30% of those stopping at the six-flavour booth purchased the jam.

Repeated experiments show that despite an expressed preference for more choice, consumers are less likely to make a choice when faced with a high number of options. And when they do choose, they have less confidence that they have made the right decision. Given more options, consumers are also less satisfied with their choice (even when this led to better objective outcomes).

Three negatives

There are three reasons for the negative effects of choice: information overload, consumers not having well-developed preferences, and negative emotions.

When too many choices are offered, consumers use a two-stage process to make a decision. First, they use non-compensatory screen or heuristics to bring the number of alternatives down to a manageable number. Then they employ a multi-attribute model to make a final choice. Often, when the wrong non-compensatory attribute – one that is not important or is negatively correlated with other important attributes – is used, the process ends up eliminating valuable alternatives.

The solution to information overload is to add an extra option only if the importance of the attribute is high and if the attribute helps maximise differential attractiveness. Alternatively, to reduce the cognitive effort, deploy decision-support systems that help consumers make choices.

Second, when consumers do not hold stable or well-ordered preferences, choice leads to cognitive conflict. Hence, they avoid making choices, or feel dissatisfied when they do make one.

The solution to unclear preferences is to add a default "best" option and give consumers the ability to opt out. Or one can provide opportunities to practice and develop a better understanding of one’s own preferences. Take the decision builder, which allows consumers to learn the impact of different savings and investment choices on their retirement incomes.

Negative emotions arise when all the choices are associated with unpleasant outcomes, or if they require a tradeoff between valued attributes such as safety and environment friendliness.

If all the alternatives lead to an unpleasant outcome, consumers prefer delegating decisions to an expert (perceived as more knowledgeable than them) and one who is trustworthy (will act in their best interests). The challenge here is for the company to build a reputation as a trustworthy advisor.

I consume hundreds of categories and brands in a month. Like many consumers, I prefer companies who make things that fit my needs, but also make life easy for me. If when selecting a new brand to consume or a new product category to participate in, I have to consider and evaluate many options every single time, then staying with the current brand or simply not buying is the optimal way to go.

In any case, providing a lot of explicit choices means abdicating responsibility in a data-intensive world. Amazon does not show its entire inventory to a consumer, or even its highest-margin products to a particular visitor. Instead, it uses a "recommendation engine" to showcase the items with the highest likelihood of purchase for that shopper. This recommendation engine, and its continuous improvement – which allows Amazon to know customers' preferences, often even better than the customers may know themselves – is the company’s competitive advantage.

Nirmalya Kumar is Member-Group Executive Council at Tata Sons and Visiting Professor of Marketing at London Business School. His Twitter handle is @ProfKumar. This article is written in his personal capacity.