Good moves

"All that matters on the chessboard is good moves." This quote comes from Bobby Fischer, winner of the "Match of the Century", in which he defeated his Soviet opponent Boris Spasski in 1972 and became the eleventh world champion in chess. But what is it that turns moves into "good moves"? And what does that have to do with customer loyalty? This post wants to clarify these aspects for you. Without the hassle of mathematical details, you will learn how a special method of artificial intelligence, reinforcement learning, helps companies optimize customer experiences in order to maximize customer loyalty and ultimately customer lifetime values.

Acting with foresight

So what is it that makes moves "good moves"? For example, is it bad to lose a piece? Certainly in the immediate aftermath. Used strategically, however, this situation can result in a later advantage that compensates for the initial loss, for example by breaking up structures advantageous to the opponent. So there are situations in which decisions should not be assessed solely on their immediate benefit, but must be placed in a long-term context. In these situations, it is important to come to an optimal sequence of decisions rather than just a sequence of decisions optimized separately.

How do you achieve this formally, in other words, what is the ratio behind algorithms that would allow us to make an optimal sequence of decisions? The answer is very simple: if the result of an individual decision within a sequence of interactions was good, you not only have to increase the value of this decision, but also the values of all decisions that have led up to it. That is, if you win a game of chess, it is not just the result of the last move. If, for example, you have sacrificed a pawn earlier on, this move may already have set the stage for ultimately winning the game. If you find yourself in a similar situation at a later game, it stands to reason to consider this pawn sacrifice again. Very experienced players have often found themselves in the same or comparable situations and know the moves that make sense in the long term and can bring them a little closer to victory.

What does this have to do with customer loyalty? Here too there is the proverbial pawn sacrifice. Let us assume that a customer is about to renew her mobile phone contract. Let us also assume that she has almost used up her monthly data volume, which would lead to throttling the transfer rate soon. According to the strategy of the mobile phone provider, she receives an SMS message with an overpriced offer for an additional data package. The customer may accept the offer reluctantly, as she might need to do some research urgently. When the contract is due to be extended soon thereafter, however, she decides to terminate it instead. As a result, the provider will miss out on important future revenue.

What went wrong here? Could it be that a pawn sacrifice, i.e. offering a discounted data package, would have made more sense in this situation? Definitely.

The provider could have chosen a better route, as outlined in the following. In a test, he could have offered a subset of all customers a discount to compare the resulting short-term losses of this strategy with long-term profits. This way, the provider could have found out that the customer from the above example and customers from the same customer segment (comparable customers) would have appreciated a little extra generosity, extended their contracts more often, generating a more sustainable revenue stream in the long run. The measure of an early, discounted offer is therefore also to be understood as an instrument to churn prevention and to foster customer loyalty. The problem of churn does not usually arise from a single instance at the end of the contract, but is often the result of a sequence of earlier suboptimal decisions. What makes things worse is that touchpoints nowadays happen in an increasing number of channels (social, push, email, SMS / MMS etc.) adding up to an overall experience. Companies are therefore faced with the challenge of orchestrating touchpoints in a variety of channels in order to increase the loyalty of their customers. People refer to this as the problem of the next-best-action (NBA). The word "next" indicates the sequential nature of the decision problem (see figure below). As often as this expression is used, in our experience it is rarely understood properly and even less so implemented in practice. For this reason, res mechanica has developed the software service goodmoves, which is based on reinforcement learning, the most modern artificial intelligence tool for solving sequential decision problems.

A typical customer journey at a mobile virtual network operator.

To orchestrate all measures in sometimes long-standing customer relationships across many channels within a highly dynamic market poses a massive problem to many companies. Due to a lack of awareness or against better judgement, companies have so far optimized individual decisions in isolation and often based on simple rules. With the help of reinforcement learning, decisions can be optimized in their entirety and dynamically adapted.

The art of generalization

The previous section introduced the notion of similar situations on the chess board or comparable customer behavior. Having a concept of comparability is of enormous advantage in both contexts. The reason is that with the large number of possible positions in chess games and the diversity of customer behavior, even with a vast experience, there are always situations that are different from those seen up to a particular moment. Someone who has learned to generalize from previous experiences, will be able to make an informed decision despite the novelty of the situation.

Formally, this means identifying invariant features of situations in which a particular decision is optimal. In this context, one also speaks of pattern recognition. In chess it may be that the exact position of a bishop who is neither threatened nor threatens another piece is irrelevant for the next best move. In the customer's example, the effectiveness of a particular upsell campaign, for example, may not depend on the age of the customer, but rather on whether the last invoice was comparatively high. The invoice amount would therefore be one of these invariant characteristics. Deep learning, another method of artificial intelligence, has the outstanding property of automatically identifying those features in data that allow the best possible generalization. As a result, in some fields of application, such as the diagnosis of leukemia, performances were achieved that are on a par with human ability. These algorithms were often able to detect and exploit features that were hidden to the human senses.

That is why res mechanica offeres to combine deep learning with reinforcement learning in its software service. Deep learning might not always be the best choice, though. Deep learning methods typically require large amounts of data to identify relevant features and learn what optimal decision can be associated with them. Not every use case complies with this requirement. goodmoves therefore relies on a family of complementary algorithms that take over the helm depending on the data at hand.

Adaptability

In addition to acting with foresight and the ability to generalize, a third characteristic is required to become a good chess player or to substantially improve customer experiences: the ability to adapt quickly to changing conditions. If you play against an unknown opponent, for example, the strategies you have learned from previous games can suddenly become less effective. In the case of a mobile operator, it can happen that, e.g., a new competitor enters the market, or an existing competitor rolls out a new product. Both actions have the potential to influence the behavior of existing customers. It also happens that special marketing campaigns attract new customers who behave in a way that is not comparable to that of existing customers any longer. The world is changing and it seems to be doing it at a faster and faster pace. In this case, it would be naive to solely rely on existing strategies and simply apply what has been learned up to some point in the past. Rather, it is a matter of careful experimentation to improve and adapt the strategy in accordance with market changes.

Formally, this means not always taking the measure that is optimal based on the current state of knowledge, but occasionally trying to make a possibly suboptimal decision. Decisions that have proven suboptimal in the past may have become optimal today and vice versa. The only way to find out is through appropriate tests.

res mechanica has integrated this subtle balance between exploitation of existing knowledge and exploration of new situations into its software service goodmoves. This makes it de facto self-learning: it autonomously recognizes trends at an early stage and prevents losses in revenue. In contrast to the advanced technologies that have become available, many companies currently still rely on so-called scores. A churn score, for example, is proportional to the probability that a customer will terminate her contract within a certain period of time. The underlying probability comes from statistical models of churn behavior. These models may be sophisticated, but there is still a problematic break here: The business department must decide which measures to take to increase customer loyalty depending on the value of the churn score. A high score, however, does not say which of all options is the best countermeasure. If an offered bonus is too high, the company is unnecessarily loosing money; if it is too low, the customer will be lost. goodmoves closes this significant gap by identifying optimal measures for each customer directly, while taking into account all business logic.

Summary

In summary, it can be said that succeeding in chess and campaign management requires three basic skills. Firstly, act with foresight, secondly, generalize, and thirdly, be quick to adapt to changes. All three skills culminate in goodmoves, a single artificial intelligence your business can benefit from. Did we spark your interest? As additional service, res mechanica offers to estimate the expected uplift for using its software service in advance with very little effort. Please contact us for a complementary consultation.

For the technology behind goodmoves, we achieved the highest score ever awarded for the renowned "EXIST" grant from the German Federal Ministry for Economic Affairs and Energy. Renowned companies from industries with long-term customer relationships (banks, mobile service providers, insurance and utility companies) are using our services - so why don't you? We are happy to talk to you about customer retention. Just get in touch!