Scarcity: Arena’s Metagame Problem

The largest issue with Arena Draft is the homogeneity of the metagame. This is a consequence of insufficient bots. I often hear players exclaim that the bots need to draft better, to draft more like people. But this doesn’t necessarily solve the real problem.

If I sit down at a draft table with a bunch of reasonable drafters, and I do this multiple times a day with the same seven other people, by the end of the week I’ll likely have an exploitative strategy. The decks that are in the Arena Draft metagame are drafted in this fashion. We all go sit with the same bots, employ similar exploitative strategies, and then play against each other with the same archetypes. Even if the bots had reasonable card evaluations, the consistency in their drafting still yields easy exploits. And while there is some deviation given that bot-personalities that bias towards archetypes/colors have been implemented, it hasn’t proven good enough. Honestly, I’m not sure there is any approach to designing Draft bots that don’t have this issue. However, I believe there are principles of Limited to include and different approaches to designing the algorithms to minimize this issue.

As the quote from Lee Sharpe (Digital Project Manager/Game Designer for Magic Arena) indicates, the goal of the bots on Arena is to provide a good Draft experience, and they don’t necessarily need to draft reasonable decks in order to achieve that. However, I don’t believe the bots succeed on the three axes Lee defines. It’s too easy to find an exploit, and even when the bots are updated, a new exploit emerges quickly. I don’t believe this issue is due to failure on the part WotC or their employees, but rather because the problem presented is just that difficult. My main goal today is to offer an understanding of the fundamental differences between drafting with bots and people. I think these differences shed light on the nature of the game, and teach lessons that are seldom taught. But before I get into this, I want to address: “Why should you listen to me? What could I possibly know about bots?”

I spend a lot of time drafting. In fact, I believe it is the activity I spend the second-most time doing. What’s the first? Devising Machine Learning and Artificial Intelligence systems and algorithms. I was introduced to programming these kinds of algorithms almost eight years ago. I pursued them in both my undergraduate and graduate degrees. I’m currently a Data Scientist/Machine Learning Scientist for PepsiCo. I’ve even partnered with DraftSim and have been working on my own Magic Drafting Neural Network pipeline. I’m certainly not the only Magic player who has this kind of background, but I doubt there are many more that have the understanding of Draft that I do and a platform to discuss it.

The reality is that nobody outside of those who work on the bots really knows how they function. That includes me. I could speculate for hours on how they function, how they’re implemented, what kind of data they see, and more. But that’s not what this article is about. It’s about why this problem is hard. What I believe is important for a successful Draft experience on Arena. And most importantly, how this environment accentuates a fundamental principle of Limited that is often forgotten: scarcity.

The Scarcity Function

Every Limited format has a feature that I refer to as its scarcity function. Magic has four rarities: mythic, rare, uncommon, and common. Rarity influences the expectation of how many copies of any given card will show up in a draft. For example, most formats have about 100 unique commons, and 24 packs with ten commons each are opened. This means that, assuming each common has the same probability of being in a pack, the expected number of a specific common in any given draft is:

24 * (10/100) = 2.4

Scarcity is when you take this expected value and consider it alongside how highly that card is picked. It’s much easier to draft multiple copies of Barrow Witches than it is to draft multiple copies of Bake into a Pie, even though both cards are equally likely to show up in packs. The difference is that Bake into a Pie is more highly evaluated than Barrow Witches, making Bake into a Pie harder to come by. That’s the scarcity function in action.

Informally, I define a scarcity function as the number of times an arbitrary player at a draft table expects to see each card in the format during the draft. Note that just because this function says that the expected number of Bake into a Pies is low, it doesn’t mean that it’s impossible to draft many, just unlikely.

Furthermore, a format’s scarcity function isn’t static. As a format evolves, the evaluation of the cards changes. In the first week of Throne of Eldraine on Magic Online, Revenge of Ravens would go late. As more players played against it and consumed content that suggested how good it is, their evaluations changed, and hence the scarcity function of the format changed along with it.

My goal in any Draft format is to learn the scarcity function as quickly as possible, and continue to adapt my strategy as the format evolves. The scarcity function is what dictates the cards that wheel, what’s open more often. And, more than any other concept, it informs how I draft.

This is the largest fundamental difference between drafting with bots and drafting with people: Arena has a different scarcity function. A recent example of this is Merfolk Secretkeeper in the previous iteration of the bots. It was effectively a super-common because the bots just wouldn’t take it, and hence Dimir Mill ran the tables. The bots behave with their own understanding of the format, and if they have fundamental misevaluations, it becomes too easy to exploit them because cards that should be scarce aren’t, or cards that keep archetypes in check are too scarce. This is what homogenizes the experience of drafting on Arena. If the bots draft specific cards much higher than humans, those cards are effectively removed from the format altogether because people won’t have the opportunity to draft them.

Note: The following sections are a bit dense. If you’re not interested in exploring how one might approach this problem with the bots, feel free to skip to the last section, which explains how the contents of this article can be applied to improve on the skill of drafting with people.

Nash Equilibrium

This poses the question: “Does each format have an ideal scarcity function?” In my opinion, this is the same thing as asking if a Draft format has a Nash equilibrium. The informal definition of a Nash equilibrium from Wikipedia is as follows:

Informally, a strategy profile is a Nash equilibrium if no player can do better by unilaterally changing his or her strategy. To see what this means, imagine that each player is told the strategies of the others. Suppose then that each player asks themselves: “Knowing the strategies of the other players, and treating the strategies of the other players as set in stone, can I benefit by changing my strategy?”

If any player could answer “Yes”, then that set of strategies is not a Nash equilibrium. But if every player prefers not to switch (or is indifferent between switching and not) then the strategy profile is a Nash equilibrium. Thus, each strategy in a Nash equilibrium is a best response to all other strategies in that equilibrium.

Consider the game of Rock, Paper, Scissors. Three potential strategies are “always rock,” “always paper,” and “always scissors.” It’s clear that none of these strategies are Nash equilibria because if your opponent knows what your strategy is, they can deviate from their strategy and beat you every time. The solution to this game is a Nash equilibrium of randomness. Select rock 1/3 of the time, paper 1/3 of the time, and scissors 1/3 of the time. This way it doesn’t matter if your opponent knows your strategy; they cannot do better than beat you 1/3 of the time. This is called a mixed strategy, a strategy where you follow a set of pure strategies (e.g. “always rock”) based on some probability distribution.

The aggregation of card evaluations according to the probability distribution of a mixed strategy yields a scarcity function. Hence, if a Draft strategy is a Nash equilibrium, it would generate an ideal scarcity function to describe that Draft format. Implementing bots that behave according to this equilibrium would mean that the player drafting with the bots is incentivized to draft according to the same equilibrium, and cannot be rewarded by deviating from the equilibrium by definition. This not only removes the issue with exploiting the bots, but it also solves the problem that the cards the bots draft don’t go into the pool of cards faced in the games.

Is this even possible? Can a Draft format have a Nash equilibrium?

John Nash proved that every game with a finite number of players and a finite number of pure strategies has at least one Nash equilibrium. Hence there exists a mixed strategy that is a Nash equilibrium for any given Draft format because there is a finite number of archetypes and ways to rank cards within those archetypes. However, it’s likely to be quite complicated because the strategy must be affected by what cards have been drafted so far. In fact, it’s probably so complicated that humans couldn’t even attempt to grok it, which is why draft rarely feels solved outside of archetype/color rankings.

This is even further complicated because pick orders must be fluid. I could decide to take So Tiny over Merfolk Secretkeeper for Pack 1, Pick 3, and then be offered the same decision in Pack 1, Pick 4 and take Merfolk Secretkeeper over So Tiny. Another example of this is synergistic cards — for example, the value of Cauldron Familiar is much higher with access to Witch’s Oven. As an extension, the value of Cauldron Familiar is higher in Pack 1 than in Pack 3 because the probability of seeing Witch’s Oven in the draft is higher at that point.

The nuance that a perfect Draft strategy employs is incredible. But Aaron Foresythe stated that this nuance isn’t necessary for the bots:

If learning the perfect equilibrium is both extremely difficult and unnecessary, can we sufficiently approximate it in a way that still solves the following issues?

1. It is too easy to exploit the bots.

2. The play experience is homogenous (because the cards the bots draft don’t go into the pool of cards played against).

Objective: Approximating an Equilibrium

One of the most successful Poker algorithms, DeepStack, “plays an arbitrarily close approximation to a Nash equilibrium.” Given that it isn’t necessary for the Arena bots to discover the Nash equilibrium of a format, I believe that optimizing for the scarcity function is the perfect way to maintain a reasonable experience drafting on Arena, and can in theory approximate a Nash equilibrium because a Nash equilibrium generates the scarcity function.

Every Machine Learning solution boils down to an optimization problem. The optimization goal is called an objective function. So in order to properly train bots for Arena, there needs to be an objective function. I will define this objective function as a minimization of the difference between the Magic Online scarcity function and the scarcity function that corresponds to the Arena Draft bots. I believe that this is a reasonable approach to designing bots because the person drafting with the bots will see cards according to a similar distribution to drafting with other people.

Scary math warning!

If this distance is zero, this means that strategy h* perfectly generates the perceived scarcity function for MTGO data, and hence drafting on Arena will provide players with the same exact card availability as drafting on MTGO!

However, h* is not guaranteed to be a Nash equilibrium. It’s just a strategy of drafting that corresponds to the frequency cards are seen in drafts with eight people on MTGO. Early on in a draft format, it’s unlikely that this is a Nash equilibrium because people are not behaving according to an optimal strategy yet. But later in a draft format it would likely be a reasonable approximation of a Nash equilibrium. If it was easy to update the bots, this wouldn’t be an issue, as WotC could just update them at a higher frequency as the MTGO metagame evolves. However, given how infrequently they update the bots, I don’t want to assume that that’s a possibility. This begs the following question:

How can we protect the bot-draft ecosystem from exploits when there is a reasonable probability that the MTGO scarcity function is generated by a non-equilibrium strategy?

Introducing Distributional Deviation

Arena bots have different “personalities” according to this article.

With this release, we’ve given the bots drafting Guilds of Ravnica personalities. This means that for GRN drafts you’ll be sitting down with a table of random bot personalities that each value color combinations differently . . . For all the personalities, including the no preference bots, we’ve added a little randomness on the top for each of their picks as well. We believe this introduces a healthy level of variability into each draft and should keep the drafts more interesting.

This was implemented as an attempt to introduce variance into how the bots make picks in order to avoid the homogeneity of the metagame. While it made exploits more difficult to find, it didn’t solve the problem. Maybe the magnitude of the issue lessened, but not by enough. However, the concept of introducing variance to the set of bots is important. If there is minimal variation in the bots’ behavior, exploiting them is easy, even if they behave with a higher level of complexity than pick order. And once an exploit is discovered, the metagame becomes homogeneous. Hence it is important to introduce variation in the bots created via the objective function described in the previous section.

Adding deviation to the scarcity function is similar to WotC’s approach of creating bot personalities. Rather than have each bot behave according to the same h*, different bots could have different strategies. Recall that the scarcity function f_MTGO (C) is just the expected number of times a person should have the opportunity to draft card c in any given draft. What would happen if this function changes? Well, then our algorithm would learn a different h*. It’s possible to run the algorithm 100 times with 100 different versions of f_MTGO (C). This would generate 100 strategies for h*, some more similar than others.

Then, for every draft, seven unique bots are selected that behave according to different strategies of h*. Furthermore, if the deviations in f_MTGO are created about a normal distribution with a relatively small standard deviation, the integrity of the algorithm likely holds. This is because the mean of the 100 different versions of f_MTGO is approximately f_MTGO. It isn’t perfect, but the goal is not to create perfect bots; it’s to create bots that provide a good draft experience. However, this approach may have computational limitations. Depending on how involved the training process is, running the algorithm many times could be expensive. Let’s consider a less computationally intense approach.

Once h* is learned, adding changes to the strategy has almost no computational cost. Hence it’s possible to introduce deviation directly to the strategy h* for every single individual bot, which allows for unique draft experiences. Unfortunately, I’m not convinced that introducing variance to the strategy h* in this manner would maintain the integrity of the algorithm, as the scarcity function is no longer guaranteed to be close to f_MTGO. But this doesn’t mean that it’s not a reasonable approach, just that it should be implemented carefully.

In the case where there are large computational limitations, I would suggest introducing some logic for deviating from h* based on the confidence that f_MTGO is generated by a Nash equilibrium. What this means is that, in the beginning of the format, bots should deviate from the strategy h* both more often and by larger amounts because the probability that they can be exploited is higher. Later in the format, this distributional deviation can be minimal, as it’s more likely that f_MTGO corresponds to a stable metagame.

The Takeaway

Scarcity is a crucial concept in Draft that is often forgotten. I believe this article demonstrates how fundamental scarcity is because optimizing for scarcity is a possible solution to creating Draft bots.

If scarcity is not something you think about while you draft, I highly suggest it. Start slowly by trying to identify cards that don’t behave according to their rarity: commons that appear at a higher frequency than any other common, uncommons that are such a high priority that they appear as frequently as a rare. Once you begin to notice these, patterns emerge. It becomes easier to devise strategies and make picks because a card may wheel. It becomes easier to identify the open colors. Everything becomes easier once scarcity is introduced to your vocabulary, and I can’t emphasize that enough.

Furthermore, I hope this article is a good demonstration for my favorite lesson I learned in graduate school: Sometimes, in order to solve a problem, it must be explored in a different space. I apply this to drafting, gameplay, and even my non-Magic job!

The intuitive way to create bots for drafting is to attempt to mimic the picks that humans make. Almost every implementation of bots I’ve looked at tries to minimize the error between what the bot would pick and what a human takes. However, exploring the space of minimizing the scarcity function opens the door for a whole new approach to the same problem.

Next time you’re in a situation, whether in Magic or not, where the problem in front of you is proving difficult, take a step back. See if you can tackle it from a different angle, in a different space. It’s a powerful tool to have in your arsenal!