Avoid these thirteen common playtesting traps and you’ll win more at Magic.
A lot more.
Everybody always wants the latest decklists, the latest tech cards to turn matchups around, the latest metagame data. These are all valuable, but there are
other elements of tournament preparation that most players gloss over that could greatly improve their understanding of the game and their chances of
winning.
These thirteen common playtesting traps interfere with our ability to learn about the game, the cards, the matchups, and the format. Avoiding them is
crucial for achieving mastery. It can be hard to let go of all of them at once, but the more you can put down, the better the results you’ll get.
If you’re serious about stepping your game up, make a list of each of these traps and which ones you do occasionally (perhaps more than occasionally).
Realizing when you’re falling into traps is the first step to dodging them in the future!
Then, the next time you do some testing or just playing games at all really, consult the list just prior to playing. Being mindful of the traps beforehand
can have a profound impact on one’s ability to get the most out of their testing games possible. Then after testing, look at your session ruthlessly with
criticism. Where would it at least be possible to accuse you of falling into one trap or another?
1.Getting Attached to a Deck…
One of the most common playtesting traps, this is also one of the biggest and most difficult to overcome. Becoming personally invested in a project can be
a powerful motivator, and that’s not a bad thing. The problem comes if your investment turns to attachment. The moment you start caring more about your
idea looking good than learning the truth, you start losing value.
Obviously, being attached to a deck shuts down a lot of possibilities that you could have moved to, but it actually goes beyond that. If you care more
about making your deck look good, you will miss opportunities to tune it, to make it the best it can be. You also miss opportunities to help your testing
partners gain the greatest understanding they could.
We must be able to take criticism from our teammates or testing partners. It’d be nice to imagine it’s never personal, but the truth is, to some degree it
is. The way some responds to you, the things they say, the stock they put into what you’re saying, it is personalized to you to some degree. The key is to
let go of that. Don’t take it personally even when elements of it feel like they are personal. Give people the benefit of the doubt. Remember, it’s not
that they don’t like you or want to hurt you (if that’s really the case, get the hell out of Dodge).
Being able to take criticism doesn’t mean agreeing with everything everyone is saying. It just means listening to it, hearing it, and being okay with
people sharing it. If people feel like they can’t criticize your ideas, you are going to miss out on a lot of valuable feedback. Even if the feedback you
receive ends up being “wrong,” and you were right the whole time, it was important that they were able to share the feedback in the first place. It’s
important not to repeatedly tear them down for having been wrong. Of course, if your testing partners are chronically in err, in some part of testing, talk
about it. What’s going on?
When you’re going into testing, make sure you are honest with your testing partners and with yourself about what your goal is. If you want to win the
tournament, remember that your goal is to win the tournament, and when you are faced with a decision, ask yourself which choice gives you the best chance
of winning the tournament. If you just want to play your own deck, that’s fine, just be honest with yourself. Maybe you just don’t want to play a control
deck no matter how good it is. Maybe you just want to play a deck you think is fun. You don’t have to justify why your priorities are the way they are.
That’s your business. It’s just useful to know what they actually are.
Know thy self.
Of course, none of this is to say that you shouldn’t try to make a deck work that feels like it might have potential to you. You have to trust your gut,
your instincts, but you also want to train yourself to seek the truth, whatever it is.
So, how can you know when to keep working on a deck vs when to let it go? This goes back to what your goals are. If you’re trying to win the tournament,
then how much time you still have to prepare is a big factor. The amount of time you have to prepare is a resource. You can gamble it on speculative
strategies, but you should figure out ahead of time, how much time you need if you have to go with plan b.
It’s useful to have a fallback plan in effect when testing speculative concepts. If your testing doesn’t end up producing something good enough, what will
you do? Maybe Affinity is your backup plan for Modern, and you know it well enough that you don’t need to test it. Then you can spend almost all of your
time searching. On the other hand, maybe you feel like if you fall back on Affinity, you need a full day to update and tune the sideboard. You should be
keeping track for yourself, what your plan is, and how much time your plan will need.
If you are going to have multiple testing sessions before your event, it’s good to ask yourself, at the end of each one, what would you play if the event
was tomorrow? If that strategy requires some amount of preparation, that’s sort of a final cut-off time for brewing.
Of course, we don’t want to spend all our time working on a strategy that isn’t going anywhere though. When you’re evaluating a deck and determining if you
think it’s got any potential, always remember the prime directive:
“Never play a bad something else.” -Michael Flores
2. Wasting Time on Hopeless Decks…
Is your brew faster? Better in the semi-mirror? Goes over the top of the format? Hits from an angle people aren’t prepared for? Exploits a powerful new
combination of cards that some people can’t beat? Is it more consistent?
There are any number of things the deck could offer to give it a reason to exist. Identifying a deck’s reason for existing is crucial. Figure out the niche
in the metagame the deck is trying to fill and ask yourself what advantages does your brew have over that deck–as well as what weaknesses.
Let’s say you like Jund because of how good it is against aggressive decks. If you end up determining that it isn’t actually better than Abzan against
aggro, but that Abzan is better against combo, you might determine it is time to give up on Jund unless something changes. The value of “being different”
isn’t zero, but it is generally overstated. It’s generally better to just play the best cards you can, the best deck you can.
You’ve got to be able to get away from a deck that isn’t going to work out. Feedback and criticism from testing partners is one of the principle ways to
figuring this out. Of course, this isn’t a call to be super harsh all the time…
3. Shooting Down People Potentially Good Ideas…
There’s a time for criticism, and there’s a time for open-minded exploration. When people are brainstorming, it can be very easy to poke holes in their
ideas. Sometimes this is called for, but sometimes you want to be able to just brainstorm what might be possible without being bound by what you can prove
is good this second.
When people are brainstorming, positive energy is a powerful catalyst. If someone has an idea that doesn’t sound that good to you, look to see if there is
something redeeming about it. Maybe you think it is unlikely to work. Well, what if it did? What would that look like or have to look like? You don’t want
to be deluding yourself or your testing partners, but sometimes it can be good to imagine, “What if?”
While you should strive to always be open to criticism, make sure to put enough positive energy into the equation to balance the criticism you point at the
ideas of others.
4. Having a Bad Take-Back Policy…
Whether or not to allow take-backs in testing is a controversial topic that doesn’t have one right answer. It really just depends on what you’re trying to
accomplish.
Generally, once my focus is on practicing with a deck, I don’t allow myself to make take-backs at all. However, when I am facing a stock deck that someone
is playing against me but that is not their deck, I would prefer them to make a take-back than to have a worse game. Now, if important information has been
revealed that changes things, it’s better not to go back; however, if someone playing a gauntlet deck just missed something, I’d rather have as tough of a
practice as possible.
If my focus is on figuring out if a deck is viable, I am more keen on using take-backs as long as no meaningful information has been revealed that changes
things. When you just want to know if a strategy works, it isn’t always useful to hold yourself to a tournament level of preparedness before you’ve had a
chance to practice. Of course, you have to be very careful, as this not only takes away from your own practice, it has the potential to interfere with your
learning about the format.
When someone is really invested in a deck, they may want it to win so bad that they give it their complete focus. They think about every play. They take
their time. They take back mistakes. However, if they are facing someone playing a stock gauntlet deck, their opponent may not be playing with the same
level of intensity, the same willingness to take mistakes back. This can lead to…
5. Not Trying with Stock Decks…
Testing against the stock decks isn’t about convincing yourself you’re already good enough. It’s not that you want the stock decks to go through the
motions of losing. If you beat them, you want it to be because of superior strategy, a superior deck or matchup.
All too many people just want to work on their brews. Somebody’s got to play the stock decks to test against! Every team is going to have members with
different strengths and weaknesses. Some people are going to spend more time brewing, and that’s fine. Having brewers is important, but so too is having
people willing to play the enemy and play it well. Even if you’re one of the members that does a lot of brewing, force yourself to play all of the gauntlet
decks some. The experience you gain with them tends to give you more tools for brewing, more insights.
I remember a few years ago, a strong player I knew was testing with a teammate, and their brew was crushing an important matchup. The thing is, not
everyone on the team believed the results, and one of the naysayers took a turn with the gauntlet deck and completely turned the matchup around.
It wasn’t that he was trying to lose on purpose. It’s just that both of the brewers wanted their brew to win very badly, while the third guy tried to his
fullest with whatever was in front of him. When you’re playing a stock deck, give it your focus. Win with it. You don’t want to take forever, as playing
very slow in testing is just a giant waste of your most precious resource. However, you do want to give it your real effort.
6. Not Keeping Track of Results
I can’t tell you how many times I’ve seen two players sit down and play a matchup for an hour, then have no idea what the score was. Keep track! It’s not
that the exact result is necessarily that important. It’s just that it takes so little effort, and the information can be valuable.
Of course, you do want to avoid…
7. Overvaluing Small Sample Sizes
Yes, you may have won that set 6-4, but a ten-game set is not actually that large of a sample size in the grand scheme of things. Even worse is playing
three games and having won 2-1, declaring it a good matchup.
I remember seeing a very strong player pick up a brew, lose his first three games, declare the matchup bad, causing him to not consider the deck only to
realize later that the matchup was at least 60-40 for the deck that had lost three times. Remember, if you’re a 60-40 favorite with a deck, you’re still
going to lose the first three 6.4% of the time! And that isn’t even factoring in possible unfamiliarity with the deck.
In order to make effective use of our playtesting time, we need to be able to look past the numbers, even when we keep track of them and use them some of
the time. What is causing the wins and losses? I remember sets with Faeries where sometimes Faeries would win 80-90%, while other times Faeries would win
just 40%. Were the minor tweaks really making that big of a difference? Then you realize the Faerie deck won almost every time it drew Bitterblossom and
lost most of the time it didn’t. Maybe that’s what’s really going on? If so, there is math we can use.
8. Overvaluing Recent Information
A corollary to this is to properly weigh new information against old. There is a real human tendency to overvalue recent information, to set aside hours,
days, weeks, even years of knowledge because of a single new data point.
Oh, the most recent tournament didn’t have any Abzan decks in the top 8? Does that mean Abzan is dead? Of course, if the field is 20% Abzan, even if it
averages 55% against the field, if you run twenty tournaments, there’s a very good chance some number of the tournaments feature top 8s completely devoid
of them.
The most insidious bias in testing because of recent information comes when people test a matchup, find it losing, change a few cards, test again and lose,
change more cards, test again and finally win 6-4, thinking they’ve solved the matchup. It might still be pretty bad, but when you just keep throwing out
results until you get the results you want, what have you really learned? That’s why the feel is more important than the exact numbers. The numbers are a
tool, they are another way of framing what you’re observing.
Along these lines, there is also a tendency in people to discount the old results that don’t conform to what they want to see, arguing, “Well, I changed
cards A, B, and C.” That may be true, but you then must discount the good results you had earlier too.
9. Counting Results After Changing Your Deck
Imagine a Sultai midrange deck that beats Affinity but loses to Twin. In an effort to improve the Twin matchup, you replace your Damnations and Spell
Snares with Slaughter Pacts and Dismembers. Now, you are getting the win percentages you want against Twin. However, you can’t just crutch on an Affinity
set you played yesterday by saying, “Well, I already beat Affinity.” Your list isn’t the same anymore, and those are meaningful changes!
10. Not Understanding What is Causing What
Sometimes, it can take a lot of changes to turn around a matchup. Where the problem comes in is when you don’t know what is causing what. You changed eight
cards and got the results you were looking for, but what did it? This isn’t to say you should only ever change one card at a time, however.
The two main approaches are to try changing as little as possible to see how much you can get out of a specific change. This is often the preferred
approach late into testing when you’ve already got a lot of results against a lot of decks and don’t want to throw all of the previous results out the
window.
Early in testing, however, you can make bigger, more radical changes. Do the most you could imagine justifying to make the matchup good. If it still
doesn’t work, you could save yourself a lot of time hunting something unrealistic. If it does work, pull back some. Try cutting out some of whatever seems
to help least (or fit least well). If you know you can turn it around with eight changes, but you liked the old cards in other matchups, maybe then try
just six (reverting two).
11. Trying to Beat Everything
At the end of the day, however, we gotta remember, we can’t beat everything (usually). In can be very inefficient to try. For instance, maybe we can
dedicate our sideboard to beating aggro and turn around an otherwise bad bunch of matchups. Is it worth it?
If aggro is just 20% of the field, all of that sideboard space might have been able to get us a lot more percentage against control, combo, and midrange
decks. It’s not about beating everything. It’s about giving ourselves the best chances to win the tournament.
An important part of tournament preparation is figuring out what you’re willing to lose to. Of course, this doesn’t mean you won’t have chances. It’s just
that, often, the best way to min-max is to take a certain matchup as a loss (for the most part), and not waste the slots trying to fight it. Falling into
the trap of trying to beat everything generally means watering your deck down against the field as a whole.
12. Getting Inbred
While it’s a trap to try to beat everything, it can also be a trap for everyone in the group to just try to beat the things other people in the group are
considering. The stuff your testing group likes is an indication of the kind of stuff some number of others will like, but it is vital to understand the
metagame as the community sees it.
If your group is all about U/B Control and Abzan, it might be tempting to slant your deck very heavily against those two decks, leaving you extremely
vulnerable to red aggro or green devotion or whatever. Learn from the stuff your group likes, but always remember to differentiate between your team’s
metagame and the real metagame.
13. Not Sideboarding…
One of the biggest leaks people seem to have in testing is not testing sideboards. Decks really are 75 cards, not 60, and you play about 60% of your games
after sideboarding. Time can be a real constraint, but making sideboarding an early part of testing is an invaluable habit to get into.
When a deck is in its infancy, it might be prudent to just play game 1s, getting a feel for the core strategy. However, when possible, it’s generally
better to include sideboard testing even in fairly early stages when the stock decks have relatively known sideboards.
In testing for Pro Tour Khans of Tarkir, it seemed clear that Abzan’s sideboard was a vital component of its ability to interact with different types of
opponents. Almost every ten-game set I played with it involved four games pre-board, six games postboard. Of course, there are some small sample sizes
possible there, but the important thing is the feel. Ideally, we want to play 8-10 game 1s, followed by 10-12 game 2s once we are actually trying to get
more solid of data.
Most tournament players actually don’t test sideboarding at all in most of their playtest sessions! That is so crazy, given how big a part of the game
sideboarding is. Remember, it’s not all about having a magical in/out list for card swaps. Getting experience with the change in pace of the game, the
change in relative importance of different cards is key.
Want to avoid this trap? Get in the habit of testing sideboards! Even if your sideboard is unfinished. Build a fifteen-card sideboard at the start of your
session, and resist the urge to change it every time you face a deck you weren’t expecting.
As you can see, there are tons of pitfalls one can encounter in testing if one isn’t careful. It’s not enough to just spend a lot of time testing. Making
the testing count has a multiplying impact on the time spent.
When you look at a list of these thirteen traps, how many do you fall into sometimes?
If it’s less than half, are you really being brutally honest with yourself? Can you really not find any instances of these?
Changing them takes time, but being mindful of them and working on them will have a profound impact on your game. It’s as simple as that.