In last week’s article, I decided to try something new. I gave you guys a untested deck I designed and asked you to playtest it and use the feedback option at the top of that article to let me know what you have learned in StarCity’s forums. The decklist I gave last week is as follows:
Star City Beats
4 Bloodstained Mire
4 Sulfurous Spring
2 City of Brass
1 Swamp
6 Mountain
4 Rishadan Port
4 Goblin Lackey
4 Mogg Fanatic
4 Jackal Pup
4 Veteran Brawler
4 Mogg Flunkies
4 Goblin Piledriver
4 Phyrexian Negator
4 Tangle Wire
3 Terminate
2 Lava Dart
3 Reckless Charge
Sideboard:
2 Threaten
1 Terminate
4 Cursed Scroll
2 Lava Dart
2 Disorder
1 Flaring Pain
2 Wasteland
1 Flash of Defiance
This project is still ongoing. There was lots of discussion on theory, but what I was really looking for was actual playtest results and suggestions based on playtesting experience. As of the time this article was written, such feedback was only starting.
This project needs more input from you (the www.starcitygames.com readers). So check out last weeks article and get crackin’!
In thinking about the task I laid out before the community, I realized that when I think of playtesting, I’m thinking of a process that’s more involved than most people’s concepts of playtesting. I decided that for this week’s article, I would share my ideas on playtesting.
Step 0: Don’t mess before you test
This is step zero as opposed to one because it calls for inaction. Many times when players get a decklist from a friend or off the net their first instinct is to customize it. They’ll modify the deck list swapping some cards for”better” cards based solely on theory and Magic intuition.
When this urge grips you, resist! It is very important to play at least a few games against every major deck type with the deck as is before you mess with it. This is especially true for decks off the net that placed well in a tournament.
Even the best pros can’t completely grasp all the ins and outs of a deck just by glancing at the list. There are reasons that deck did well enough to be listed on the web. Make sure you fully understand them before you muck with the deck. Often it takes actually playing the deck to realize the interaction between cards, or the usefulness of a spell against an opposing archetype.
Once you’ve gained insight into the deck though play, feel free to make your modifications. Then when you test your new version you will be able to compare and contrast the results with the original and see if your changes are actually improvements.
Step 1: The Gauntlet
The first step in testing any deck is knowing what to test it against. What are the decks you are likely to face in whatever tournament you are testing for? The best way to figure this out is to look to what people have been playing in similar tournaments, and what has done well in big events.
For example, if you are preparing for a PTQ, find out what people played in the last PTQ in the area. Many players will stick to the same deck for the next event. Some players, however, will want to switch to whatever deck type has been doing well around the world; the top 8 from recent GPs and decks that have been winning PTQs are often copied.
Once you think you know what you are likely to face, build a gauntlet for your deck of choice to test against. It should include every deck you feel will have a significant presence at the event. Don’t waste your time on”rogue” decks that only one or two players will play: You’ll need to spend your time concentrating on the popular decks.
Step 2: The Timeline
This step is about organizing the time available to you so you can get the maximum effectiveness out of your testing. Eventually, this will become second nature. At that point it won’t really be a step in the process. You’ll do it without thinking about it.
How much time do you have to test the deck? How many playtesting sessions can you squeeze into that time? Once you have answered those questions, make a schedule that brings your deck from raw material to finished product.
As a rough guide, put about 25% of your time into maindeck testing, 25% tournament-style testing, tweak, 25% more tournament style, tweak again, the final 25% tournament style then, decide on your deck.
Step 3: Main deck testing
In the beginning, you should concentrate on playing the main deck against as many opposing decks as possible. This will give you familiarity with how the deck works, and you can get an idea if the deck is worthy of further testing.
A question that often comes up in playtesting is,”How many games of a match up do I need to play to have a good feel for it?” The answer is a little more complicated than you might think.
If you just look at wins and losses, the number of games needed to have reliable results is very large. To give you an idea of what I mean, I’ll use a simplified example:
Say you have a fair coin, and you want to determine the odds of it coming up heads or tails solely based on flipping it and looking at the results. What happens with different numbers of flips?
1 Flip
- 50% chance – heads : You would conclude this coin comes up heads 100% of the time.
- 50% chance – tails : You would conclude this coin comes up heads 0% of the time.
You’re off by 50% (as bad a possible) 100% of the time. Average error is 50%.
2 Flips
- 25% chance – 2 heads : You would conclude this coin comes up heads 100% of the time.
- 50% chance – 1 heads, 1 tails : You would conclude this coin comes up heads 50% of the time.
- 25% chance – 2 tails : You would conclude this coin comes up heads 0% of the time.
You’re off by 50% half of the time. Average error is 25%.
3 Flips
- 12.5% chance – 3 heads : You would conclude this coin comes up heads 100% of the time.
- 37.5% chance – 2 heads, 1 tails : You would conclude this coin comes up heads 66.7% of the time.
- 37.5% chance – 1 heads, 2 tails : You would conclude this coin comes up heads 33.3% of the time.
- 12.5% chance – 3 tails : You would conclude this coin comes up heads 0% of the time.
You’re off by 50% 25% of the time and by 16.7% 75% of the time. Average error is 25%.
4 Flips
- 6.25% chance – 4 heads : You would conclude this coin comes up heads 100% of the time.
- 25% chance – 3 heads, 1 tails : You would conclude this coin comes up heads 75% of the time.
- 37.5% chance – 2 heads, 2 tail : You would conclude this coin comes up heads 50% of the time.
- 25% chance – 1 heads, 3 tails : You would conclude this coin comes up heads 25% of the time.
- 6.25% chance – 4 tails : You would conclude this coin comes up heads 0% of the time.
You’re off by 50% 12.5% of the time and by 25% 50% of the time. Average error is 12.5%.
As you can see, with a relatively small number of coin flips (a small sample size), you are not likely to be able to put together an accurate picture of the odds.
Fortunately, we’re not dealing with something as black and white as a coin flip or a die roll: In a game of Magic, you don’t just have a win or a loss. You have close games, lopsided games, and blowouts. You can use this information on what kind of win or loss each game was to get a better idea of the odds of a matchup without having to play hundreds of games.
For example, take a set of four games of Deck A versus Deck B. Deck A wins one in a blowout, wins two close games, and loses one in a blowout. If you conclude that deck A wins 75% (or most) of the time, you’re misleading yourself. From those results it would be more accurate to say the match-up was about even, maybe slightly in Deck A’s favor.
Keep this in mind when you analyze a deck performance.
Step 4: Tournament-Style Testing
It is very important that early in your testing you start playing sideboarded games. In the event you are preparing for, you will be playing more games with your sideboard than without. This means that how a deck performs in a matchup post-sideboarding is even more important than its pre-sideboarded games.
Despite this fact, many players spend far less than half their time on sideboarded games. Sometime players will go so far as to spend weeks testing a deck, only to throw the sideboard together the night before the tournament.
A good way to avoid this pitfall is to playtest in matches instead of games. Do best two out of three, with sideboarding after game one – just like in a tournament. This will force you to develop your sideboarding plans in every matchup, and give you a more accurate picture of how the deck performs.
Step 5: Tweaking
Now that you know how your deck performs against the field, you’re armed with the information you need to make modifications to the deck. Figure out where you are strong, and where you are weak. Revamp the main deck and board to address those weaknesses.
A few cards that are good in matchups where you have a large advantage might be swapped for cards that would help your tough matchups.
Once you have made changes, go back to”step 4″ and retest this new version against the entire field. You need to determine if changes you have made for one matchup weaken (or strengthen) the deck against other matchups – and by how much.
When trying to determine if the changes were worthwhile, keep in mind the amount of each deck type you expect. A 5% improvement in a matchup you expect to be 20% of the field is just as valuable as a 20% improvement in a match up you expect to be 5% of the field.
Step 6: Make Your Determination
The final step is to decide if the deck is what you should play in the tournament. Look at each deck you’re considering. Determine how each does against the field you expect.
I’m going to give you the long-winded versions of how to make that determination. You probably won’t have numbers accurate enough to make it worth plugging through the formula. It’s just meant to give you the idea of what are the important factors in deciding how a deck does against the field.
- Take the win percentage of your deck versus an expected deck in the field. (Example: this deck beats Tog 60% of the time.)
- Multiply that by the percent of the field you expect that to make up. (Example: Tog is expected to be 20% of the field. 60% x 20% = 12%)
- Repeat versus each deck in the field.
- Add all the products together.
- The total is your deck’s win percentage against the field as a whole.
The deck that has the highest win percentage versus the field is your best deck. When deciding which deck you should play, keep in mind your play style and preferences. It’s okay to play a deck which didn’t do quite as well in your testing if it’s more your style.