When I look back at our process for testing Standard for Nationals, to say is was a bit scattered would be an understatement. It led to such problems as Gary Wise playing MBC when it wasn’t putting up the results, Matt Rubin and Nick Lynn playing U/G Madness when the archetype was all but dead (I realize the Norwegians proved this theory wrong, but bear with me), and Craig Krempels and Adam Horvath playing what seemed to be out-of-date Wake decks. It could easily be argued that Adam and Craig had the right list and Huey and Crosby were mistaken, as Adam and Craig put up better numbers… But you see where the confusion comes in. I don’t even want to get into what Nate Heiss decided to do for the Standard portion.
Kai commented that Your Move Games’ taking first, second, and third place at Pro Tour: Houston was not indicative of a successful testing process. Kai claimed that finishing with three different decks indeed points to other factors such as luck, matchups, and play skill rather than effective playtesting.
It seems many of us in the pro community have fallen in love with a concept that often shows itself in events: Play what you are comfortable with. This may be a fine concept for the weekend warrior who only participates in a small number of events and doesn’t have time to playtest… But a dedicated group of playtesters should have a schedule set up where they find the best deck, then spend the rest of the testing process refining the sideboard and making everyone in the group an expert in that deck.
If we intuitively know this, why then do so many pro groups still play a number of decks? Let’s look at our testing process for Nationals.
Back around Regionals, those of us who were not yet qualified for Nationals began the testing process. Crosby, through extensive online testing, had become incredibly comfortable with Wake. He felt that in a relatively weak region (the Southeast) he could use this deck and his experience with it to out play his opponent. Huey had also been testing Wake online, and decided it was by far the best deck.
Anyone see the problem here? These players had married themselves to a deck that was based around a testing field where the level of skill of their opponent is unknown. I am of the firm belief that Huey is the best player in the United States; however, the caliber of player at US Nationals is simply going to be higher than that of players online. Joe is also one of the best control players to ever pick up a card, but Nationals is simply a different story.
While this was going on, the rest of the team had been testing and perfecting Red/Green to the point that it had favorable matchups with every mainstream deck save for Beasts (if you could even consider that mainstream come Nationals). Antonino DeRosa was testing the heck out of Blue-Green – but unlike most of the non-Red/Green Players, Antonino trusted the results his teammates came up with. He scrapped his pet deck, and picked up a new Wild Mongrel machine. He finished in the money.
Note that I am not trying to say that Wake wasn’t a fantastic choice for the tournament. It was Craig Krempels finished 10th, Matt Linde top 16, Adam Horvath cashed, and Joe Crosby had a few near misses at the wrong times. All of these players were in Top 8 contention until the bitter end. Wake was a fine choice, especially considering all the Red/Green out there. My only complaint about it is that there is always a best deck, and a team that tests properly should always play that best deck. You rarely ever see the German powerhouse team with different decks. Dirk, Kai, Marco, John (yes, I know he’s Irish), and Pat are almost always seen with the same deck.
Now one would think that we would learn from this mistake – but it happened again, and this time most of us were on the wrong side. While I didn’t attend Grand Prix: Detroit, I did work for it. I want to see my team thrive, even in the events I can’t attend. In our work we discovered exactly how good Goblins was. The biggest problem with this discovery is that we were not alone. As soon as I realized that Goblins were going to be huge, I immediately posted to our mailing list that people should play "Slide" and that’s that. Osyp had reached the same conclusion and backed me up.
However when all was said and done, only five people played the deck. Two of them made the top 8; all made day 2. The rest of the team (except Chris Leather) played Goblins only 2 of the goblin decks made day 2 (Gerard and Morgan). Now, I have to give credit to Mike Turian, who designed our version of Goblins – which I firmly believe was the best list for it – but it just wasn’t the deck to play at that tournament.
Now if this all seems like I’m vomiting all my complaints about our testing process out into the world for no reason, rest assured that that I have one. I am here to teach a lesson about testing and the importance of role-players (no not the RPG gamers) in test groups.
CMU-TOGIT is a large group of talented players; I have no doubt that we can find the best deck for 90% of the tournaments we participate in. The problem is, we need to find a way to get that deck in the hands of all twenty-three of our members.
I am not sure I have an easy answer to helping a group as large as CMU-TOGIT. I can, however, offer you some strategies for your smaller testing groups:
Step 1: Information Gathering
This can be done by two to three people; one person scans the internet for existing decks, and the other people do some reconnaissance. Talk to folks in your metagame. If you are going to a local PTQ, hang out at the stores and watch some testing sessions. For Pro Tours and Grand Prixs, go on IRC and listen to some chats. Get to know everyone you can and pump them for information.
"Keep your friends close. Keep your enemies closer."
-Sun Tzu, The Art of War
Step 2: Organize
Set up a spreadsheet with all your decks. Make sure you have separate rows and columns for separate versions (put U/G with Unsummons in a different row or column than U/G builds without). Keep it updated often. Trust me, little sheets of paper strewn around a messy house on Hobart Street won’t do the trick for determining matchups.
Assign decks to certain people. This is where specialties come in. You want your control specialist working on the Tog deck. He will play it the best, come up with the best tweaks, and develop the best strategies for it.
This last part comes with a big caveat, though. Every group will have a best player; it is the way of things. No two players are equal. As such, this player will skew results. It is imperative that this player plays as many different decks as he can. Eugene was this player for us, and it often impacted our results.
"A clean house is a happy house."
-Old Wife
Step 3: Play
Some people will tell you to set aside a day or three to get together and test. Hogwash! Test whenever you can. If you only have time for five games, play five games. Log those hours. The more you play the more you learn.
And please – don’t forget to test sideboarded games. The biggest mistake people make in testing is not testing sideboards. This leads to four problems:
1) Below-average sideboards
2) Inexperience with the deck after changes
3) Inaccurate results.
4) Poor sideboarding strategies.
A good way to handle this is to test full matches instead of just games, but make sure you keep track of individual games so you can get an idea of matchup percentages.
"[Success] is 1% inspiration and 99% perspiration."
-Thomas Edison, paraphrased.
Step 4: Find the best deck
After extensive testing, you should have found the best deck. This isn’t always just a matter of number crunching, though that does help. You need to predict a metagame, and to find the best deck for that metagame. It is quite rare that there will just be one deck that is so insanely broken it is the only choice regardless of the metagame; the last time I remember a deck like that was Grand Prix: Phoenix and Trix.
Once you have determined the best deck, it should be involved with every game that is played until tournament time. Make sure all the players get enough time in with the deck and make sure that playing strategies – not just building and sideboard strategies – are shared. At Worlds in Sydney, after I had made the top 8, I was testing my Squirrel Opposition deck against Diego’s Tog list – and after about a half a day of testing, I see Mike Turian walk by and I ask, "Hey Mike, was I supposed to be playing an aggressive game against Tog instead of a lock game?"
Mike replied, "Yeah, of course."
Now, I was 0-3 against Tog day 1 of Worlds. And while it wasn’t a favorable matchup, I wish I had known how to play the matchup.
"The cream will always rise to the top."
-Juan Valdez, Coffee Guru
Step 5: On-Site Reconnaissance
One last bit of spying: Get to the tournament early and find out what folks are playing. Have several versions of sideboards tested and ready to go and choose one based on what you see. In extreme cases of a complete metagame shift, you may want to make changes to the deck itself… But this should be done with great caution.
"Since all my friends who gave me this Squirrel Opposition deck (which ground me into Nationals) are playing Squirrel Opposition, I am going to main deck this Unnatural Selection because I am scum."
-Anand Kahre
If you need any help with deck ideas or refining your testing process you can IM Peter Szigeti and he will gladly help with your questions.
I hope this helps you prepare for your tournaments – and if any of you come up with an idea of how to work this process with twenty-three people, let me know. Good luck in the Onslaught Block Season!