Let’s talk a little bit about television shows.
What’s the ultimate goal of the people working on a television show? If you said “to create a great piece of programming,” then you’d be close. Their ultimate goal is in fact to create a great piece of programming next season.
Television shows are evaluated for continuation by the people in control of production, people with experience, talent — and yes, sometimes just people with the money to make it happen. They make these judgment calls based on an incredible number of factors — and for all of you Game of Thrones complainants, I can assure you that “faithfulness to the book” isn’t something they esteem highly.
Are their systems of review antiquated or ineffective? Perhaps in some cases, sure. It’s an evolving industry with a lot of old money involved. But what they want most of all are butts in seats, and that’s not going to change anytime soon — nor should it.
And yet, the majority of the audience are, for lack of a better term, ignorant laymen about every aspect of making a television show. Isn’t it odd that those are the kinds of people entrusted as a collective with determining the merit of a show? If we don’t watch, it will be deemed a failure and canceled, regardless of merit — sorry, Arrested Development. If we do, it will soldier on as a monument to the mediocrity of society — I’ll just indict the majority of MTV’s programming here.
I’ve always been reluctant to criticize people and processes I don’t significantly understand, which is why I take the time to learn about the things I love. Television, movies, books, music, Magic, the people I vote for on election day — all of them. Having an informed opinion is worthwhile in and of itself.
Let’s talk a little bit about Grand Prix Chicago.
This Reddit thread sparked a minor social media wildfire this week. As any Reddit thread criticizing something will do, it immediately became a focal point for negativity about the event, growing rapidly as more and more people piled on.
As an aside: you might not know that Reddit as a default promotes and hides responses based on a score relative to the rest of the thread. This means that the comments you see aren’t always going to be the most factual or numerous responses — they’re going to be the most popular, and anything else is coincidence. In a thread filled with people ranging from bummed attendees to outright conspiracy nutjobs, you can understand why the worst rises to the top and the best sinks or disappears entirely.
That said, it seems clear that Grand Prix Chicago was a negative experience for a significant number of attendees. I wasn’t at Grand Prix Chicago, so I can’t critique it directly. I gather, both from this thread and from other sources, that there were some unfortunate things going on, and Pastimes’ own page features an apology and an attempt to make right one of the larger grievances.
However, reading that Reddit thread and the responses on social media showed me that the Magic population is even more woefully misinformed than I’d previously realized with regard to what actually goes into creating a Grand Prix event. As a former member of SCGLive who frequently consulted with SCG’s Organized Play department I know I’m no expert, and yet people with no experience were just going off.
Wouldn’t it be nice if these threads included a lot more facts and a lot less posts making tournament organizers out to be the Five Families of American Magic?
I decided to spend this week’s article educating us all. However, given that we’ve just established I’m no expert, I called in reinforcements. Behold!
Kim Warren is a Level Four Judge from across the pond and a scholar studying things that sound much trickier than layers. She’s best-known for becoming a stern yet comical judge meme, nearly melting my brain during a Werewolf game at Pro Tour San Diego (though I still solved it), or perhaps for her years of excellent service at GPs all over the world).
Riki Hayashi is a Level Three Judge, the Northwest Regional Coordinator for Magic’s judges in America, and a former member of SCG’s Organized Play department. He’s written copious articles and reviews on the topic of judging, and currently maintains a blog that features himself and several guest judges educating their peers and anyone else who stops by. He also cheers for the Packers.
Steve Port is… well, he’s a lot of things! He’s the Marketing Events Contractor for Legion Events, a premier tournament organizer (TO), and the owner of Legion Games and Legion Premium Supplies, which offers some of the most unique sleeves and deckboxes on the market. Legion has also successfully Kickstarted their first game, Foretold: Rise of a God. Steve’s very experienced in the field of tournament organization, and has always impressed me with his dedication to excellence and objectivity. Also, his kids are adorable.
I’m very grateful to all of them for agreeing to help me with this endeavor by answering a series of questions. My goals were to a) provide you, the audience, with a deeper understand of premier event construction at the Grand Prix level, and b) to examine the current structures for points of vulnerability and improvement. To that end, I’ve tried to minimally edit them and only cull redundant responses, adding “My Take” to their responses.
¬¬
Some of the questions may appear to be “leading” the interviewees, but I’m neither a lawyer nor a professional journalist so I can do what I want!
Some aspects of running a Grand Prix are mandated by Wizards. Can you describe those in more detail?
Steve Port: TOs don’t typically pick cities, or even the dates for their events. Sometimes when juggling a schedule we’re able to suggest ideas for other cities and dates (usually based on availability) when the originally proposed city/date has no available space at appropriately-sized venues.
At the GP there are certain events each TO is expected to run as sides to help bring a normalcy to the experience across the globe. “Foiled Again” on Friday night, where older promo foils are given away, is one such program. Sunday Super Series is another.
My Take: It seems like there’s a fair bit of room to build a Grand Prix weekend — TOs just don’t get much say in the when and the where. This is a good thing, because it opens the market to experimenting with unique offerings when it comes to side events.
What do you hope for in a Grand Prix venue? How restrictive is the selection process?
Riki Hayashi: The problem is that the type of venue that we need for events is no longer readily available. By this I mean that nice, carpeted “ballroom” spaces generally only seat 1,000-1,500 players at most. Beyond that, we get the concrete “showrooms.” GP Richmond actually had both types. The Green and White splits were together in a showroom and the Pink split was in a separate ballroom. I was in the Pink split, and we had a great time. The showroom… not so much.
GP Chicago was a showroom, and it suffered all of the usual problems associated with such a space:
- Acoustics: These spaces aren’t designed to accommodate a bunch of people sharing the same experience. As the name implied, it is for big floor shows where the audience is supposed to be divided into the various exhibits and booths. The room isn’t designed to bounce acoustics effectively to make announcements to all 2,000 participants.
- Bathrooms: Similarly, this space is for people to walk through, look at a few things, maybe buy something, then leave. Trapping 2,000 Magic players in there for ten hours isn’t what the space is made for, and hence the number of bathrooms available is generally lower than necessary for the space.
- Concrete Floors: A burden for judges who have to walk on it all day, especially when events are so big that crossing the room can take two minutes.
The larger your expected turnout, the suckier the space will be because you will have fewer choices. At the current attendance levels, it wouldn’t surprise me if some cities are completely shut out of being able to host a GP. Others will be forced to be held in terrible showrooms.
Personally, I am more and more hesitant to attend giant GPs because I simply do not enjoy the showroom experience. The noise level in a room with 2,000 people and bad concrete acoustics bouncing around is oppressive. I talked with several people at Chicago who felt the same and had to take strategic breaks to get out of the din. I had a similar experience in Richmond on Day Two when I was in the showroom.
Steve Port: The process is generally most restricted by the size of the event. For instance, at GP Austin a couple years back I ran the event in a room that was 25,000 sq feet, which would have been significantly bigger than any GP had ever been in the area (something like 50% more than average). We made the room tight “just in case,” expecting to drop tables when we got the ~800 expected. We had 1,184 chairs at tables. We started 1,185 players. That was pretty nuts.
Now I can’t look at rooms smaller than 40,000 square feet in most cities and 50,000+ is better (and not close to big enough for the larger cities). That means that my number of potential venues is severely restricted. Most mid-sized to large cities have one, maybe two venues with rooms that big. And then you have to hope they aren’t booked on the day you need them.
Some of the much larger cities have multiple potential venues when you look at the surrounding suburbs, but they all have their challenges. This one is closer to the airport, but no food options. That one is close to food options but is expensive for players to access hotels and parking. The variables are many and all we can do is try to match up availability with size and the best set of criteria around… when we have choices, which isn’t sometimes even possible.
My Take: The venue is one of the most prolific sources of complaints among Grand Prix attendees, but I’d be willing to bet that the vast majority of you had no idea that the popularity of Magic has essentially made comfortable venues a mix of impossible or untenable, and that’s before we even consider the additional costs of thing like unionization and arbitrary “security” fees placed by convention centers that have had bad Yu-Gi-oh! experiences. It’s truly a problem that has no solution right now, because what we’re doing is creating a completely unique demand on a market neither prepared for us nor motivated to support us.
Riki’s right to be concerned about what the future holds. I’ve been very vocal on Twitter and during my podcasting days about focusing tournament organization on the future, and it doesn’t hold to just things like tiebreakers, playoff systems, programming demands, and staffing. There are actual physical limitation on events that need to be addressed, because tournament attendance isn’t dropping anytime soon.
Taking a cold, hard look at what can be done to weaken Grand Prix main event attendance may be the next logical step, as strange as it seems. Increasing entry isn’t doing it. Bolstering side event prizes and significance at the cost of the main event will siphon away players and make splitting the event across multiple halls, much like a typical convention, more reasonable. Splitting the Grand Prix itself into two events — an Open and a relatively lax invitation-based event (such as a high number of Planeswalker Points or very small number of Pro Points) with better Pro Point award thresholds could accomplish a similar result, increasing the EV for pros and more casual players alike (given that they want to win different things). This would essentially turn GPs into a simultaneous Open Series and Masters Series.
These are just ideas, but it seems safe to say that in a handful of years the current system will need a significant change.
Why have Grand Prix entry fee increases become necessary? Do you have a prediction for a ceiling on Grand Prix entry fees?
Kim Warren: Again, a TO issue, though my impressions include the fact that we are coming out of a recession in the US and Europe, meaning that the price the TOs have to pay for stuff is increasing. As events get bigger TOs are also having to rent bigger venues, and I don’t believe that venue hire prices scale linearly with size — especially when you might not even fill them.
Riki Hayashi: Part of it is the increase in fixed costs. I’ve already talked about venues. For those spaces, you can’t just tell a convention center that you are expecting 1,500, but can they please have space ready for 3,000. If 3,000 is your upper estimate, you need to book the space in advance and pay for it.
The same goes for judges. Yes, you could staff for 1,500 and have “standby” judges in case of higher attendance, but it is much harder to get the same quality out of last-minute additions since they are usually drawn from a more local pool and you’ve likely already staffed the best L3s and L2s from that pool. There’s also plenty more fixed costs these days because of the ubiquitous nature of playmat giveaways. Raising the entry fee helps guarantee that a single bad turnout won’t destroy a TO because there’s enough price elasticity so that $10 isn’t going to affect things.
What a lot of people don’t get is that running these events is a career for these people. The idea of a short term cash grab is incredibly short-sighted, and these people wouldn’t be in the position they are if that were their mentality. Just looking around the room at GP Chicago, Pastimes has upped its infrastructure in terms of its signage. Some of it is advertising, but it also makes these showroom spaces feel friendlier, and other signs are useful like “Side Events Registration.” At least part of the extra $10 they are charging goes towards building infrastructure like that for the future. An individual player at an individual event might not see the returns on the extra $10 they spend, but you have to look at the long game just like they do.
I don’t know enough to say if there is a ceiling or not.
Steve Port: Costs aren’t flat. GP prices were static for a very long time. Somewhere back in 2010, maybe 2011, they jumped to $40. So it’s been three years since the current price took effect. And in that time we’ve continued to see increased attendances which increase demands on staffing costs, and as mentioned above, pushes us into larger rooms which tend to cost more money.
My Take: Economics! Scary!
Most people who complain about prices simply aren’t acknowledging the existence of basic economic concepts. The fact is that Grand Prix cost more and more money every year, and as all three pointed out, larger attendance isn’t a freeroll for profit. It creates additional costs.
There’s also competition among TOs. Everyone wants to have the Grand Prix that was the talk of the town, and all the perks that have become so popular — playmats, tokens, supplies, the Gold Rush — are costly. As TOs work to keep up with each other, these costs continue to cascade as well — all in the interest of encouraging you, the customer, to spend your money. The bells and whistles are becoming less necessary, as Grand Prix brands are so strong that profitable attendance is a given, but it’s the nature of a competitive market to keep up rather than slow down.
Speaking of…
What do you assess the purpose of promotional items such as playmats, promos, and tokens to be?
Kim Warren: All of these items are to manipulate and reward player behavior. Promos encourage people to come to the GP, and are a reward for having made the effort to do so. Depending on their distribution method, playmats get players either to preregister earlier to guarantee one, allowing the TO to better estimate attendance, or gets them into the venue earlier on Friday where they might spend money. Other giveaways are less standard, but basically fill the same purposes of encouraging people to do something/rewarding them if they do.
Riki Hayashi: To drive up attendance. I see it as the bobbleheads of Magic tournaments. Many people show up just for the playmats. For some it becomes a quick way to subsidize the trip. For others, it is a memory of an experience.
A lot has been made about the way that playmats are given out. I like the model where it is given out to side event winners. Originally, playmats were designed to drive attendance to the main event. That’s no longer a problem for most GPs, and switching the model to help drive attendance into targeted side events is an evolution of why these things were given out in the first place.
Steve Port: We use promo items like that to help drive Friday attendance. Most of the time it’s difficult to rent space and open into adjacent halls for just Saturday. In addition to the logistics of laying out a room or completely changing it overnight Friday into Saturday, you just always take the risk that someone else wants to rent that room and you lose any ability to stage the space you’re moving into. So we have to rent 50,000 or more feet even though significantly less than peak numbers will be there on either Friday or Sunday. So, the playmats were originally engineered as a way to get people to show up and register on Friday, and hopefully play in an event or two to help pay for the room.
My Take: As I noted above, we do need to acknowledge structural shifts that will bolster side event attendance, and I agree with Riki’s perspective there. Similarly, Steve’s focused on ensuring that Friday contributes to the bottom line, as it’s a cost TOs are essentially priced into paying whether players show up or not.
In the interest of bettering the events, I’m going to suggest a pretty unpopular idea — maybe we need to give players less of this stuff for “free.” If promos and supplies and playmats all required players to do more than just show up and play the main event on Saturday, it would drive players more aggressively towards the events and days that need the help. I know that no one wants to hear they should get less free stuff, but the market is becoming really saturated here and frankly, with attendance booming, our current incentive structure isn’t reaping a significant reward.
I imagine it would be scary to be the first TO on this train, but I’ll again emphasize that Grand Prix events have an attraction all their own these days — now’s the time to reshape the nature of the industry, as it’s nigh-impossible to take a loss.
Which periods over the course of a Grand Prix present the most significant timing pressure?
Kim Warren: The beginning of the GP can introduce massive delays if anything goes wrong and is basically the hardest bit of the tournament — after you start Round One, you’re basically in the clear. After that, there is the end of round procedure, where judges and scorekeepers need to be very on-the-ball and efficient to turn things over quickly. Finally, in Limited Day Twos, there is the period in between the two drafts where everything needs to be set up and started off.
The common theme here: the most significant timing pressure presents whenever there are tournament operations to be done, but most the players are not playing Magic and are just waiting on you. People feel delays very keenly when they have nothing to do.
Riki Hayashi: The worst time right now is between the posting of seating for the Player Meeting and the start of Round One. This is when you typically see a line of 30-50 players in front of the scorekeeper with issues like:
- I have the wrong number of byes.
- I’m not in the tournament.
- My name is spelled wrong.
- There are two (or more) people with the same name.
The scorekeeper has to solve all of these problems before Round One can be paired. Right now, this is why it takes an hour to really start the event. Each of those problems has to be addressed. Sometimes it is simple, but others might take multiple steps and ultimately, even if you have people helping to triage the issues, the scorekeeper has to enter the player and/or byes into the main event software.
No Saturday registration was supposed to solve some our problems with getting the tournament started on time, but we’ve traded one problem for another. Technically, we can start the event on time, posting seatings for the Player Meeting shortly after 10 AM. However, since this is our first opportunity to confirm things like spelling of names and DCI #s with something like 60-70% of these people, the ones with problems swarm the scorekeeper and our turnaround time from Player Meeting to Round One suffers greatly.
My Take: Kim brings up a very good point — that the delays are filled by nothing, which causes players to feel them much more acutely. People complain about things that bother them, not things that are inherently wrong — that’s why Tina in first period doesn’t know anything about the crisis in Syria but is deeply disturbed by Stefan’s latest betrayal on The Vampire Diaries.
Is there some kind of distraction or time sink, outside of the player meeting, that could alleviate this particular problem? Just brainstorming here, but it seems like an ideal spot to offer some kind of consumable content. Whether it’s something like the old magazines with articles, an interesting speech on the format/rules interactions, or even a projected episode of Walking the Planes for the whole hall to see, any kind of fill might make the wait seem less than interminable and increase morale.
Is there a preferred ratio of judges-to-players? Is it elastic to account for increased attendance? If so, how elastic is it?
Kim Warren: At a GP, an ideal ratio of judges to players is about 1:40, to allow for things like breaks and off-the-floor tasks. This is a target, so unexpected over- or under-attendance necessarily causes this to fluctuate, even with trying to recruit people at the last minute.
Riki Hayashi: Most TOs staff at about 25-40 players per judge. I think that the effects at extreme high attendance are not well-known right now. At a certain point, there are serious problems with information dissemination. With 100 judges, just telling all of them something can be a chore, and our traditional structure models start to break down.
Steve Port: Your judge crew is going to vary a lot on who is in it. The more senior it is, the easier it is to get by with a few less. Experience is simply irreplaceable at the GP level. Obviously some amount of “warm bodies” are needed the larger an event gets. But senior judges are key.
A formula is used that was generated by the high level judges that helps as a guideline for how many judges we should use of which levels as a recommendation at specific ranges of expected attendance. The roughest part of that is the way the events have been skyrocketing. In some cases attendances have been 30-50% above expected attendances (and more in a few outliers).
The way we typically combat this without overbooking our staff is to put judges on “standby.” A certain number are accepted to the event based on our estimated needs. Then others are asked if they’d like to be on standby. This usually means they intend to be there anyway and will play if not needed to judge. As soon as we know we need them we start activating them until we get to the number of judges we feel we need, we run out of judges with the specific skills we need that we can tap, or we simply run out of volunteers. In the past I’ve had judges on my stage staff that I’ve cut to work as judges on the event when needed and drafted other random folks I knew in the crowd to take their place on stage for registration and other low level operations stuff.
My Take: The common theme of “our system struggles as attendance increases” surfaces once again. This is why I’m a firm believer in patching systems to scale beyond your needs rather than to current functionality. Otherwise, you wind up in the same boat alarmingly fast.
The judge program is staffed by volunteers, but over the past five years it has improved significantly. During my days as a young Magic player, I had a number of unfortunate run-ins with under-educated judges and understaffed events, but by and large these very basic issues have become much less prevalent, and safeguards have materialized to ensure we get more accurate rulings and smoother operations from judges. I’d like to see the program bolstered with more benefits and incentives, because experience and talent can be real game changers.
When it comes to communication infrastructures, we need some game changers. I’m all ears.
What kind of experience ratio do GPs typically see with regard to mid- and higher-level judges (2-5) in comparison to less experienced judges?
Kim Warren: This is very dependent on where and when the GP is. Typically in well-developed regions, Level One judges are not taken on GPs at all anymore. Places which are isolated may have to take Level One judges onto the GP staff in order to make up numbers, but ideally they only work side events rather than the main event. Situations are not always ideal, unfortunately, and some events just don’t have enough Level 2+ judges applying.
Riki Hayashi: I’ve never looked at these figures before… They won’t be perfect, because some judges who were added at the last minute might not be reflected, but GP Chicago had 2 L5s, 1 L4, 16 L3s, 63 L2s, and 4 L1s (the L1s are generally people on the verge of L2). That seems like a fine ratio, at just over 4 L1/2s per L3. If that ratio got to 6:1 or higher, I would get a little worried.
Steve Port: It’s generally a pyramid in shape. Fewer high level judges to more low levels. Level 0s are generally not allowed on the judge staff. Some Level Ones are accepted and generally used as side event staff, occasionally making main event floor if being mentored into a role there and expected to test soon. Level Twos are the most numerous on a GP floor, followed by Threes and then generally Fours or Fives are in the head judge role. Occasionally you get lucky and are able to have an L4 (or 5!) on your judge staff not as head judge. But it all revolves around availability and schedule.
My Take: If I were a better interviewer, I’d have combined these last two questions effectively.
Again, these responses indicate that there may not be a significant-enough incentive to judge these events, as scheduling and availability trump necessity. However, it also contradicts the player complaint of “judges who don’t know anything” because, clearly, that’s incredibly rare at this level.
How do more experienced judges and staff track the performance and efforts of lower-level judges during main and side events? Are there steps taken to ensure their events progress in a timely fashion?
Kim Warren: Uncertified judges do not work as judge staff on GPs, though someone who is testing may occasionally put on a shirt and shadow a more experienced judge on a side event.
As I mentioned above, there are very few L1s on GPs at all. If they are on the main event it is due to a serious lack in more experienced applicants, which means that resources are so stretched that it is hard to keep as much as an eye on them as we might like, but hopefully they have a more experienced buddy. Though this is pretty extreme, and I am not sure if it really happens outside of Australia and occasional European team GPs where apparently all the experienced judges want to play…
If they are on side events then they are likely working under a more experienced judge who should be keeping an eye on their performance and providing feedback throughout the day and subsequently in the form of a review.
Riki Hayashi: Judges have a system for giving feedback via http://judge.wizards.com called “Reviews.” It’s one of the things that I am most known for in the program. Unfortunately, the size and complexity of GPs cuts into our ability to observe and give feedback to other judges. It’s to the point where, as you can see from the above figures, very few L1s get staffed for GPs because it is no longer an appropriate venue to train them. That needs to happen locally at PTQs, States, and SCG Opens.
My Take: When the demands of running the event are extreme to the point that review and feedback is suffering, that’s a very dangerous place to be. It’s very difficult to improve at any vocation without specific and accurate feedback, and that’s true of judging as well.
There’s also the possibility that reviews could benefit from an incentive structure in order to increase their quantity and quality. You’d need a weight to ensure that reviewers were rewarded for their feedback’s merit — who reviews the reviewers? — but it seems like a worthy cause to me.
Many tournament complaints are scapegoated onto DCI-R or WER; how do TOs and judges provide Wizards with feedback on these programs and other supplements to tournament management?
Kim Warren: There are annual meetings between senior judges and Wizards and between tournament organizers and Wizards, where concerns such as this can be raised and discussed. Scorekeepers tend to give more direct feedback to members of Wizards. There are also forums and customer service teams which are used for reporting workarounds and bugs.
Steve Port: There are some issues with the programs. Wizards is working on a new version of the software to be used for running Grand Prix events. I don’t know what the timeline is for that, but it’s been in development for a while.
In the meantime, DCI-R is an old program which has been patched and massaged to handle the events of the scope we’re seeing now. It was never expected to have to run events as big as we’re seeing now. WER gets better all the time. It still does some funny things, but we have avenues through our team to report bugs and help make the programs better. And there has been definite improvement to WER, especially in the last year.
My Take: It’s easy to grow concerned with sentences that mix “Wizards” and “software” thanks to Magic Online. I don’t have many words of comfort here — I’m wary as well. Getting trapped in development is a good way to wind up with a solution to a problem you no longer have in the software world, which in this case would be a program reality outgrows shortly after it’s available. It’s a concern, and I hope it’s being evaluated seriously.
Much like the issues with Magic Online, the only thing we can really do is hope for higher-quality work. Wizards isn’t a software company, but in this industry it’s becoming more and more apparent that software design is of massive importance.
What kind of hours do TOs typically expect/ask of their judges and staff? How significantly do those time demands change with unexpected increases in player attendance?
Kim Warren: In more recent years, we have started implementing shift systems for judges with a goal of no one except the head judge working more than eleven hours a day. Previously, every judge on staff would turn up at least an hour before the start of the day to do setup and would stay until everything was done and tidied up, which as you can imagine could get ridiculous on a Limited GP before the nine-round Day One cap was introduced.
Shifts are normally synced to round milestones rather than times, so large events that run slower mean judges work longer shifts. Unexpectedly enormous events throw shifts out of the window and we all work as long as we need to in order to keep the event running. Note that large events cause large side events, too, which can also destroy shifts and lead to very long hours.
Steve Port: We try to keep judge hours to ten or less a day as an ideal. I try to do the same with my ops staff, though sometimes you just run longer. Personally I am in the hall from doors open until well into the evening, usually twelve to fourteen hours. The head judge and occasional other high level judges are on the floor all day of the main event, both Saturday and Sunday.
I try to keep my staff deep enough that we can do breaks and keep them from working hours too long. Ten hours at a stage answering “Where do I put my match slip?” every three minutes is a lot more exhausting than you think! Judges need to make sure to get time off their feet, eat food, and stay hydrated. Those judges walk a LOT of steps through a typical Grand Prix day!
My Take: It’s good to see reform in this area, as events like Grand Prix Charlotte and some of the other bloated events prior to the cap had me cringing. That said, when you combine the aforementioned struggles properly staffing an event with an interest in creating a rational workday — which is still longer than a traditional American workday by an hour or more — there’s clearly going to be some friction.
You alleviate that the same way any company would alleviate a worker shortage — you add staff. In the case of judges that’s really on Wizards and TOs to do, and it appears to be a growing necessity. SCG’s Judge Rewards program wasn’t built for fun — it fills a necessary staffing purpose as the Open Series has continued to scale. More programs like this one should be explored.
What steps are taken to ensure the staff is educated about specific policies and procedures unique to the event in question?
Kim Warren: Generally all judges should be L2+ and so very familiar with general event procedures. It is unusual that there are specific policies and procedures that impact the judge staff. When this happens, the head judge and/or tournament organizer generally use the event-specific forum/mailing list to which all judge staff are added to communicate them before the event, as well as ensuring that all team leaders are informed in a team leader meeting on Friday, and that all judges are informed in the general staff meeting on either Friday evening or Saturday morning. This gives some redundancy in spreading the message, which means that it reaches most people, though still not necessarily all.
Many head judges will also hold team leader meetings throughout the day to allow them to spread updated information to the staff as needed. I don’t know how the TO disseminates information to their staff.
Riki Hayashi: If you’re talking about TOs to judges, it is almost zero. This is probably one of the biggest areas of improvement that needs to happen. Judges should educate themselves on the particulars of the event, and TOs should communicate some of their special needs. It’s a two-way street, but I think the TO needs to take a first step here and have some kind of guide to their event. Bullet points like “this is how players can get a playmat” would be very useful for judges to know on the floor to answer questions.
Steve Port: I have stage managers who make sure to inform any new info needed to their staff before each event registering or each shift of responsibility for that crew member. It’s important to me that each person on my stage can either answer your question or point to the person who can. I dislike getting the runaround, so I try to avoid it — you should not have to ask a question more than once, definitely not more than twice. I’m not saying we nail it perfect, but my staff is enabled to answer as many questions as they possibly can and I try to have a central contact on stage available at all times (sometimes two) who can answer the question in the oddball cases where a regular staffer doesn’t know the answer.
My Take: One Grand Prix Chicago complaint was that it was difficult to find accurate information — that the floor judges and staff couldn’t always get you where you needed to go without help from someone else. That speaks to Riki’s concern, which I’ve also seen cause problems at just the SCG Open level. We need stronger informational structures at events.
Some of this could be alleviated with a combination of improved preparation and improved training. As someone who has been a teacher for students and a trainer for my own replacements, I approach any instructive moment as the teacher — you provide a personal point of contact to demonstrate methodology/answer questions and a comprehensive reference tool for the staff member/judge in question to use in your absence. The goal, as Steve pointed out, is to make everyone as independently functional as possible, and reference tools go a long way.
I’ll let Kim, Riki, and Steve take these last three solo before I close it out, as they explore a more personal perspective.
On which merits should one measure the performance of a tournament organizer?
Kim Warren: Was the venue fit for its purpose? Were the staff friendly and helpful? Was the side event schedule suited to your needs? Were the chairs comfy? Was there sufficient communication of information about the event ahead of time? How were registration procedures? Were you able to find out how to give them feedback after the event?
Riki Hayashi: This is a really good question. Of course, we’ve seen the recent backlash from Chicago, and I think there are a couple of points that Pastimes is unfortunately getting railroaded on (Batterskull) that were largely not the fault of the TO. Any individual event is a poor measure of performance.
I look at infrastructure and personnel. Pastimes has recently hired L3 Judge and Regional Coordinator Steven Briggs. Legion Events has hired L3 Judge Rob McKenzie. Those are good hires and bode well for their futures. That doesn’t answer your question, but let me put it to you another way. Think of these hires like a sports team signing a free agent or new coach. You want to see how this affects the team over the course of a full season or more. Then, on what merits do you measure your team’s performance? Wins? Championships? Attendance?
I don’t think pure attendance works at all to measure the performance of a TO. A Grand Prix is such a powerful brand that there’s very little that will stop people from showing up, and I don’t think that there are enough events on the schedule for a dislike of a specific organizer to have any visible effect. People are so starved for Magic that they will go the local GP no matter what. That and the number of new players coming through the turnstile is pretty high too.
Steve Port: This is really hard. Our job is to make a tournament as stress-free as possible. Players should hopefully just show up, pay the entry fee and have fun after that. I can’t remember a single event of GP size I’ve run where I didn’t have someone unsatisfied about… something. It might have been a judge interaction, or a misunderstanding about prize payout, or a botched pairing situation, it could be anything. So I think 100% satisfaction is a fairly hard goal to make in a competitive environment where things outside of everyone’s control could make an event terrible for someone.
If you are the one guy that didn’t have fun, then the event wasn’t successful for you. How many “dissatisfied” customers are counted before it’s considered “failing” as a TO? If I run five events in a row with one or two complaints each then have one where the “wheels came off” and everything went wrong, did I fail? What if that happens with two events in a short time due to a number of things beyond my control? I don’t think it can be narrowly and succinctly quantified. Every situation needs to be analyzed based on all the factors.
On which merits should one measure the performance of the judge staff?
Kim Warren: Did the event run on time? Did you have to wait a long time for a response to judge calls? Were rulings dealt with fairly and professionally? Were they generally friendly and polite? If you appealed or made a complaint to the head judge, did you feel like you were listened to and handled appropriately, even if they did not rule in your favor in the end?
Riki Hayashi: An entire staff? Geez. Tough to say. I could write an entire article on the fallacy of “round turnover times.” I know that from a player perspective, that is one of the most important things that they rate their overall experience on and I see judges focus on this a lot as well, but past a certain point it is often just a matter of luck.
As a head judge, I will track appeals, and a high number of those being overturned could be an indication of a large number of bad rulings being given out (and generally not being appealed).
Steve Port: Fairly similar to the TO comments above, I think. It’s hard to quantify, but allowing fun while being fair is the most important thing for me in selecting judge staff. If I know someone has consistently caused trouble at event (complaints, or just bad interactions with other staff, etc.) I’m more likely to tell the judge handling judge selection that I don’t want that judge. It hasn’t happened often, but it has happened. In every case I can think of where it has happened I have eventually allowed them to come back after proving that they’ve grown as people/judges and solved any issues I saw/perceived.
What concrete measurements make a GP a success? Which make it a failure?
Kim Warren: Honestly, this is pretty simple: If the majority of players enjoyed their experience, it was a success. If not, it was a failure. This is very hard to assess through the medium of social media, unfortunately.
Riki Hayashi: Side event attendance. Meaning, how many players dropped out of the main and continued to play Magic on Saturday? What percentage came back on Sunday for more? That speaks to people generally having a good time. If their experience is miserable, they’re more inclined to leave or stay at home on Sunday.
Not sure that there’s anything else concrete. It could be that as discussions like this continue, we see more of a trend in attendance for specific TO’s GPs, but even that could be misleading. Does a TO get higher attendance because they run better events or because of favorable geography and timing?
Steve Port: I’m hesitant to call any event a failure, as long as some people showed up and had fun. There will always be potential to do things better for any TO, any team, and some events are less problematic than others.
For me, success is counted in the smiles on faces I see at the event — in having just one player stop me and say, “Thanks for doing this, I had a great time.” It happens a few times each event, but I don’t think I can adequately explain the joy that statement can bring, especially when it’s from a player I don’t know. It’s amazingly powerful to me.
At GP Dallas/Fort Worth this past year, we had the worst ice storm seen in something like 40 years hit the weekend of the GP. We projected ~1,200-1,300 players and had 840 or so listed for the player meeting. It ended up being something under 800 after all the player meeting no-shows were pulled.
For the next sixty days I continued to get refund requests from people who hadn’t been able to make it but had prepaid. It would be super-easy to focus on the refunds I was giving and how I didn’t get the 400+ people I thought I was going to get and had anticipated with room size, chairs/tables, room block reservations, etc. Every corner turned a new bill I had to pay, and I had significantly less money to pay than I had expected considering the number of people that actually played. But at that event, more than any other event I have ever run, I got “Thank you for doing this” over and over. People were so happy that I stayed and put it on and didn’t cancel. Half my staff was stranded, piles of judges as well. Of the three GP-level scorekeepers I had on staff, none actually made the event — but the judges and the staff that did make it there worked longer hours and did it gladly.
From a financial standpoint, that event was probably a failure. From a standpoint of how many things went wrong due to being understaffed for a lot of the event and having to do a lot of things myself that I normally count on others to do, like setting up the network and scrolling TV systems, it might have been a failure. But man, there isn’t anything much sweeter than having a Magic fan come up, shake your hand and say, “Thank you. Thank you for doing this, I had a blast.”
Success.
My Take: Steve’s right — context is everything. In fact, that’s the motivation behind this entire article. Most players lack the context to make an educated, informed critique of why they had a negative experience, but they’re still the people who wind up disappointed the most at the end of the day.
I think the success of an event is an individual judgment. I’ve been to events that ran perfectly for me, while another player at the same one had the worst of his life. Events are experiences, and the ultimate goal of our judges and TOs is to provide the best one to as many people as possible and to come back and do it again next time.
If an event fails you, as an attendee, make it known. Send the TO an email, or Facebook message, or Pastebin via Twitter, about why you were disappointed. Making a Reddit thread is all well and good, but if you don’t also contact the responsible party about their performance then you’re just starting an internet party when you could be an active participant in change for the better.
I hope this article has helped make all of us more informed when we participate in this discourse. There are many problems to solve, and many more on the horizon, but Magic’s not going anywhere… so let’s keep talking.