(author’s note: this is a version of a document I originally wrote in 2011 explaining, among other things, why Evo uses a double elimination format instead of single elimination. It was never really finished, but since the topic of seeding is coming around again I thought I would post it up. At worst, it’s a (sometimes poorly written) brain dump of the decisions we’ve made in our 15 year history, the mistakes we’ve made, and how we got to where we are today. Hopefully someone can find this useful in structuring their own tournament ruleset. Feel free to leave your questions/comments below and I’ll try my best to respond.)
The tournament format currently used at Evo has been refined 13-years of experimentation, sticking with what worked, and abandoning what didn’t. This article the Evolution tournament is structured, why it looks this way, and what you should expect if you’re playing in the tournament. It’s a long read, but hopefully you’ll find it interesting to know why the tournament is structured the way it is and how it reached its current form.
Tournament Format Theory
There are an endless number of ways to structure a tournament. Popular examples include single elimination, double elimination, round robin, and swiss. Each format has its own unique properties, but they generally hold this same, basic property: formats which allow the player to play more games can more accurately determine the best players.
Choosing a tournament structure is often a difficult balance between finishing your tournament on-time and getting a reliably good outcome, where I’ll define “good” to mean that the best players finish very highly in the bracket. Over time, we’ve also included how friendly the tournament format is to players who have to navigate it into that definition of “good”, but put that aside for the moment. Let’s just look at tournament formats on the merit of how accurately they predict the strongest players given the amount of time they take.
Consider the single elimination tournament format. It only takes 999 matches to determine the winner of a 1000 person tournament. That’s super fast, but you pay for that speed with a loss of accuracy in the “best” (i.e. most skillful) players placing highly in the results. For example, if by chance the best player in the tournament is matched against the second best in her very first match, one of those two players will lose and tie for last with 499 other players. That’s not a very good result! You can mitigate that effect with proper seeding, but even the best seeding systems come with some element of bias. Ideally, we would like to minimize how tournament organizer bias can influence the results, and relying on seeding to make sure the 2nd best player that day doesn’t place last doesn’t sound very good.
Also remember that even the best players will not always win every match they’re favored in. No one, no matter how good, wins all of the games they’re supposed to win. Even if the best player in the field wins 9 out of 10 games against a random opponent, he’ll only win a single elimination 1024 man tournament a little more than 1/3rd of the time. That’s not a good result either! Real world, Japanese fighting game tournaments routinely use the single elimination format, and some of Japan’s top players (including Daigo “The Beast” Umehara) have publicly said that’s one of the biggest reasons they prefer foreign tournaments to Japanese ones.
Generally speaking, you can get more reliability in your tournament results by adding more games to smooth out the variability caused by player inconsistency . For example, if we make two players play 2-out-of-3 games instead of a single game in a Single Elimination tournament to determine the winner, the theoretical chance of our best player winning a 1024 man tournament jumps from just over 33% all the way up to 75%. Another way to improve the reliability is to switch to a format which has more games built-in, like Round Robin.
In Round Robin, everyone plays everyone else. The winner is determined by who wins the most games, with some (usually super complicated) scheme for breaking ties. In a perfect Round Robin 1000 man tournament, the best player will have a perfect record (crushing 999 of his foes), the second best player will go 998-1 (losing only to the best player) and will place second. Sounds great, but round robin tournaments are extremely time consuming. A 1000 man round robin tournament would require 499000 matches (almost 1/2 a million). Very small round-robin tournaments can finish very quickly, but the bigger they get, the longer they take (increasing with the square of the tournament size, which means they get longer much faster as they get larger). Because of this, a lot of people like to use the a hybrid format which starts with small Round Robin pools to seed a tournament and then moves to a single or double elimination format, the grand-daddy of all of these is the World Cup, which leads to the greatest weakness of Round Robin: the potential for cheating.
Regardless of how noble or honorable your player base is, some percentage of them will attempt to game the system for the benefit of themselves and their their peers. Many, if not most, of these will be top players. Round Robin tournaments often run into situations where players can deliberately lose to influence the rank of other players in their pool, which can give them or their peers a better position in future rounds of the tournament. One of the most notorious examples of this occurred in the 1982 World Cup where West Germany and Austria (allegedly. ha!) colluded to fix the score of their match to ensure both teams advanced. Here’s a summary of the issue on Wikipedia.
To quote the wikipedia article:
“This performance was widely deplored by all observers. German ARD commentator Eberhard Stanjek at one point refused to comment on the game any longer. Austrian commentator Robert Seeger bemoaned the spectacle and actually requested that the viewers should switch off their television sets.”
The long tournament time and potential opportunity for collusion are deal breakers for Evo. We have tried it in the past, we found the tournament to be rife with collusion (unsurprisingly!)
Evolution’s Tournament Format
Enough about theory. What does Evolution use? We’ve used the Double Elimination format for almost every tournament we’ve ever done. We find that it strikes the best balance between having stable results, running quickly enough to finish large tournaments in a single weekend, and has minimal odds of being gamed by the player. In the Double Elimination format, each player must lose to an opponent twice before they are out of the tournament. In order to increase the reliability of the results, we also make sure that each match between any two players is determined by the best of 2-out-of-3 or 3-out-of-5 games, depending on the standard tournament rules adopted by the community for that game.
The Evolution Pool System (pre 2011)
(author’s note: unless you’re specifically interested in the tournament format evolution or just nostalgic, feel free to skip this section)
One of the challenges with a large double elimination bracket is that the order when people play their games may be spaced out. For example, in a 1000 man double elimination bracket, there could easily be over 500 matches between the time a player has to play their first match and their second. That’s a lot of time just standing around waiting! No one wants to stand around waiting for hours for their match to be played, so we have traditionally divided the tournament field into pools.
A Pool is a simply a section of the double elimination bracket where players in the pool play exclusively against each other. For example, suppose we divide the 1000 player tournament into 16 pools. Each pool
would have roughly 64 players in it, and the time a player would have to wait between any two matches would drop from 500 matches to around 32-40 at the latest. That’s a lot less standing around, and has worked very well for us in the past.
As mentioned, a Pool is simply a section of a larger double elimination bracket. The Pool will graduate 2 players (one from the loser’s side and one from the winners side) into a Semi-Finals Pool which is just final 16 winners and final 16 losers from the original bracket. The Semi-Finals Pool is traditionally played down to the Final 8, which takes place at the Tournament Finals Super Sunday for all games. Losses do not reset, and if you ever lose more than 1 match in either your pool, the Semi-Final or the Finals bracket, you are out of the tournament. The Pool system doesn’t change the structure of the Double Elimination tournament, it just changes to order that the matches are played.
The Evolution Pool System (post 2011)
The 2 level Pool System has served us very well in the past, but it’s time to evolve the system yet again. With the growth of the tournaments at Evolution, even the Pool brackets are becoming large and unwieldy. For Evolution 2011, we will be moving to a 3 level Pool system to make sure players get to play all their matches in a reasonable timeframe.
At Evolution 2011, all Pools will contain a maximum of 16 players. All your games in the initial pool will be played in a 2 hour session. The top 2 players from each Pool will move on to the Quarter Final round in the
tournament schedule for 6:00 PM. The top 2 players in the Quarter Final round will move onto the Semi Finals scheduled to start immediately after the Quarter Finals finish (espected start time of 8:00 PM).
For players that don’t make it out of their initial pool, this means you can sit down, play your games, and be out of there in under 2 hours. You can spend the rest of your time checking out the EVO panels, visiting our sponsor booths (where we have all sorts of un-announced surprises for you), getting your game on in the BYOC, or just checking out the sites in Vegas. If you’re in the top 15% and actually do make it out of your pool, the rest of the day is free: just be back at 6:00 to play the rest of your matches.
Even with 3 levels of pools, you can (and should!) think of the bracket as just one, big double elimination bracket. If you lose twice at any stage in the tournament, you’re out!
Seeding by Rank
If everyone wins the matches their supposed to, the Double Elimination format can only reliably determine the top 2 players. For example, if you place your top players in the bracket incorrectly, the #1 and #2 players could face each other first, as well as the #3 and #4. If you were really unlucky, the losers (#2 and #4) could face each other first in the Losers’ Bracket, knocking your 4th best player out of the tournament in last place (0-2). This brings to light something very important to remember: the position of players below 2nd in the tournament is influenced both by the player’s kill AND the position in the bracket. Very frequently, the result of who get’s 8th vs 5th place in the tournament is determined more by the player’s position in the bracket than by those players individual ability. For that reason, you must be extremely careful when building your bracket, both to help ensure you get the best results possible and to remove any of your own personal biases from influencing the results. The best way to do this is with an good seeding system.
To accomplish this, we created the Road to Evo where players could play to earn a seed at this year’s Evolution. Players gained points for placing highly in events leading up to Evolution, and the players with the most point will get the highest seeds. This system is not perfect, but at least the players are aware of the rules on how to get seeded well in advance of the tournament. This helps to greatly helps reduce the potential for tournament director bias (real or imagined) influencing the results.
Seeding by Region
Evolution is the largest, open fighting game tournament in the world. Everyone who enters has an equal shot at testing their might against players from all over the world, running the gauntlet and coming out victorious in the end. To supplement that goal, we also try to avoid a situation where players who routinely play against each other are scheduled to face off early in the tournament bracket. No one wants to fly halfway around the world only to lose to the same opponents they’ve been training against when there are so many other good players they haven’t even met yet We deliberately arrange the bracket to make sure that doesn’t happen. We call this system “Seeding by Region”
When we Seed by Region, we try to make sure the distribution of players from the same geographic location is equal across every initial Pool in the tournament. For example, if the tournament has 1000 players and 120 of them are from New York, each pool will have approximately 2 New Yorkers in it (16 players per pool, means 63 pools, or 2 New Yorkers per pool). This rule is enforced in addition to the rank based seeding.
Our mechanism for seeding US players is simple: we ask for your zipcode. US zipcodes are designed to speed the routing of mail. A side-effect of this is that contiguous zip codes are geographically close to each other, while zipcodes which look nothing like each other tend to be spread far apart. So for Evo, we try to make sure each bracket has the same number of same-looking zipcodes, so (for example) people with zip codes which look like 95014 should be evenly spaced throughout the tournament. For international competitors, we simply use their country name. The zip code is a good compromise between getting good geo-spatial separation between players and anonymizing each player’s location.
Advancing in the Bracket, Reseeding, and “Floating”
In the double elimination format, the bracket soley determines who plays whom at what time, all the way down to the final match to determine the winner. This is a sacred property: once the bracket has been set, it is entirely up to the skill of the players to determine the victor. In the past, we’ve found two seemingly compelling arguments for tinkering with a bracket to improve the tournament experience for the players. In practice, we’ve found this not to be true. You never, ever want to tinker with the bracket, even if your motivations are good.
The first motivating case is what we called “Floating”, where you swap two players positions in the bracket to all players’ advantage. Suppose you have 4 people left in the tournament. The bracket as written has has scheduled A vs. B and C vs. D. Now suppose A and B are both from Japan while C and D are both from Canada. Suppose further that they’re all best friends and don’t want to play each other. To please all parties, you decide to swap A and C, so now C will play B and A will play D. Now you’ve got 2 Canada vs. Japan matchups and everyone’s happy, right?
Wrong. Everyone is happy until the matches are played. At this point, you are going to have 2 people who lost, and they both have legitimate claims that in an alternate universe where you did not float them, the may have won. Suppose A beats D and B beats C, resulting in an A vs. B match in the finals. Here are the types of things people will be upset about:
- C could make the claim that he could have easily won his original match against D, and by floating him to play B you have robbed him of a higher placing. Is he right? We’ll never know. D can make a similar claim of his match against A.
- Everyone else at the tournament can complain that by moving the order around, you have robbed them of an exciting Canada vs. Japan finals, and now they must suffer through Japan vs. Japan.
Whether or not these complaints are legitimate is irrelevant. In practice, It’s nearly impossible to please everyone by floating the bracket. Even if you can please everyone in the next 5 minutes, it’s almost a certainty that someone will complain about something in the tournaments future which ultimately may not have happened if you hadn’t floated.
Most importantly, if your rules allow for the possibility of Floating, whether or not you choose to float, someone will accuse you of bias. If you didn’t float in the situation above, someone will accuse you of not floating to guaranteed a top 2 finish for Canada, even though the best two players in the tournament were Japanese. If you did float, someone else will accuse you of trying to eliminate both Japanese players so Canada could take the top 2 spots. Are they right? That’s not important. It’s important to avoid both bias and the possibility of bias in the tournament, so that everyone feels fairly treated at the event. We do not want anyone at Evo to feel that the tournament organizers had an undue influence on the results of the tournament. People with allegations of bias against the tournament organizers feel slighted, regardless of whether or not their allegations are grounded in reality. So to avoid the possibility and the perception of bias, we simply do not float anymore. Ever. For any reason.
The second motivating case for tinkering with the bracket is the “Reseeding” method. This one is actually pretty popular. It’s where at some point in the tournament, you decide to throw away the bracket and make an entirely new one. This new bracket is seeded by some set of rules (usually by region or rank) which changes the destiny determined by the original bracket. Some variations of this rule also wipe out all losses from previous stages of the tournament.
This is really just a special case of the “Floating” rule where you float everyone in the bracket. It has all of the downsides, magnified by the fact that everyone in the tournament is affected! No one will ever agree on what the “correct” seeding is when you do this, so it’s inevitable that someone will feel slighted.
Moreover, reseeding the tournament actually makes it much harder for non-seeded players to win. A correctly seeded bracket has an equal player-power distribution at all levels of the tournament. For example, in a 16 man tournament, the first match of the #1 player will be against the #16 player. #2 will have to play #15. #3 will play #14, etc. The average rank of each match is 8.5 (e.g. (3+14) / 2 = 8.5), for all matches in the tournament. This means that if you added up the “skill” of each match in the bracket, you’d always get the same number. Each matchup has the same “power”.
Similarly, if you look at the semi-finals, the bracket predicts that #1 will play #5 (average rank: 3) and #2 will play #4 (average rank #3). Same power distribution! One important side-effect of this is that the bracket is, roughly speaking, equally difficult to advance in at all levels. If you have the match of your life and beat the #1 player, you will have a relatively easier time in your sub-bracket because you’ve knocked out a significant percentage of the power contribution on your side. Your sub-bracket is now easier than sub-brackets with the #2 player, the #3 player, etc. and it should be! You’ve earned it by pulling off the upset, and you should reap the rewards.
By reseeding the bracket, the tournament director is taking the “easy roads” earned by every upset in the original bracket and re-distributing it to the players he thinks should be seeded highly at the next stage. They are robbing from the poor and giving to the rich. We did this at Evo 2004 (I think) and most people were greatly dissatisfied with the result. I have seen it done at other tournaments (e.g Devastation) with equally poor reception from the players. So we don’t do this anymore, and I wouldn’t recommend it to anyone else.
Dealing with Collusion
The dictionary definition of collusion is a secret or illegal cooperation or conspiracy, especially in order to cheat or deceive others. When I talk about collusion, I’m talking about exactly that. Collusion is super bad for any tournament. I don’t think I need to explain why.
Less damaging than collusion is match fixing. I’ll define match fixing as players using some factor other than a genuine display of skill to determine the outcome of a match. There are lots of reason players will fix a match: some legitimate and some not. For example, a player may need to leave the tournament early and purposefully lose his match to an opponent so he’s eliminated. I don’t think anyone would complain about that. Similarly, players have been known to purposefully lose a match to “clean up” the losers bracket, to ensure a specific matchup in the losers bracket against a rival, or to let a teammate advance. Reasonable minds can disagree as to whether or not these should be considered rule infractions. Finally, player may simply decide to throw a match because they’ve been paid off, or because they are splitting the pot. Almost anyone you talk to will agree that these are pretty egregious offenses. Even people who believe there’s nothing wrong with them will rarely, if ever agree to doing them, even if questioned directly. I’m aware of all of these events happening at tournaments, and guess what: you cannot stop them.
You can’t stop match fixing, but you can stop collusion. The rule at Evo we’ve settled on is, “do not play fake matches”. If you have decided the outcome of a match for a particular reason, you must refuse to play your match and tell us who the winner is. This sounds really bad, but puts the responsibility for match fixing where it belongs: on the player forfeiting the match. It’s up to them to explain to their friends, family, fans, and all the other players watching the tournament why they lost without playing the match. This is greatly preferable to having everyone feel cheated, taken advantage of, and lied to when having to suffer through (in most cases obviously) faked matches where one player is purposefully trying to lose in the game.
There is no punishment for forfeiting a match at Evo. Conversely, the punishments for collusion are severe, including multi-year bans from competing at Evo events. Since we instituted this rule, it has worked out quite well. People sometimes forfeit matches (sometimes late in the tournament), but there have been almost no instances of collusion.