Real life puzzle that puzzlers can't solve.

[PuzzleScot]

Slighly embarrassing, and most definitely real-life.

The UK Puzzle Association (of which I am chairman) needs to choose a method for how to pick the four best players from our shores for the next World Puzzle Championship.

Is there a method that's both simple and fair?

Options mooted so far are

1) A single qualifying tournament. Top 4 are in.
+ Black and White, no arguments
- Best competitors (from past performance) may not be able to make that day, or have a bad day.

2) Multiple stage tournament. Top 4 in resulting table are in.
+ Competitors practice competitively. Best should rise to the top.
- Players may not be able to participate in all rounds. How is a player scored?

The preference would be for the later option, if we could agree on a scoring system that was fair, and could (in the extreme) compare one player who does all odd-numbered contests against another who only does even-numbered contests. There are multiple international contests held on-line. (eg, see http://forum.ukpuzzles.org/viewforum.php?f=5 ) These vary in difficulty, time and scoring systems.

My proposal was taking a player's percentage of the top score achieved by anyone in that contest, and count their top 5 scores. However, this can be distorted in lots of ways,
eg
Popularity of contest. If the top 5 in the world all take part, everyone who participates gets a lower relative score than if they didn't. That's unfair on those that happened to compete in these contests

Exclusion. If there are, say, 10 ranking contests of varying difficulty, and you have to compete in at least 5, late comers are excluded from the contest.

Cherry-picking. If someone enters all 10, they have a better chance of finishing ahead of someone who only did 5

Another proposal was like a Formula 1 points system, where top UK finisher got 25 points, with the next getting 18,15,12,10,8,6,4,2,1
The Exclusion and Cherry-picking distortions are still here. An additional distortion appears when there is low participation levels. (Sometimes we have had only 2 or 3 people enter what seems like a hard contest, or occurs at a very inconvenient time).

Do any of these seem fairer than any other, or is there another more complicated (but fairer) way of coming to our ultimate decision?
_________________
http://www.ukpuzzles.org/

[Zag]

If you were completely making your own, I would design, say, 5 contests, and attempt to make them roughly equal difficulty. Declare that every contestant must compete in at least 3, and may compete in 4 of them (but not all 5). Then base the choice on each contestant's top 3 scores. This gives the advantage that all the members can help create/administrate one of the contests without hurting their own chances. (Obviously, you could tweak these numbers -- 6 contests total, top 3 scores, take 4 max, etc.) I like the idea of each contestant having the option to toss out one bad score, which eliminates the 'bad day' issue. (If a contestant has two bad days, he becomes a significant risk for another at the world competition, anyway.)

However, I assume that it is not an option to build your own multi-day competition, because it is too much work to create and administer, so anything that depends on more than a single competition requires you to base results on existing tournaments, right? I agree with the evaluation that this could produce skewed results, but probably not really. Identify the major tournaments that are roughly equal prestige, and then just use the normalized results of each (normalized based on the overall results of the tournament, not just the UK players in it).

If some tournaments are significantly more prestige (and therefore have harder competition, so normalized results will tend to be lower) then you could re-normalize based on the common UK players that attended that and others. This only works if you have something resembling a statistical sample of common attendees, however.

Also, I don't know if you get raw scores to work with, or only ranking. If all you have is ranking, then you can start ranking every contestant on a ladder based on common tournaments. Then you have a one competition that you create to resolve ties. It's a little simple-minded, because it assumes that if A beats B in one competition, he will beat B in every competition, but you have to base it on something.

[PuzzleScot]

Thanks Zag,

It would be nice to run 5 of our own contests of a similar difficulty and scoring structure. That would indeed be an excellent solution. We simply don't have 5 puzzle designers capable of producing good quality (let alone comparable) contests, so we're forced to use international contests.

There is a monthly contest at Logic Masters India, and frequent contests at the German equivalent. Usually guest authors. Several other countries have a 'national championship' that anyone can play along with. Yes, we can see everyone's raw scores.
(As a side note, we can't use India's ranking system, as we need to separate true puzzles from Sudoku - they're separate teams)

Say we categorised the tournaments into 'high' and 'low' importance. Take best 3/6 high importance contests and best 2/4 low importance.

I think the issue is with evaluating the norm and marking everyone against this. There's always going to be an issue with assessing the difficulty level of a contest, especially if the contestant list (not just UK) in each one is different (albeit very unlikely). Maybe we should keep a 'world ranking list' even if only for our own comparative purposes?

I was thinking of try to compare our situation to another field, and see how they do it - eg, running. However, in that example, times are fairly consistent, whatever the event, regardless of competition. Perhaps that proves that ranking alone is not an indicator of ability...
_________________
http://www.ukpuzzles.org/

[Jack_Ian]

Does the championship involve team challenges?
If so, you might want to choose a good team rather than the best 4 individuals.

How about a final round where qualifying players (those who reached the cut-off point) risk a portion of their points on some outcome. Allow players to confer and even lie to each other.

This will maintain the fun for everyone right until the end.

e.g. What will the total amount of risked points be? The player (or players) who get nearest to the value, but not below it, gets their risked amount back doubled. All others lose their risked points.

[Lepton*]

Echo what Jack_Ian said about teamwork. You simply cannot afford a breakdown in relations, and ice-breaking between introverted strangers can take months.

When I choose our national high school debating team, I get a pool of the top competitors (determined by nomination or strict criteria) to work in frequently-shuffled teams, and then eliminate some using external judges. Toward the end, I accumulate all the feedback and try to be as impartial as I can about the final cuts. Ultimately, I end up as the football coach who crushes 'lil Jimmy's dreams... but that's the burden of team selection.

[PuzzleScot]

Yes, the championship does involve team rounds. (4 of 16 ish)
However, with the geographic disparity of contestants, it's nigh on impossible to assemble the final candidates in a room anywhere...

Strangely, we actually had a fractious situation at a WPC once with 'personality clashes', that didn't do the overall team any favours, so I agree with that sentiment.

One solution to this, that I considered to be an overall best strategy, is that team selection is entirely subjective, where I just choose who I think will make the best team. Almost certainly, that would produce the best combination, but I don't think I could stomach the ensuing anger from those that missed out!

Another solution to that is to have a non-playing captain (with past experience) on-site to make sure the team runs smoothly, and a WPC-experienced player actually IN the team. This is the arrangement we will be going with to best avoid a repeat episode. Team exercises can be done at the WPC, before and during the event.

My latest theory is to implement a 'normalisation' system (akin to moving a point on a length of elastic) to arrive at a percentile: S1 = raw score, M = median score (on 1-100 scale), S2 = final score.
S1<M: S2 = S1 * 50/M
S2>M: S2 = 100 - ((100-S1) * 50/(100-M))

Under this system, the median score is the percentage of average score to maximum achieved score. How does that sound?
_________________
http://www.ukpuzzles.org/

[Zag]

If there are team events, then you want a good mix of skills as well as personalities. Math, cryptoanalysis, visual, etc. all need to be well-represented. With this in mind, you could go ahead and choose the team that you think will be best, and then reverse-engineer a category-based scoring system that arrives at the group of people you want.

[PuzzleScot]

Ha-ha! It's largely an individual contest, with a team championship bolted on.

The subjective choice method is certainly the most effective, but I'm going to have to resort to an objective method unfortunately. I think I have enough ideas now to formulate something workable.

The types of puzzle generally mean that (specific type practice aside) if you are good at one type, you're 'good' at all of them - it's kind of a World IQ Championship, but we all detest those 2 letters, don't we?

The instruction booklet for the 2009 and 2010 WPCs are available for free download if you're interested...

Thanks for the advice. Any other advice will still be much appreciated
_________________
http://www.ukpuzzles.org/

[Tinhorn]

2 events, 2 weeks apart. Take top 2 from each event (in 2d week, that's the top 2 that did not win in first week).

or

2 events, 2 weeks apart. Take winner of each, plus top next scorer as wildcard (try to make them same difficulty, but if people want to be sure, they should attend both). Then, those 3 people *pick* the fourth person from among the top 3 finishers in both weeks.

[Quailman]

Dave10000! Good to hear from you. How are you doing?

[Tinhorn]

Oh, can't complain, QM, thanks for asking. Gearing up for the Mystery Hunt, now only 6 weeks away. And otherwise keeping busy, workwise and puzzlewise. Hope all's well with you.

In case anyone's interested, here's a page of the puzzles I've written in the past 5 years or so, since I joined the National Puzzlers League and substantially cut back on my time at the Labyrinth:

http://puzzles.shukan.net

Cheers.