Making contest problems and writing tests

I'm now well behind in Advent of Code. Traveling up to Michigan to see my son and my brother and his family will do that.

I was planning on working to catch up but got distracted earlier by this thread on the Advent of Code subreddit. Specifically this comment by Eric Wastl, the contest creator. The thread is about the level of difficulty of the various problems and Eric's comment talks about some of the factors that go into creating a good collection of problems for a contest and the difficulties involved.

It made me think of a conversation I had twenty years ago with Rob Kolstad. We were chatting at a programming competition while the students were hacking away. We were discussing that there were only so many questions that a typical high school or college competition can ask:

1. The Mathy question
2. Recursion / divide and conquer etc.
3. Graph stuff
4. Dynamic Programming

and I think one or two more. The actual questions were just window dressing on these fundamental question types.

You can see this in many competitions including Advent of Code - particularly if you look over all three years. This is not a criticism. I love Advent of Code and think that Eric and the people he's working with are doing a great job . Read Eric's comment to get a deeper dive into what he thinks about as a contest designer.

I also got to thinking how similar creating a competition is to creating an exam. Maybe that's because I'm currently writing the final my students will take in a couple of days. In class tests are certainly not the best of assessments but they do have their place and in any event, they're frequently required by school or departmental policy.

There are many similarities between what teachers consider when making tests to what contest designers consider. What concepts and techniques must the students use? Are the problems too similar? Do they ramp up in difficulty? Are they all or nothing or can a student incrementally work their way to a solution? Does one problem lead into another?

There's no big reveal or surprise here today. Just something I've been thinking about.

A couple of busy days coming up but maybe over the weekend I can get back to Advent of Code.