Writing Quiz Questions Without Bullsh*t

I recently found a book in my local library called Writing Without Bullshit: Boost Your Career By Saying What You Mean (2016) by Josh Bernoff.  Despite its wonky premise — since when has sincerity and straight talk ever boosted anyone's career? — it contains a lot of good advice pertinent to a concept I've termed rhetorical intelligence: the ability to communicate in a way that raises that probability that a) people understand what you mean and b) people remember what you say and are affected by it.  (It's closely related to what another great how-to book terms stickiness.)

The fact that we call quizzing "trivia" in this country allows us to categorize it as "trivial," which in turn allows terrible writing to flourish.  No one bothers to emphasize good writing because we convince ourselves that "trivia" writing isn't real writing, and therefore principles that obviously apply in other contexts somehow don't in "trivial" ones.

Well, they do.

Take Bernoff's "ROAM" model, which he calls "The 4 questions to ask before you write anything."

* Readers: Who is the audience?
* Objective: What are you trying to do?
* Action: What do you want the readers to do?
* iMpression: What will they think of you?

Let's apply this to writing "trivia" questions.

Readers

I spent a good portion of my time at NMSU working on the "Quality Initiative," a project intended to improve the quality of student writing.  My job involved coding class syllabi, and one category I was asked to code that turned out to be wildly underrepresented was writing assignments targeted at a specific audience.  One of the most important skills for any writer is to match their approach to anticipated readership, and yet we don't teach that at all in schools.

Putting yourself in the shoes of your audience is a vital skill practiced by all question writers worth their salt.  Many people don't understand this, judging from the number who try to tell me that I can't be trusted to write mass-audience quizzes because I "can't write easy questions."  In truth, of course, my "Super Hard Quizzes" are just that because they are intended to challenge my Facebook friends — a coterie that includes many of the best quizzers in the world.  I've written plenty of quizzes for less intense groups that are, well, easier.

Don't ever begin writing a quiz until you have in place a mental model of your audience.  What excites them?  What repels or numbs them?  How much do they already know?  How hard are they willing to work?

Objective

There are people with fragile egos who just can't handle questions to which they don't know the answer.  If for any reason you get trapped into writing a quiz for them, just toss off a bunch of gimmes and move on.  But people like that aren't really the core audience for quizzes, because they have no interest in learning new things or pushing themselves.  They might attend a quiz if they're dragged to it but they won't pursue quizzing as a hobby.

The kind of people who could potentially enjoy quizzing as a hobby are searching for three things.

* Validation: Not just questions whose answer lies within their knowledge base, but questions that validate that the things they've chosen to attend to and remember are worth attending to and remembering.
* Challenge: Questions that lie outside their primary knowledge base but which they feel they can get if they follow the right bread crumbs and assemble various pieces of knowledge they do possess.
* Education: Questions that lie outside their knowledge base but are inherently interesting and inspire them to learn more.

In other words: easy, moderate, and hard questions.  There are varying schools of thought about the ideal proportions, but in general "hard" should be the smallest category (unless, I suppose, you're writing your own "Super Hard Quiz").

All this is complicated by the fact that what counts as "easy," "moderate," and "hard" varies drastically based who your target audience is.  It's up to you assess the target level of difficulty properly.  Once you do, however, it's not all that hard to tweak questions so as to raise or lower their difficulty.

Take, for example, Q6 from SHQ24:

Prior to her breakthrough role as Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016), Alison Sudol had a music career as A Fine Frenzy — a stage name taken from Act V, Scene 1 of which Shakespeare play?

I toyed around with this question for a while before finally settling on the hardest possible version of it, one which requires either very deep knowledge of Shakespeare or very deep knowledge of Ms. Sudol's biography.  Yet consider these variations that simply involve rearranging the same facts:

Prior to her breakthrough role as Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016), Alison Sudol had a music career under what stage name taken from Act V, Scene 1 of A Midsummer Night's Dream?

Prior to her breakthrough role as Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016), who had a music career under the Shakespeare-derived moniker A Fine Frenzy?

Alison Sudol, also known as the musical artist A Fine Frenzy, broke through as an actor after being cast in what role in Fantastic Beasts and Where to Find Them (2016) and its sequel?

Although she had already established a music career under the name A Fine Frenzy, Alison Sudol broke through as an actress with the role of Queenie Goldstein, who has a romance with the "no-maj" baker Jacob, in what 2016 film?

Prior to her breakthrough role as Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016), Alison Sudol had a music career as A Fine Frenzy — a stage name taken from Act V, Scene 1 of which Shakespeare play that features the fairies Oberon, Titania, and Puck?

Alison Sudol, also known as the musical artist A Fine Frenzy, broke through as an actor with the role of Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016). Who wrote both that movie and the book it is based on?

Each of these questions teaches quizzers the same interesting set of facts, but each is a graded step down in terms of difficulty ... and thus a graded step up in terms of accessibility.

Now, more accessible is not always better.  People are often proud of their specialist knowledge and — understandably — want it to be rewarded more than their competitors' non-expertise.  But too little accessibility makes people lose engagement and become bored.  It's a tricky balance to hit, and the more information you have about your audience, the better equipped you are to manage that balance.

Action

Yes, every time you write a quiz you are trying to motivate an action.  At a bare minimum you want people to keep playing.  You want them to come back to your future quizzes.  Ideally, you want them to recommend you to their friends, and perhaps even be willing to pay for your services.

For reasons I described above, simply feeding them easy questions they know the answer to will not cut it.  (Also, it will destroy your soul.)  Another pitfall to avoid is boring questions.

What do I mean by boring?  Let's consider another version of that Alison Sudol question:

Who played Queenie Goldstein in Fantastic Beasts and Where to Find Them (2016)?

A lot of people won't know the answer to this, but more importantly they won't care.  "Who played [X] in [Y]?" is the kind of question that could be written by a computer.  There's nothing compelling about it.  There's no sense of surprise.

Conversely, if you sprinkle in the fact that Alison Sudol is also A Fine Frenzy, then even people who instantly get the answer might learn something new.  Those who know her as an actor might not be familiar with her music, and those who are into her music may not realize that she's transitioned into a movie career.  Questions that tie together multiple domains make people feel as though they're journeying on a voyage, rather than just sitting through a tedious test that requires spitting things back from memory.

Now, it's possible to imagine a version of this question that goes into exhausting detail about Alison Sudol and her career in a way that overwhelms the interest of all but the most dedicated fan (who is likely to know that information already!).  Even if you love all of her albums — which, incidentally, are called One Cell in the Sea (2007), Bomb in a Birdcage (2009), and Pines (2012) — there's just no reason to mention them unless you think they will genuinely lead someone in your audience to the answer.  The important thing is not to shoehorn in facts, but to shape a narrative.

Capturing your audience's imagination is especially important when it comes to the moderate and hard questions, the ones they may not answer correctly but hopefully will be intrigued by and want to learn more about.  It's especially vital for these questions to tell a story.  I'll go into this in more detail below, but remember: Every interesting fact has a narrative that goes, "The world is cooler and more intriguing than you imagine."  Find that narrative thread!

iMpression

There are two rookie impression-related mistakes that many quizmasters make.  One is that they think they can come off as smart by writing difficult questions.  In reality, it's very easy to come up with a question that virtually no one can answer.  For example: What was the only single released from A Fine Frenzy's third studio album, Pines?  The answer is "Now Is the Start," and I don't know that because I'm admirably knowledgeable about music; I know it because I know how to use Wikipedia.  (And now I'm going to forget it.)

A better alternative is to turn your obscure fact into a solvable puzzle.  I may have implied above that I think that all questions about Alison Sudol's deep cuts are inaccessible and uninteresting.  But not so fast!  Try this one for size: A Fine Frenzy's 2009 single "The Things We Did Last Summer" comes from His Way, Our Way, a tribute album based around the songs of which legendary musical artist?

This is a great "challenge" question, in that most people don't know it off the top of their head but sufficiently insightful lateral thinkers can, with some effort, link "His Way, Our Way" to the artist most associated with "My Way."  Good quiz writing doesn't try to "stump" people; it tries to nudge them into realizing that they are smarter than they think they are.

The other common rookie mistake is to attempt to be "entertaining."  There is a prominent quizbowl personality who is a horrible human being but did deliver one rhetorical masterstroke when he gave aspiring question writers this piece of advice: "You are neither cute nor clever."

A less confrontational way of putting this is that if you select interesting clues and interesting question topics, your core audience will be entertained.  But if you set out to self-consciously "put on a show," you're likely to just come off as awkward and self-indulgent.

So what impression do you want to create?  In two sentences: Quizzing is a conversation.  Be a good conversation partner.

What is a good conversation partner?  Someone who respects your intelligence, who exposes you to new and interesting things by connecting them to what you already know, who shows you new angles of things you thought you were familiar with, and above all someone who DOESN'T WASTE YOUR TIME.

So much of good writing, honestly, boils down to "Don't Waste Your Audience's Time."  In question writing, that means don't pack in more information than necessary.  Don't bloat your sentences with adjectives and subjective opinions.  Don't convolute your sentence structure to sound smart.  (Oh, and don't mess around with "trick" questions.)

At the end of the day, the best method to ensure that you are not wasting your audience's time is to ruthlessly cut away every part of your questions that does not serve one of four functions.  I present these very much non-randomly, in descending order of importance:

* Clues: Is it a clue that, realistically, might actually help a member of your audience figure out the answer (and isn't redundant with other clues)?
* Interest: Does it contribute to letting your audience know why this particular topic is worth caring about?
* Pinning: Every good question must be pinned to a single factual answer or group of answers.  Sometimes extra words are required to do this.  But as far as possible, try to make the clues sufficient so that no extraneous pin is necessary.
* Rhetorical flavor: I also recently read a book that reminded me that figures of speech are the spice of discourse.  Like all spices, they should be applied sparingly.  But a little bit of added processing fluency never hurt anyone.

So how can you do this without sounding dull or mechanical?  First of all, don't write questions unless you can make them interesting.  "Interesting" is subjective, yes, but the principles underlying it are not.  As Murray Davis stated way back in 1971, interesting propositions contradict weakly held assumptions of their audience.  They are simultaneously counterintuitive and ring true.  We're writing general knowledge questions, not dissertations, so it isn't necessary to be massively groundbreaking.  Just make an effort to complexify your audience's worldview in some minor way, to break them out of their usual frames.  Remind them that sometimes Hindu statuary shows up in ancient Roman ruins, or the great-grandson of a regicide grows up to play Henry VIII on TV, or the Atlantic end of a famous canal lies west of its Pacific end.

The main impression you want to leave people with, in other words, is not about you.  It's about the world.