How voting works

Why ask the crowd

One leader, or one workshop, hears from a few voices. A crowdsourced question hears from everyone it is sent to, in a few days, in their own words. When many people judge independently, their combined view is usually more reliable than any one opinion, and it surfaces ideas that nobody in the room would have thought to ask for.

Why compare two ideas instead of voting for favorites

The obvious way is to list every idea and let people tick their favorites. It sounds simple, but it goes wrong in familiar ways:

• Ideas posted first collect the most ticks, simply because more people saw them.
• People tick their friends' ideas, or their boss's, or their own.
• Nothing forces a trade-off, so everything looks "good".

Comparing two ideas is quick, forces a choice, and shows each person a mix of ideas. Fifty or so quick choices add up to a full ranking. You never see your own idea, and you never see who wrote the ones in front of you.

How the ranking is built

Every idea starts with the same score, 1,000 points. It works like a chess rating: beating a stronger idea earns more than beating a weaker one, and losing to a weaker idea costs more than losing to a stronger one.

Scores settle as ideas meet more opponents. The leaderboard shows how each idea ranks and how often it was chosen when it was compared.

Do late ideas have a fair chance?

They should, and the system is built so they do. An early idea has had more time to collect points, so a late idea needs help:

• New ideas are shown more often at first, so they meet enough voters quickly.
• New ideas move faster. Their score changes about six times as much per comparison, easing off over their first 40 comparisons.
• New ideas meet opponents near their level, then tougher ones as they win, so they find their true place instead of being stuck.

We tested this by simulating thousands of projects with a known best idea. Share of runs where a truly great idea finished in the top 5:

A simulation, not a guarantee. It assumes people usually, but not always, pick the better idea. A merely good idea is not lifted by the faster start: it still finishes around where its quality puts it. Re-run it with node scripts/simulate.mjs.

What keeps it honest

• You never see your own idea, and each pair is shown to you once.
• Votes are anonymous while the project runs, so nobody votes for the boss or a friend.
• Each person has a vote limit, so no one can push an idea by voting thousands of times.
• Ideas that three people flag are hidden for review, so the crowd moderates itself.
• People choose whether their name is shown. Names appear only for top-10 ideas, only after the project ends.
• Results are only broken out by group when at least 5 people are in it.

What it can and can’t tell you

The ranking shows what this group prefers, as a group. It is a strong starting point for decisions, not a verdict. It works best with a good question, enough people (the more the better), and leaders who act on the top ideas and say so afterwards.

Questions people ask