Pitch The Switch: And the Winners Are…

Note: This post was archived from the Infusionsoft Blog. More details here.

Pitch the SwitchWe’ve finally concluded the end of our month-long contest to discover who created the best video pitch in their own words for Infusionsoft. However, it was a tough one. First, I’ll announce the winners. Then, I’ll explain what we’ve learned from this contest — anyone considering to run an online contest needs to read this.

After tallying all 6,397 votes, here are the results of the Pitch the Switch contest:

FIRST PLACE, GRAND PRIZE
Scott Griggs, Trainz.com
$10,000 Cash
1 year of Infusionsoft

SECOND PLACE
Nathalie Lussier, Raw Foods Witch
Apple iPad

THIRD PLACE
Dustin Mathews
Apple iPod Touch

FOURTH PLACE
Marketing Ninjas
Flip HD Video Camera

Congratulations to the winners and thanks to everyone who participated. We saw some great activity and of course, it has been exciting to see how Infusionsoft has helped our customers grow their businesses and improve their lives.

Now that the winners have been announced, let me explain a few things that happened along the way and share a some lessons we learned.

Shortly after the second phase of the contest, where contestants were encouraged to drive votes on their videos, we noticed a couple of weird things. Some contestants were receiving far more votes than others. After some investigation, we discovered that those contestants had created bots (software programs) that would automatically submit votes. As soon as this activity was discovered, we issued a communication to all contestants stating that we would not tolerate cheating and adamantly stated that fraudulent votes would be disqualified and we may disqualify cheaters.

Amazingly, we received a formal apology from one of the contestants. He was unaware that one of his programmers had employed this tactic. He was sincere in apologizing and withdrew from the contest. Stand-up guy. I love our customers.

Our communication also mentioned methods that were okay for promoting their video, such as Twitter, blogging, emailing family and friends, etc. All of those would be what we label as “organic” traffic. However, we didn’t account for the possibility of paid traffic.

So, right before Thanksgiving, an email was sent from one customer (let’s call her Jill) informing us about another customer (let’s call him Jack). Jill showed us evidence of Jack paying for votes. Since it was the day before Thanksgiving, I was out of the office and not checking email. We didn’t see the email until early this week — which was a problem since the contest ended on Tuesday (yesterday).

We had some healthy debate (minor yelling and screaming) on whether or not allowing paid votes was acceptable or not. Initially, we leaned toward allowing those votes since we never explicitly stated that paid traffic was against the rules. In an effort to be as fair as possible, we decided that paying for votes was outside of the intent of the contest and we disqualified Jack’s paid votes.

You can imagine the quandary we were in. If we allowed the paid votes, then we would have a flurry of activity (which was already kicking up) on the social networks of people upset because we allowed votes that they considered fraudulent even though technically it wasn’t in the rules. We’d also have contestants that would feel disenfranchised because one customer won using a tactic that they felt was inappropriate (or that they would have used if they knew it was OK). On the other hand, if we chose to disqualify Jack because of his paid traffic, he could easily feel cheated (by us) for not holding to the letter of the law — our own rules didn’t explicitly ban paying for votes. Both arguments can be strongly justified and trust me, we had quite a debate here about it.

We were in a lose-lose situation.

Ultimately, we decided to disqualify Jack’s paid votes, but not disqualify him as a contestant since we had never had said specifically that paying for votes was against the rules.

As it turns out, Jack’s votes totaled only 8% of the total votes and didn’t skew the results at all. By the way, I called Jack this morning and had him show me through GoToMeeting the votes he paid for. He logged into the system he used to pay for the votes and showed me live. This verified the data we had on our side. With $10,000 on the line, we felt it was necessary to take every measure possible to ensure that we were counting votes appropriately and being as fair and equitable as possible.

In light of the above, here are the top five lessons we’ve learned from this contest. If you want to do a contest, please heed these pieces of advice.

  1. Don’t allow anonymous votes. We probably should have required a valid email address to vote. This would have reduced the chance for fraud. It wouldn’t have eliminated the possibility of people creating bots or paying for votes, but we would have had one more data point to track.
  2. A big cash prize makes the contest wonky and doesn’t produce results. People get very emotionally attached when they think they have a chance at winning a big cash sum. So, they do things they wouldn’t normally do and they get really let down if they see the prize getting further out of reach. The intent of our $10,000 prize was to encourage a ton of people to enter the contest. The reality was that we didn’t get any more entries to the contest than we’ve received for other contests where we didn’t offer a cash prize. So, the cash prize didn’t produce results and it caused the contest to get a little out of hand. The difference between $1000 and $10,000 didn’t necessarily render a better outcome in this contest.
  3. Explicitly state every possibility in the rules. It’s impossible to think up all the possible ways that contestants might try and game the system. But, it’s worth the time and effort to really think through all the possibilities you can possibly imagine and explicitly state them in the rules. It will save you a ton of time, potentially money, and probably your reputation if you take the time to do this. Consider asking some people to look over the rules and listen to them if they can find loopholes in your policies in it. (We learned this by failing to mention paid traffic. We assumed everyone would drive organic traffic.)
  4. Provide a sample entry. We realized after the fact that we probably would have had more entries if we had provided a sample entry for people to see what was required of them. We’ve seen contests be effective when a low quality sample entry is displayed as an example. The key is to provide a low quality entry. When the example is low quality, potential contestants automatically think, “I can do better than that”, and that mindset gets them motivated to enter. This resulted in a somewhat-uncomfortable delay until the first few videos were published.
  5. Contests based on voting decrease participation. When winning a contest is based on voting, many people automatically discount themselves because they don’t have a big reach or the ability to find many people to vote for them. A contest where the winners are chosen at random or by the contest holder is more likely to encourage participation because potential contestants don’t immediately discount them self. This also limits the scientific approach for winning and enables people to become creative with their submissions.

Running the Pitch the Switch contest was a blast for us. I’m sorry that things got wonky in the end, but I feel like we made a good decision, and I’m even more glad that in the end the results were not skewed by questionable activity.

Congratulations to all the winners and everyone who participated. I hope our experience will help any others who are planning to run an online contest.

Oh, and by the way, watch their videos – they were all great!

5 thoughts on “Pitch The Switch: And the Winners Are…”

  1. Don’t be too hard on yourself. I saw the “It’s impossible to think up all the possible ways that contestants might try and game the system. But, it’s worth the time and effort to really think through all the possibilities you can possibly imagine and explicitly state them in the rules.” statement. There’s been a Federal Election Commission coming up with rules on voting for government offices for hundreds of years and governeing God knows how many elections and tallying countless votes…yet it is a fluid system that continues to grow every election as you say politicians try to “game the system”

  2. What a great blog post. It’s great to see you publish the “Behind the Scenes” action to help other business owners who are considering running a contest so we can learn from your experience. I was surprised at how many videos were submitted on the “due date” of the contest. Infusionsoft has creating some pretty cool contests in the past. How often do you typically run contests for clients?

  3. Great post and just in time for us, we are about to announce the contest for our annual Enterprise Social 2.0 conference in Europe and I can use some of your advice, for example, we did not think about providing a sample entree.
    About point 5 I can argue, it can discourage some people who have a small number of followers, from the other side it can encourage member who have a large amount of followers and if they do, they can create a big buzz online and this is what we need by rolling out the contest, isn’t it? My solution here is: the winner should be selected by both criteria: number of votes AND the final word will be by the jury.

  4. To be honest, I prepared the concept of my video, when I reviewed again the rules and saw that part of it is driving traffic to the video, I said ok I don’t think it’s worth for me to even try, next time I would suggest have the contest on the best video only leave alone the traffic part.

  5. Skip the valid email address requirement as in point #1 Don’t allow anonymous votes. There is already automated software that can create hundreds of email accounts on the fly and automatically click a verification link return emails. So that wont solve any voting issues.

    I think the contest was great in concept, but I knew I would not have a chance to win since I did not have enough people to tell and watching 15 2-5 minutes videos to figure out which one was the best consumed far too much time to warrant watching all of them to make a valid decision.

    Perhaps a random drawing of all videos sent in and having LOTS of winners might have produced more videos for you.

    A for effort though. As they say, if you never step up to the plate, you will never hit a home run.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.