Smoked by their own marketing

Smoked by Windows started life as a clever series of challenges at CES this year. The demo guy, wielding a practiced hand on his WP7 device, was betting a hundred dollars that volunteers couldn't match his phone at common tasks. He lost a few times, but he won the majority of the time and Windows Phone came out looking aggressive, hip, and most of all fast. Microsoft generate a lot of free publicity at the even (minus a couple hundred dollar bills), so it wasn't surprising when the event morphed into an advertising campaign. Then, to crown it all off, Microsoft decided unveiled in-store challenges that placed a $1,000 laptop on the line.

And that's where Microsoft may have tried too hard. We understand that with stakes of this magnitude Microsoft doesn't want to lose, so we expect them to stack the deck in their favor. Still, as long as they were showing off the some of the world-beating features of Windows Phone, who could blame them? Alas, new information has come forth that casts the in-store version of the challenge in a less favorable light.

Earlier today we reported on a small brouhaha that arose when Sahas Kattaa beat the “Smoked by Windows” challenge with his Galaxy Nexus. An employee at his local Microsoft store challenged Katta to see who could more quickly display the temperature in two different cities. Mr. Katta had two weather widgets already configured to do so, and had further turned off the lock-screen, allowing him to win the challenge with the push of his power button. The store employees, apparently caught off guard, claimed his victory “didn’t count” because the two temperatures had to come from different states, even though that wasn't a previously stated part of the challenge.

As you'd expect in this day and age, the interwebs erupted with people calling Microsoft out, and someone with a higher pay grade soon made the executive decision that looking like a poor sport was worse than admitting the loss and providing Katta with his $1,000 laptop. We commend Microsoft for stepping up and doing the right thing here, but we wonder why this issue wasn't anticipated - surely they knew that such a loss would happen eventually?

It turns out that Microsoft may not have been prepared to lose; they had constructed the challenge in such a way that employees with the proper training could avoid losing altogether. They circulated a training guide with detailed instructions on how to conduct the challenges - a guide that fell into the hands of writers at The Verge.

Given the steep price and high embarrassment of a loss, it’s not really surprising that Microsoft would have guidelines on how to conduct the Smoked by Windows challenge. What may surprise you, however, is how bluntly the guide admits that Windows can be smoked by other phones, and how it teaches their employees to "select a challenge appropriate to your customer" so they won’t lose.

The weather challenge, for example, is not recommended for use against Android devices. The guide warns explicitly that “Android users have access to Widgets that can, if properly configured, display the weather and other information in real time.” Apparently they should have spent more time on that aspect of the training. Owners of Apple’s iPhone, on the other hand, do not have access to homepage widgets and as such are perfect candidates for the weather challenge.



A better challenge for a Galaxy Nexus would be the “Pocket to Picture to Post” challenge, which tests who can take a picture and post it to Facebook faster. Windows Phone's Facebook integration lets them upload a photo to Facebook in fewer steps than Android handsets. On the other hand, employees were warned that this isn’t a good challenge to pit against the iPhone 4S, because it has similar Facebook integration and a faster camera. To show you the extend to which Microsoft stacks the deck in their favor, they even warn against pitting this challenge against the mid-range HTC Status (the HTC “Facebook phone”) because the fast camera and auto-posting Facebook button would lead to a lot of free laptops.

Not only are the challenges rigged to be in their favor, but Mircosoft actually supplies a list of phones that you have to pick specific challenges for. How does an employee know what challenge to use? The guide encourages employees to spend time talking to challengers before hand, to ferret out what type of phone they have so they can pick a winning challenge.

This raises the question of whether in-store challenges are a good idea as conceived. With a $1,000 prize on the line and a much larger pool of potential challengers, Microsoft should have realized that sooner or later someone would figure out how to beat the system - especially since success depends on floor representatives memorizing a list of challenges and the correct phones to use them against. Human error was going to lead to a victory sooner or later. Microsoft also decided to take a picture of the "losers" of the in-store challenges; this makes for good theater, but if a customer like Sahas Katta feels cheated this type of public embarrassment will only encourage them come forward with their story.

On top of the potential for embarassment, it's no longer clear what exactly is being tested. In the original $100 CES challenge it was to lose once in a while - no one really thinks that Windows Phone is the best at everything, but by winning the majority of the time Microsoft's mobile OS came out looking good, and the challenge felt legitimate. With loses no longer really an option with $1,000 laptops on the line, Microsoft steered the challenge too far astray - if you have to exclude the HTC Status from some of your challenges, you aren't testing all-around speed and utility.

We all knew that the Smoked by Windows challenges were designed to show off the benefits of Live Tiles and other Windows Phone features (we wouldn’t expect them to create a challenge they would normally lose), but we think this part of the marketing campaign was poorly conceived. The high price of a loss required Microsoft to rig the challenges too much. Requiring employees to chat up a challenger ahead of time to discover his or her phone model so you can draw from a memorized list of the "right" challenges is bound to go wrong eventually - and even when it doesn't it smacks more of a parlor magician playing tricks on an audience than an actual comparison between phones.

source: The Verge