Testing shows Siri beating out Google Now and Cortana for accuracy, user satisfaction and more

Which of the three top mobile platforms has the best virtual voice activated personal assistant? A company called Experts-Exchange had iOS, Android and Windows Phone users in the U.S. each ask four questions to Siri, Google Now and Cortana, respectively, and have them complete three tasks. The participants were told to score each response for accuracy, and keep track of the number of times they were given follow-up questions. Other metrics were also scored by the phone users.

The four questions that testers asked their assistant were: Who won the Super Bowl in 1978?, How many calories in a muffin?, Where can I buy tires? and When is "Kung Fu Panda 3" coming out? The three tasks were: Ask your phone to set an appointment, Turn the Wi-Fi setting on, and Send a text message.

Siri received high marks from users for answering the questions. For example, when the three assistants were asked the question about who won the Super Bowl in 1978, Siri answered correctly 95.3% of the time. 86.1% of users were satisfied with the response. While Google Now had the correct answer 88% of the time, 80% of the time it responded with a web search. Cortana gave the wrong answer 3.3% of the time.

When users asked their personal assistant to set up an appointment, Siri did it correctly during 92.3% of the attempts. 89.2% of users were satisfied with the response. Those two figures for Google Now were 81.3% and 80%. Cortana set the appointment up correctly 73.3% of the time, with 70% of the Windows Phone users satisfied with her response.

Overall, Siri had the fewest incorrect answers (2.4%), the highest number of satisfied users (81%), the lowest number of follow-up questions (5.2%) and was tied with Google Now for having the lowest percentage of answers that were in the form of a web search (5%). Google Now was second in the test, with Cortana third. Google Now (68%) and Cortana (57%) had low satisfaction scores compared to Siri, which has to be concerning for Google and Microsoft.

Interestingly, those living in the Western states gave higher satisfaction scores to Siri, while those living in the East gave higher satisfaction scores to Google Now and Cortana. Additionally, women using Cortana and Google Now received follow-up questions twice as often as men. This might have to do with the way that women using the two personal assistants structured their questions and task requests. Siri, on the other hand, asked follow-up questions of men more often than women.



source: ExpertsExchange via WMPoweruser