Siri finishes last in test against other smart speaker virtual assistants

A smart speaker is made up of a speaker and a virtual assistant. Apple finally got around to launching its HomePod smart speaker yesterday, and while it delivers what users are calling a premium audio experience, the device is apparently falling short in the virtual assistant department according to a test run by a venture capital firm named Loup Ventures. The test featured four virtual assistants including Apple's Siri, Amazon's Alexa, Google Assistant and Microsoft's Cortana.

In a report released on Saturday written by former Piper Jaffray analyst Gene Munster, it was disclosed that the aforementioned quartet of virtual assistants were asked 782 queries. HomePod's Siri answered 52.3% of the questions correctly. That placed it dead last among the four smart speaker assistants. Slightly above Siri, answering 57% of the questions correctly, was the Microsoft Invoke's Cortana. Alexa, found on the Amazon Echo line of smart speakers, had the right answer 64% of the time. And Google Home's Google Assistant was on the top of the list as the virtual assistant answered 81% of the queries correctly.

Munster pointed out that part of the problem with Siri is that on the HomePod, Apple's virtual assistant cannot handle tasks or answer questions related to navigation, calendar, email, and calling. Siri would answer questions related to these "domains" with a stock response that said, "I can’t ___ on HomePod." Removing queries from these domains took Siri's accuracy from 52.3% to 67%. That score would have placed it above Alexa and Cortana.

The analyst said that Siri understood 99.4% of the questions asked, and despite answering the fewest questions correctly among the four assistants tested, he wrote "Overall, Siri performed above our expectations given the limited scope of HomePod’s music focus."

The report adds that HomePod has better listening abilities compared to other smart speakers, and that Siri sounds more human on the device than on the iPhone. Communications with Siri are more human-like according to Munster, since she doesn't repeat every query like Alexa and Google Assistant both do. Apple was also praised for the packaging of the device.


A survey conducted by Loup Ventures suggests that demand for the product will be similar to demand for the Apple Watch. Munster expects that HomePod will grab 12% of the market for smart speakers in 2018, compared to a leading 52% for Amazon's Echo, and 32% for the Google Home. Other units will nab 4% of the market.

source: LoupVentures via AppleInsider