Sony Motion Eye camera explained, or how the Xperia XZ Premium shoots epic slow-motion videos
Remember the bullet time special effects introduced by the Matrix movie? Remember the action scenes of Neo caught in a mid-air battle against the evil, all-powerful agent Smith? Man, those were epic! Such stunts required some serious gear to pull off, and even though it has been two decades since then, I doubt that a modern smartphone would be enough to reproduce them. But perhaps there's a couple of phones that could get close.
At this year's Mobile World Congress, Sony introduced the Xperia XZ Premium and Xperia XZs, both of which feature a brand new camera that can perform some nifty tricks. Their Motion Eye camera, as Sony calls it, shoots dramatic slow-motion videos – four times slower than any other phone on the market – and can capture photos of moments in time before the shutter is pressed. Impressive, right? But how does this fancy new camera work? Well, that's what we're going to explain in the paragraphs below.
How is slow-motion video made?
Back in 2015, Sony announced the fourth generation of its RX-100 compact digital camera – a camera that did not go unnoticed by photography enthusiasts. It could shoot stills at a rapid rate, video at a whopping 4K resolution, and – perhaps most impressively – slow-motion clips at a mind-boggling 960 frames per second (fps). To this day, the slowest slow-mos out of a commercially available phone do not exceed 240 fps.
Now, I better put these figures into context in case it's not entirely clear what they mean. In a nutshell, video is little more than a sequence of images/frames that change fast enough to create the illusion of motion. Traditionally, movies are shot at 24 frames per second, with some exceptions like Avatar or The Hobbit shot at 48 fps. The great majority of YouTube videos are streamed at 25 or 30 fps.
But here's where the magic happens: if you play a video at the rate at which it was recorded (e. g. playing a regular 30fps video out of a phone at 30 frames per second) it would appear to run at a normal speed. A higher framerate would make motion in the video look a bit smoother and more realistic (possibly giving you a mild feeling of dizziness along the way), and that's pretty much it. However, if you play video at a framerate lower than the original, that would create a slow-motion effect. For example, playing a 960fps video at 30fps would make objects appear as if they're moving 32 times slower than in real life. Here's what that looks like:
How is this technically possible?
That was a cool video, right? Sony's RX-100 camera was able to achieve the said framerate because it had an image sensor designed for speed. You see, shooting at 960 frames per second generates tons of data over a short period of time, so to mange that data effectively, a special DRAM memory element was built into the sensor. That DRAM acted as a buffer, storing the visual information until it was ready to be handled by the image processor.
Unsurprisingly, the Sony Motion Eye camera in the Xperia XZ Premium and Xperia XZs is built on the same principle, only applied on a smaller scale to make the module fit into a smartphone. The image sensor in use consists of three layers: a pixel section at the top, circuitry at the back, and a DRAM element sandwiched between them.
A Conventional 2-layer stacked CMOS image sensor (left) vs the new 3-layer stacked CMOS image sensor. DRAM element shown in orange
So, how do you shoot Super Slow-motion video on the Xperia XZ Premium and XZs?
It's quite the process, actually, but you should be able to get the hang of it after a few practice turns. Two things you will absolutely need, however, are good timing and plenty of light.
As seen in the picture below (and in the video, at the 1:14 mark), 960fps videos are accessed from the video camera screen. Above the record button, there's a smaller button that makes Super Slow-motion accessible. But even after pressing it, you're still not recording anything. You have to hit thе record button to start taking regular video, and then pressing the button a second time enables Super Slow-motion 960fps video for a period of 0.182 seconds (or about a fifth of a second). This is then transformed into 6 seconds of dramatic, Super Slow-motion video. Pressing the button again will capture another 0.182-second portion at 960fps.
That small button above the big one activates Super Slow-motion videos on the Sony Motion Eye camera
Again, timing is of importance here, as you have to hit the Super Slow-motion button at the right time to slow down that special moment. Of course, it would have been much more convenient if we could manually select the portion of the video that is to be slowed down, but alas, that's not technically feasible. As we mentioned above, 960fps slow-mos are buffered in the camera's DRAM, and that, despite being 128 megabytes in size, gets filled up in under 2 seconds. Another downside of Super Slow-motion videos is that image quality is greatly reduced due to the increased sensitivity (ISO) of the camera. You'll need plenty of light to capture good-looking 960fps slow-mos.
For the record the Sony Motion Eye camera can shoot 120fps videos to which you can later add slow-motion effects for a specific portion of the footage.
What about that "taking photos before the shutter is pressed" part?
Ah yes, super slow-motion movies aren't all that Sony's Motion Eye camera has to shine with, and in fact, this feature is looking a lot more practical. Long story short, the cam starts taking photos automatically if it detects motion in the frame (hence the Motion Eye naming of the camera). Once you press the shutter, you're given not only the picture that you took, but also three other full-resolution frames that were captured a few moments earlier.
To be clear, there's nothing that you have to do in order to activate these special Motion Eye abilities. Designed with user friendliness in mind, the feature is integrated seamlessly into Sony's Intelligent Auto mode (set by default), so there's no need to fiddle with modes or settings. Of course, you do get a cue letting you know that the camera used its special abilities – a little "Predicted" notification pops up in the corner of the camera UI.
But what about image quality? Are regular photos any better?
That's an excellent question. Up until now, Sony's smartphone cameras were characterized with having big sensors with lots of megapixels, but with relatively small individual pixels. Smaller pixels make a camera more susceptible to digital noise (that fuzzy distortion you see when you zoom in) and less sensitive to light, which is likely one of the reasons why the 23MP Sony Xperia XZ struggled against the 12MP iPhone 7 in a recent camera comparison.
The Sony Motion Eye camera makes a shift to a lower number of megapixels – from 23MP down to 19.3MP – and larger individual pixels – from 1.12um up to 1.22um. As a result, we're expecting to see superior image quality compared to last years' Xperia models, especially in low light. And yes, we will do a camera comparison to confirm whether or not that's the case, but until then, the gallery with images below is all we can provide. Friendly reminder: the Sony Xperia XZs unit used to take the images was a pre-production unit.
FCC OKs Cingular\'s purchase of AT&T Wireless
Things that are NOT allowed: