The philosophy behind the PhoneArena Camera Score: what we're measuring against

After we unveiled the PhoneArena Camera Score benchmark this week, I figured it’s worth it to say a bit more about the philosophy behind the Camera Score.

In other words: what are we measuring against? What are these phone cameras trying to reach? What is our standard for the greatest camera out there? In simpler terms, what does a higher result mean in PA Camera Score parlance?

We struggled with this question for a long time as we were working on different components of the rating. And from looking around in the camera benchmarking space, which is a rather uninhabited space, that was like the question to answer. Why? Because evaluating a camera, let alone a smartphone camera, is an inherently subjective affair.

So, what do you measure against? Based on what principles do you label one camera better than another?

One of the logical ways to go, which was also our intention for a while, is to have “reality” as the benchmark. In other words, this concept would have you take brutally-realistic photos of the test environment, as close to how our eyes see it, and then measure the shots taken by the phone cameras we test against that realistic reference.

This all makes sense, but there’s one big problem with this concept: it goes against what users demand out of their phone cameras. And what they demand is a camera that produces the most impressive-looking photos with the least effort possible.

Aside from the several purists out there, the vast majority of us want photos that make us look great, that make the environment we’re in look great, and that are ready for sharing right away. As we got convinced ourselves, nobody wants a dark, gloomy, blueish photo, only because it was taken in overcast weather. If that happens to be the case, we’d like things in our photo to turn out a little more… balanced, a little warmer and likable. Otherwise, what are we to do with that gloomy photo?

We absolutely think that phone manufacturers are overdoing it with the processing nowadays.
We’re fed up with the artificially oversharpened look, the always bright and sunny colors, the skin tones that make people appear orange…

But, at the same time, we realize we cannot live in the land of make believe, and we can’t walk against human nature and insist that phone cameras should take photos that are as realistic as possible, because if we’re completely honest – we don’t want photos that are as realistic as possible.

What we want are photos that just look great! And what does great mean? Well, I think that’s what phone manufacturers are trying to figure out. Year after year, they tweak, rework, shake up, improve, upgrade, all in the pursuit of a formula that customers would take preference to.

So, we decided to take the road less traveled so far, which is the middle road. It’s the middle road because we don’t think we’ll ever go back to pursuing realism in phone photography, but we also dislike the overprocessed look many popular phones offer these days. This is why we based our benchmark target on a concept we refer to as “Reality Plus”, which simply means that we want things to be grounded in reality and not look like they belong to some other planet, but at the same time we acknowledge the fact that we do want photos that simply look great and are ready for both viewing and sharing.
The PhoneArena Camera Score measures multiple objective aspects, like a camera’s ability to capture true detail, or the reliability of its preview, or the level of undesired oversharpening.


We hope this philosophy makes sense to you. Let us hear your thoughts about what modern phone photography should look like in the comments!