Camera blind test results: Pixel, S8 and iPhone are top, while LG, HTC and Sony flop

Camera blind test results: Pixel, S8 and iPhone are top, while LG, HTC and Sony flop

Over the past week, the AndroidPIT community has been participating in a great camera blind test. The flagships of the moment were included: the Sony Xperia XZ Premium, the HTC U11, the Samsung Galaxy S8 and the LG G6. Also included in the poll were the Apple iPhone 7 Plus and the Google Pixel, for comparison. Here we reveal the results!

The overall principle hasn't changed since our last blind test. The Xperia XZ Premium and the HTC U11 were added as challengers this time around. The expectations for the Sony smartphone were high given its breakthrough DxOMark rating and that Sony is the market leader in image sensor technology.

We've captured and then cropped the same scenes with each of the six smartphones, each with their factory settings, apart from setting them to their maximum resolutions and to automatic mode. To provide a reference shot, we've also taken the same photos with a Canon DSLR. Then, we invited everyone to vote for the best in each category.

The smartphone with the most votes in a category got six points. Second place got five points, third place got four points, and so on. In the case of a tie, both phones received the points that corresponded to the respective place, but the next place was omitted. 

blindtest q2 2017 1f
This image belongs to the Google Pixel and collected 43 percent of the votes. / © NextPit

The results

For the two new contenders, the Xperia XZ Premium and U11, the results were more (Sony) or less (HTC) a huge disaster. While the U11 made it to third place twice, the Xperia XZ Premium only made it there once - all other times these two were either in last or second to last place.

The overall top spot was won by the Google Pixel, closely followed by the Galaxy S8 and iPhone 7 Plus. Trailing behind a bit were the HTC U11 and LG G6, while the Xperia XZ Premium fell flat in last place.

  • 1st place, 20 points: Google Pixel
  • 2nd place, 18 points: Samsung Galaxy S8
  • 3rd place, 16 points: Apple iPhone 7 Plus
  • 4th place, 11 points: HTC U11
  • 5th place, 11 points: LG G6
  • 6th place, 9 points: Sony Xperia XZ Premium

You can look back at all the pictures in the original poll here, as we've now updated the captions to reveal which photos belonged to which phone.

blindtest q2 2017 2e
This was the LG G6's only first place winning photo, which took 30 percent of the vote. / © NextPit

Software over hardware

It is intriguing that in the past half a year, no one has managed to surpass the Google Pixel in terms of photo quality. The 1 / 2.3-inch image sensor of the Pixel is quite considerable in size, however, the IMX378 is also found in the BlackBerry KeyONE and the Xiaomi 5S. Yet, these two phones don't come anywhere near the picture quality of the Google flagships. While Android is more or less open, the proprietary camera app of the Google Pixel is not.

This could be a way to keep other manufacturers at bay, because unlike Google Photos or Gmail, no data is generated for Google to leverage to train neural networks for its "AI-first" corporate strategy. User photos come to Google through Google Photos now anyway. This strategy shows in another new technology, which will be in Android O, called Google Lens, which allows cameras to understand what they see. Now, back to the cameras...

blindtest q2 2017 3b
In this scene, the iPhone 7 Plus and Pixel did similarly well, with 23 and 34 percent of the vote respectively. This image comes from the Apple smartphone. / © NextPit

Smartphone cameras in the future

Right now, most manufacturers are already tinkering with their own camera apps, like those of Samsung's Galaxy S8 and HTC's U11. Although both the camera sensors and the operating system are freely available, you must be aware that Google has its own agenda and no reason to share its optimizations with the competition.

With the competition from Windows Phone and BlackBerry OS out of the way, and iOS sticking around 20 percent market share, things are pretty fixed. With no other alternative available, Google has tremendous leverage. This leverage will be used again in Mountain View as the next generation of the Pixel is diligently crafted. The Pixel 2 is already announced.

It's tough for smaller manufacturers like HTC and Sony to devote a lot of resources to outdoing Google in the software department, with either their camera app or their image processing. I don't believe we will see HTC, Sony or even LG reach the top ranks in the near future for this reason.

Even if the camera sensors and the Android operating system are freely available, Google isn't in the business of giving anything away for free unless it benefits the firm. Android is not a donation to needy manufacturers. Android has been, and probably always will be, a tool for Google to knock competing platforms aside and enforce its own strategy.

We saw this strategy in action this week in the Pixel's superior picture quality. Were you surprised with the results? Let us know what your think in the comments.


Write new comment:
All changes will be saved. No drafts are saved when editing
Write new comment:
All changes will be saved. No drafts are saved when editing

  • Dean L. Jun 10, 2017 Link to comment

    It's no surprise to me of the top 3. And most people I know use the stock app on their phones so from that perspective this is a totally valid comparison. But maybe use Snapchat for a test to see how the hardware does using a third party app.

  •   12
    Deactivated Account Jun 9, 2017 Link to comment

    Very good article. Lots of hurt feelings are fairly obvious.

    The vast majority of smartphone camera users pick up their phone, open the camera, and hit the shutter button to take a picture of whatever they want to, same for video.

    Of course, all the enthusiast photographers use any third party app and any endlessly variable settings of their choosing, but that's not really a feasible comparison or test.

    This is an out of the box camera comparison. You know, the way that most smartphone buyers leave the camera. That is why I think this is a very good article.

    Also, I own a Pixel xl and an iPhone 7 plus, but if it's for more than day to day shots because it's the device that you most always have with you, I reach for dedicated cameras with much larger sensors and more capabilities.

  • Albin Foro Jun 9, 2017 Link to comment

    "We've captured and then cropped the same scenes with each of the six smartphones, each with their factory settings, apart from setting them to their maximum resolutions and to automatic mode."

    Sorry, but that says more about OEM factory settings and quality of the preinstalled camera app than real capability of the hardware. There are too many excellent camera apps that far exceed the OEM defaults - a fairer test of the camera hardware, as such would be of different models using the same third party app. Even then, using standardized settings for different hardware components instead of an experienced photographer's best efforts - in some challenging situations - is an artificial random handicapping of some models that can do much better. No question, it's not easy to do a "fair" comparison of smartphone cameras, but buyers seriously interested in that aspect of the phone are not going to be very impressed by the OEM's out of the box newby experience in easy lighting.

    • Bastian Siewers Jun 9, 2017 Link to comment

      Well if you test a smartphone camera, it is not only about the hardware, also the software. We wanted to compare the results the manufacturers would achieve with their offering, and that obviously includes the software. And yes, you can easily see how important the software actually is, as the article actually states.

  • Leonard Morrissette Jun 9, 2017 Link to comment

    i still attest that the S8 imx333 sensor for camera supersedes the pixel 2 fold. period. my note 7 was great and s8 plus sensor is better than note 7.

  • Synthetica Jun 9, 2017 Link to comment

    As an owner of a G6 I can say that in real world usage the camera is quite good.

  • Vibhu Kaushal Jun 9, 2017 Link to comment

    Proud to be a pixel xl user. Camera in best in world processer fast

  • David Martrano Jun 8, 2017 Link to comment

    I thought Sony would fare much better. How does the HTC11 come just about in last with a dxo score of 90?? I'm beginning to wonder about all the tests they run on the cameras! Yikes!

  • Peter Harwood Jun 8, 2017 Link to comment

    What total waste of time and floored test. How about next time get a professional photographer and let him adjust settings to get the best out of each camera then compere results, only then will I take notice.

    Deactivated Account

    • Bastian Siewers Jun 9, 2017 Link to comment

      And how exactly do you know that Stefan, who took care of the pictures and stuff, doesn't know what he's doing? ;)

    • Leonard Morrissette Jun 9, 2017 Link to comment

      i agree on all that. ANY CAMERA TWEAKED ENOUGH CAN beat another. for all we know a dslr camera was used, in place of a phone.

      • Bastian Siewers Jun 9, 2017 Link to comment

        And why exactly would we do that when we compare cameras with each other? :O

Write new comment:
All changes will be saved. No drafts are saved when editing