The iPhone 11 and the power of “computational photography”

Photo and Video have become major differentiators in the world of smartphones – and the three-way competition between Google (Pixel), Samsung (Galaxy) and Apple (iPhone) has led to huge improvements in the last few years. Each new generation is markedly better at making pictures than the previous one.

A few weeks ago, I could not resist any longer, and took advantage of a promotion of my favorite carrier to buy the brand new iPhone 11 for $350.00 – I just had to surrender my old iPhone 7 in exchange. The truth is, I needed more internal storage, but I also wanted to see whether the new camera was as good as promised.

IMG_5131
iPhone 11 – sunset at the pool

And I was not disappointed. The photo section of that thing is incredible. Simply having access to any focal length between the ultra-wide (13mm equiv.) and the wide angle (26mm equiv.) at full resolution, with a digital zoom to bring you a bit closer to the subject if you need it – is literally a game changer: how often have amateurs access to a  13-35mm zoom lens on their full frame digital camera?

Of course, with a 12 Megapixel sensor and a “normal” lens limited to the equivalent of a 26mm wide angle, it can’t beat a medium format digital camera for large prints, or a DSLR with a fast telephoto lens for sports photography. And because of the way Apple has tuned noise reduction and HDR, the pictures are a bit light on contrast to my taste, but that’s nitpicking.

For subjects which are considered “normal” for an amateur photographer: selfies, family shots, portraits, street photography, urban landscapes, interiors, and for the “normal” destination of most of today’s pictures  (instant messaging, social networks, on-line photo galleries, prints up to 11x 8) it’s so good that I doubt I could get better pictures out of camera with any of the digital cameras I own.

Even if I spent big money on the latest and greatest full frame mirrorless camera, bought more lenses, and dedicated a lot of time to practicing and testing in order to seriously step up my technical game, I’m still not sure I would get significantly better results out of camera than what this iPhone gives me effortlessly. (*) (**)

So is the power of “computational photography“…

Very high level, an iPhone takes many different versions of the same shot (just before and just after you press the shutter release), with different focus and exposure settings, and uses artificial intelligence to decompose the image in sections (main human subject, background, sky, …). Each segment of the image is then optimized (exposure, contrast, noise reduction, focus, white balance…) and integrated into the “final picture” presented on the phone’s LCD. They call that “semantic rendering”.

All of this happens in a fraction of a second – the iPhone’s processor is a 64 bit / 6 core chip with a “machine learning accelerator”, and it can process 1 trillion operations per second.

Canon, Nikon and Sony don’t disclose many details about the architecture of the electronics of their top of the line cameras – but I doubt they have anything that even remotely compares to the processing power of the best of the smartphones.

apple-iphone-xs-camera-layers-keynote
iPhone 11 presentation – the processing pipeline of an image – segmentation is what’s new.

Ultimately, “amateur photography” is about the pleasure of taking and sharing pictures.  I’ve been so pleased with the iPhone’s pictures that I’ve not used any other camera (digital or film) since I bought it. The novelty will wear off, and at some point, I’m pretty sure I’ll get tired of a “neural engine” making “semantic rendering” decisions for me. I’ll want my pictures to be really mine, not a quilt of segments massaged by an algorithm running on a chip with 8 billion transistors. Maybe I’ll just go back to black and white film, and process the images in a dark closet at home.

In the meantime….

Happy New Year.


Out of the camera pictures taken on the iPhone 11 – minor adjustments in the iPhone’s photo app.

IMG_5134
Florida – Sunset at the pool – iPhone 11

 

IMG_4977
Las Vegas – the Paris – iPhone 11 – with the 13mm equiv. Wide Angle lens

 

IMG_5009
Las Vegas – Night scene (iPhone 11 with optical image stabilization, handheld)
IMG_5024
Las Vegas – Mandalay Bay hotel. Interior Photography – iPhone 11
IMG_4994
Las Vegas – Street Photography – iPhone 11
IMG_4865
iPhone 11 – portrait mode. at night, at the terrace of a restaurant
IMG_5154
Florida – Landscape at sunset – iPhone 11 –

 

(*) Of course, the key restriction here is “out of camera” – with the help of Lightroom, and a set of Lightroom plug-ins, a recent full frame digital with a good set of lenses beats an iPhone – but we’re not in the realm of candid amateur photography anymore.

(**) I don’t do videos. But I’ve read multiple comparative reviews of the iPhone 11 opposed to good mirrorless cameras: for still images, a dedicated camera will ultimately yield better results than Apple’s latest smartphone (think large prints, action photography, …) – but for videos there is no discussion that the iPhone – because it has enough processing power to enhance each individual frame in a real time – is the better widget.


More about Apple and computational photography

https://coolhunting.com/tech/apple-iphone-xs-camera-computational-photography/

https://blog.halide.cam/inside-the-iphone-11-camera-part-1-a-completely-new-camera-28ea5d091071