Don’t look now, but Microsoft is becoming a serious player in the world of iPhone apps.
It has already brought over Office and subsumed well-regarded email app Acompli and calendar app Sunrise into a revamped Outlook for iPhone. Now, Microsoft is looking to offer up a rival to the built-in camera app.
Microsoft Pix, a free app from Microsoft Research, focuses first and foremost on delivering better pictures of people. It does this by continuously taking pictures when the app is open and using an algorithm to choose the best shot or shots from among 10 images (seven just before the camera button is pressed and three after).
“If you see it happen and you aren’t already taking pictures, it’s too late, you missed the shot,” said Josh Weisberg, a Microsoft researcher and former team photographer for the Seattle Seahawks.
It turns out often the shot we really want is one just before our finger manages to hit the capture button, both in terms of composition as well as the shake that can occur when a finger hits the screen. Choosing from multiple shots also allows the app to weed out the ones where someone is blinking or goes from a smile to a grimace, serving up the best image, or the best couple of images if they are significantly different.
Microsoft uses artificial intelligence in a variety of ways to provide a better picture. It draws from stills it discards to improve the best shots by reducing noise and also automatically adjusts contrast and white balance with a focus on making sure the faces in shots look great.
The app is part of two big trends at Microsoft. First, and best publicized, has been Microsoft’s move into iOS and Android. Less well known is a big shift inside Microsoft Research, which for a long time was seen as a pure research house. Historically, one of Microsoft’s existing product groups would have to decide it liked a research concept and then do the work of commercialization. With Fix, Hyperlapse and other recent releases, Microsoft is showing it is willing to let the research team directly bring products to market.
Weisberg said that while Microsoft isn’t charging for Pix, it still reaps benefits as more consumers gain an appreciation for the company’s ability to lead in the burgeoning field of machine intelligence.
Weisberg and Microsoft Research colleague Neel Joshi showed off the app at a tricked-out loft in San Francisco’s South of Market neighborhood last week, using a specially built rig that allows two iPhones to shoot simultaneously, one using Pix and the other using Apple’s camera app.
Using a combination of backlighting, some breakdancers and pyrotechnics, the two showed off Pix at its best — capturing the people you want in complicated scenes and lighting conditions.
Microsoft Pix is not just for still photos. Fix also incorporates Microsoft’s Hyperlapse technology to turn any video into a high-speed time-lapse video. That works whether or not the video was shot in Microsoft Pix. Hyperlapse can also be used just to stabilize video at normal speed.
Microsoft also has its take on the Live Photo feature Apple introduced with the iPhone 6s. Microsoft’s “live images” represent the same general concept — mixing still and live imagery — but not the exact same approach. Microsoft’s versions are stored as short video clips and lack sound, but can be compelling in their own right, especially when the result fixes much of the action with only some of the image moving.
Unlike Apple Live Photos, Pix only takes live images when it thinks there is enough action to be compelling. Plus, Microsoft Pix works on the iPhone 5s, 6 and 6s, as well as newer iPads.
And while there are other alternative camera apps, one thing Microsoft Pix has on its side is that it seamlessly saves images and videos in Apple’s camera roll as well as in its own gallery. Microsoft says it does so by synchronizing its photo database with Apple’s native camera roll.
Microsoft says it isn’t done with Pix and has a bunch of things on its roadmap, including adding support for Android and slower-speed video, adding panorama capabilities and bringing over the beauty mode originally developed in the Microsoft Selfie app.
Here’s a look at a shot taken at the same time on identical iPhones using the iPhone’s camera app and Microsoft Pix.