Months after Microsoft released a tool designed to guess the age of someone based on a photograph, the company has unleashed another round of fun tools designed to show off its machine learning and AI capabilities. The most appealing tool in the new batch promises to detect what emotion someone is feeling in a particular, photographed, moment, based on cues in their facial expression.
Since every photograph of me contains the primary emotion of “I dislike being photographed,” and that’s not one of the emotions Microsoft programmed into the tool, I tried it out on an animated GIF of The Doctor gazing sadly through the rain:
The tool certainly recognized that the poor Time Lord was sad, but incorrectly concluded that he was more neutral than anything else. Let’s try it on a still photograph from a different fandom, with a different emotion:
That’s … well. Okay. Much clearer.
But why? Well, these APIs are designed to show off Microsoft’s growing AI and machine learning capabilities for developers who might be interested in incorporating them into their own projects. The age guesser, and this month’s Movember-themed facial hair analyzer, are supposed to show how developers might be able to get creative in using these tools, and the range that’s capable here.
And, for users to opt in, Microsoft says that some of the tools can improve themselves using your input.
Microsoft does not claim ownership of the materials you provide to Microsoft (including feedback and suggestions) or post, upload, input or submit to any Services or its associated services for review by the general public, or by the members of any public or private community, (each a “Submission” and collectively “Submissions”).
However, by posting, uploading, inputting, providing or submitting (“Posting”) your Submission you are granting Microsoft, its affiliated companies and necessary sublicensees permission to use your Submission in connection with the operation of their Internet businesses (including, without limitation, all Microsoft Services), including, without limitation, the license rights to: copy, distribute, transmit, publicly display, publicly perform, reproduce, edit, translate and reformat your Submission; to publish your name in connection with your Submission; and the right to sublicense such rights to any supplier of the Services.
To try and cool some of those concerns, Microsoft’s outward facing machine learning sites — like How Old — now contain a note stating that Microsoft doesn’t keep the photos you upload, unless you check a box and give them permission to do so — which they’re calling an opportunity to “donate” to science.
Although programs like these learn by processing more and more data, Microsoft says that not every photo uploaded to one of these tools is used for that purpose. “The APIs don’t get better just from people using them. However, if people opt in to letting us keep their photos, we would like to use those photos to help us improve,” Ryan Galgon, senior program manager for Project Oxford, said in an e-mail to The Post. Otherwise, Galgon said, the company doesn’t keep the photos you upload, or use them to improve the service.
That will work a little differently when it comes to independent applications that developers build using these APIs, a spokeswoman for Microsoft said, because what they choose to keep from users of their own apps will be mainly up to them.
“We have a Code of Conduct to help developers understand our expectations about protecting people whose data they collect and legal terms to help us enforce good behavior if necessary,” spokeswoman Melissa Hovis said in an e-mail, “Our legal terms also require developers to get consent.”
Those independent apps could very well keep the data they collect on you for any number of purposes, but Microsoft said it’s taken steps to avoid privacy violations on their end. “Microsoft has processes in place that help de-identify user generated content in audio, video, images and other data when it is processed by Project Oxford APIs,” Hovis added. “For example, any images that are provided by an application to the Face API are stripped of any metadata, session data or IP addresses to protect the privacy of people in those images.”
When more developers start using these APIs, though, Microsoft might have a bigger pool of user-submitted information to learn from. Microsoft “may use the data” sent from an independent app to the API to “perform and improve our services,” the company said.
If you want to look at the other tools they’re working on — including some related to speech and spelling, you can look and demo here.
Liked that? Try these!