Face recognition: an even more personalized experience

At Mirabeau and Cognizant we love to demonstrate our face recognition demo to visitors of our offices to show the power of today’s Cognitive AI services. It is an appealing demo that suprises and makes people smile. This concept has already been applied to different custom client implementations: e.g. we connected the technology with a coffee machine. Serving coffee with a smile ;)

In this update we took the demo to the next level. Based on feedback we added some neat features, made the experience faster and even more stable. In this blog I’ll explain how we made this new 2.0 version.

With help of my colleagues I started looking at different platforms that would provide possible benefits over the current Microsoft Windows setup. For this project we were looking for a compact, but visually powerful platform to run our AR-like software. I quickly found that the Android platform (yes, the phone OS) also drives numerous other devices!

One Android device seemed like a match made in heaven: Android TV boxes. These are used as an entertainment system and can provide “smart” functionalities for your TV. One pretty amazing feature is that they provide high-end specs including built-in GPU’s, at a relatively low price point.

After some research, we decided to order a Nvidia Shield and started developing. In addition we switched from Microsoft HD webcam to a Logitech Brio 4k camera. Although 4k is not yet supported by the Nvidia Shield this camera provides a better image quality and wider camera angle.

nvidiashield Nvidia Shield, great performance for about 200 euro

Faster hardware, a rebuild system, all to improve quality and create more interesting features

Technology Stack

The next step was to design the best fitting software stack for our use-case. Performing face detection and recognition requires lots of computing power which could be provided by our Android Shield, but there are also some great cloud services out there that can do this for you. After some testing, I concluded that the best option was a hybrid approach. Which means that we do the face detection locally and using a cloud service for the recognition part (e.g. detecting facial features like emotion, beard, glasses, make-up etc).

mirabeau facerecognition explainerboard High level flow of the face recognition demo

Google Mobile Vision + Azure Face API

To achieve a real time experience we do face detection for each camera. I decided to make use of the Google Mobile Vision library. This library is lightweight, easy to implement and gives the ability to quickly detect faces for each camera frame and tracks the faces while visible.

For the recognition of facial features the demo still makes use of the Microsoft Azure Face API which has been used for many different projects at Mirabeau with great results. So... whenever our application detects a new face in front of the screen, an Azure API call is made to return facial feature data and to perform similar face search query.

New UI

The new demo provides a general smoother and faster experience because of improvements in performance. Compared to the previous version the improved face detection shortened the time to recognize a person and accelerated the camera feed refresh rate.

Mirabeau Face Recognition v2 Face Recognition demo 2.0 with new styled employee info

Finally the complete UI has been redesigned to create an appealing more personal experience. Our designers came up with the idea to change the UI according to the recognized person’s job profile. In this way facial features of a developer are shown in program code and the lay-over of a marketing person contains social media elements. Very fun!

mirabeau facerecognition v2 UI Different User Interfaces for a personal experience

Face recognition demo for your industry?

The demo shows the true power of today’s Cognitive Cloud Services powered by AI and can easily be enriched with your own data. Several use-cases can be thought of such as supporting your employees with relevant data, like their next appointment. Attracting visitors to your booth at an event is another great example. And because the system is capable of logging statistical data, you could provide insights like the happiness level per day, top visitors, average age, etcetera.

Read this article on how we applied face recognition.