Making data visible using AR

Gizing Khalandi

Our offices these days are filled with sensors that produce a constant stream of data that gives us a deep understanding of usage and conditions. However, we are not yet using this data to its full potential. My mission was to give people real-time insights into the health and comfort of any given room in our Digital Studio. To make it relatable (and exciting) for people, we used AR to create a digital-twin persona for each room. Maccy was our first prototype and you can experience it now! This is how Maccy came to be.

Environmental data like the amount of CO2, temperature and sound level can only be accessed through clumsy dashboards by experts like building managers. Getting to true insights about the comfort level of a room and being able to act in real-time is near impossible. Data is hidden, and isn't smartly integrated into a full circle feedback system. So.. what if you actually can see the data?

Drawing with Gizing and Rosti - Mirabeau

How it started

During a previous project, Pjotr made a Sensor box for every meeting room on the first floor of our office. This Sensorbox can detect humidity, luminosity, CO2 levels, sound levels, and temperature. Our challenge was: how to present that data to our facility managers and room visitors at a glance, in real-time.

To give you an idea where my coaches were thinking about: what if the room itself could talk to the people in the room and show clear actionable information. So, when people have been workshopping for hours, you can imagine that the temperature and CO2 levels are high. Time for the room to say: "Let's crack open a window and have a little break." That might be a bit intrusive, so instead of that we decided to visualize the data in a fun and actionable way: VR and with a personality per room.

Making data visible using AR - interns of Mirabeau

A collaboration between design and engineering

Together with Rostislav, I designed an Augmented Reality (AR) interface for the meeting room called “Macintosh”. We started with research over the Sensorbox, Augmented Reality and the employees. After this, we noticed that most employees book the same rooms because of the acoustic, capacity, light and they just need room to discuss sensitive information.

Every week I made sketches of how the data should present itself. Normally this is a lengthy process, so that's why we decided to do a Design Sprint. In it, we went to Dutch Design Week to get inspired by other projects. I made a mood-board and made sketches on see-through paper to try out AR concepts without programming.

For the meeting room Macintosh, we created a personalized AR assistant in the same style as the interface of the first Macintosh. This assistant can inform you about the comfort level of the room and how you might be able to improve it.

Making data visible using AR - Mirabeau

Learning and building a prototype

At first, we tried out a 2D prototype of this idea to see if the communication was clear. After three different ideations, we had our final concept, ready to be tested with people. When we were testing the demo we saw that people couldn't make sense of the prototype. Based on the feedback we made a priority list and re-engineered our prototype.

  • We made the data boards more clear and gave it more context
  • We split up the experience in three ways to see the data
  • We introduced features to make the application smarter
  • And we introduced pop-ups for urgent messages

The final version

The final version of Maccy can tell you about the current quality of the room and how you can improve it. After downloading and opening the application, you get an introduction and mini-course through a couple of popups.

There are three ways to see the data in our room:

1. Quick tips: The assistant gives tips to improve the quality of the room environment.

2. Dashboard: This mode provides visualized data and an overall comfort score. The dashboard is projected in front of you and follows your view, no matter where you look. By tapping on the screen you can move the dashboard.

3. Explore: In this mode you can walk through the room and discover various data points! To help you we use arrows to guide you. To make the experience as easy as possible the data will increase its size when you gaze at it for a while. You can also use the classical pinch to zoom.

If conditions such as temperature, or CO2 levels change rapidly, the assistant will give you suggestions to improve the comfort level. This way the people inside stays healthy, happy and productive.

Making data visible using AR

Next up

In the future we're going to create a custom made personality for the assistant of every room based on its name and location. This makes the experience for each room a bit more fun and interesting.