People tracking has been a major topic in recent years, especially because people are more worried about their privacy than ever. This is why a more privacy secure method, e.g. an approach where no identification takes place, is very desirable. Realizing this in a cost-effective IoT solution will enable many applications, including security, investigating customers movements, and managing the distribution of employees over an office building. In this blog post I examine the latter: the goal is to count the number of occupants in each meeting room so that the space can be used more efficiently.
It’s a common problem in office buildings: you want to book a room for your meeting, but all suitable rooms are booked. However, not every room is actually occupied. Or, the room large enough for your team is occupied by a single employee. Inquiring which rooms are actually occupied and by how many occupants can resolve this issue.
Nowadays there are many ways to detect people. Cameras are excluded because they collect far too much data about the detected. I propose a method with a far infrared thermal sensor array, or simply put, a thermal camera. To keep the cost low and the possibility of identification to a minimum, a sensor with a very low resolution is used. The data produced by the sensor is read in and processed by a single-board computer, in this case a Raspberry Pi.
The data is first converted into a simplified image on which computer vision algorithms can be applied. The computer vision library OpenCV offers many image processing features for this. A gaussian filter is used to remove some of the noise in the picture which is produced by the sensor. A false color map can be applied to display a heatmap.
Both pictures represent the same situation where a person stands directly in front of the sensor. The image on the left represents the data taken directly from the sensor. This is processed into the picture on the right. Using a colormap a more recognizable image is produced.
The peaks in temperature are detected. The color map won’t be used for this, as the colors don’t actually represent anything: it’s just a way to make the image more recognizable for the human eye. Finding the peaks has a number of benefits: the size and intensity of an object are not relevant (this is greatly affected by how far away a person stands from the sensor), the center point of each object is easily determined, and making a distinction between people who are close to each other becomes surprisingly easy.
Detection on blobs varying in intensity
People standing right next to each other
Below you see the thermal image of four people entering a room and what that looks like after applying some filters.
Using the image below, the software determines where the peaks are. After finding contours, the center of the most inner contour is used as a location of a heat source.
To demonstrate the solution, a camera is added so that observers can see whether they are being detected. Keep in mind that this takes up a large part of the performance: without camera feed, the device refreshes about three times more frequently, removing many false readings and enabling more filtering. Also, an estimation of their body’s surface temperature is displayed. The temperature has to be calibrated with the distance from the sensor to the source, so this is not very accurate yet.
One of the challenges is detecting people standing behind other people. This can be partially solved by placing the sensor as high as possible in the room, so there is less overlap. This is of course limited by the ceiling height, so in rooms with a low ceiling the readings might be less accurate. The used sensor utilizes a wide-angle lens with a 110 degree horizontal viewing angle. This means that, when the sensor is placed near the corder and appropriately aimed, an entire rectangular room can be captured.
Other warm objects can be detected as well. Think of laptops, phones or your coffee! The system can be extended to detect items that have been left behind. The device can also serve as a platform for more sensors: temperature, humidity, sound level and air quality sensors can easily be added. Gaining knowledge about these parameters can help users choose a suitable room for their meeting, or find a workplace that fits the users personal preferences.