Now that voice is starting to become a mature platform, more people will start to interact with voice assistants. Gartner predicts that by 2021, at least one-third of enterprises will deploy a multi-experience development platform to support mobile, web, conversational and augmented reality development. In the future, many people will accept voice as a medium and expect it to be a platform through which they can connect to a business. Because of predictions like this, we looked into a fun use case with which we can teach the new generation about voice assistants. In this blog, I will talk you through the creation of our Guess Who voice assistant that we use during visits (pre COVID-time) at primary schools to teach children about voice and technology.
The origin of the Guess Who project started with a brainstorm for a good use case where we could show off the power of voice using machine learning and voice interfaces. Our team’s brainstorm started with an idea of a voice assistant that could recognize colleagues based on a description of their looks or projects that they worked on. This was an interesting use case because, we have a big company and it is difficult to meet every person or know them by their name, so describing a person, and getting their name would be very helpful in a call or meeting.
During the brainstorm, the idea of guessing a person based on their features became the concept that we kept revolving around and it did not take long for us to get the popular board game “Guess Who” made by Hasbro. Guess Who was a great choice, because it combines a recognizable concept with a very important ingredient, it is fun to play! Many people have played Guess Who at some point in their life, so we knew it grabs people’s attention and shows that you can use voice to create new experiences, even for a board game that is 40 years old.
From this point on, we decided to create a voice version of Guess Who. Now the next question we needed to answer was “For who are we going to create this?”. Creating the game using all our 250.000 colleagues was a fun idea, but not very practical. It would have taken us a lot of time to create 250.000 characters and most people in our company already know about voice, so it would not be the right audience, because we wanted to introduce voice to people who never used it before. This is why we decided to focus on children as our target audience for the assistant.
Children and games go hand in hand, so we knew this would be something they could enjoy. One of our team members opted to see if we could use this at schools. This would be a great opportunity to combine our project with an educational lesson for the kids and give us the opportunity to create a unique voice assistant, which could play games with a complete classroom. Because of this, we looked into opportunities to create an experience that would be fun for an entire classroom. Some features that we came up with were:
- Adding fun sounds to the game
- Give the children card cutouts so that they can play the game in their classroom.
- Customizable players so that a teacher could create personalized boards for the children.
- Include hobbies as customizable feature, so that children can get to know each other while playing the game.
- Many more..
At the start of the project, we decided to start small and focus on getting the voice assistant to work with a version of the board that we purchased. From that point, we expanded the features of the voice assistant to make it easier to use.
For this project, we used Google’s NLP Dialogflow; it offers a good integration with Google Assistant. Together with that, we have a back-end written in Typescript and use the Actions on Google SDK to control the conversation. At this moment, we use the Dutch language as the main language for our action, since our target audience are Dutch children at school. Besides the technical requirements, we also bought a real version of the game, for testing purposes of course.
The first thing we focused on was making the assistant understand the rules of the game. We did this by building a service that acts as a digital version of the player’s board. The service remembers which characters are still in play and removes characters when a user asks a question or guesses a name. At the start of a game, the assistant picks one character on the board and makes that as the character that you need to guess to win the game. When a player guesses the correct character’s name, the game ends and the player can start a new game. At this moment, the assistant does not ask questions back, so you can only play a game where you are guessing which character the assistant has in mind. During the game players have to ask the assistant about features of characters on the board. Because there are many different ways to describe people and the things they wear, we took a lot of time to teach our voice assistant to understand all the characters features. We did this by building an extensive description entity in Dialogflow.
The description entity represents a description that a player can use to ask the assistant a question about a character on the board. Some examples of this could be:
- Is the character wearing a hat? (accessories)
- Is the characters nose big? (body part & size)
- Is the character a girl? (gender)
Using this entity, the player can ask a number of different questions and use sentence variations to play the game. Thanks to the example phrases, Dialogflow can convert user input into this description entity during the game. The player’s digital board uses the description entity and tries to match it with the character that the assistant has in mind. It checks the property of the character, informs the user if it matched the character in mind, and after that, it will remove any characters from the players digital board according to the rules of the game.
Besides the technical challenges of making the assistant understand the rules of Guess Who, one of the biggest challenges during the build of the game was an ethical challenge. Due to the nature of a game such as Guess Who, you spend most of your time describing characters and looking for common features between characters. During some early tests, we noticed that children use skin color as an easy way to get characters off the board. Most of them asked questions such as “Is he white” or “Does he have black skin”. While these are effective sentences to use during the game, we did ask ourselves if it is okay to teach children to think this way. The children themselves did not seem to have a problem with the questions they asked since they are playing a game, but we could understand if a teacher or parents would take offence to the game if we teach kids you can win by asking questions like that.
We decided to teach the assistant only to respond to political correct phrases such as “Is he Caucasian” when it comes to describing a characters race or skin color. We want this game to be fun for everyone and we do not want to teach kids that it is okay to describe people like that, as it might upset others.
After a challenging build process, we managed to create a version of the board game that you could play with the voice assistant. It allows you to play the board game without having the need for a second player. You can see an example in the below video.
Now that we made an amazing and fun voice assistant, we wanted to share it with as many people as possible, but before we could do that, we needed to see if others like the assistant as much as we do. We contacted the local primary school, gathered a good amount of energy and went to see if children enjoyed our voice assistant during a couple fun mornings where we talked to them about the internet and security. We ended each session with a couple games of Guess Who together with the children.
We created card cutouts of all the characters that you would see on the board and handed all the children a character so that they would becoming a living Guess Who board. The teacher would choose someone who could ask Google a question. The assistant would hear the description and then instruct the classroom who should sit down because their character was not the character that Google had in mind.
The children loved playing Guess Who and the word about the assistant quickly spread through the entire school, so much that the next morning cheering kids awaited us in each classroom. They heard about the assistant and were excited to be able to play Guess Who too.
Children from primary school De Stapsteen. Due to privacy reasons, the faces of the children are covered
After a couple of successful and very fun mornings at a local primary school, we can conclude that the children loved the Guess Who assistant. We learned a lot about how familiar young children are with voice assistants and that they interact with them quite frequently. Having the possibility to test the voice product with school children has been an eye opener. Children interact with assistants in such a different way, so being able to see what that is like was a lot of fun.
Now that the first version of Guess Who has been a success, we are looking into the next steps. We are looking into new features and ways to use the assistant. So stay tuned for more!