If you’ve ever wanted to cook in the kitchen from Ratatouille, travel to the world of Harry Potter, and even in the past during the reign of Peter the Great and enjoy a journey in your favorite universe, you can now do that with projects developed by students and graduates of ITMO’s Faculty of Software Engineering and Computer Systems. Read on for backstage development stories inspired by virtual realities and cartoon-like sets.
Multiverse Kitchen VR app
https://www.youtube.com/watch?v=BoZNMx6NJrI
Created by Oleg Fedyukovich:
"Everything you do in Multiverse Kitchen happens in Paris, in a cozy small flat belonging to Alfredo Linguini, a young chef from Ratatouille (2007) and Remi, a rat and also the main character of the story. However, this flat is also located in the multiverse, where different worlds are linked by portals, which is why here you can cook Krabby Patties, Minecraft cakes, or any other dishes from your favorite cartoons, games, series, or anime.
I chose a cooking game as the best setting to demonstrate how haptic gloves work, seeing as we rely on fine motor skills during our kitchen experiments. Haptic gloves and other similar force-feedback devices bring another dimension to virtual reality, complementing the traditional image and sounds of VR headsets. When you are wearing the gloves, they stop your fingers in the shape of the object you are trying to grasp, thus making you feel that you are holding it.
However, the project had gone a long way before I could implement the gloves into it. First, I developed the first level for the game and artist Polina Golubova created the setting. She has really done a great job: after the final rendering, I started to think our graphics were better than in the original cartoon. When this groundwork was done, I assembled the prototype of my gloves: chose the configuration I needed, downloaded the blueprints, radio circuits, and software, tweaked them, printed them out, soldered them together, and calibrated them. At first, the gloves worked with a test app from LucidVR, so my next step was integrating force feedback and finger tracking into the app. When everything was running properly, I still had enough time to develop a second level for the game.
The game is now fully playable, but there is still room for improvement – for instance, I could add more levels. As I was working on the project, I reached my goals of acquiring the experience of working with such technologies, as well as the knowledge I can use in the future. For now, I am not planning to develop the game further."
Fireworks on January 1, 1704 – a VR location
Created by Anastasia Bagritsevich and Maria Levkovskaya.
Anastasia shares:
"When I saw the topic of my groupmate Maria Levkovskaya, I got really interested and decided to join her project. I had worked in 3D Max and Blender before, so I was curious to develop special effects and fireworks animation for the location. Moreover, I was excited to learn more about this historic period, which I wasn’t really an expert on before.
Our project allows users to see what fireworks were like back in the day. At the heart of the VR location are over 20 street lamps with different images. If you approach them, you will see glowing candles inside, just like in real life. Apart from watching a colorful fireworks display, users will have the chance to learn some interesting titbits integrated into the location as fact notes.
We decided to share responsibilities within the project. While I was responsible for finding fireworks references to develop the animation on Unreal Engine 5, Maria was responsible for the location’s 3D model. Even though the project is complete, we can still polish it further by making the fireworks interactive, tweaking the camera settings to make visual perception more realistic, adding more detailed animation, and so on. In other words, we can implement nearly anything we want!"
Harry Potter VR Game
https://www.youtube.com/watch?v=xHYvMzB2Ufs
Created by Anastasia Tsitelova and Artur Dzyuba, students of ITMO’s Faculty of Software Engineering and Computer Systems. The game recreates the Burrow, the home of the Weasley family, in VR.
In the game, the fans will find many Easter eggs with allusions to the franchise. However, the game will also have something for those who just love everything magic and fantasy. Users will get to play for Ron Weasley, who gets a letter from his brothers, the twins Fred and George, asking him to solve puzzles and send various objects over the floor network (a magical communication system connecting wizards’ fireplaces) to the Weasleys' Wizard Wheezes shop. In the end, players will learn a special new spell for all their efforts.
“For me, personally, the Burrow always felt like a place of warm family gatherings and pleasant buzz. That’s why I decided that the objects our players will have to find will be items from Weasleys' Wizard Wheezes, a joke shop. Each of these objects is unique and produces a certain effect when touched by the player. For instance, a yo-yo will squeal like a pig and the Peruvian Instant Darkness Powder will, well, cast total darkness for several seconds,” says Anastasia.
Gesture and voice control are two of the game’s main game mechanics. For example, hand tracking technology allows players to pick up objects and move between rooms by pointing up or down with their left hand. Some objects in the puzzles are hidden in locked locations. To open them, players will have to first find plaques with spells strategically placed all around the house. Players will be able to cast any of the game’s five spells by conjuring up their wand with a specific gesture and then drawing symbols with it in the air.
After a player has collected all the objects, they will have to send them back to the shop by pronouncing its name into the mic (just like when using the “actual” floo powder). The game is completed when all the objects have been successfully “burned” in the fireplace.
Completing a project in VR isn’t an easy task, so Anastasia was joined by Artur Dzyuba, a first-year Bachelor’s student in the Computer Technologies in Design program and an employee of the Center of Usability and Mixed Reality.
The students decided to divide their responsibilities. Anastasia was responsible for the game’s concept, storyline, and location. In developing the visuals, she relied on the books and films, as well as sketches from Brian Sibley’s book Film Wizardry, and Harry Potter Wiki. However, as not every corner of the house was described or shown in the films, Anastasia had to rely on her imagination to make the space feel whole and real.
“One of my main challenges was fitting together the information from all the sources. For example, the interior of the Burrow greatly changes between the films. We also needed to find a way to make the location recognizable and authentic, while keeping in mind that we couldn’t transport all of it into VR. In my pursuit of authenticity, I searched the books for Ron’s childhood nickname, mentioned only once in the series, and relied on actual Weasley twins' quotes to make the letter as close to the one they could’ve written as possible,” shares Anastasia.
Artem Smolin, head of ITMO’s Center of Usability and Mixed Reality and supervisor of the project, helped the students with some of the game’s puzzles and features. According to Anastasia, thanks to their collective input, the game came out engaging and fun to play.
In the student duo, Artur Dzyuba was responsible for developing the game mechanics. He experimented with different technologies to see how they would function and how much time they would take to implement. After settling on the concept and technological aspects, the team started testing them “on location,” then moving on to polishing the details.
“Every part of the game’s mechanics, starting from transporting the model to VR and all the way to voice control, was done by me. I think that it is precisely the combination of voice and gesture control that allows the player to experience the atmosphere of witchcraft and wizardry. Moreover, it’s canonical, according to the books and movies. The hardest part for me was to tackle voice control,” says Artur.
According to the students, even though the project is complete, there is still room for improvement. In the future, they are hoping to release the game to a wider audience.
Made with the help of ITMO.NEWS
