In Košice, the second largest town in Slovakia, resides a small innovative company called Matsuko. Just a dozen or so experts from the video game industry, machine learning and mixed reality bring into life something, which up until now was seen in science-fiction movies: they bring humans into 3D. Their aim is to make revolution in remote communication between people.
You can chat with your partner or employee through Skype, but it is not so personal. People live in 3D world, so they prefer to communicate with 3D person. Using human hologram could be a totally new experience.
“It is like communicate with somebody real in the room not through the PC screen,“ Mária Virčíková, the leader of research and development team in Matsuko, explains for InnovateCEE.
Join knowledge
Virčíkova was a researcher in the field of artificial intelligence and machine learning at Technical University in Košice. She met Matúš Kirchmayer, Matsuko CEO, when he became a tutor for her research project Emomime which was 3D avatar (virtual character), that was meant to replace mannequins in shop windows. Kirchmayer’s background is the in development of video games where he used virtual, augmented and mixed reality.
The influence of video games and Japanese culture also helped creating the name of the company.
“Matsuko is Japanese girl name, but we like to play with words and my first name is Matus,“ Kirchmayer explains the name of the company.
“The project Emomime was focused on advertisement, but I want to do something more that ads,“ explaining her decision to to leave the academic soil and join the company Matsuko in 2015.
However, the company was established in 2010 and since then young entrepreneurs have tried to bring various 3D characters to life. They managed to install a mixed reality dinosaur hologram that can interact with visitors in the Zoo in Bordeaux; another animal was a 3D penguin in a toy shop that was able to entertain kids.
“Now we are focusing solely at human holograms where we combine 3D and artificial intelligence,“ Kirchmayer claims.
The challenge ahead for the company was to find the solution to three questions: How to create human hologram? How to transmit it in real time? How to show it in other parts of the world? They have already solved the first problem-creating a3D hologram. Now they are in the second stage – transmitting it in real time.
Trained network
Their technology is based on a developed method of how to train neural networks to capture and reconstruct the 3D head and body from a single 2D image. Neural networks work like the human brain, it is basically a software with artificial intelligence that can learn new things from experiences. Matsuko experts developed software and trained it, to learn how to quickly extrapolate the shape of a face from a camera stream. If the camera does not see all your head, it can image how it should look based on experience. It is like a machine learning process.
Usually to get good quality 3D object you need an expensive hardware such as many cameras shooting the object from many different angles or special depth sensors.
“We use just a single web camera or mobile phone camera that can capture the human body and motion as well,“ Kirchmayer explains the advantage of the company in comparison to its competitors, among which are big brands such as Microsoft.
Matsuko have already managed to achieve photorealistic quality in 3D.
“Imagine talking to your friend or colleague in 3D as if you really were with him at the same place,“ Kirchmayer says.
Commercial applications
The human holograms can be used in many areas and they can be monetized as well. Images in 3D are suitable for video games, remote coaching or medicine, or for telepresence in companies. Holograms can become a tour guide in museum or shop assistant in retail.
In cosmetics it could work like a good marketing vehicle. The make-up company can recommend you some treatment just according to your picture. Hologram or 3D face will be made from your picture and professionals can present live to their customers how their face will change, a similar approach can be used in plastic surgery. Matsuko have successfully integrated their high-quality 3D face reconstruction technology into the cosmetics segment.
Challenges ahead
Now the company from Košice owns the technology to deliver very good rendering quality of image. Next challenge is to transmit it in real time.
“We are now working at optimizing the speed of communication,” Vircikova says.
Matsuko is also constantly rising money via crowdinvesting and business angels.
“We are looking for funds not just in central Europe, but in the West,“ Kirchmayer says.
Another challenge is the hunt for qualified people.
“We are looking for experts who have lust for building the company with us. There are not many of them at the market, but there are some,“ Virčíková.
Therefore they are looking for experts also outside of Slovakia.
“ We have good experience with westerners who like the dynamic of our country and small company,“ Kirchmayer.
The Slovak company is on the path to making communication between people more personal. Transmitting humans in 3D holograms does not exist, so if they manage to do it, it will be a breakthrough in this field.
