Developing a video streaming plugin for UE4 that allows simultaneous playback of several high-resolution video streams. The plugin is used in a VR application called Headwall created by ITI Systems. Headwall is a command center video wall virtualized in augmented reality.
Integrating eye tracking features into computer games
Designing and implementing various eye tracking features such as Extended view, Aim at gaze, Interact at gaze, and others in games based on Unity and custom game engines.
Idea, development and software architecture for Tobii Ghost and Twitch Extension
Tobii Ghost allows content creators to overlay their gaze direction on the video stream. The gaze bubble is either getting blitted into a video on the streamer's side or rendered in the viewer's browser as a Twitch video overlay extension. The project included rendering in DX11/WebGL, creating a client application, and Twitch extension frontend and backend. On this page, you can find a list of streamers who are now live on Twitch with Tobii Ghost Twitch Extension:
The software is widely used by content creators on Youtube and Twitch, and you can find lots of videos with multi-million views by searching for "eye tracking" on Youtube.
Tobii Ghost is also used in many esports tournaments including ESL, ELeague, WESG, United States Nationals and more. https://gaming.tobii.com/broadcasting/
Developing Tobii Game Hub and Unity mods
Tobii Game Hub is a software that adds eye tracking features to PC games without doing source code integration. That is achieved by using open protocols and creating custom mods for Unity games. The team created 100+ mods for Unity engine-based games.
Developing Tobii Unity SDK
Designing the API and creating demo scenes for Tobii Unity SDK.
Tobii Game Analyzer is a discontinued tool that allows analyzing player's gaze behavior through their gameplay sessions in Starcraft 2 and Dota 2. The software is displaying an HTML/JavaScript overlay inside the game. It is deeply integrated into the game's replay functionality through the use of binary injection. Rendering is done using DX11 and CEF.
Later these analyticial concepts were reworked and integrated into Mobalytics.
Mobalytics can perform a much broader analysis based on not just what the player did, but what they were actually looking at during each second of the match.
If you know where someone is looking, it’s the closest thing to knowing what they’re thinking.
Also:
Generating product ideas and driving them to MVP.
Fast prototyping new concepts and demos.
Generating leads for software partnership/game integrations.
Supervising student projects.
Developer support on Tobii Developer Zone.
Helping out with Tobii Eye Tracker reviews in media.
September 2013 - July 2021
Summer Worker
Scania
Development of pedestrian tracking and collision warning algorithms.
June 2013 - August 2013
Master Thesis Student
Scania
Development of a pedestrian tracking algorithm, which utilizes data from a ToF camera.
September 2012 - March 2013
Web Developer
Studio 84
Management of website and design projects.
Website development (Joomla/Magento).
April 2009 - March 2013
Research Engineer
Kungliga tekniska högskolan
Working on an international robotics project CogX Dora by testing, debugging, and implementing new features in navigation and AI.
I felt like Superman annihilating things with my eyes. The eye-tracking was fast and accurate and felt natural.
User can just use their glance to target enemies. The player looks where they want to shoot and simply presses the RT button to fire with no need to use the scene changer joystick for aiming.
Hearthstone Eye Tracking Mod
This mod allows you to play Hearthstone handsfree with your eyes using Tobii Eye Tracker. #AccessibilityInGames
Faeria Eye Tracking Mod
This mod allows you to play Faeria handsfree with your eyes using Tobii Eye Tracker. #AccessibilityInGames
Rusteroids
Asteroids game made in Rust.
Education
Kungliga tekniska högskolan
Master of Science (MSc)
Systems, Control and Robotics
Areas of interest: digital image processing, artificial intelligence, robotics, control systems, programming.
The video shows how the mazes-solving robot wins silver in the final competition. The project included the development and implementation of AI, localization, navigation, and image recognition algorithms, as well as the development of robot hardware.
2010
ITMO University
Master of Science (MSc)
Automation and control
The video shows the sokoban game solver robot. The project included the development and implementation of artificial intelligence and navigation algorithms, as well as modeling in the Webots software package.
