What is Lemmings?

Lemmings is an API that uses simple spatial relationships to create remappable gestures for immersive experiences. Most gestures in immersive experiences are created using machine learning, which is very effective, but very difficult to adjust manually. Some gesture systems allow for gestures that can be adjusted but they rely on very complex libraries that might break with unexpected updates, making them difficult to update or transition to new projects.

Remapping is a fundamental requirement for making accessible experiences. It is the process by which a user can make an experience work for them, suitable to their unique needs. While this applies to users who might be missing a hand, or have a condition that makes traditional gestures difficult, the truth is that remapping benefits all users. Any gesture made repeatedly can potentially create a painful condition such as carpal tunnel. The ability to make adjustments to the interface ensures that users can make adjustments to maintain their comfort and joint health.

Lemmings, by its nature, is a work in progress, meant to be flexible and robust for a variety of projects, which is why it is open source and why this guide exists for people to navigate it more effectively to make adjustments and changes. As a single person, with a small team for support, there are limits to what I can accomplish, but collectively we can create better solutions for all.

What ISN’T Lemmings?

Lemmings is not a body tracking solution. It does not calculate the position or rotation of objects in space, such as fingers or hands. This is a feature and not a bug.

The reason Lemmings does not try and solve this problem is that there are many excellent solutions already in place, and some of them are supported by large teams of well-compensated engineers focused solely on this particular challenge. The problem is that these solutions do not always work for a particular project and they often rely on a wide variety of additional tools that all need to operate in concert in order to function. Switching from one solution to another is a cumbersome process and in many cases it is easier to rebuild a project from scratch than it is to manually convert each piece of code and each set up. Is this a problem for large teams and well-financed operations? Sure, but it is a much larger problem for small teams or even single individuals that might be trying to solve local challenges specific to their communities.

Why Lemmings?

Lemmings in nature are small rodents that travel in large herds. A common myth associated with lemmings is that they follow the herd so blindly that they will even follow it over the edge of a cliff. Lemmings, the software, operates in a similar fashion, only instead of following a herd, they follow the position of Unity gameObjects, such as the joints and other mapped features of body tracking solutions.

Even when a body tracking solution is the right one for a project, it is not rare for the solution, or one of its dependencies, to update in a way that ultimately causes complex problems. This is not solvable, but it is made more manageable, by having a tool that sits on top of such systems. Because the interactions and their corresponding logic are reliant on Lemmings, instead of the underlying solution, developers do not need to change their code, only the system that it is referencing.