IRL this has been somewhat tried with the Humane AI pin, with gestures, less than instantaneous response is noticeable, so latency has to be paramount.
Don't mean to rain on the parade as it's a nice piece of work, but comparing to say chording (low cpu requirements, better accuracy, faster, but you have to learn how to chord) shows the different tradeoffs.
By anchoring virtual buttons to your physical hand, your proprioception kicks in and you can rapidly and accurately hit keys. It's pretty much trivial for most humans to tap the same spot on their palm without even looking, but most people can't do this with an arbitrary spot in free air.
The VR/XR industry has tried floating virtual keyboard many times in basically every form you can imagine. All attempts have been abandoned because they suck really bad. This is an idea that is fundamentally incompatible with human psychology and physiology.
rspoerri•8mo ago
But completely unusable for any real application where you'd do a lot of typing.
CGamesPlay•8mo ago