This was just a fun little project I made. Currently, the website doesn't work on screens less than 1200x728 (Sorry mobile users!) It also might struggle on lower end devices.
For best results, have a webcam pointing right at you. I tested my website with a MacBook camera.
Any comments, questions, or suggestions are greatly appreciated!
blog: https://blog.andykhau.com/blog/eyesite
check it out: https://eyesite.andykhau.com/
- Papoutsaki, A. et al. (2016). "WebGazer: Scalable Webcam Eye Tracking Using User Interactions." Presented at the International Joint Conference on Artificial Intelligence (IJCAI). https://www.ijcai.org/Proceedings/16/Papers/540.pdf
While we are at it, you may also find the following research publications relevant to this discussion:
- "Improving User Perceived Page Load Times Using Gaze" (USENIX NSDI 2017) https://www.usenix.org/conference/nsdi17/technical-sessions/...
- "No clicks, no problem: using cursor movements to understand and improve search" (Huang, White, Dumais from Microsoft Research, ACM SIG CHI '11) https://dl.acm.org/doi/abs/10.1145/1978942.1979125
- Virtual gazing in video surveillance (SMVC '10) https://dl.acm.org/doi/abs/10.1145/1878083.1878089
1 - Calibration. Looking at static dots is BORING. The best idea I've seen is Tobii's gaming calibration where you look at dots to make them wobble and pop. This makes the whole process feel like a game, even when you've done it a hundred times before. I would love to see more ideas in this space to give a much more natural-feeling calibration process - even better if you can improve the calibration over time with a feedback loop, when users interact with an element.
2 - Gaze feedback. You are absolutely right that seeing a small, inaccurate and jumpy dot does more harm than good. Again, Tobii have led the way with their 'ghost overlay' for streamers.
For an example, see the following video. After calibration the ghost overlay is used to give approximate feedback. This is enough that some naive users are able to make small adjustments to a constant calibration error, or at least give feedback that the gaze is wrong, not that the UI is not responding.
1 - I experimented with some calibration involving staring at a point, but I found it troublesome as blinking would make lead to some inaccurate calibration data (webgazer doesn't have blink detection). It was also a little more fatiguing since the user would have to really focus on staring the entire time. I found that it was less mentally fatiguing if the user could control their own calibration give themselves room to blink or just rest their eyes for a second.
2 - Ghost overlays is a really good idea. I'll see what I can do to implement that feature.
I really appreciate you taking your time to write this!
The challenge is recording and syncing the motion at a higher frequency and being able to save without much drift and the performance of these landmark/gaze models is often slow.
One more option to speed it up is not to do the eye tracking at record time, just record a crop video of the face and the screen first at 60Hz and then run the model on each frame and update the metadata of the dataset.
Why?
cwmoore•1d ago
jeffhuys•1d ago
noduerme•1d ago
People who make products and sell services need to advertise. They in turn pay taxes. The many layers of parasitism in the advertising world historically relied on conning these people and taking their money in exchange for an unprovable proposition, namely that if you run this ad we tell you to run, right here, your sales will go up - but we'll never be able to actually tell you for sure by how much, or whether it was a good deal for you. From that perspective, more and better viewership data helps undermine the advertising bullshit machine and close the gap between people who run businesses and the people they're trying to sell their services to.
jll29•23h ago
Italians deploy fearsome SPY MANNEQUINS to win Fashion Wars (2012) https://www.theregister.com/2012/11/22/bionic_mannequins_are...
According to the media at the time, the mannquins quickly disappeared again after the scandal broke. If you are more cynical, you might question that narrative.
noduerme•23h ago
catlifeonmars•22h ago
Do they though?
No one is entitled to better data. It should be on the advertisers to figure out more privacy preserving ways of getting feedback.
Ubiquitous technical surveillance is, in economics parlance, a negative externality.
jeffhuys•15h ago
I have epilepsy. It's managed, I mean, I'm allowed to drive again. But what about people who don't have it managed (for some no meds will work)? They're just f'd when they walk down the street and Nike NEEDED to switch the screen every 0.2s?
If I can in _any_ way inhibit their ability to grow, I will.
If I need your product, I'll find it. I'm against any and all ads - I know it's unrealistic, but I'll do everything in my power to lower the amount of ads I see or to be a nuisance to them.
Really don't care about morality in this case.
pixl97•14h ago
Heh, only 30 years behind the US.
Of course our rate of people dying at younger ages over here seems to be a lot higher too.
pzo•12h ago
- sound of phone ringing to keep you out of your zone
- loud annoying music with annoying lyrics
- putting cat in advertisement and meowing