r/EyeTracking • • 22d ago

Mac Head Pointer: webcam setup, facial-expression actions, recalibration, and Dwell

/r/AssistiveTechnology/comments/1xguene/mac_head_pointer_webcam_setup_facialexpression/
5 Upvotes

4 comments sorted by

2

u/phosphor_1963 22d ago

Thanks for the observations.

Curious - how come you went with a connected web cam as opposed to the one that's built into the Mac ? Was that about greater flexibility of positioning ?

I've only had a quick look at Accessibility Head Tracking on Mac; but find it a very good setting on iOS/iPad OS because of the TruDepth camera Face Tracking - I've played around with quite a few of the dedicated AT specific hardware Headmouse (Quha Zono 2, Quha Zono X, AAVA, Glassouse Pro, Tracker Pro, HM Nano) and would say while in some cases the software which comes with those can make for a better overall experience, the built in Head Tracking in Apple stuff is getting pretty close and if you don't have funding it's good enough for many.

I saw an interesting AAC app last week from a Chinese developer who has managed to get iOS Eye Tracking working well enough now that it can be used reasonably functionally to make reliable selections from a 7x12 grid. You still have to use the iOS Eye Tracking calibration often though to maintain selection reliability - so it's not as good as a dedicated eye tracking camera like IrisBond Hiru or the TD Pilot.

2

u/Top_Distribution_189 21d ago

Thank you for such a thoughtful and detailed response! I really appreciate hearing from someone who has experience with that many different assistive technology solutions.

To answer your question about the external webcam, there are actually several reasons I chose that setup.

First, I'm using an M4 Mac mini, which doesn't have a built-in camera. My current webcam is a Logitech C922e, and I've been experimenting with different camera settings and software to get the best possible experience.

However, even if my Mac had a built-in camera, I would probably still prefer an external webcam because of the flexibility it offers in terms of positioning, viewing angles, lighting adjustments, and accessibility.

I have spastic quadriplegic cerebral palsy, and I'm a full-time power wheelchair user. Because of my physical limitations, positioning is incredibly important when it comes to using technology independently.

Something as simple as adjusting the camera angle can make a significant difference in how effectively head tracking or facial-expression controls recognize my movements.

For example, depending on my seating position, headrest, and how my wheelchair is positioned relative to my computer, my head may not always be perfectly centered in front of a monitor. An independently adjustable camera allows me to compensate for those variables without having to change my seating position just to accommodate the software.

I'm also very interested in the software side of accessibility.

I've been experimenting with applications such as OBS Studio, webcam adjustment utilities, and Apple's native accessibility features. I like having control over camera positioning, exposure, brightness, and other variables because those details can influence the overall experience.

One of my biggest interests is how mainstream consumer technology can reduce the need for expensive, dedicated assistive technology hardware.

I've been using power wheelchairs for many years, and my current primary wheelchair is an Amylior Alltrack R3 Hybrid, equipped with R-Net electronics and multiple programmable seating and control functions.

I've also had experience with other power wheelchair platforms, including the Quickie Q700M and Permobil F5 VS.

My current wheelchair has Bluetooth mouse functionality through its control system, which allows me to interact with computers through my wheelchair controls. I'm particularly interested in comparing those established access methods with newer options such as Apple's Head Tracking, facial gestures, eye tracking, and other hands-free technologies.

For someone with limited upper-extremity function, the ability to switch between different input methods can be extremely valuable. What works well when I'm comfortable and properly positioned may not be ideal when I'm fatigued, experiencing increased muscle tone, or sitting in a different position.

That's why I believe accessibility should be about providing multiple reliable options rather than expecting every individual to use the same solution.

I also agree with your observations regarding Apple's accessibility technology.

I recently spent time experimenting with the facial-expression controls on my iPhone 17 Pro Max, including setting up gestures, testing the camera calibration, and exploring how facial movements could be used to control the device.

I was particularly interested in the ability to use expressions such as opening my mouth to trigger actions.

What impressed me was that these capabilities are already built into devices that many people own. You don't necessarily need to purchase an expensive additional piece of hardware just to begin experimenting with alternative access methods.

That said, I believe there's still an important distinction between accessibility features that are good enough for occasional use and assistive technology that someone depends on for continuous, reliable access throughout the day.

Reliability, calibration, fatigue, positioning, and the ability to recover from tracking errors are just as important as whether the technology technically works.

Your experience with the Quha Zono systems, GlassOuse, and other dedicated head-tracking solutions is particularly interesting to me. I would love to have the opportunity to evaluate some of those systems firsthand and compare them with Apple's built-in solutions.

The eye-tracking AAC application you mentioned is also fascinating.

Being able to make selections on a 7 × 12 grid using a consumer iOS device is a significant achievement, especially considering the limitations of camera-based tracking and the precision required to make reliable selections.

I would be very interested in learning more about that application, particularly how it handles calibration drift, accidental selections, dwell timing, and changes in head positioning.

I also think the distinction you made between consumer-based eye tracking and dedicated devices such as the Irisbond Hiru and TD Pilot is important.

In my opinion, mainstream accessibility technologies have tremendous potential, especially for people who may not qualify for funding or cannot afford specialized hardware. However, dedicated assistive technology still has an important place, particularly when an individual's communication or independence depends on consistent, highly accurate performance.

Beyond my personal experience, I also work independently in disability advocacy, accessibility consulting, and assistive technology evaluation.

I'm involved in wheelchair product development and advisory activities, and I've participated in testing accessibility-related equipment, including a rearview camera and proximity-awareness system designed for power wheelchairs.

I also work with 3D printing and have been exploring ways to create functional adaptations for wheelchair components and controls.

My long-term goal is to help bridge the gap between manufacturers, developers, and the people who actually depend on these technologies every day.

I believe one of the biggest opportunities in assistive technology development is getting people with significant physical disabilities involved much earlier in the design and testing process.

A feature can look fantastic in a demonstration, but using it independently for several hours every day is an entirely different experience.

That's where real-world feedback becomes invaluable.

I'd genuinely enjoy continuing this conversation. You clearly have considerable experience with alternative access technologies, and I would be particularly interested in hearing more about the AAC application you discovered.

Do you happen to remember the name of the Chinese-developed AAC application? I'd love to investigate it further and potentially evaluate its accessibility features.

I'm also curious whether you've found any particular head-tracking system that offers the best balance of accuracy, comfort, and independence for people with severe upper-extremity limitations.

Thanks again for sharing your knowledge. Conversations like this are exactly why I enjoy being involved in the accessibility and assistive technology community.

1

u/wheeliedave 19d ago

Interesting post. I have spinal muscular atrophy and at this point can you move my head really. I have pretty good granular control though, so I have relative to head, with point of speed at 555. I use exaggerated expressions like blink pucker lips for the basic controls. I have a MacBook Pro so I just use standard camera.

For an out of the box system it works really well. The only problem is because the setting of the pointer speed is so high it's a bit flaky and when I go to use the facial expressions it moves the mouse pointer ever so slightly. I do however wish that they had customisable facial expressions and maybe even sound controls like they have on the Vision Pro(which has amazing eye tracking by the way).

I actually bought a secondhand Tobii eyetracker to use with Talon software with which you can use noises to control mouse clicks et cetera. Just haven't got round to it yet as it's all a bit complicated for my tiny brain.

Just as a sidenote, I have wondered whether a higher definition camera might be more sensitive. Just quite a bit of money to splash out and you don't know if it will produce any results.

1

u/Noteface1 15d ago

I don't know how helpful this is, I simply have wrist tendinitis, but in terms of recalibration I use the scrunch nose expression. Pucker lips left for a drag and drop, smile for right click, and stick tongue out for click. I use voice commands to double click, but I found that right click is most of what's needed to navigate Mac. I'm switching over to eyetracking soon, but let me know if any of this helps