Webcam gesture control for Jira? Yes!

[_A9_ William Kennedy]

I was scrolling Web APIs and Interfaces at Mozilla.org and explaining the general idea behind web application development to my daughter (ELI8)

We looked at standard things like the Touch events API, https://developer.mozilla.org/en-US/docs/Web/API/Touch_events, WebGPU, WebMIDI and others.

As I explained digital interfaces, serial ports, the GPU and touch inputs to a girl with a shortening attention span, I pass an idea to one of the product agents...

Prove that a Forge Custom UI app can safely turn keyboard/switch/gesture input into real Jira board actions without scraping or manipulating the native Jira board DOM.

The prototype should answer four questions:

1. Can Forge Custom UI run the camera + gesture stack reliably?

2. Can we map noisy gestures into safe, confirmable commands?

3. Can we move Jira work items through official APIs?

4. Can we maintain a strong privacy posture: no video storage, no remote inference, no unnecessary egress?

So, it is possible to "bite" and move cards or pinch or grab and move them from status to status, and this is reflected back in the source board.

You can reuse the code, revisit the spec, README or whatever.

It works, and has advanced beyond this first demonstration of moving a card, but it likely isn't going to improve anyone's experience using Jira in its current state, and there are probably better ways, but sometimes you just have to try something. I'm not an A11Y researcher, and I instantly saw the benefit to how the Wiimote tracks movement. Hmm.. I wonder...

https://github.com/wjkennedy/A9GesturesForJira/tree/main