patrickz
Before you start
Paper or a simple web prototype, a chat assistant and a consenting tester. Use fictional phrases for the first pass.
Why this lesson exists
The posts behind “The Communication Tool That Needed No Mouse and No Voice” describe a prototype that ran on a webcam, a tablet and open-source software. The lesson centers on a user’s reliable movement and meaningful messages. Interface design and individual testing matter as much as the AI component.
The idea this guide practices: Build for one real person first. Prefer a story? Read the comic The Mouse Has Left the Building.
Do the exercise
Start with a person’s choices
Ask which messages are useful and which input is comfortable. Do not assume that speech difficulty means a lack of understanding. Begin with four large picture-and-text choices.
Keep the first interface simple
Include select, clear, repeat and pause. Make the selected message visible before or while it is spoken. Keep ordinary touch or keyboard input available.
Add one alternative input
Try a deliberate dwell (selecting by resting the pointer on an item for a set time) or switch-scanning (items highlight in turn and one button selects) prototype before adding camera control. Make timing adjustable and show which item is active. Prevent accidental repeated selection.
Test comfort and recovery
This can be a separate short session. Use familiar yes/no questions and an easy way to stop. Record mistaken selections and fatigue. Revise with the person; do not claim clinical validation from a technical demo.
A prompt to adapt
Replace the bracketed parts with your own details.
Help design a four-message communication-board prototype. User preferences: [consented observations]. Include touch/keyboard access, one optional alternative input, visible focus, adjustable timing, clear/repeat/pause controls and a test plan. Use fictional messages and avoid assumptions about cognition.
What this looks like
A dwell selection needs a visible progress cue and a way to move away without activation. A button that works once but repeats uncontrollably has not passed the interaction test.
Check your result
Use evidence from your output. A confident explanation from the AI is not enough.
If it isn’t working
If an item fires by accident, lengthen the dwell time and require the pointer or scan to leave an item before it can select again. If the tester tires or seems uncomfortable, stop, cut the board to two choices and record both as design findings. This is a learning prototype, not a clinically validated device or an emergency communication method; involve a qualified communication professional for real care decisions.
Optional. Progress stays in this browser.
Where this came from
Adapted from the archived LinkedIn theme “The Communication Tool That Needed No Mouse and No Voice.” See the post coverage and editorial method.
Prepared September 2026. Tools and interfaces change; use current official setup instructions. Session lengths are estimates.