All posts by James

Tobii Eye Tracker 4C Review

Tobii has released a new consumer eye tracker, the Tobii Eye Tracker 4C for $150. Although I haven’t found eye tracking to be nearly as helpful as speech recognition, it is handy for those occasional situations where you just want to click a button or change context and you don’t have any command to do so (see my earlier post for details). I have been pretty happy with the Tobii EyeX, but it isn’t perfect, so I was excited to try out this new device. Continue reading Tobii Eye Tracker 4C Review

Hands-Free Gaming

Not related to coding, but hands-free coders need to have some fun too. 🙂

I discovered recently that Hearthstone can be easily played with eye/head tracking and minimal voice controls (move pointer and click), thanks to the turn-based interface, large click targets, and a high thinking-to-clicking ratio. I don’t even use a custom grammar and it works very well. If you have a good experience, you can thank Blizzard on this thread I started. Hopefully I didn’t just set the voice coding community back by a few months!

If you know of other games that play well with hands-free control, please post in the comments.

Getting organized with Org mode

Like many an Emacs user, I am enamored with Org-Mode. Every great coding session begins with organizing your thoughts, and Org-Mode is an excellent tool for the job. If you’re tracking New Year’s resolutions, it’s great for that too. Since Org-Mode already has an excellent compact guide, I’ll focus on my voice bindings and finish with a bonus section on how I like to structure my personal to do lists. Continue reading Getting organized with Org mode

Designing Dragonfly grammars

UPDATE 9/8/2018: This is a technical introduction to designing grammars with Dragonfly. If you are looking for recommendations on words and phrases to use as commands, see this post.

As you build on your grammars over time, you start to run into all kinds of problems. Commands get confused with each other, latency increases, and your grammars become giant disorganized blobs. This is particularly challenging with Dragonfly, which gives you the power to repeat commands in a single utterance, but leaves it up to you structure your grammar accordingly. In this post I’ll discuss the techniques I use to wrangle my grammars. Continue reading Designing Dragonfly grammars

Dictating Code

For a site titled Hands-Free Coding, I haven’t written much about How To Actually Write The Code. It turns out this is easier than you might expect. Before reading this post, please familiarize yourself with my getting started guide and how to move around a file quickly.

There are two basic approaches to dictating code: using custom grammars such as Dragonfly, or using VoiceCode, (not to be confused with for Mac, which I just discovered and haven’t used yet). VoiceCode is much more powerful out-of-the-box, but is also harder to extend and more restrictive in terms of programming language and environment. You might say that VoiceCode is Eclipse, and Dragonfly is Emacs. You could also consider Vocola for your custom grammars; it is more concise but not quite as flexible because you can’t execute arbitrary Python. Since I prefer Dragonfly, I’ll cover that approach. Continue reading Dictating Code

My GitHub Repository

I’ve been a little hesitant to publish a complete repository of all my Dragonfly commands because I think the journey that got me there is more useful than the raw code. If you just read the code, you’ll miss out on why I made certain decisions, you won’t know about all the stuff I tried and deleted, and you won’t know how I actually use all the commands in combination. That said, I do think it is a helpful supplement to this blog, so I decided to go ahead and make it available on GitHub. You can find it here, or linked from the navigation sidebar on every page.

While I’m laying down disclaimers, I should also mention that the code is a work in progress and isn’t as clean and modular as I would like, but I decided it was better to just get the code out there and improve it later. If you make improvements, please send me pull requests!