Dictating Code

For a site titled Hands-Free Coding, I haven’t written much about How To Actually Write The Code. It turns out this is easier than you might expect. Before reading this post, please familiarize yourself with my getting started guide and how to move around a file quickly.

There are two basic approaches to dictating code: using custom grammars such as Dragonfly, or using VoiceCode, (not to be confused with VoiceCode.io for Mac, which I just discovered and haven’t used yet). VoiceCode is much more powerful out-of-the-box, but is also harder to extend and more restrictive in terms of programming language and environment. You might say that VoiceCode is Eclipse, and Dragonfly is Emacs. You could also consider Vocola for your custom grammars; it is more concise but not quite as flexible because you can’t execute arbitrary Python. Since I prefer Dragonfly, I’ll cover that approach. Continue reading Dictating Code

My GitHub Repository

I’ve been a little hesitant to publish a complete repository of all my Dragonfly commands because I think the journey that got me there is more useful than the raw code. If you just read the code, you’ll miss out on why I made certain decisions, you won’t know about all the stuff I tried and deleted, and you won’t know how I actually use all the commands in combination. That said, I do think it is a helpful supplement to this blog, so I decided to go ahead and make it available on GitHub. You can find it here, or linked from the navigation sidebar on every page.

While I’m laying down disclaimers, I should also mention that the code is a work in progress and isn’t as clean and modular as I would like, but I decided it was better to just get the code out there and improve it later. If you make improvements, please send me pull requests!

Browsing the Web

Update 11/22/2017: See my new post for the current extensions I use.

One of the best ways to get started writing Dragonfly macros is to set up web browsing by voice. Thanks to the extensibility of modern browsers, this works surprisingly well. Note that Dragon does have built-in support for web browsing, although I find it doesn’t work very well. The extension tends to cause pages to hang, and it requires that you speak the link you want to click on, which introduces ambiguities and doesn’t work well for all clickable elements. And of course, it’s not very customizable. I do recommend you try it first to see if it works for you, and to think about what you would like to improve in your custom version.

To begin with, you’ll want to decide between Firefox and Chrome. Both of these support the extensions you’ll need, so it is really a matter of personal preference. Firefox is probably the easiest to get started with, although I prefer Chrome. Continue reading Browsing the Web

Zipping around a file with ease

To be an efficient hands-free coder, you’ll need to learn how to move the cursor around a file quickly. There are two challenges: first, since you can’t use a mouse, you can’t just click to the location to move to. You can try using an eye tracker to accomplish this, but the precision isn’t quite high enough. Second, with a keyboard you can hold a movement key and release when you reach your location, but this doesn’t translate well to voice control, which has too much latency (although you might try measuring the latency and adjusting for it). Continue reading Zipping around a file with ease

Getting Started with Eye Tracking

Update (12/18/2020): I now use a different method to integrate the tracker. See Efficient UI interaction with OCR and Gaze Tracking for the most up-to-date instructions.

You can do a lot just using your voice, but there are still a few times you’ll find yourself reaching for a mouse. It’s often for the silliest little things, like clicking in the empty space within a webpage to change the keyboard context. If you’re serious about not using your hands, you can use an eye tracker to eliminate these last few cases. This post will teach you how to get started. Make sure you’ve read my introductory post on voice coding, since will be building upon that. Continue reading Getting Started with Eye Tracking