Once you’ve gotten used to the basic commands and extensions to control Google Chrome, you may start to hunger for a faster way to control websites you use frequently. Some sites have keyboard shortcuts you can bind easily, but others don’t. This post will describe how to set up commands to control any webpage. Continue reading Custom web commands with WebDriver →

I’ve been a little hesitant to publish a complete repository of all my Dragonfly commands because I think the journey that got me there is more useful than the raw code. If you just read the code, you’ll miss out on why I made certain decisions, you won’t know about all the stuff I tried and deleted, and you won’t know how I actually use all the commands in combination. That said, I do think it is a helpful supplement to this blog, so I decided to go ahead and make it available on GitHub. You can find it here, or linked from the navigation sidebar on every page.

While I’m laying down disclaimers, I should also mention that the code is a work in progress and isn’t as clean and modular as I would like, but I decided it was better to just get the code out there and improve it later. If you make improvements, please send me pull requests!

When I first started using Dragon, I was bummed to be restricted to Windows. Fortunately, there are lots of ways to work around this limitation and use it with whatever operating system you want. I’ll cover the method I use and describe some alternatives. Continue reading Using Dragon with Linux →

Update 11/22/2017: See my new post for the current extensions I use.

One of the best ways to get started writing Dragonfly macros is to set up web browsing by voice. Thanks to the extensibility of modern browsers, this works surprisingly well. Note that Dragon does have built-in support for web browsing, although I find it doesn’t work very well. The extension tends to cause pages to hang, and it requires that you speak the link you want to click on, which introduces ambiguities and doesn’t work well for all clickable elements. And of course, it’s not very customizable. I do recommend you try it first to see if it works for you, and to think about what you would like to improve in your custom version.

To begin with, you’ll want to decide between Firefox and Chrome. Both of these support the extensions you’ll need, so it is really a matter of personal preference. Firefox is probably the easiest to get started with, although I prefer Chrome. Continue reading Browsing the Web →

To be an efficient hands-free coder, you’ll need to learn how to move the cursor around a file quickly. There are two challenges: first, since you can’t use a mouse, you can’t just click to the location to move to. You can try using an eye tracker to accomplish this, but the precision isn’t quite high enough. Second, with a keyboard you can hold a movement key and release when you reach your location, but this doesn’t translate well to voice control, which has too much latency (although you might try measuring the latency and adjusting for it). Continue reading Zipping around a file with ease →

Update (12/18/2020): I now use a different method to integrate the tracker. See Efficient UI interaction with OCR and Gaze Tracking for the most up-to-date instructions.

You can do a lot just using your voice, but there are still a few times you’ll find yourself reaching for a mouse. It’s often for the silliest little things, like clicking in the empty space within a webpage to change the keyboard context. If you’re serious about not using your hands, you can use an eye tracker to eliminate these last few cases. This post will teach you how to get started. Make sure you’ve read my introductory post on voice coding, since will be building upon that. Continue reading Getting Started with Eye Tracking →

Updated on 1/9/2021 to discuss newer software versions.

A hands-free coding environment has a lot of moving parts, which can be overwhelming at first. This post will teach you how to set up the basic voice recognition environment. I also use eye tracking, but I’ll cover that in a separate post. Continue reading Getting Started with Voice Coding in Dragonfly →

Coding without a mouse? Easy, just use any decent text editor. But coding without a mouse or a keyboard? That’s where things get interesting. Continue reading Adventures in Hands-Free Coding →