Adventures in Hands-Free Coding

Coding without a mouse? Easy, just use any decent text editor. But coding without a mouse or a keyboard? That’s where things get interesting.

About three years ago, I set up Dragon NaturallySpeaking on my home computer to give my hands a chance to heal from RSI (Repetitive Strain Injury) on the weekends. I mostly used it for web browsing and occasionally writing emails. It was a great relief; finally I didn’t have to feel guilty for using my computer on weekends. But it was also very limited, especially when it came to coding. I’m a professional software engineer, and I liked my job, but I also wanted to work on side projects. I tried one of the leading Dragon extensions to help with this, VoiceCode, but after a few hours I decided it was too frustrating and slow, and the software just wasn’t there yet.

A couple years later I watched a great talk by Tavis Rudd, Using Python to Code by Voice. I was put to shame. Not only did he manage to do it, he did it professionally! He managed to cure his RSI completely over several months, but he never stopped using his voice. He was even faster using both voice and keyboard than keyboard alone.

I decided to give it another try, using the Dragonfly library that Tavis recommended. Unlike VoiceCode, which is a complete solution, Dragonfly is just an improved Python API to Dragon. It doesn’t come with too many built-in commands, but it’s extremely easy to extend. With a powerful Python API to Dragon in hand, I started experimenting with different commands and grammars to use in Emacs and elsewhere. For the first time I started to see that it was possible to become productive. This was good timing, because my RSI was only getting worse.

About six months ago, I decided it was time to start using my voice environment at work. At first I was very slow. It took a lot of mental effort to do the most basic things, and I definitely had moments where I questioned whether I would ever become efficient. But with each week I got faster and faster. And something else changed: I started having fun! Adding new features was building my future. Getting faster was gaining freedom. I felt like Bruce Wayne building his Batcave!

I’m not Batman yet, but I think it’s time I start to share what I’ve learned to help others do the same. I could just publish my code, but that wouldn’t reveal all the little lessons I learned along the way that you will need to know to build your own custom environment. I’m also hoping to hear from others, so please post your own ideas in the comments and tell me what I could do better!

5 thoughts on “Adventures in Hands-Free Coding”

  1. Hey, I want to start using speech recognition to write documents in LaTeX and some basic stuff like e-mailing. A problem I’ve run into is that my voice seems to get tired too easily. Do you get used to this after a while, or is there something you can do to last longer (I’m only managing half an hour currently)?

    I was also wondering how your RSI is doing currently.

    1. Hi Max! Sorry for the slow/disorganized reply, I’m sending this from a phone on vacation. I do still have RSI but fortunately I’m able to use my voice full-time (and I often work 9+ hours a day, although that’s not all coding).

      In general I was happy to find that my voice naturally became stronger over time (unlike my RSI experience!). That said, don’t push it early on. Stop if you feel hoarse. Also I highly recommend using sugar free lozenges.

      I went to a speech therapist for one session and didn’t get much out of it. Just don’t speak too quickly and speak smoothly and at a normal volume.

      I still occasionally get a hoarse throat but it typically recovers by the next day without even slowing down much. It used to take a weekend or so to recover, and I would have to stop all computer usage. Hopefully you see similar improvements!

  2. James, I just wanted to post to thank you for this blog. I discovered it in May 2015, and it was extremely helpful while I was setting up my voice coding system.

    I’ve been coding by voice full time now for almost 9 months and with Emacs the system is powerful. I don’t know if I would have been able to set it up without all the info you have here.

    I wanted to take the time to thank you explicitly. You definitely saved my career. Hopefully I can convey my gratitude. Thanks man. 🙂

    1. Wow thank you for the feedback, it means a lot! I started this blog in hopes that I could save people’s careers, I’m happy to hear it has actually happened!

      1. Hey, no worries. I figured you would want to know!

        I’ll probably put my system on github at some point. I’ll let you know when I do. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *

Markdown is supported. Make sure raw < and > are wrapped in code blocks. You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>