Voice control systems come and go. Most fail to attract a significant following and eventually fizzle away. Every once in a while, though, one persists and shapes the community forever: Vocola, Utter Command, Dragonfly, and others before my time. In the last couple years, Talon has gained momentum as a new system targeted at power users. About a year ago it added Windows and Linux to the list of supported OSes (after Mac), and that’s when I started paying closer attention. For a while, I watched it improve from the sidelines. As someone who spent years developing my own grammar for Dragonfly, I wasn’t eager to rewrite it all or learn a new community grammar. On top of that, I had concerns that much of Talon was not open source. Despite this hesitance, I saw enough promise in my early experiments that I decided to take the plunge and test it out in earnest. My effort has been undoubtedly worthwhile. I’ve been impressed both by what Talon can do today and the velocity with which it is improving. I can confidently say that I’m more productive than ever, and have never been more excited about the future of voice computing. In this post, I’ll examine Talon’s capabilities and conclude with a detailed discussion of the implications of its partially closed-source model. Although the focus will be on Talon, I’ll occasionally compare it to Dragonfly as a helpful reference point, since it is a system that I (and many of my readers) have used for years.
Using Talon
Having helped many people with Dragonfly’s notoriously fickle installation process over the years, I was relieved by the simplicity of Talon’s installation. It is freely available for Mac, Linux, and Windows, although I have been using the bleeding edge Beta version on Windows, which costs $25/month for early access to new features (and consequently, influence on those new features). Simply run the installer and clone the community grammar into the user directory it creates for you, and you are ready to go. Talon bundles its own Python environment, running a recent version of Python 3 64-bit, in contrast to NatLink’s experimental-only support for Python 3 and restriction to 32-bit. Updates to Talon are even easier: it checks automatically on startup and prompts you to apply the update if you wish.
Talon bundles a lot of useful functionality out of the box. Like Dragonfly, it can integrate with Dragon, but it does so using its own custom integration (Draconity) instead of reliance on NatLink. By enabling communication with Dragon from a separate process, Draconity is generally able to provide a more robust experience and access to 64-bit Python. The only issue I’ve noticed is occasional dropped commands immediately after a context switch. (Already fixed!) Additionally, Talon offers its own speech recognition engine, Wav2Letter Conformer, which, impressively, is competitive with Dragon. Like Dragon, it can guess the pronunciation of made up words with uncanny accuracy. It responds with very low latency, even lower than Dragon when context switching between different grammars. In my experience, though, accuracy still lags a bit behind Dragon, although it is catching up and getting better with each release. Dragon’s strength is its language model, which enables it to make better guesses when the speech audio is ambiguous. Currently, I’m using Dragon for work, but I use Conformer at home to track its development and ensure everything I do is compatible with both. I’m writing this post with Conformer.
In addition to speech recognition, Talon includes support for noise recognition, custom keyboard shortcuts, and eye tracking. The built-in noise recognition support is currently pretty basic (pop and hiss), but Beta users can try out an experimental integration with parrot.py, which lets you train your own custom noises, integrating seamlessly into Talon’s grammars. Astoundingly, the author of parrot.py used it to reach Diamond League in StarCraft … with videos to prove it! In comparison, the custom keyboard shortcuts are less exciting, but still useful as they let you bind keyboard shortcuts to arbitrary Python code, just as you can with words and noises. The eye tracking support works with Tobii eye trackers, bucking their heavyweight software in favor of a lightweight custom integration (in fact, you need to kill the Tobii services or uninstall their software for this to work). It also extends OS compatibility to include Mac and Linux, provided you have access to Windows for initial setup. It works nicely out of the box as a mouse substitute, using popping sounds to zoom in on the screen and then to click. You can also make use of gaze coordinates in your custom commands.
The de facto community grammar, knausj_talon, is an essential part of Talon’s functionality. It even implements some building-block interfaces that Talon provides, so everyone should start with this. Fortunately, it is very well-built. It loosely follows the Utter Command style that I gushed about in an earlier post, in contrast to the Dragonfly community grammar Caster which mixes in some made up words. Additionally, it includes a voice-searchable GUI-based help system which doubles as a gentle introduction to the Talon script syntax (more on that later). This help system works with both the bundled commands and any other commands you use, whether you write them yourself or they come from another third-party bundle. You’ll be using this help system a lot, because there’s a ton of functionality in knausj_talon. Out of the box, this grammar has its own versions of ~80% of the Dragonfly commands I’ve written over the years, along with plenty more I never got around to. The major gaps I’ve noticed are gaze + OCR, accessibility API, and webdriver integrations. Of these, I use gaze + OCR the most by far, and it wasn’t too difficult to get this working with Talon (I plan to release my work on this once I’ve made it easier to install). Of course, much of the overlapping functionality that comes with knausj_talon did not map to the exact same phrases I had been using. For the most part, I’ve been trying to learn the knausj_talon commands, except for my custom monosyllabic alphabet because it works well and would be a pain to relearn. This was a one-line change to integrate (really!) and it immediately worked everywhere without exception.
Knausj_talon also includes a system for prose dictation formatting. Thanks to Draconity, it is able to effectively override Dragon’s own dictation system, because it can bind commands that match any arbitrary utterance — no need to preface with a command word. This can be a double-edged sword, because Dragon has pretty good prose dictation formatting, and knausj_talon lags behind in some respects (e.g. automated number formatting). That said, this extra power means the ability to rethink rough edges in Dragon’s dictation, and standardize on commands that work across multiple recognition engines (e.g. Conformer). For example, knausj_talon bundles its own custom vocabulary system, which lets you make simple edits to a file to add custom words instead of dealing with Dragon’s clunky UI (and you can bind your own commands to edit this file, if you wish). If you just want to use Dragon’s own dictation and vocabulary system, you still can, just as you would with Dragonfly. Knausj_talon leverages a modal system for switching between commands and dictation, so you’d simply avoid entering Talon’s dictation mode and use Dragon’s own dictation mode (or mixed mode) instead. Personally, I’ve been enjoying iterating on knausj_talon’s dictation mode myself in an effort to build the system I always wanted … stay tuned for pull requests!
Recently I’ve started using a couple more community-contributed grammars that work seamlessly alongside knausj: Talon HUD and Cursorless. Talon HUD adds a more full-featured GUI layer, including an indicator of the current mode and a better display of recently-recognized commands. It’s easy to move this interface literally anywhere onscreen; I put it in the taskbar adjacent to the notification area so it doesn’t overlap any content:

Talon HUD also includes a professional-quality tutorial that walks you through all the available functionality. Finally, it doubles as a library for building additional HUD elements, so I’m excited to see what gets built atop this.
Cursorless is a brilliant system for editing files in Visual Studio Code. It decorates the text with colored “hats”, making it possible to efficiently manipulate nearly any sequence of onscreen code with a single short command. Watch the intro video to see how powerful this is. It’s also under active development with new features regularly introduced. Cursorless is one of those lovely examples that show off how voice is not only a viable alternative to mouse and keyboard, it can actually be faster. It also pairs with a full-featured grammar in knausj for working with the rest of VS Code, making it a popular editor to use in combination with Talon. I’ve been very impressed with VS Code so far, although I wish I could have this functionality in Emacs too (the Org Mode extension in VS Code is no substitute for the real thing).
Extending Talon
Just as Talon is easy to use right away, it’s also easy to extend. All user scripts (grammars) are organized into Talon files (.talon) and Python files (.py). Talon files mostly consist of simple mappings from commands to actions, following a grammar syntax that’s very similar to Dragonfly. Talon originally used Python for everything (like Dragonfly), but the author decided to introduce a special file format because it brought several advantages. By focusing on command mappings, it enables a very terse and simple syntax. For example, you can bind “save file” to “ctrl-s” with the intuitive line: save file: key(ctrl-s)
. This will automatically support chained recognition with all other commands. If you accidentally introduce a syntax error, it will generally only affect a single command (instead of breaking parsing of an entire Python file). The format is sufficiently self-documenting that it’s used as-is in knausj’s GUI help system. The other major advantage of the Talon file format is that it encourages clean separation of concerns. Complex behaviors should be defined in a Python file and exported as an action. These actions can then be used from any Python file or Talon file without the need to explicitly import the action or worry about circular dependencies. You can reorganize your files into directories however you would like without breaking anything, and if you edit and save a file, it’ll be reloaded everywhere instantly. Additionally, any action (including the built-in ones) can be overridden in any context, such as within a single app or website. These factors make it easy to modify a community grammar to suit your needs, whether you want to change the grammar (just adjust the Talon files) or override functionality to work differently in a particular app (just override the action within that context). The only tradeoff I’ve noticed versus doing everything in Python is that there is some overhead when moving functionality from a Talon file to Python, e.g. to refactor a command into an overridable action, or remove redundancy from a bunch of related commands.
Perhaps the weakest aspect of Talon scripting is its documentation. The official documentation is mostly focused on the Python API, and it gives a bunch of isolated examples without really introducing the concepts and how they fit together. Fortunately, there is decent unofficial documentation that covers most of the functionality you will likely use. There is also a built-in REPL that can be used to reveal more hidden functionality, although this too is not formally documented. I recommend running actions.list()
and pasting the output somewhere that’s easy to access and search. You can also run sim(“an utterance”)
to see how Talon will parse an utterance. Since this is typically context-dependent, however, you are better off saying the knausj command “talon test last” after a command goes awry to print debug information to the log.
The best way to learn about Talon’s undocumented functionality is to engage with the community Slack. You’ll not only learn a lot by asking questions and reading the (sadly truncated) archives, but you’ll also realize that you don’t need to build everything on your own. The community is very interested in building shared functionality, and tends to be responsive to feature ideas. For example, I handed down the idea of modulo line numbers to the Cursorless maintainers (just as Mark Lillibridge did to me many years ago), and they had prototyped this the next day with plans to release it. If you do end up building interesting functionality yourself, I encourage you to discuss it on Slack and send a pull request.
Open vs. closed source
As described above, a lot of Talon’s functionality is present in its community grammars, and those are completely open source. Talon itself, however, consists of a mix of open and closed source code. For example, the awesome Conformer speech recognition engine is being developed in the open, building on Facebook’s wav2letter project. Conversely, the nuts and bolts of the Talon Python APIs are generally closed source. This was, for a while, the main thing that made me hesitate to switch from Dragonfly to Talon. Now, I should note that the typical Dragonfly stack is not fully open source either, with Dragon as a prime example of proprietary software. But the Dragonfly community has long been working towards a fully open source system by replacing Dragon with alternatives like Kaldi. My biggest concerns with Talon’s partially-closed model were practical, however, not ideological. I completely respect the Talon author’s decision not to release all of his source code. He made a daring decision to quit his job and work full time on voice control (something which I fantasized about but never did), and he needs a working business model to sustain this. Nevertheless, I had first-hand experience with both a closed source system that had rotted to the detriment of its users (Dragon) and an open source system that had been resuscitated only because the source code was available (Dragonfly), so this worried me. Fortunately, the Talon author, Ryan Hileman, was kind enough to discuss my concerns in depth and the mitigations he has in place. Hence, I will break down the possible implications of a closed source model, and how these do (and don’t) affect Talon in practice.
My top concern was that Talon seemed like a one-person project, carrying risks if Ryan lost interest or if something unexpected were to happen to him (i.e. a bus factor of one). To be sure, there’s a great community building atop Talon, but Ryan is undoubtedly the genius behind the project, something which became manifestly clear after my conversations with him. Fortunately, Ryan says he has privately shared source access with a few others to protect against the most unexpected outcomes — hopefully that’s never needed. A bigger worry for me is simply that Ryan loses interest at some point. He assured me that if he ever stops development, he would establish a continuation plan with the community, which could mean open sourcing the code. That’s comforting — and backed by his strong track record in open source — although there could be a gray area where development slows down but doesn’t stop. This is a difficult question to discuss in the abstract, especially when today Ryan is so devoted to this space: to my knowledge he is the most productive person who has ever worked on voice control. But it’s something to consider in the long term.
Another concern we discussed was community involvement in fixing bugs and adding features. As productive as Ryan is, he’s still one human being, so he has to aggressively manage his time and prioritize between competing feature requests and bugs. At this point, Ryan has fixed or responded to all the bugs I’ve logged, but if the project were open source, I could have opened the hood and tried to fix them earlier myself. Ryan has pointed out that nearly everything in Talon is user-overridable, but in practice it’s much easier to fix the code for keypresses directly than to reimplement them from scratch (for example). Personally, at this point in my life I have little free time, so I’m happy to hand issues off to someone else, but it’s something that might have felt limiting a few years ago. Notably, most of the bugs I’ve logged had workarounds, and in the rare cases where something nasty cropped up in Beta, Ryan speedily fixed it (and it’s also easy to roll back to earlier Talon versions). Similarly, I once shared a feature idea only to have Ryan implement it in Beta minutes later. Ryan also noted that he has shared the Talon source code with individuals and accepted contributions on a case-by-case basis — it’s just not a decision he takes lightly. Overall, due to Ryan’s extraordinary development speed, this concern feels mostly theoretical right now — Talon is improving at a rate exceeding that of its fully open source predecessors.
Fixing bugs and adding small features is one thing, but what if there’s ever a more fundamental disagreement about Talon’s direction or design philosophy? If the project were fully open source, one would have the option to fork. From our discussion, Ryan sees this as an advantage of Talon’s model: by preventing forks, it prevents fragmentation of the community and simplifies development by ensuring that all user scripts are coded against the same backends (e.g. the same Python version). Additionally, he notes that the parts of Talon which are closed source are generally the parts which are difficult to maintain or least interesting to fork. You might not agree with every design decision in the Talon file format, for example, but you probably wouldn’t want to maintain a fork for that reason. Deep philosophical disagreements are most likely to apply to the community grammar, but that’s all open source. This reasoning does depend on Ryan being an effective gatekeeper with good judgment. Fortunately, that’s something that you can easily assess his track record on, and my assessment would be overwhelmingly positive, as evident from the earlier sections of this post.
Cost is another consideration. Today, Talon is free for the public release and $25/month for the Beta (up from $15/month earlier on, with early subscribers grandfathered in). This seems like a very reasonable pricing model, but it’s something that could possibly change. Again, it helps to consider Ryan’s track record here. He’s been closely engaged with the community from the start and he exudes a hacker ethos, so it seems unlikely that he would make a tone-deaf change to pricing.
It might be nice if Talon were fully open source so its source could be reused in a wider variety of contexts or as educational material. But that shouldn’t impact your decision as a user deciding whether to adopt Talon or another system. Also, some of the most complex and interesting parts of Talon are already open source: the Conformer speech recognition model and Draconity for integrating with Dragon.
In summary, much of the above discussion comes back to one point: trust in Ryan — to keep developing Talon, to be an effective gatekeeper, to keep the cost reasonable. Personally, from my biweekly conversations with him over the past year, and from his impressive track record on Talon, he has earned that trust. It may also be comforting to consider the absolute worst case scenario, to see it’s not the end of the world. Suppose something happens that makes Talon a complete no-go. You’ll still have your grammar source code, and you can always migrate it elsewhere. I was able to convert about 80% of my Dragonfly grammar to Talon in a weekend, and this would likely be even simpler starting from Talon with its neat separation of concerns and easily-parsed Talon files. Even in this worst case, I have faith that the “Talon period” of voice control development will have been a huge net win for the community, having already led to great progress in a variety of areas.
Conclusion
I highly recommend trying out Talon and its community grammars if you haven’t used them recently. You may be surprised at how easy it is to get up and running — even if you’ve built a lot on your own over the years. I hope that the Dragonfly devotees continue their noble work towards a fully open source alternative, but I plan to use Talon on a day-to-day basis for the foreseeable future. It’s unfortunate that the power user community now seems to be split across these two systems. For my part, I plan to develop any major contributions to work with both platforms, starting with porting my gaze_ocr library to Talon, and I encourage others to do the same.
Once you’ve had some time to play around with Talon, please let me know what you think in the comments!
Hi James, thank you for taking the time to do the tests and make such a nice post for us to read. I really liked how you credited Ryan with “to my knowledge he is the most productive person who has ever worked on voice control.” It speaks not only to his output but also to the depth of your community perspective that has developed over time. It’s always nice to see such progress – and admire his ability to ship products on both Macs and Windows; multiplatform development is never simple.
Thanks! Yes, it is very freeing to not be tied to one platform … or speech recognition engine.
With Talon, how easy is it to 1) say something of my choice up to 10 or 15 words, 2) have Talon recognize them all reasonably accurately (does it have a custom vocabulary?), and 3) have Talon shell execute my program and pass me the recognized words as arguments? That way, I could do whatever I liked with the utterance. Thank you.
This is all very well supported: it’s the bread and butter of what Talon does. Let me clarify each of those pieces of bit:
1) Talon can either recognize arbitrary text or specific commands, or some mixture of the two (e.g. commands with free-text parameters). If you specify commands, it biases recognition a little towards those commands.
2) The accuracy depends on whether you are using Dragon or Conformer, although both are good. Both have support for custom vocabulary. Dragon optionally lets you train the pronunciation, whereas Conformer just guesses it — with high accuracy.
3) Yes, you can definitely have parameters in your commands which Talon will pass along to you (these are called “captures”). And you can do anything with them that is possible within Python (including starting separate process if that’s necessary).
James great post as always. I’m starting to explore this on my Mac since open till now there hasn’t been a truly viable solution there. Specific question regarding the alphabet change. I see that’s defined in keys.py, did you just change it there, or is there a way to overwrite it and one of the talon files?
I just changed it there, and then it will have effect anywhere that the “user.letter” Talon list is used (and it is consistently used throughout the community grammars). There is a feature proposal to make it possible to override talon lists without touching Python, but in this case it’s probably simpler just to do it this way even if that feature were implemented.
One other question, cursorless definitely looks interesting. Kind of like an advanced version of Ace jump. Do you know if it’s tightly coupled to talon? Or is it possible to use it with other frameworks through some keyboard commands?
It is possible in principle but would be tricky. Looking at the Talon side of Cursorless, there are thousands of lines of code here just because the grammar is that powerful/complex. It is ultimately calling functions in the vscode extension which I believe could be bound there to keyboard shortcuts, but you run the risk of breaking changes upstream because it currently designed to be used in tandem with the talon grammar. They do have note saying that they plan to add keyboard support, but don’t think this is a top priority. If you just want to bind a couple movement commands (e.g. “pre” and “post”), though, you could add some logging to see what the Talon grammar is sending to vscode, and then try binding that yourself.
have you by chance run run into a scenario where Cursorless does not seem to listen to the colors? for example I say “take etch” and it highlights the word associated with the letter ‘e’ and the gray dot. however if I say ‘take green etch’, it always misrecognizes green. I will dig around, and maybe check the slack channel but thought I’d ask here as well. Thank you.
Several of the colors are renamed. Pink is plum, red is rose, and squash is yellow. Green should be green, though, and these are all configurable via csv.
Yes I was aware of that. It just ended up being that I needed to restart Talon and VS code. When all else fails just reboot right? :-). I’m still trying to figure it out but it certainly looks promising. I’m a bit surprised that it doesn’t have ways to select by specifying a paren or quote type. Although there do seem to be close to equivalents. Definitely looks interesting.
One other question that popped up is what the professional tier vs beta tier is? Does it imply that to use the software for commercial purposes you need the $100 a month version? It’s a bit hard to tell from the site. That’s definitely a big step up in cost, and I’m not sure what the delta is there.
What do you mean by “select by paren or quote type”? You can definitely have it select a full string or expression (e.g. an arg). As for pro tier I believe this is nothing more than a way to show the developer additional support. You can use any tier (including free) professionally.
Yes, when I looked over here
https://github.com/pokey/cursorless-talon/tree/main/docs#syntactic-scopes
there isn’t a direct obvious way to say select everything inside of the () or {} or []. it does look like map, list, string do the equivalent(?), but there’s the extra processing in my head of “what is a map exactly in this context?”.
Hah, looks like the new version released today supports this! “”take pair” expands to include the nearest containing pair of any kind”
Nice!
Regarding the keyboard support, or rather in addition to that, do you know if the developer is aware of dragonfly and has looked at supporting a dragonfly grammar?
Yes he’s very aware and at one point was trying to add Talon support but unfortunately it didn’t work out: https://github.com/dictation-toolbox/dragonfly/pull/326
Actually I was referring to Cursorless support for dragonfly rather than talon :). I had seen some conversations between Ryan Dane and Aaron on the gutter back in the day.
I meant gitter of course… Darned auto correct..
Wow! Thanks for the detailed review, I needed to rebuild my voice coding set up and this can just in time. My first impressions of Talon are: really impressed!
Quick and easy install, no hitches, including interfacing with my current Dragon installation.
Getting a new .talon file working for Android Studio was superquick and much easier than dragonfly.
Bonus: I made a change to a .talon file with my commands/macros, as soon as I saved the file, the Talon Log viewer showed the change and immediately pointed out a syntax error!
Seems I can do all my basic needs with the free version (so far)?
Do you use their new alphabet (air, bat, cap…)? I know I can replace it, but I’m guessing they chose those words based on what was best for voice recognition, rather than human hearing over a radio(alpha, bravo, charlie…)
Android Studio doesn’t seem to be 1 of the supported IDE’s in the knausj_talon library, so I’ve started making my own just to get going. I can trigger the commands with keyboard shortcuts eg “ctrl-z”, but when I looked in the jetbrains.talon I see object references eg ‘play recording: user.idea(“action PlaybackLastMacro”)’
Do you know what the difference is, and where they find those object references?
Thanks!
Great to hear it’s working well for you!
I just spoke with Ryan yesterday about the Talon alphabet. He said he designed it with speed, accuracy, and ease in mind. It’s probably better than the arch/brov/char alphabet I use (maybe I’ll invest the time to relearn at some point …)
I haven’t tested Talon with Jetbrains, but I took a look at the code and it’s sending those commands through an RPC layer, bypassing the need for keyboard shortcuts. Presumably those commands are the IDs in JetBrains. I’m not sure offhand how to get those — maybe try the keyboard shortcut editor? The README says you need to install this extension to use it: https://github.com/knausj85/knausj_talon#jetbrains-commands
If that doesn’t work out of the box with Android Studio, I’m guessing it’s a small tweak to get it going. The code seems to support a wide variety of JetBrains IDEs: https://github.com/knausj85/knausj_talon/blob/master/apps/jetbrains/jetbrains.py#L16
I’ve been working through those tutorial exercises, trying out his new alphabet, initially with dragon but it wasn’t working great. Lots of misreads and it was struggling to string together more than one letter at a time.
The I installed Conformer through the menu and tried the new alphabet, and it worked MUCH better, I rattled off the whole new alphabet three letters at a time with zero errors!
Do you know how to configure conformer? Now that it’s installed and running I don’t see it in the start menu, and even searching my pc for “conformer” or “w2l” yielded nothing.
Conformer is built into Talon so there isn’t any public facing configuration aside from Talon scripts, which let you adjust various settings such as how long it waits for a pause. Described here at the bottom: https://talon.wiki/unofficial_talon_docs/
Thanks James, I went through that but didn’t see any mention of vocabulary, how is that amended with Conformer?
Thanks for all the feedback!
If you load knausj_talon it will create a “settings” subdirectory. Check in there and you’ll see files that let you add and replace vocabulary words. It’s doing all this as user script features so it’s not exactly the same as editing the core Conformer vocabulary, but thanks to the flexibility of Talon and the fact that it lets the user build their own dictation mode, it’s mostly equivalent to being able to directly affect the Conformer vocabulary.
Excellent! Thanks for that
Is there a trick to enabling the programming language grammars?
According to the knaus readme page, programming should be enabled by default for vscode but I’m not seeing that. I tried saying things like “force see plus plus”, but the commands inside of lang/programming.talon like ‘state else if’ ‘block’ don’t seem to be recognized even after that.
It “just worked” for me. Try opening a Python file and saying “state else if” (that’s something I just confirmed). I also have the Talon VS code extension activated, maybe you’re missing that? https://marketplace.visualstudio.com/items?itemName=pokey.talon
It is strange indeed. On the mac, which is where I’m trying it, it seems to work in a Python file now, but not on a C++ file. I do have the VS code extension activated.
it looks like the log is complaining that
NotImplementedError: Action ‘user.code_block’ exists but the Module method is empty and no Context reimplements it
2022-01-04 10:29:09 ERROR 5: talon/scripting/talon_script.py:590|
I don’t see a cpp folder in the lang/
although it’s definitely in the extension map. may be it is not implemented by default?
Sounds like that’s it. I suppose you could add cplusplus to the header of the talon file for C to pull all that in.
Okay after fiddling around for a while I figured out how to add it and it looks like you need added to both the c.py and the .talon files! Searching the slack channels it also looks like somebody created a cpp.talon file, so I’m not sure why that hasn’t been added to the knaus repo? In any case this is definitely going to be a learning curve and some of the grammar choices feel quite verbose. But one step at a time 🙂
Don’t hesitate to add terse grammar alternatives for commands you use often. You may run into some merge issues later on but they are easy to resolve.
Also do you find that it is a feasible solution for dictation? Having to switch out of dictation mode to do any sort of commands, and also having no select that I could find at least, and no correct seems like a pretty serious impediment vs Dragon eg. Or do you use it with Dragon and somehow get both?
I recently pushed some (but not all) of my dictation mode improvements out to knausj. In particular, I use an experimental feature (which I requested) that’s only in beta which makes mixing dictation and regular commands work well, by allowing some regular commands be used in dictation mode, but only allowing dictation commands to be chained with other dictation commands. Ryan recently pinned information about this to the beta slack channel (it’s called “anchor-file”), so take a look there (or at my repo: https://github.com/wolfmanstout/wolfmanstout_talon).
The other big improvement is gaze_ocr. I’m talking to Ryan about how we can get this shipped in a way that doesn’t require pip installs, but it’ll take some time because it will depend on new Talon APIs being exposed.
I take it there’s still nothing equivalent to the “correct [through ] in dragon though?
Well, I have “replace X with Y” through gaze_ocr, but I don’t think there’s anything like this in vanilla knausj. I haven’t used it, but there is a “draft mode” that lets you edit your text in an editor, and I’ve seen people use this with VSCode+Cursorless, which can easily do this sort of edit (albeit with a different grammar syntax).
wrt “idea of modulo line numbers”. I don’t see this anywhere in the Cursorless docs or recent commits. Has it been committed yet? I’m curious if there’s an equivalent to the commands I currently have ( inspired by Mark’s win32pad). Hightlight [through ]. Or even the one that copies then pastes the range into the current cursor position.
I don’t think they have pushed this out yet. In the meantime, you can refer to a line by referring to one of the marked characters in the line. For example, “take line arch past brov” would select a range of lines (I’m guessing that’s what highlight does?). One nice thing about this feature is it even works if you have multiple panes open, because each of those is marked up. They’ve been pretty busy launching new features so I’m guessing this is somewhere down the queue.
There are also Cursorless commands “bring” and “move” which copy/paste and cut/paste ranges, respectively.
Correction, they do now support modulo 100 as of 0.24! You say “row”, as in “take row 5 past row 10”. I have an FR out so that the second “row” isn’t needed (and the maintainer said he plans to pick this up): https://github.com/pokey/cursorless-vscode/issues/438
Awesome thanks! Yes I was actually chatting with them on their slack channel. I’ll have to dig around and see if I can customize the commands. They also documented a request I had to reduce the frequency of for and color updates to make it less distracting and help with string together commands. Now if they could add a natlink based Grammer that would be the icing on the cake. 🙂
How does Talon do with multiple monitors? I have 2 24″ plus a laptop screen that I all use for work. Home is same but w/o laptop.
I’ve tried Eye trackers but they seem to do only 1 monitor.
I’m not a regular user of multiple monitors, but I believe that support is robust. In general, the maintainers are quite thoughtful about code quality and robustness for a wide range of users. Also, I think I remember reading or hearing somewhere that eventually Talon may support multiple eye trackers (one per monitor). That’s an expensive solution but considering the field of view of eye trackers it seems like the only possible solution from a hardware standpoint.
I’m using 2 monitors with Talon, but not with eye tracking.
I switch app focus (on Win) with “focus Chrome” or just “focus” to bring up the Windows app selection dialogue.
I can also move a window to the relevant screen by saying “snap screen x”, and then “super up/down” to restore/minimise/maximise.
Works fine. I actually have an eye tracker and haven’t tried it with Talon yet, I will do that sometime soonish and report back.
The snap support, both single and multi- monitor, is nice, especially for Mac that does not have that built-in.
https://github.com/knausj85/knausj_talon/blob/master/code/window_snap.py
https://github.com/knausj85/knausj_talon/blob/master/misc/window_management.talon#L12
It breaks the commands up in talon and py files, similarly to how Vocola has vcl/vch and extensions. (although the lines are slightly different, eg, lists must be defined in py)
Hi James, did you get the eye tracking working with talon? I have a Tobii 4c that I got earlier, but wasn’t happy with it. I’m interested in seeing if it works better with talon. Thanks!
Definitely, it’s an essential part of my setup. I’ve used both the 4C and 5 and they both work well. As noted in my post, you will need to kill the Tobii services for it to work (which can be done from the ctrl-alt-delete task manager) because Talon uses its own direct integration with the hardware.
Don’t expect an increase in overall accuracy, though. There is some nice out of the box functionality in knausj for using it to control your cursor, although that’s not a daily driver for me.
I haven’t used it but I saw a recent announcement in the talon slack about an experimental modification that improves the accuracy or maybe extends the size of the monitor you can use. I suspect it’s only available for the beta, but you may want to go over to the slack and check. There’s an eyetracking channel.