How iOS and macOS Dictation Can Learn from Voice Control’s Dictation

Originally published at: How iOS and macOS Dictation Can Learn from Voice Control’s Dictation - TidBITS

Oddly, Apple has two seemingly independent dictation capabilities available for both iOS devices and Macs. The standard dictation feature is, unfortunately, pretty weak in comparison with the dictation feature that you get when you enable Voice Control. Adam Engst looks at the differences and suggests features that Apple could move over from Voice Control.

macOS Big Sur beta 1 was released at WWDC on Monday, June 29, 2020. I coughed up the money for a developer account, and downloaded and installed it on Tuesday, because I was keenly interested in what new functionality/features Apple added to Voice Control. I went through the command list to see if Voice Control had any new commands. Specifically looking for the spelling command/mode I’ve been asking for. Remember I have no use of my limbs. So if Voice Control misrecognizes a word, and does not have an appropriate alternative in its correction list, I can’t just grab the keyboard and type in the appropriate word. I need to make the correction by voice and if Voice Control has spelling functionality I could make the correction by voice. Unfortunately no spelling commands/mode yet.

There are a few new commands in macOS Big Sur beta 1. The new commands by category are as follows:

Basic Navigation:

  • <item name>
  • Find next text <phrase>
  • Go to sleep Mac
  • Wake up Mac

Overlays & Mouse:

  • Show numbers continuously
  • Show grid continuously

Dictation:

  • Press Space key

Text Selection:

  • Select <phrase> emoji

Text Editing:

  • Insert <phrase> after

Accessibility:

  • VoiceOver activate
  • VoiceOver interact
  • VoiceOver read all
  • VoiceOver stop interacting
  • VoiceOver select last item
  • VoiceOver select next item
  • VoiceOver select previous item
  • VoiceOver actions
  • VoiceOver applications
  • VoiceOver commands
  • VoiceOver contextual menu
  • VoiceOver item Chooser
  • VoiceOver notification menu
  • VoiceOver rotor
  • VoiceOver next heading
  • VoiceOver previous heading
  • VoiceOver next link
  • VoiceOver previous link
  • VoiceOver find
  • VoiceOver find backward
  • VoiceOver find forward
  • VoiceOver verbosity
  • VoiceOver help
  • VoiceOver more help
  • VoiceOver hint
  • VoiceOver describe image
  • VoiceOver where am i
  • VoiceOver speak summary
  • VoiceOver stop speaking

Note: Accessibility category is brand new in Big Sur. It was not there in Catalina.

In addition to the new commands, Big Sur Voice Control seems to be quicker. Show numbers in Safari now numbers links on web pages. This makes it much easier surf the web completely hands-free.

3 Likes

If you think that it is frustrating to use the voice options, you try to do it without sight. My father’s macular degeneration has finally reached the point where is both legally and practically blind. While he can see enough to navigate the house, he can no longer interact with the computer or phone. The OS is a real disappointment when it comes to solving these problems. You should try to read some of his email or text messages, Siri really butchers things quite often. You have to develop a bit of skill at deciphering cryptic messages to be successful.

-Chip

The ability to do mouse actions by voice: click and double-click.

I work a lot in Blackboard, which requires a great deal of clicking to do most things a teacher needs to.

Extra Scripts used to be able to do that, but then a system upgrade disabled it.

If anyone has any ideas how to click by voice, I would love to hear about them!

Thanks!

Thanks so much for the comparison list, @tscheresky!

I’m glad you noted that there is no spelling mode, since that’s a capability I’ve wanted as well, though I didn’t know enough to know what to look for or how to tell it wasn’t there.

I’d encourage everyone interested in Voice Control to submit the lack of spelling mode as feedback to Apple.

@pellerbe, it’s built in! With Voice Control turned on, move the pointer to the right spot and say “Click” or “Double-click.”

Well, I’m glad to hear that! [so are my arms].

Which is the earliest system that has voice control with that included?

Thanks so much!

Polly

PS I now have a reason to buy a new Mac, never a bad thing to have.

I would assume, though I don’t know for sure, that it was part of the major Voice Control revamp in Catalina.

Voice Control, the Accessibility feature, was first introduced with Catalina.

Because I could not find the complete list of Voice Control commands online, and I wanted a complete list I could review to come up to speed on Voice Control more quickly, I created the following documents and shared them:

2020 iPadOS 14 Voice Control Commands
https://drive.google.com/file/d/1qD_V3YlZmSJ5UOJJlP47-PYNnk1OTDsr/view?usp=sharing

2020 macOS Big Sur Voice Control Commands
https://drive.google.com/file/d/1P4dh1H9pzEedCv2-1xXyE37Ej0QW_7-U/view?usp=sharing

Please note: the tabs (a.k.a. sheets) at the bottom of the spreadsheet (a.k.a. workbook) represents the categories for the voice commands. Each tab has the commands for the particular category.

5 Likes

Check out the 2020 macOS Big Sur Voice Control Commands link I just shared. It contains all the voice commands by category. Including mouse commands.

1 Like

Dictation is something of great interest to those of us with manual dexterity problems, like me. Having used dictation in both macOS and iOS since its inception for this reason, it’s been interesting to track the ups and downs of its usability over this time period, and developing workarounds for its most annoying foibles. (I’m using “interesting” in its most diplomatic sense here.)

This article is greatly appreciated — while I am unwilling to upgrade my Mac to Catalina yet, I’ll see how Voice Control behaves on my iOS devices using the latest system.

By the way I used to use Dragon Dictation and I’m a little surprised the article did not mention it at all. It had pretty good voice editing capabilities.

That’s because I’ve never used it. :slight_smile: I can’t pretend to be an expert in dictation in general—what has piqued my interest is how it has become sufficiently available to everyone now, and sufficiently good that even those who don’t need to use it might be tempted.

Another problem with dictation in iOS and macOS is that it works for only a certain amount of time—about 60 seconds (iOS) or 40 seconds (macOS) in my testing. As a result, you cannot dictate a document, or even more than a paragraph or two, without having to restart dictation by tapping that microphone button.

Editing after the fact is much easier on the Mac, of course, but you can’t so much as click the mouse while dictating without stopping the dictation.

I haven’t experienced any time limitation with dictation for MacOS–through Sierra anyway. I do download ‘enhanced dictation’ for offline use (a GB or two depending on system). Editing by voice was easy on El Cap, though you need to turn on “Enable advanced commands” in Accessibility/Dictation/Dictation Commands… There’s a list of the editing commands there, and you can keep a list open while dictating for reference. In principle it should work the same way on Sierra, but for for some reason instead of editing it just parrots back the editing commands on my system. I may have installed a conflict of some sort.

Clicking the mouse and otherwise editing via mouse and keyboard doesn’t dismiss dictation on either el cap or sierra. It might time out after awhile if you wander off or spend a lot of time just thinking, but mine has just been open and idle for about 5 minutes and is still there.

I haven’t tried dictation in Mojave or Catalina yet since my modern mini has no microphone, but all of the preferences look the same. Has dictation really regressed so much? If so is it because Apple severed relations with Nuance a while back?

Enhanced dictation, as you point out, does behave slightly differently. I’ve tried it many times over the years, especially when my power goes out (as it does frequently) and I lose Internet connectivity, but I always end up going back to the Internet-based dictation because it’s slightly more satisfactory for my use. I do appreciate the reminder, though, because I recently upgraded to high sierra and I don’t think I’ve tried it yet.

I would say dictation has definitely regressed, although of course it’s impossible to say why. For example, lately on my Mac I have not been able to get it to capitalize anything to save my life. Dictation on my phone tends to work a lot better.

I can’t comment on El Capitan and Sierra—I haven’t used them in many years. :slight_smile: I don’t remember them as being sufficiently functional for dictation that I ever considered using it.

Perhaps @tscheresky is more familiar with what was possible in the past, though he would have been relying on Dragon Naturally Speaking then.

“Sufficiently functional” is relative. In my case I can only type with two fingers, so even in its early days, dictation was a lifesaver. :slight_smile:

Adam, you are right. The only part of macOS built voice recognition I’ve used, going back to Snow Leopard, was its ability to turn Dragon Dictate for Mac’s microphone on after I have turned it off, and restarting Dragon Dictate for Mac after it has crashed.

I don’t see a need for macOS Dictation because Voice Control has everything Dictation has and more. macOS Voice Control is only missing a few features that would make it a complete replacement for Dragon Dictate for Mac.

I still use Dragon dictate for the Mac to this very day and it has a very high accuracy rate. The accuracy rate is the key thing you want in voice recognition because as you noted, it gets frustrating to have non-sensical words show up in your text. One thing that you did not mention in the article is the use of a high quality microphone to improve accuracy on the Mac. A high quality microphone can greatly increase the accuracy rate. When errors do occur, I find it much easier/intuitive and far more efficient to edit errors with the keyboard and mouse.

I do too. I’m using it on my primary computer under Catalina. I even have it working on my secondary computer under macOS Big Sur beta. However those days are numbered. Not only has Dragon for Mac not been supported since October 2018, Dragon speech engine is x86 based. Therefore I’m guessing anyone wanting to move to Apple Silicone (me) won’t be able to do so unless Apple’s Voice Control comes up to speed and adds the missing features needed to become the alternative for Dragon for Mac.

As I see it, here is the path for running Dragon on the Mac. You tested it on Big Sur and it worked and that was something I was personally wondering. Next on Apple silicon it should work just fine with Rosetta 2. If Apple removes Rosetta 2 in the future, then we head for virtualization. Will Parallels and Fusion have a solution for Apple Silicon? My guess is yes. So I think Dragon for the Mac will be sustainable for the foreseeable future.