1

BRIEF: When speech recognition software occasionally inserts extra blank spaces at the beginning of lines that are being dictated into emacs fundamental or textmode, how can the bad stuff below be avoided in the 1st place, or fixed up most conveniently?

 blah blah blah
   blah blah blah
    blah blah blah

---+ DETAIL

I use speech recognition to dictate text into EMACS; also to invoke emacs commands, sometimes directly, but usually from "speech commands" in the speech software package.

Specifically Dragon speech recognition from Nuance (now owned by Microsoft).

Dragon typically inserts one blank space between words, and 1 or 2 blank spaces before the next word after the end of the sentence, if Dragon thinks appropriate. E.g. if Dragon thinks you are in the same flow of text, not at the beginning of a paragraph or after a hard newline. This works for "speech-ready" applications that allow Dragon to determine the state or context of the dictation.

Unfortunately, EMACS is not speech aware. Dragon will infer that it is at the beginning of a line if Dragon has seen the user dictate "newline" or some other line ending utterance. But Dragon's notion of state or context will fall out of sync (a) if the user types a key, rather than going through Dragon, or (b) if the user executes an emacs command like goto-line or kill-line that Dragon, and (c) in similar circumstances.

These extra spaces cause problems. (As similarly does Dragon's inferred capitalization of words, but that's not part of this question.)

This question seeks advice on how best to mitigate one particularly annoying aspect of this problem: when these extra spaces are inserted at the beginning of a line of text.

Below I provide a probably too large example of some of the artifacts that occur when dictating into EMACS fundamental mode or text mode. Both in my default setup, but also with all emacs startup files disabled. With electric-indent-mode both on and off.

The example is probably too large, and I may reduce its size in subsequent updates of this question. I'm somewhat reluctant to actually edit the example - I would prefer to have the example be exactly what I dictated - but that's a bit hard to make concise.

The short version is:

In the default configuration, with electric-indent-mode on, dictating text text, While also dictating new lines, mostly looks OK

 this text is dictated,  and Dragon sees the command  that ends this line
this is the new line
and this is another line, again with Dragon seeing the end of line
 but  here the previous line was terminated by physically pressing Enter
  here as well
    and so on
      because with electric-indent-mode on,
       each new line is aligned to the previous
         And then the extra space indents further.
 the above had electric-indent-mode on.
 with electric indent mode off
 we don't get this incrementing alignment behavior
 but we do get extra whitespace at the beginning of the line
 if Dragon  is not aware of the end of line.
  • It's not that hard to fix, e.g. by pressing TAB to invoke indent-for-tab-command, which typically invokes indent-relative.
    • although that does not remove the extra space of indentation that might've been inserted at the very 1st line.
  • I am a bit embarrassed to admit that I frequently resort to indent-rigidly
  • but I'm even more embarrassed to admit that quite a few of my git commit messages in my private repo exhibit such damage - especially when I had just started using Dragon and emacs.

I am sure there are other, better, ways to prevent this problem from happening and/or repair it once it has happened.

  • There are probably some standard ways to repair this damage that I don't know about: settings and/or commands that are especially convenient.

    • I look forward to answers that tell me that I am an idiot for not knowing about them.
  • I suspect that avoiding this damage requires that new code be written.

    • Although perhaps somebody has already solved this problem
  • And I seek advice as to the best ways to approach coding was to avoid this problem in the 1st place, or at least where it gets too bad.


I've already tried some things, and considered other approaches. I will post some of these as pseudo-answers, but I will not do so immediately.

Posting to https://emacs.stackexchange.com because the answer is almost certainly to write elisp code, since emacs is much more flexible than Dragon. some of the approaches that I have considered are purely elisp based, but some are a hybrid of elisp code and Dragon configurations, possibly Dragon extensions or plug-ins.

This is one of the of what will be several questions related to making emacs friendlier to speech recognition and voice control software. In particular Dragon speech recognition software from Nuance (now owned by Microsoft), because that is what I'm currently using, but I'm open to other speech recognition and voice control software packages.

---+ BIG EXAMPLE

 blah blah blah
  blah blah blah
   blah blah blah

 ---+  PROBLEM:  incrementally incrementing indentation

    frequently text files that I have dictated into using Dragon speech recognition
     look like this, with successive lines frequently having one extra blank  on the left
      being progressively more indented
       because Dragon has not seen  me physically pressing the enter key
        so Dragon thinks I'm still dictating on the same line or in the same paragraph

    however if I say "newline" or "press enter" on the end of the line
    no problem occurs, because then Dragon has seen the actual transition

     the extra indentation occurs because I frequently hit enter on the keyboard,
       forgetting to say "newline" or "press enter" to  Dragon
 it also happens when I
       go back to a line
  i.e. when I'm editing nonlinearly
  I have emacs set to delete white space at the end of lines
    so if I move to a blank line
    pres TAB to align
     I still get an extra blank.

---+  BACKGROUND

 I use speech recognition because  it hurts me to type too much.
 But I can still type, and I often use a "hybrid mode"  where I dictate most
  but where I will  press a small number of physical keys
   typically  large easy to find keys like and enter
    and I will usually use a trackball
     rather than using the IMHO very annoying ways of moving the mouse by voice.
 at 1st I thought I was being "impure" by using such a hybrid mode
  for speech control, but it turns out a lot of people do this
    if they can.

 Observation:
 + speech recognition is pretty efficient for dictating text.
 +  Also for invoking "speech commands"
    + including emacs commands by voice
    + far fewer problems avoiding key-binding conflicts
 +  but speech recognition is somewhat inefficient for simple keypresses
    +  compare saying "press enter"  to simply hitting a large enter key


Yes: I know about  things
+ demos of emacs by voice such as
  + https://whalequench.club/blog/2019/09/14/strange-loop.html
  + Tavis Rudd's famous  "coding by voice...  using a combination of grunts and animal sounds"
+ I also know about things like
  + talon
  + DragonFly
"Know of",  I've chosen to go a slightly different direction.
   this is not the place to discuss overall approacesh to  voice control.
I just want to talk about this 1 specific problem  that arises when I
 use Dragon to dictate into emacs fundamental mode.

 blah blah blah
  blah blah blah

 if I  keyboard 2 blank lines,  I return to the right

 emacs help tells me that the key-binding "RET (translated from <return>) runs the command newline"

 if I say
  or press
   TAB
    on such an indented line
     it gets fixed (which I'll go back and demonstrate)
      because
      "TAB (translated from <tab>) runs the command indent-according-to-mode"

I forget
 if I have
 auto indent on
  indeed it looks like I do

---+ no startup files

OK, I just  started emacs --no-init-file --no-site-file
to get an absolutely fresh set up
 but see what happens now
  Yep I still get the  extra indentation
   if I start new  from the keyboard
 what happens if I do goto-line in emacs (target this line)
  Yep I get the indentation
  The above is with electric-indent-mode t,  i.e. on, the default in recent emacs

---+ disabling electric-indent-mode

Now I have disabled electric indent mode.
no extra space when Dragon saw the new line
 this is what I get when  actually press the physical key enter
 but of course I don't get the incrementing  behavior
 I just get one extra space at the beginning line.
 But that one extra space can cause problems e.g. in some  markup languages.

---+ org-mode

 now I have switched to Org mode
+ 1st list item
+ 2nd list item
  + again, if I  hit the enter key

  but it is not just  hitting the enter key.
+ Dragon also does not see the state of the buffer when I have
 executed an Emacs command, whether invoked from the keyboard or from
 the voice command, 
  + e.g. goto-line
  +  e.g.  killing a line and then typing new contents

1 Answer 1

1

Some things I've tried and/or considered

If fully fleshed out these might be accepted answers, but I'm hoping for something better.

---+++ OK, but...: fix-+1-indentation

I've written elisp to compare the indentation of the current line to that of the previous line. If the current line is 1 space more indented than the previous line, I can remove that extra space of indentation.

I can then invoke this fix-+1-indentation command

  • E.g. I can invoke fix-+1-indentation via a new \n or Enter keybinding, or as defadvice on an existing.

    • a new line may be incorrectly indented
    • but the indentation will be fixed if you press enter at the end of the line
    • doesn't fix incorrect indentation when you use other emacs commands like goto-line
    • every time I encounter a new mode has a slightly different newline binding (org mode, and pop-org mode, I'm looking at you), I have to do it again
  • E.g. I can bind it to blank space, or...

    • gets invoked a lot
      • all not as bad a performance problem as I feared
    • works immediately
    • but requires the user to type ^q space or something similar to actually insert a space that will not get deleted

This actually works fairly well

  • both the generic fix-+1-indentation command

    • and its siblings align-to-previous-line and align-to-next-line (which take arguments)
    • and can be invoked on regions one line at a time
  • and the implicit versions invoke fix-+1-indentation on Enter or space or other key bindings.

But they aren't really transparent - they amount to creating a new indentation mode. one where indents of +1 wrt the previous line are disallowed or mapped to +0, but where indents of > +2 are allowed.

It's certainly not the same as the present electric-indent-mode behavior when typing normally. Actually, in many ways I prefer this "forbid +1 indentation mode to the present electric-indent-mode behavior. But it is different, so I won't approve this answer right away.

---+++ Have emacs drop the extra space - but how does emacs tell?

---++++ Distinguish by timing?

If emacs could tell that a space character at the beginning of a live text was coming from the Dragon speech recognition software, elisp could do something about it. ... Probably by extra behavior in the key-binding for or advice added to space.

But I don't know how emacs to distinguish a space character coming from Dragon from a space character produced by a keyboard.

It might be possible to tell via timing: the speech recognition system emits the {space}blah-blah-blah with much tighter timing than humans can. But AFAIK Emacs does not make character arrival time available to elisp.

Moreover, although usually much closer together, not always. This would be slightly flaky.

---++++ Distinguish by ... Dragon plug-in?

More background: most users of Dragon speech recognition, dictating into non-speech-aware/ready applications like emacs, go through a "dictation box". Dragon provides the default dictation box, but there are quite a few third-party dictation boxes such as DragonCapture or Speech Productivity Pro (who sells several in the same package).

If I had the source code to such a dictation box I could have the dictation box change all of the space characters that Dragon emits into something else to be sent to emacs. And then emacs can handle them specially: suppressing and beginning of line, converting back to ordinary space elsewhere.

This avoids the slippery slope of changing the key-binding for the space key, in every mode that loads its own non-self-insert version.

Unfortunately, I don't have the source code. I don't know of any open source "dictation box" for Dragon. As far as I know the open source tools such as Talon and DragonFly that connect to Dragon via NatLink/Python plug-ins only handle speech commands, not dictation.

  • I that somebody can point me to and open source "dictation box" for NatLink
  • or for documentation as to how to build one.

I don't think it would be quite fair to the programmers who developed and sell the closed source dictation box substitutes like DragonCapture and Speech Productivity to reverse engineer their stuff.

What else?

I've certainly considered many different approaches, but they all boil down to something that resembles `fix-+1-indentation` (or more general), bound to differing events, and/or distinguishing Dragon spaces at beginning of line.

I look forward to other, better suggestions.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.