You're going to want to set both bidi-paragraph-start-re and bidi-paragraph-separate-re to "^":
(setq bidi-paragraph-separate-re "^"
The documentation for these variables says:
If you change the value of this variable, be sure to change
the value of ‘bidi-paragraph-separate-re’ accordingly. For
example, to have a ...
I've since find out that wrapping the phrase on both ends with the LEFT-TO-RIGHT MARK gives the desired result. To do that position your cursor appropriately and C-x 8 RET LEFT-TO-RIGHT MARK. The visual display will fix itself and the text will be shown correctly.
You need to make sure that your Emacs is built with support for text shaping. AFAIK, it's the --with-m17n-flt option to configure. This option is activated automatically but only if the relevant include files are found.
On Debian-derived systems a good way to make sure you have all the relevant packages installed is to use something like:
(Converting running comments to an answer.)
This is a partial solution. It's unclear why AUCTeX perpetually tells you "You should run LaTeX again..." after compiling (but I can confirm that it happened for me as well). Best guess is that it's something about the fact that polyglossia requires either xetex or luatex, and perhaps the latter don't play as ...
From what I can see, largely in src/bidi.c and src/xdisp.c. The latter implements the bulk of the redisplay, and there are numerous relevant comments about the bidi implementation in its header commentary (search for "Bidirectional display").
Bidi reordering is controlled by the buffer-local variable bidi-display-reordering. You make it default to false by adding the following incantation to your init file:
(setq-default bidi-display-reordering nil)
The variable bidi-paragraph-direction is not the answer to your problem, since it affects the whole buffer. Instead if you want to change the directionality of a particular piece of text, you'll need to insert special control characters.
E.g. you could insert an LRM (see https://en.wikipedia.org/wiki/Left-to-right_mark) character just before the opening $ ...