How to alphabetize strings with diacritics?

Question

Q: how do I alphabetize strings with diacritics?

Problem

I have a long list of author names, some of whom have letters with diacritics in their names (e.g., "á" or "é"). I want to sort this list alphabetically.

The problem: sorting the list with string-lessp does not sort them alphabetically.

According to the accepted answer in this post from a sister site, English ignores the diacritics in sorting except to break ties. (Other languages do it differently.)

Toy Example

Here's a toy example. The list of letters that I start with is already ordered alphabetically. When I sort this list with string-lessp, however, it sorts them in what I presume is unicode point order rather than alphabetical order:

(let ((letters '("a" "à" "á" "â" "b" "c" "e" "é" "ê")))
  (sort letters #'string-lessp))
;; => ("a" "b" "c" "e" "à" "á" "â" "é" "ê")

What Do I Do?

How can I alphabetize strings with diacritics in them?

At a minimum, I'd like to respect the "ignore diacritics except to break ties" rule described above. Ideally, I'd like to be able to alphabetize according to an arbitrarily defined language, but I'll happily settle for English.

Answer 1

If your system locale is set to something that will properly collate diacritics (not POSIX), this should work for you:

(let ((letters '("é" "a" "à" "c" "â" "b" "á" "e" "ê")))
  (sort letters #'string-collate-lessp))
;; => ("a" "á" "à" "â" "b" "c" "e" "é" "ê")

If that doesn't work, you can supply a locale string as the third argument to string-collate-lessp to get what you want. With American English on a POSIX system, for example:

(let ((letters '("é" "a" "à" "c" "â" "b" "á" "e" "ê")))
  (sort letters (lambda (a b) (string-collate-lessp a b "en_US.UTF-8"))))

(For American English on MS-Windows, replace "en_US.UTF-8" with "enu_USA.1252".)

If you want to dig into this a little more to see what happens under the hood, I recommend taking a look at the function definition of str_collate in src/sysdep.c.