Why is dovecot necessary?
Dovecot is not necessarily necessary; it depends on your setup, needs and preferences.
Can one just read a maildir directly into gnus?
Of course one can, using the
nnmaildir select method. See also the group parameters it recognises and a comparison of Gnus mail backends. Here is an example of my previous configuration:
i dont really understand why dovecot is needed for maildit+gnus so would also love to get insight into that
I, too, use
mbsync to synchronise multiple Gmail mailboxes with Maildirs under a single parent directory on my system. Up until a couple of days ago I used
nnmaildir and was happy with its performance, even in groups containing thousands of articles. The only slow operation was starting Gnus, which on my machine and with 3GB of mail took 9 seconds with the default
gc-cons-threshold and 6 seconds with a temporarily increased value. This may be related to mutbuerger's comment; I am not sure. More importantly, though, I never faced any issues like the ones described in this SO question and the discussion thread linked therein (I use the latest version of Gnus in the
master branch of the Emacs development tree).
Since then, however, I have configured Dovecot to serve said
mbsynced Maildirs, which Gnus reads via the
nnimap select method, for two main (possibly related) reasons concerning Gnus (as opposed to unrelated reasons like personal/academic interest):
Sometimes Gmail's special labels like
[Gmail] All Mail,
[Gmail] Bin, etc. did not behave like other labels/folders (unsure as to the correct terminology here). For example, I sometimes would not be able to select them from the group buffer without first rescanning them for new articles. Or, more annoyingly, I sometimes would not be able to move articles to them from a different summary buffer without first having selected or scanned them. I have no idea why this was the case or who was to blame, and I haven't yet found the time to investigate.
The straw which broke the author's back was
nnir. This select method provides excellent OOTB support for
nnimap and can even be easily configured to use the Gmail search key
X-GM-RAW, in case you connect to their servers directly from Gnus.
nnir supports and can be extended to support multiple search engines for local Maildirs, but I only had time to try one. I picked
notmuch because it is the newest and most actively developed project out of the supported engines. I soon discovered, however, that the relevant
nnir code has not been updated in years, does not work at all with my multiple-Maildirs-under-single-parent-directory setup and has a few bugs which I will report and attempt to fix when I find the time.
I even resorted to rewriting some of the
nnir-run-notmuch code and registering it as a new search engine in my
gnus-init-file as an initial foray into debugging and improving the original code.
This is where I discovered that
notmuch does not handle the aforementioned special Gmail labels very well OOTB either. In particular, filtering with the
folder: keyword on such a label causes the whole query to return nothing. This made it impossible to search across the whole Maildir, let alone be confident in the thoroughness of the results. One day I would like to investigate all this further, but right now I just need a working mail setup.
Which brings me to my main justification for Dovecot. It is a large, mature, feature-rich yet modular and performant, specialised yet configurable and widely used project. I found it quite easy to configure as it has sensible defaults. More importantly, it has and will probably continue to have better IMAP support than Gnus, and IMAP in turn is a more mature and widely supported standard than the Maildir format. I would indeed like to see and help Emacs' IMAP and Maildir support improve, but Dovecot is simply the more stable and hence sensible option for (seemingly) a lot of people at the time of writing.
Having said that,
nnmaildir is better than
nnimap at filling in threads (i.e. looking up articles in any group by
Message-ID), unless you enable the registry, which in turn results in some space and time overhead when selecting groups, etc.