BRIEF: like the title says, I want to know how best to version control newsrc files such as Gnus .newsrc.eld and possibly the legacy .newsrc

By "version control" I mean something like git/hg/rcs/cvs. NOT simply .~1~, .~2~...


I am religious about version control systems like git/hg/rcs/cvs. Tools that allow me to keep multiple versions, diff them to see what has changed, comment changes, and possibly restore to an earlier state.

I do not mean simply keeping extra copies around in the file system. in particular, not emacs's .~1~, .~2~, ... controlled by the unfortunately named variable version-control. Although I suppose the main difference is that proper version control tools make it easy to manage.

Restoring to earlier state is particularly important in situations such as what happened to me last night, when I seem to accidentally have killed most of my Gnus/IMAP/Gmail groups because of an unfortunate interaction of Gnus unmodified key-bindings and Dragon speech recognition/dictation.

(Gnus groups == IMAP folders == Gmail labels)

I spent quite a bit of time organizing several hundred Gnus groups into Gnus topics. It was quite annoying to have to redo it.

This is not just a Gnus/topics problem. I recall having similar issues in the last century using newsreaders that used the older .newsrc and .newsrc.el files -- no topics, but remembering which of several hundred Gnus groups you want to subscribe to is also something you want to version control.

Unfortunately many of the tools that use .newsrc* files can suffer file corruption problems. I remember this from the last century, and I have suffered several instances since I restarted using emacs Gnus a month or so ago.

One problem is that the .newsrc and .newsrc.eld files keep track of several different types of information:

(SLOW) Information that changes relatively slowly, under user control

  • Gnu's mapping of groups into topics, and their sort orders
  • The list of subscribed (news)groups / IMAP folders / Gmail labels

(FAST) Information that changes quickly, as messages are received and read and deleted

  • which messages have been read or not in the various groups
  • other metadata

Moreover, the (FAST) and (SLOW) information is interleaved at fairly fine granularity within the .newsrc.eld file. multiple instances on the same very long line. line oriented grep tools need not apply, but at least it's elisp so parsing tools should be available.

I most emphatically want to version control the (SLOW) information -- the stuff that I might've spent quite some time developing, which I would lose if the .newsrc* file(s) were destroyed or corrupted.

I do not need to version control the (FAST) information. It changes too quickly, behind the curtains. Sure, sometimes it's nice to be able to recover it after problems, but there's always the option of just prefetching all messages in your subscribed groups in your existing topic hierarchy, and catching up. Some lossage, but fast, certainly faster than rebuilding your subscriptions and topic hierarchy.

Similarly, the list of subscribed (news)groups, the topic hierarchy, and the sort ordering thereof, is reasonably portable between systems. Whereas the (FAST) information, e.g. which message numbers have been read or not, is much less portable between systems. (E.g. different NNTP servers accessing the same USENET newsgroups, and/or slightly inconsistent mirrors or archives of the same mailing lists.)

Of course, one can simply git checkin .newsrc.eld in much the same way one can version control binaries. Note that this is more like version controlling a dynamic binary database; version controlling a static set of binaries has reasonably good value, because you can tell when things were changed infrequently, even if it is hard to tell what the changes were. Whereas version controlling the (FAST) information will always indicate changes. It is just hard to tell if the changes matter or not because diffs do not work very well.

I suppose that in some ways what I'm really asking for is a good tool to diff .newsrc ... .newsrc.eld files, where I can filter out the (FAST) changes and only look at the (SLOW) changes that I care about. Which might be integrated with a version control tool like get to avoid doing you checkins if the (SLOW) information has not changed.

Summary: what are the best ways to manage files that have mixtures of fast and slow information such as that in .newsrc.eld? generic and/or emacs specific.

  • Perhaps pretty printing the.newsrc.eld. Elisp putting different data types/elements on different lines. Diff|grep -v message number ranges, distinct from values I care about. I already transform XML in UTF 16 to UTF-8 to diff —- this goes just a step further to avoid checking in if the filtered stuff has not changed. // I wish I knew of a good tool to diff tree/nested data, that avoids miss leading deaths when an entire sub tree is moved. Many data formats need such.
    – Krazy Glew
    Dec 31, 2021 at 1:38


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.