Delaying mail delivery

My current mailfilter has two features which increase my day-to-day productivity:

a “tickler,” which is a reminder system inspired by the tickler file component of David Allen’s Getting Things Done action management method: I can submit emails and notes to the tickler along with a timestamp in the future, and the tickler delivers the mail (or note) to my inbox when the timestamp has passed.
the ability to delay certain types of mail (e.g. Debian mail is held until the weekend, while news items and the like are only delivered at night).

My current implementations work alright, but they’re brittle and crap. What’s worse is that the two features could be combined and handled by one and the same tool, but I implemented them differently for now:

The tickler consists of a Maildir, to which I can submit mails, either by mailing a note to a specific email address (currently broken, thus not linked from here), or by adding a tickle stamp (X-Tickle header) to a message with this script_, saving it to the local tickler mailbox and asking offlineimap to shove it to the server.

On this server, the tickler queue is regularly scanned by a script, which resubmits mail whose timestamp has expired to my mailfilter, where it is treated somewhat specially as a resubmitted mail.

After almost three months with this setup, I can identify the following shortcomings:

Since the queue is only processed on the server, I cannot register a mail with the tickler without going online and synchronising my mail before the timestamp expires. So if I am on my way to a conference on Monday and tell the tickler to resubmit a given message on the next day, but I don’t connect before Wednesday, I won’t get the mail on Tuesday (because the mail does not actually reach the tickler until Wednesday, when it’s immediately resubmitted).

I have not found an algorithm that would let me run a queue processor on the server and my laptop without the two causing mail duplication and potentially interfering with each other.
I keep all my mail in a single Maildir to make searches easy. Since mail registered with the tickler is stored in a separate Maildir, I often need to search two Maildirs if I am looking for a specific message. This is awkward.
The queue processor has to iterate the entire queue, which is fine for a hundred messages, but it does not scale.
I have not found a way to ask mutt to tickle me about a message I send without going to the sent messages folder and scheduling the message for tickling manually. One approach would be record=no and a custom sendmail script, which would store mail to the store or tickler Maildirs, depending on whether the X-Tickle header is present, but I want to avoid a custom sendmail script for reasons unknown. Another approach might be to set bcc to the appropriate tickle address, but this would result in duplicate mails, unless I set record=no individually.

When I implemented the delay queue, I knew of these problems and went a different way: delayed mail is stored in a Maildir and a (msgid, timestamp, filename) tuple is inserted into an sqlite3 database on the mail server. This script_ regularly processes the queue.

If I just sent you the chills, at least we have the same taste. There are numerous problems with this approach, the foremost being that Maildir filenames are not guaranteed to be constant: mails jump between the new and cur directories, and tags, such as seen are encoded in the filename (thus, symlinks also cannot be used). My script now uses an ugly heuristic, which at least makes it work. I should investigate whether inodes could be used instead as I think those wouldn’t change throughout the lifetime of a mail, at least while it’s not moved between folders.

I initially considered just dumping messages to files and encoding the timestamp in the files’ mtime, but then I would not be able to access the queue with mutt in case I needed to fetch a delayed mail prematurely, or if I wanted to synchronise the queue with offlineimap as well.

The past few days, I’ve been condensing experiences from both approaches and am working out a new technique to combine both features. In essence, I think the database/index approach is the best, if I can figure out a way to uniquely identify mail message files, ideally across folders. Assuming I can use inodes for that, delayed mail would then be stored into the delayed Maildir and an (inode, timestamp) tuple saved into the database. Tickler mail would be stored along with all other mail in my store Maildir and would get a similar entry in the database.

This approach solves some problems and leaves others. Assuming I synchronise the store Maildir remotely (which I do), then I can easily fathom making modifications via IMAP which causes orphan records in the database (if only IMAP would allow me to store key=value pairs for mails…). Furthermore, I’d have to submit mails to the tickler by bouncing them to an email address, and deleting the local copy, unless I want duplicates. If now the mail is somehow dropped, I’ve lost mail.

Still unhappy about all of this, still searching for a better implementation, I’d appreciate any feedback!

NP: Luminous Flesh Giants: Duma I Upadek