6:45 p.m.
Hi,
Of course, very OT.
I have been mulling a move from sylpheed to neomutt (because it is not clear to me that
sylpheed's development is happening all that much which is a worrisome sign for its
future) and so am also deciding on whether to keep the mh folder structure or move to
Maildir. I do have a lot of subfolders so it is a bit of a pain to convert everything, but
I think that I can probably write a script for doing so. But I wanted to ask if it is
worth it to make this change. Now, I know that sylpheed's mh structure is not exactly
the same because it puts in some marks and sequences in each directory and does not touch
the traditional .mh_sequences touch, but I do not see if that will matter much in terms of
the conversion.
Anyway, thoughts? I simply want to know if it is worth making the change which will
require some scripting (and some inevitable disruption).
Many thanks,
Ranjan
10:17 p.m.
On 14Oct2020 18:45, Ranjan Maitra <maitra(a)email.com> wrote:
I have been mulling a move from sylpheed to neomutt
Try regular mutt. A lot of development has happened in the last year or
so, and neomutt may no longer have many features not now available in
the mainline.
Also, join the mutt-users mailing list:
http://lists.mutt.org/mailman/listinfo/mutt-users
http://www.mutt.org/mail-lists.html
(because it is not clear to me that sylpheed's development is
happening all that much which is a worrisome sign for its future) and so am also deciding
on whether to keep the mh folder structure or move to Maildir. I do have a lot of
subfolders so it is a bit of a pain to convert everything, but I think that I can probably
write a script for doing so.
Plenty of us have scripts for this kind of thing, and you can even batch
invoke moutt for such a conversion (with some scripting around it).
But I wanted to ask if it is worth it to make this change. Now, I know
that sylpheed's mh structure is not exactly the same because it puts in some marks and
sequences in each directory and does not touch the traditional .mh_sequences touch, but I
do not see if that will matter much in terms of the conversion.
Mutt will read mbox, Maildir and MH. So you don't need to make both
changes at once.
Anyway, thoughts? I simply want to know if it is worth making the
change which will require some scripting (and some inevitable disruption).
I moved to Maildir from MH folders many years ago. Maildir at least has
good concurrency support (race free message insertion).
My current setup is Maildir for folders I'm active in (primary inbox,
folders for things I use a lot - python, mutt, so forth). I use mbox for
infrequently accessed folders and archive folders, as it is more compact
in terms of disc usage.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
10:42 p.m.
Thanks very much! I have not been able to see a recent listing of differences between mutt
and neomutt so that is good to know that mutt may be enough for me (in terms of features).
I will take a look there.
Thanks also for your thoughts on my original topic.
Ranjan
On Thu, 15 Oct 2020 14:17:00 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 14Oct2020 18:45, Ranjan Maitra <maitra(a)email.com> wrote:
>I have been mulling a move from sylpheed to neomutt
Try regular mutt. A lot of development has happened in the last year or
so, and neomutt may no longer have many features not now available in
the mainline.
Also, join the mutt-users mailing list:
http://lists.mutt.org/mailman/listinfo/mutt-users
http://www.mutt.org/mail-lists.html
>(because it is not clear to me that sylpheed's development is happening all that
much which is a worrisome sign for its future) and so am also deciding on whether to keep
the mh folder structure or move to Maildir. I do have a lot of subfolders so it is a bit
of a pain to convert everything, but I think that I can probably write a script for doing
so.
Plenty of us have scripts for this kind of thing, and you can even batch
invoke moutt for such a conversion (with some scripting around it).
>But I wanted to ask if it is worth it to make this change. Now, I know that
sylpheed's mh structure is not exactly the same because it puts in some marks and
sequences in each directory and does not touch the traditional .mh_sequences touch, but I
do not see if that will matter much in terms of the conversion.
Mutt will read mbox, Maildir and MH. So you don't need to make both
changes at once.
>Anyway, thoughts? I simply want to know if it is worth making the change which will
require some scripting (and some inevitable disruption).
I moved to Maildir from MH folders many years ago. Maildir at least has
good concurrency support (race free message insertion).
My current setup is Maildir for folders I'm active in (primary inbox,
folders for things I use a lot - python, mutt, so forth). I use mbox for
infrequently accessed folders and archive folders, as it is more compact
in terms of disc usage.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
_______________________________________________
users mailing list -- users(a)lists.fedoraproject.org
To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
--
Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt.
Please respond to the mailing list if appropriate. For those needing to send personal or
professional e-mail, please use appropriate addresses.
1:21 a.m.
On Wed, 2020-10-14 at 18:45 -0500, Ranjan Maitra wrote:
am also deciding on whether to keep the mh folder structure or move
to Maildir
Looking at some (elsewhere) conversations, Maildir was developed to
deal with problems some people found with mbox and mh. That may
suggest its the best way forward. Though, that'll depend on what those
problems were, and what your preferred mail client is best at.
Long ago I made the move from using mbox to maildir with my Dovecot
mail server. Since my mail clients use IMAP to access the mail server,
they (should) only temporarily cache what I'm reading, so I don't
really care what the mail clients use internally.
I looked at scripting to do a conversion, but the documentation was
dreadful, and I didn't care for all the experimenting I'd need to do to
figure out how to do it. So, I set up the new mail server using
maildir, while the old mbox server was still running, and dragged and
dropped huge batches of mail from one folder to another through my
Evolution mail client. Not the most elegant solution, but damn simple.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
11:06 a.m.
On Thu, 15 Oct 2020 16:51:42 +1030 Tim via users <users(a)lists.fedoraproject.org>
wrote:
On Wed, 2020-10-14 at 18:45 -0500, Ranjan Maitra wrote:
> am also deciding on whether to keep the mh folder structure or move
> to Maildir
Looking at some (elsewhere) conversations, Maildir was developed to
deal with problems some people found with mbox and mh. That may
suggest its the best way forward. Though, that'll depend on what those
problems were, and what your preferred mail client is best at.
Long ago I made the move from using mbox to maildir with my Dovecot
mail server. Since my mail clients use IMAP to access the mail server,
they (should) only temporarily cache what I'm reading, so I don't
really care what the mail clients use internally.
I looked at scripting to do a conversion, but the documentation was
dreadful, and I didn't care for all the experimenting I'd need to do to
figure out how to do it. So, I set up the new mail server using
maildir, while the old mbox server was still running, and dragged and
dropped huge batches of mail from one folder to another through my
Evolution mail client. Not the most elegant solution, but damn simple.
Thanks, yes, I used to use mbox long ago, when I did not understand things clearly, and I
used pine then, but I moved to mh with the move to sylpheed. An added benefit is in
backups. I have never lost any mail on mh so far, but I wonder if I am simply tempting
fate by not moving to Maildir (I do not completely understand the issues).
Separately, I am still trying to figure out if there are still major advantages to using
mutt vs. neomutt.
Best wishes,
Ranjan
6:14 p.m.
On 15Oct2020 11:06, Ranjan Maitra <maitra(a)email.com> wrote:
Thanks, yes, I used to use mbox long ago, when I did not understand
things clearly, and I used pine then, but I moved to mh with the move to sylpheed. An
added benefit is in backups. I have never lost any mail on mh so far, but I wonder if I am
simply tempting fate by not moving to Maildir (I do not completely understand the issues).
Mostly the races.
To add, remove or flag in MH probably requires a locking mechanism while
updating the message number lists (and correspondingly, allocating new
message numbers). You also don't know an arriving message is complete unless
it isn't numbered yet; I imagine an MH insert goes
save-completely-to-tempfile, allocate-number, rename-tempfile-to-number,
update-number-lists. A lock would need to be held over the last three
steps.
Maildir is race free. Messages get unqiue filenames (composed of various
sufficently unique values combined), are created in the "tmp" subdir,
and renamed into the "new" subdir. Read messages are renamed from
"new"
to "cur". No shared number lists, no locks. You only look for messages
in "new" and "cur".
Separately, I am still trying to figure out if there are still major
advantages to using mutt vs. neomutt.
My advice would be: start with mainline mutt (because I know the maintenance
situation is active) and consider neomutt if there's some specific missing
feature you want (if neomutt has it, of course). But ask on the mutt-users
list about features; some things are available and/or doable without always
being immediately obvious.
You can also try both - mutt, hmm missing feature, neomutt. You may need
to maintain distinct config files for each, but they can share mail folders.
Also, config files can source other config files, so you might have:
.muttrc-common-config
.muttrc
.muttrc-neomutt
with the last 2 both sourcing .muttrc-common-config and then adding
a few specific tweaks.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
12:15 a.m.
On Fri, 16 Oct 2020 10:14:48 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 15Oct2020 11:06, Ranjan Maitra <maitra(a)email.com> wrote:
>Thanks, yes, I used to use mbox long ago, when I did not understand things clearly,
and I used pine then, but I moved to mh with the move to sylpheed. An added benefit is in
backups. I have never lost any mail on mh so far, but I wonder if I am simply tempting
fate by not moving to Maildir (I do not completely understand the issues).
Mostly the races.
To add, remove or flag in MH probably requires a locking mechanism while
updating the message number lists (and correspondingly, allocating new
message numbers). You also don't know an arriving message is complete unless
it isn't numbered yet; I imagine an MH insert goes
save-completely-to-tempfile, allocate-number, rename-tempfile-to-number,
update-number-lists. A lock would need to be held over the last three
steps.
This may be a stupid question, but does this problem still happen if I use fetchmail to
pull mail and store messages in files via procmail?. In my experience, messages have been
given unique numbers in their respective folders. (Of course, my MH is really not MH, but
rather sylpheed-mh, since they do not update the .mh_sequences but use .sylpheed_mark.
Maildir is race free. Messages get unqiue filenames (composed of various
sufficently unique values combined), are created in the "tmp" subdir,
and renamed into the "new" subdir. Read messages are renamed from
"new"
to "cur". No shared number lists, no locks. You only look for messages
in "new" and "cur".
Thanks very much for this. I have found a tool that can convert mh to mailbox:
https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
It seems to work, but can not handle a second level of subfolders: brings them all out as
individual folders at the first level, so Ihave to fix that. Also, I don't like the
new folder names, seem too unnecessary for me. (I was expecting to the old MH folder names
inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
I have to look into this some more. I am not sure if this is the standard way to store
Maildir format messages.
One aspect of MH that I have liked is that I pull mail on two machines (using fetchmail
via a POP server) and they are assigned the same filenames (numbers). Then, if I use rsync
with delete, I can delete the corresponding message in the remote machine if I have
deleted it on my local machine. It has worked like a charm over the past 15 years (I would
say).
>Separately, I am still trying to figure out if there are still
major advantages to using mutt vs. neomutt.
My advice would be: start with mainline mutt (because I know the maintenance
situation is active) and consider neomutt if there's some specific missing
feature you want (if neomutt has it, of course). But ask on the mutt-users
list about features; some things are available and/or doable without always
being immediately obvious.
You can also try both - mutt, hmm missing feature, neomutt. You may need
to maintain distinct config files for each, but they can share mail folders.
Also, config files can source other config files, so you might have:
.muttrc-common-config
.muttrc
.muttrc-neomutt
with the last 2 both sourcing .muttrc-common-config and then adding
a few specific tweaks.
Thanks, I think that I am also coming round to the view that I should stick to mutt. The
listed neomutt features are: https://neomutt.org/feature. I do not know what is not also
in mutt. But in any case, I probably won't know the difference. (I don't
completely understand all the features.)
I want to read my e-mail locally. Basically, run fetchmail or mbsync in the background to
get mail and store in local folders at intervals, then have mutt read them, etc. I like
storing my e-mail locally so that I can get to it offline. Which is one of the reasons I
have never liked all these web-based mailers that almost all my friends and colleagues
(and especially those on gmail) seems to like nowadays.
Thanks again!
Ranjan
1:02 a.m.
On 10/15/20 10:15 PM, Ranjan Maitra wrote:
Thanks very much for this. I have found a tool that can convert mh to
mailbox:
https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
It seems to work, but can not handle a second level of subfolders: brings them all out as
individual folders at the first level, so Ihave to fix that. Also, I don't like the
new folder names, seem too unnecessary for me. (I was expecting to the old MH folder names
inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
I have to look into this some more. I am not sure if this is the standard way to store
Maildir format messages.
Maildir has all the folders at the top level with dots to indicate
subfolders. And that is also the standard filename format.
One aspect of MH that I have liked is that I pull mail on two
machines (using fetchmail via a POP server) and they are assigned the same filenames
(numbers). Then, if I use rsync with delete, I can delete the corresponding message in the
remote machine if I have deleted it on my local machine. It has worked like a charm over
the past 15 years (I would say).
That sounds either amazingly risky or it's a one-way sync and what's the
point of having the email on the remote machine?
3:56 a.m.
On 16Oct2020 00:15, Ranjan Maitra <maitra(a)email.com> wrote:
On Fri, 16 Oct 2020 10:14:48 +1100 Cameron Simpson
<cs(a)cskk.id.au> wrote:
> Mostly the races.
>
> To add, remove or flag in MH probably requires a locking mechanism while
> updating the message number lists (and correspondingly, allocating new
> message numbers). You also don't know an arriving message is complete unless
> it isn't numbered yet; I imagine an MH insert goes
> save-completely-to-tempfile, allocate-number, rename-tempfile-to-number,
> update-number-lists. A lock would need to be held over the last three
> steps.
This may be a stupid question, but does this problem still happen if I use fetchmail to
pull mail and store messages in files via procmail?
Well, it is an issue regardless of the tool. procmail may do the initial
placement, but your mail reader probably moves messages around, if only
from one folder to another, which is the same task (new message in
target folder).
In my experience, messages have been given unique numbers in their
respective folders. (Of course, my MH is really not MH, but rather sylpheed-mh, since they
do not update the .mh_sequences but use .sylpheed_mark.
Well, the requirement for MH is of course that they have unique numbers
because their filenames are so named. The important thing is that
delivery tools cooperate - more than one tool might be trying to deliver
to the folder at once, and only one can work on the .mh_sequences or
.sylpheed_mark files at a time. So: locking.
Maildir lets you do this without locking because of the noncolliding filename
approach, and the "is this message file complete?" issue by doing all
preparation in the tmp subdir, before renaming the completed file into the
new subdir.
> Maildir is race free. Messages get unqiue filenames (composed of
various
> sufficently unique values combined), are created in the "tmp" subdir,
> and renamed into the "new" subdir. Read messages are renamed from
"new"
> to "cur". No shared number lists, no locks. You only look for messages
> in "new" and "cur".
Thanks very much for this. I have found a tool that can convert mh to mailbox:
https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
It seems to work, but can not handle a second level of subfolders: brings them all out as
individual folders at the first level, so Ihave to fix that. Also, I don't like the
new folder names, seem too unnecessary for me. (I was expecting to the old MH folder names
inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
Yah. I wrote one when I made this switch:
https://hg.sr.ht/~cameron-simpson/css/browse/bin/mh2maildir?rev=tip
Hmm, some years ago now, looking at the opening comment. And I'm using
procmail for the conversion (!!!), so indeed quite a while ago. This
script moves the MH folder sideways and makes an empty Maildir in its
place, then delivers every message from the MH folder into the new
maildir.
These days I'd use mutt for the bulk conversion instead of procmail. You can
see an example of that approach in this script:
https://hg.sr.ht/~cameron-simpson/css/browse/bin/mboxify?rev=tip
I have to look into this some more. I am not sure if this is the
standard way to store Maildir format messages.
mutt doesn't care - there's no "standard". There was a recent discussion
on mutt-users with another user moving to Maildir, who had nested
folders. He's still got nested folders, exactly as before. Just don't
name a subfolder like one of the three reserved names: "tmp", "new",
"cur".
So do it how you like it.
One aspect of MH that I have liked is that I pull mail on two machines
(using fetchmail via a POP server) and they are assigned the same filenames (numbers).
Then, if I use rsync with delete, I can delete the corresponding message in the remote
machine if I have deleted it on my local machine. It has worked like a charm over the past
15 years (I would say).
Ah. This is a little trickier with Maildir, because message flags are
stored in the filename. If you change the flags on both machines
without an rsync in between you might run into trouble. With MH the
flags are in the .mh-sequences file IIRC. You've still have an
equivalent problem though - I presume you're excluding the .mh_sequences
files from the rsync?
Is one of your machines considered the "main" machine where you read and
maintain email, and the other a backup? Or do you delete at both ends?
I use getmail via a POP server to collect my email to my laptop. I use
rsync with Maildirs in my backup process to the home server backup volume.
Thanks, I think that I am also coming round to the view that I should
stick to mutt. The listed neomutt features are: https://neomutt.org/feature. I do not know
what is not also in mutt. But in any case, I probably won't know the difference. (I
don't completely understand all the features.)
Mainline mutt has the sidebar patch and I think the trash folder. I'm
unsure about the rest. I index my mail with notmuch, but outside of
mutt.
I want to read my e-mail locally. Basically, run fetchmail or mbsync
in the background to get mail and store in local folders at intervals, then have mutt read
them, etc. I like storing my e-mail locally so that I can get to it offline.
Same!
So I run getmail every 30s to poll my mail and pull it locally. I have
postfix configured on my laptop, so sending email is just done with
sendmail. If I'm online it goes straight out, and if not it queues until
I am online.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
3:58 a.m.
On 15Oct2020 23:02, Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/15/20 10:15 PM, Ranjan Maitra wrote:
>It seems to work, but can not handle a second level of subfolders: brings them all out
as individual folders at the first level, so Ihave to fix that. Also, I don't like the
new folder names, seem too unnecessary for me. (I was expecting to the old MH folder names
inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
>
>I have to look into this some more. I am not sure if this is the standard way to store
Maildir format messages.
Maildir has all the folders at the top level with dots to indicate
subfolders. And that is also the standard filename format.
This isn't strictly true. Some IMAP _servers_ require you to use such a
scheme, but if you're not presenting the folders via IMAP there's no
need for such a system.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
8:35 a.m.
Thanks, very much!
On Fri, 16 Oct 2020 19:56:11 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 16Oct2020 00:15, Ranjan Maitra <maitra(a)email.com> wrote:
>On Fri, 16 Oct 2020 10:14:48 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
>> Mostly the races.
>>
>> To add, remove or flag in MH probably requires a locking mechanism while
>> updating the message number lists (and correspondingly, allocating new
>> message numbers). You also don't know an arriving message is complete
unless
>> it isn't numbered yet; I imagine an MH insert goes
>> save-completely-to-tempfile, allocate-number, rename-tempfile-to-number,
>> update-number-lists. A lock would need to be held over the last three
>> steps.
>
>This may be a stupid question, but does this problem still happen if I use fetchmail
to pull mail and store messages in files via procmail?
Well, it is an issue regardless of the tool. procmail may do the initial
placement, but your mail reader probably moves messages around, if only
from one folder to another, which is the same task (new message in
target folder).
I see, yes, that is possible, but I have not seen this problem for me. Perhaps the way I
have (inadvertently) used it so far may be the reason, or I have been plain fortuitous.
>In my experience, messages have been given unique numbers in their respective
folders. (Of course, my MH is really not MH, but rather sylpheed-mh, since they do not
update the .mh_sequences but use .sylpheed_mark.
Well, the requirement for MH is of course that they have unique numbers
because their filenames are so named. The important thing is that
delivery tools cooperate - more than one tool might be trying to deliver
to the folder at once, and only one can work on the .mh_sequences or
.sylpheed_mark files at a time. So: locking.
Maildir lets you do this without locking because of the noncolliding filename
approach, and the "is this message file complete?" issue by doing all
preparation in the tmp subdir, before renaming the completed file into the
new subdir.
I see, this makes sense. Thanks for the explanation.
>> Maildir is race free. Messages get unqiue filenames (composed of various
>> sufficently unique values combined), are created in the "tmp" subdir,
>> and renamed into the "new" subdir. Read messages are renamed from
"new"
>> to "cur". No shared number lists, no locks. You only look for
messages
>> in "new" and "cur".
>
>Thanks very much for this. I have found a tool that can convert mh to mailbox:
>
>https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
>
>It seems to work, but can not handle a second level of subfolders: brings them all
out as individual folders at the first level, so Ihave to fix that. Also, I don't like
the new folder names, seem too unnecessary for me. (I was expecting to the old MH folder
names inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
Yah. I wrote one when I made this switch:
https://hg.sr.ht/~cameron-simpson/css/browse/bin/mh2maildir?rev=tip
Hmm, some years ago now, looking at the opening comment. And I'm using
procmail for the conversion (!!!), so indeed quite a while ago. This
script moves the MH folder sideways and makes an empty Maildir in its
place, then delivers every message from the MH folder into the new
maildir.
Thanks very much for this! I will try it out, but it appears to be for individual folders,
is that correct? So, I will maybe write a script to call it.
These days I'd use mutt for the bulk conversion instead of procmail. You can
see an example of that approach in this script:
https://hg.sr.ht/~cameron-simpson/css/browse/bin/mboxify?rev=tip
I see, this is for converting to mbox and needs to be modified for
>I have to look into this some more. I am not sure if this is the
standard way to store Maildir format messages.
mutt doesn't care - there's no "standard". There was a recent
discussion
on mutt-users with another user moving to Maildir, who had nested
folders. He's still got nested folders, exactly as before. Just don't
name a subfolder like one of the three reserved names: "tmp", "new",
"cur".
So do it how you like it.
Yes, that is what I would like: I do not have any subfolders with one of the three Maildir
subfolders as a name.
According to this post here: https://www.systoolsgroup.com/maildir/ the filename has this
"gator3018.hostgator.com" which I presume comes from the hostname. How do I get
this changed to something else (unique, but not to the hostname)? Is this where mutt or
procmail is to be told to do this, how?
>One aspect of MH that I have liked is that I pull mail on two machines (using
fetchmail via a POP server) and they are assigned the same filenames (numbers). Then, if I
use rsync with delete, I can delete the corresponding message in the remote machine if I
have deleted it on my local machine. It has worked like a charm over the past 15 years (I
would say).
Ah. This is a little trickier with Maildir, because message flags are
stored in the filename. If you change the flags on both machines
without an rsync in between you might run into trouble. With MH the
flags are in the .mh-sequences file IIRC. You've still have an
equivalent problem though - I presume you're excluding the .mh_sequences
files from the rsync?
So, sylpheed does not use or update the .mh_sequences but makes its own mark which is
where it keeps all the information on the status of a message (read, replied, flagged,
etc). I do send it up.
Is one of your machines considered the "main" machine where you read and
maintain email, and the other a backup? Or do you delete at both ends?
The reason for how I set things up, and that has worked reasonably well, is (you sort of
guessed it) that I read e-mail at work and home, but the work machine is the one that I
consider to be reliably backed up. It is also bigger in terms of disk. So what happens is
that I fetchmail with keep, process e-mail at work using sylpheed and then fire up my home
machine (a laptop) and fetchmail with keep from the POP server and the rsync it down
(including the .sylpheed_mark and .sylpheed_cache). Then I work on the home machine,
continue to fetchmail process e-mails, etc with sylpheed and when I am done (before I
hibernate), I rsync it all up before I go back to work, so that when I go to the other
(work) machine, I have the same status as I left at home/work. It has worked reasonably
well.
I use getmail via a POP server to collect my email to my laptop. I use
rsync with Maildirs in my backup process to the home server backup volume.
>Thanks, I think that I am also coming round to the view that I should stick to mutt.
The listed neomutt features are: https://neomutt.org/feature. I do not know what is not
also in mutt. But in any case, I probably won't know the difference. (I don't
completely understand all the features.)
Mainline mutt has the sidebar patch and I think the trash folder. I'm
unsure about the rest. I index my mail with notmuch, but outside of
mutt.
>I want to read my e-mail locally. Basically, run fetchmail or mbsync in the
background to get mail and store in local folders at intervals, then have mutt read them,
etc. I like storing my e-mail locally so that I can get to it offline.
Same!
So I run getmail every 30s to poll my mail and pull it locally. I have
postfix configured on my laptop, so sending email is just done with
sendmail. If I'm online it goes straight out, and if not it queues until
I am online.
Thanks, I have currently been using postfix too, but will probably try mutt's smtp
(now that they have enabled office365 oauth2 support). Postfix has not yet done so, they
have done some with gmail, but it is not completely clear to me yet how to use that with
MS Office365. Same with msmtp, it seems.
Thanks again for all the nice discussion!
Best wishes,
Ranjan
8:44 a.m.
Thanks, Sam!
On Thu, 15 Oct 2020 23:02:24 -0700 Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/15/20 10:15 PM, Ranjan Maitra wrote:
> Thanks very much for this. I have found a tool that can convert mh to mailbox:
>
> https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
>
> It seems to work, but can not handle a second level of subfolders: brings them all
out as individual folders at the first level, so Ihave to fix that. Also, I don't like
the new folder names, seem too unnecessary for me. (I was expecting to the old MH folder
names inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
>
> I have to look into this some more. I am not sure if this is the standard way to
store Maildir format messages.
Maildir has all the folders at the top level with dots to indicate
subfolders. And that is also the standard filename format.
> One aspect of MH that I have liked is that I pull mail on two machines (using
fetchmail via a POP server) and they are assigned the same filenames (numbers). Then, if I
use rsync with delete, I can delete the corresponding message in the remote machine if I
have deleted it on my local machine. It has worked like a charm over the past 15 years (I
would say).
That sounds either amazingly risky or it's a one-way sync and what's the
point of having the email on the remote machine?
So, yes, there is some element of risk involved, but I have been been able to manage that
reasonably, I think. My bigger risk has been to accidentally delete an e-mail (myself), in
which case, I go in and fix the .fetchids manually so that it can go fetch the e-mail
again, and then life is seemingly well.
As I said in the other response, the reason for how I set things up, and that has worked
reasonably well, is that I read e-mail at work and home, but the work machine is the one
that I consider to be reliably backed up. It is also bigger in terms of disk. So what
happens is that I fetchmail with keep, process e-mail at work using sylpheed and then fire
up my home machine (a laptop) and fetchmail with keep from the POP server and the rsync it
down (including the .sylpheed_mark and .sylpheed_cache). Then I work on the home machine,
continue to fetchmail process e-mails, etc with sylpheed and when I am done (before I
hibernate), I rsync it all up before I go back to work, so that when I go to the other
(work) machine, I have the same status as I left at home/work. Of course, I need to be
careful and vigilant for the reasons you alluded to.
Thanks again!
Best wishes,
Ranjan
10:20 a.m.
On Fri, 2020-10-16 at 08:44 -0500, Ranjan Maitra wrote:
As I said in the other response, the reason for how I set things up,
and that has worked reasonably well, is that I read e-mail at work and home, but the work
machine is the one that I consider to be reliably backed up. It is also bigger in terms of
disk. So what happens is that I fetchmail with keep, process e-mail at work using sylpheed
and then fire up my home machine (a laptop) and fetchmail with keep from the POP server
and the rsync it down (including the .sylpheed_mark and .sylpheed_cache). Then I work on
the home machine, continue to fetchmail process e-mails, etc with sylpheed and when I am
done (before I hibernate), I rsync it all up before I go back to work, so that when I go
to the other (work) machine, I have the same status as I left at home/work. Of course, I
need to be careful and vigilant for the reasons you alluded to.
TBH I long ago gave up trying to juggle multiple machines and syncing
mail. I just use IMAP and let somebody else worry about it. In fact I'd
do that even if I was the somebody else, as I once used to be. People
say there are reasons for using POP, and perhaps there are for some
people, but frankly I've never been convinced by any of them.
Just my 2p.
poc
10:34 a.m.
On Fri, 16 Oct 2020 16:20:27 +0100 Patrick O'Callaghan <pocallaghan(a)gmail.com>
wrote:
On Fri, 2020-10-16 at 08:44 -0500, Ranjan Maitra wrote:
> As I said in the other response, the reason for how I set things up, and that has
worked reasonably well, is that I read e-mail at work and home, but the work machine is
the one that I consider to be reliably backed up. It is also bigger in terms of disk. So
what happens is that I fetchmail with keep, process e-mail at work using sylpheed and then
fire up my home machine (a laptop) and fetchmail with keep from the POP server and the
rsync it down (including the .sylpheed_mark and .sylpheed_cache). Then I work on the home
machine, continue to fetchmail process e-mails, etc with sylpheed and when I am done
(before I hibernate), I rsync it all up before I go back to work, so that when I go to the
other (work) machine, I have the same status as I left at home/work. Of course, I need to
be careful and vigilant for the reasons you alluded to.
TBH I long ago gave up trying to juggle multiple machines and syncing
mail. I just use IMAP and let somebody else worry about it. In fact I'd
do that even if I was the somebody else, as I once used to be. People
say there are reasons for using POP, and perhaps there are for some
people, but frankly I've never been convinced by any of them.
Thanks, Patrick! True, but I want my e-mail stored and acted upon locally. In that case, I
do not completely see a way out of my situation, though there are likely alternative
approaches to the same problem.
I use POP simply because the only time I used IMAP with fetchmail it pulled all the mail
from the server (marked it as seen) and it did not work for my situation. Likely there was
some issue with my setup, but I did not bother investigating and went quickly back to
POP.
Ranjan
2:14 p.m.
On 10/16/20 8:34 AM, Ranjan Maitra wrote:
On Fri, 16 Oct 2020 16:20:27 +0100 Patrick O'Callaghan
<pocallaghan(a)gmail.com> wrote:
> TBH I long ago gave up trying to juggle multiple machines and syncing
> mail. I just use IMAP and let somebody else worry about it. In fact I'd
> do that even if I was the somebody else, as I once used to be. People
> say there are reasons for using POP, and perhaps there are for some
> people, but frankly I've never been convinced by any of them.
>
Thanks, Patrick! True, but I want my e-mail stored and acted upon locally. In that case,
I do not completely see a way out of my situation, though there are likely alternative
approaches to the same problem.
I run my own mail server, so all the emails are in my house which is
local enough. I access my email from many devices, so this keeps them
all in sync. dovecot does some initial email sorting for me and all the
devices download the emails so they're available offline if necessary.
2:48 p.m.
On Fri, 16 Oct 2020 12:14:13 -0700 Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/16/20 8:34 AM, Ranjan Maitra wrote:
> On Fri, 16 Oct 2020 16:20:27 +0100 Patrick O'Callaghan
<pocallaghan(a)gmail.com> wrote:
>> TBH I long ago gave up trying to juggle multiple machines and syncing
>> mail. I just use IMAP and let somebody else worry about it. In fact I'd
>> do that even if I was the somebody else, as I once used to be. People
>> say there are reasons for using POP, and perhaps there are for some
>> people, but frankly I've never been convinced by any of them.
>>
>
> Thanks, Patrick! True, but I want my e-mail stored and acted upon locally. In that
case, I do not completely see a way out of my situation, though there are likely
alternative approaches to the same problem.
I run my own mail server, so all the emails are in my house which is
local enough. I access my email from many devices, so this keeps them
all in sync. dovecot does some initial email sorting for me and all the
devices download the emails so they're available offline if necessary.
Thanks :-) But my employer provides my e-mail that I need to use for professional reasons,
otherwise I would likely be making different/simpler choices.
Ranjan
3:36 p.m.
On 10/16/20 12:48 PM, Ranjan Maitra wrote:
On Fri, 16 Oct 2020 12:14:13 -0700 Samuel Sieb
<samuel(a)sieb.net> wrote:
> On 10/16/20 8:34 AM, Ranjan Maitra wrote:
>> On Fri, 16 Oct 2020 16:20:27 +0100 Patrick O'Callaghan
<pocallaghan(a)gmail.com> wrote:
>>> TBH I long ago gave up trying to juggle multiple machines and syncing
>>> mail. I just use IMAP and let somebody else worry about it. In fact I'd
>>> do that even if I was the somebody else, as I once used to be. People
>>> say there are reasons for using POP, and perhaps there are for some
>>> people, but frankly I've never been convinced by any of them.
>>>
>>
>> Thanks, Patrick! True, but I want my e-mail stored and acted upon locally. In
that case, I do not completely see a way out of my situation, though there are likely
alternative approaches to the same problem.
>
> I run my own mail server, so all the emails are in my house which is
> local enough. I access my email from many devices, so this keeps them
> all in sync. dovecot does some initial email sorting for me and all the
> devices download the emails so they're available offline if necessary.
Thanks :-) But my employer provides my e-mail that I need to use for professional
reasons, otherwise I would likely be making different/simpler choices.
Ok, but my other point was that mail clients like Thunderbird can be
configured to download all the emails, so they are available locally and
offline. Then you have the benefits of your POP/fetchmail config
without the risky syncing methods. I just realized though that I forgot
you were using mutt which doesn't have that capability. However,
there's an application called offlineimap which might work for you.
3:46 p.m.
On Fri, 16 Oct 2020 13:36:02 -0700
Samuel Sieb wrote:
Ok, but my other point was that mail clients like Thunderbird can be
configured to download all the emails, so they are available locally and
offline.
I use fetchmail to download all my mail locally, where I store
it in my own dovecot imap server instance. That way I get local
copies, but can read mail from any client that supports IMAP
(though once I finally decided I liked claws-mail best, I
haven't switched mail clients).
This has the advantage of allowing me to also use the dovecot-pigeonhole
package to provide the "sieve" mail filtering scripts which is
incredibly more useful than any of the filters in any mail
clients I've ever seen.
4:46 p.m.
On 10/16/20 1:46 PM, Tom Horsley wrote:
This has the advantage of allowing me to also use the
dovecot-pigeonhole
package to provide the "sieve" mail filtering scripts which is
incredibly more useful than any of the filters in any mail
clients I've ever seen.
Yes, that was what I was referring to in my previous email about Dovecot
doing some sorting for me. There's an extension for Thunderbird that
lets you edit the sieve scripts on the server. I had been using
procmail for a very long time, but I recently adjusted my mail server
configs so that Dovecot could use sieve and it's really nice.
5 p.m.
On Fri, 2020-10-16 at 10:34 -0500, Ranjan Maitra wrote:
> TBH I long ago gave up trying to juggle multiple machines and
syncing
> mail. I just use IMAP and let somebody else worry about it. In fact I'd
> do that even if I was the somebody else, as I once used to be. People
> say there are reasons for using POP, and perhaps there are for some
> people, but frankly I've never been convinced by any of them.
Thanks, Patrick! True, but I want my e-mail stored and acted upon locally. In that case,
I do not completely see a way out of my situation, though there are likely alternative
approaches to the same problem.
You can do that with IMAP as well, e.g. in Evolution there's an option
to "copy folder locally for offline operation", which will take care of
that automatically. Other IMAP-supporting MUAs presumably have similar
features, though possibly using other terminology.
There is literally nothing you can do with POP that you can't also do
with IMAP.
poc
5:38 p.m.
On 16Oct2020 08:35, Ranjan Maitra <maitra(a)email.com> wrote:
> >This may be a stupid question, but does this problem still
happen if I use fetchmail to pull mail and store messages in files via procmail?
>
> Well, it is an issue regardless of the tool. procmail may do the initial
> placement, but your mail reader probably moves messages around, if only
> from one folder to another, which is the same task (new message in
> target folder).
I see, yes, that is possible, but I have not seen this problem for me. Perhaps the way I
have (inadvertently) used it so far may be the reason, or I have been plain fortuitous.
As long as they share an agreed locking protocol it can all be fine.
> >Thanks very much for this. I have found a tool that can
convert mh to mailbox:
> >
> >https://github.com/vuntz/mh2maildir/blob/master/mh2maildir
> >
> >It seems to work, but can not handle a second level of subfolders: brings them
all out as individual folders at the first level, so Ihave to fix that. Also, I don't
like the new folder names, seem too unnecessary for me. (I was expecting to the old MH
folder names inside my Maildir.) Also, the mails get stored as something like:
1602799622.116065_21187.hostname:2, not sure if this is the recommended way that files are
stored in the Maildir format. I was expecting to have something that I could have control
over.
>
> Yah. I wrote one when I made this switch:
>
> https://hg.sr.ht/~cameron-simpson/css/browse/bin/mh2maildir?rev=tip
>
> Hmm, some years ago now, looking at the opening comment. And I'm using
> procmail for the conversion (!!!), so indeed quite a while ago. This
> script moves the MH folder sideways and makes an empty Maildir in its
> place, then delivers every message from the MH folder into the new
> maildir.
Thanks very much for this! I will try it out, but it appears to be for individual folders,
is that correct? So, I will maybe write a script to call it.
Yes. I just wrote a for loop on the command line. Something like:
cd ~/mail
for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break; done
because I feared disaster. Easy as.
> These days I'd use mutt for the bulk conversion instead of
procmail. You can
> see an example of that approach in this script:
>
> https://hg.sr.ht/~cameron-simpson/css/browse/bin/mboxify?rev=tip
I see, this is for converting to mbox and needs to be modified for
Yep. But the conversion is pretty trivial. Mutt autodetects the folder
type. If a file, mbox. If a dir with tmp,new,cur, a Maildir. Probably MH
otherwise.
So provided you _make_ an empty Maildir (mkdir $d $d/tmp $d/new $d/cur)
mutt will deliver into it correctly - no special modes. You only need to
instruct mutt when _it_ creates the mail folder, by setting:
set mbox_type=maildir
as your preference for new folders.
> >I have to look into this some more. I am not sure if this is
the standard way to store Maildir format messages.
>
> mutt doesn't care - there's no "standard". There was a recent
discussion
> on mutt-users with another user moving to Maildir, who had nested
> folders. He's still got nested folders, exactly as before. Just don't
> name a subfolder like one of the three reserved names: "tmp",
"new",
> "cur".
>
> So do it how you like it.
Yes, that is what I would like: I do not have any subfolders with one of the three Maildir
subfolders as a name.
According to this post here: https://www.systoolsgroup.com/maildir/ the filename has this
"gator3018.hostgator.com" which I presume comes from the hostname. How do I get
this changed to something else (unique, but not to the hostname)? Is this where mutt or
procmail is to be told to do this, how?
Not sure. The hostname is, IIRC, an optional component for making the
filesnames; I'm not sure mutt offers any control over that. Here're a
couple of names from my main inbox:
1386376685.65689_1131.fleet:2,S
1602208593.#2087M806900P64256:2,S
The former has a hostname in it, the latter does not. [looks...] Hoo,
the one with a hostname is Very Old. Likely delivered by procmail and
spamassassin. These days I collect with getmail and file with my own
mail filer. The code for making a new Maildir folder in the latter is
this:
def newkey(self):
''' Allocate a new key.
'''
now = time.time()
secs = int(now)
subsecs = now-secs
key = '%d.#%dM%dP%d' % (secs, seq(), subsecs * 1e6, self.pid)
assert self.validkey(key), "invalid new key: %s" % (key,)
return key
I believe once made, clients (mutt etc) don't muck with the names except
to change the suffix (",S") which is where message flags are
implemented (means you can set flags without editing the message file to
fiddle a header).
> >One aspect of MH that I have liked is that I pull mail on two
machines (using fetchmail via a POP server) and they are assigned the same filenames
(numbers). Then, if I use rsync with delete, I can delete the corresponding message in the
remote machine if I have deleted it on my local machine. It has worked like a charm over
the past 15 years (I would say).
>
> Ah. This is a little trickier with Maildir, because message flags are
> stored in the filename. If you change the flags on both machines
> without an rsync in between you might run into trouble. With MH the
> flags are in the .mh-sequences file IIRC. You've still have an
> equivalent problem though - I presume you're excluding the .mh_sequences
> files from the rsync?
So, sylpheed does not use or update the .mh_sequences but makes its own mark which is
where it keeps all the information on the status of a message (read, replied, flagged,
etc). I do send it up.
>
> Is one of your machines considered the "main" machine where you read and
> maintain email, and the other a backup? Or do you delete at both ends?
The reason for how I set things up, and that has worked reasonably well, is (you sort of
guessed it) that I read e-mail at work and home, but the work machine is the one that I
consider to be reliably backed up. It is also bigger in terms of disk. So what happens is
that I fetchmail with keep, process e-mail at work using sylpheed and then fire up my home
machine (a laptop) and fetchmail with keep from the POP server and the rsync it down
(including the .sylpheed_mark and .sylpheed_cache). Then I work on the home machine,
continue to fetchmail process e-mails, etc with sylpheed and when I am done (before I
hibernate), I rsync it all up before I go back to work, so that when I go to the other
(work) machine, I have the same status as I left at home/work. It has worked reasonably
well.
Ah, so you've always got an rsync between transitions. That should work
just as well with Maildir.
I'm also aware of a number of people who remote to their main machine: ssh in
and run mutt there.
Thanks, I have currently been using postfix too, but will probably try
mutt's smtp (now that they have enabled office365 oauth2 support). Postfix has not yet
done so, they have done some with gmail, but it is not completely clear to me yet how to
use that with MS Office365. Same with msmtp, it seems.
Fair enough. One advantage of postfix is that local programmes like cron
can also send email. But of course, nothing prevents you having a
working local postfix for the system and using mutt-smtp or one of the
various "send with smtp" standalone tools as well.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
5:39 p.m.
On 16Oct2020 16:20, Patrick O'Callaghan <pocallaghan(a)gmail.com> wrote:
On Fri, 2020-10-16 at 08:44 -0500, Ranjan Maitra wrote:
> As I said in the other response, the reason for how I set things up, and that has
worked reasonably well, is that I read e-mail at work and home, but the work machine is
the one that I consider to be reliably backed up. It is also bigger in terms of disk. So
what happens is that I fetchmail with keep, process e-mail at work using sylpheed and then
fire up my home machine (a laptop) and fetchmail with keep from the POP server and the
rsync it down (including the .sylpheed_mark and .sylpheed_cache). Then I work on the home
machine, continue to fetchmail process e-mails, etc with sylpheed and when I am done
(before I hibernate), I rsync it all up before I go back to work, so that when I go to the
other (work) machine, I have the same status as I left at home/work. Of course, I need to
be careful and vigilant for the reasons you alluded to.
TBH I long ago gave up trying to juggle multiple machines and syncing
mail. I just use IMAP and let somebody else worry about it. In fact I'd
do that even if I was the somebody else, as I once used to be. People
say there are reasons for using POP, and perhaps there are for some
people, but frankly I've never been convinced by any of them.
Local email lets me read and respond while offline, such as on a train.
And it is so much faster :-)
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
5:42 p.m.
On 16Oct2020 14:48, Ranjan Maitra <maitra(a)email.com> wrote:
On Fri, 16 Oct 2020 12:14:13 -0700 Samuel Sieb <samuel(a)sieb.net>
wrote:
> I run my own mail server, so all the emails are in my house which is
> local enough. I access my email from many devices, so this keeps them
> all in sync. dovecot does some initial email sorting for me and all the
> devices download the emails so they're available offline if necessary.
Thanks :-) But my employer provides my e-mail that I need to use for professional reasons,
otherwise I would likely be making different/simpler choices.
Just a side note: in a former life my boss suggested to me strongly that
I should separate my work and personal email. That has proven to be good
advice, and I recommend it to you. Keep two accounts.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
5:56 p.m.
On 10/16/20 3:39 PM, Cameron Simpson wrote:
On 16Oct2020 16:20, Patrick O'Callaghan
<pocallaghan(a)gmail.com> wrote:
> TBH I long ago gave up trying to juggle multiple machines and syncing
> mail. I just use IMAP and let somebody else worry about it. In fact I'd
> do that even if I was the somebody else, as I once used to be. People
> say there are reasons for using POP, and perhaps there are for some
> people, but frankly I've never been convinced by any of them.
Local email lets me read and respond while offline, such as on a train.
And it is so much faster :-)
Sure, but that is not something specific to POP3. Most IMAP clients can
do that as well.
6:17 p.m.
On Sat, 17 Oct 2020 09:42:06 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 16Oct2020 14:48, Ranjan Maitra <maitra(a)email.com> wrote:
>On Fri, 16 Oct 2020 12:14:13 -0700 Samuel Sieb <samuel(a)sieb.net> wrote:
>> I run my own mail server, so all the emails are in my house which is
>> local enough. I access my email from many devices, so this keeps them
>> all in sync. dovecot does some initial email sorting for me and all the
>> devices download the emails so they're available offline if necessary.
>
>Thanks :-) But my employer provides my e-mail that I need to use for professional
reasons, otherwise I would likely be making different/simpler choices.
Just a side note: in a former life my boss suggested to me strongly that
I should separate my work and personal email. That has proven to be good
advice, and I recommend it to you. Keep two accounts.
Thanks, Cameron! I have four accounts, I think it is de facto required for us! But my work
e-mail is still the bulk, and in any case, I read all e-mails on the same mailer
(sylpheed, but will move to mutt).
Ranjan
6:19 p.m.
On Fri, 16 Oct 2020 15:56:19 -0700 Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/16/20 3:39 PM, Cameron Simpson wrote:
> On 16Oct2020 16:20, Patrick O'Callaghan <pocallaghan(a)gmail.com> wrote:
>> TBH I long ago gave up trying to juggle multiple machines and syncing
>> mail. I just use IMAP and let somebody else worry about it. In fact I'd
>> do that even if I was the somebody else, as I once used to be. People
>> say there are reasons for using POP, and perhaps there are for some
>> people, but frankly I've never been convinced by any of them.
>
> Local email lets me read and respond while offline, such as on a train.
> And it is so much faster :-)
Sure, but that is not something specific to POP3. Most IMAP clients can
do that as well.
Agreed, and I did not want to imply otherwise. When I used IMAP long ago, it somehow did
not keep mail on the server enough for fetchmail to recognize that I still wanted it
downloaded on the other machine(s). That is why I gave up. Likely there was an error in my
configuration.
Thanks,
Ranjan
6:31 p.m.
On 10/16/20 4:19 PM, Ranjan Maitra wrote:
On Fri, 16 Oct 2020 15:56:19 -0700 Samuel Sieb
<samuel(a)sieb.net> wrote:
> On 10/16/20 3:39 PM, Cameron Simpson wrote:
>> On 16Oct2020 16:20, Patrick O'Callaghan <pocallaghan(a)gmail.com> wrote:
>>> TBH I long ago gave up trying to juggle multiple machines and syncing
>>> mail. I just use IMAP and let somebody else worry about it. In fact I'd
>>> do that even if I was the somebody else, as I once used to be. People
>>> say there are reasons for using POP, and perhaps there are for some
>>> people, but frankly I've never been convinced by any of them.
>>
>> Local email lets me read and respond while offline, such as on a train.
>> And it is so much faster :-)
>
> Sure, but that is not something specific to POP3. Most IMAP clients can
> do that as well.
Agreed, and I did not want to imply otherwise. When I used IMAP long ago, it somehow did
not keep mail on the server enough for fetchmail to recognize that I still wanted it
downloaded on the other machine(s). That is why I gave up. Likely there was an error in my
configuration.
That is definitely not how IMAP is supposed to work. There must have
been something deleting the email on download which is very strange.
The whole point of IMAP is that all the mail stays on the server and can
be accessed from multiple locations at the same time.
6:53 p.m.
Cameron,
Thanks!
>>
https://hg.sr.ht/~cameron-simpson/css/browse/bin/mh2maildir?rev=tip
>>
>> Hmm, some years ago now, looking at the opening comment. And I'm using
>> procmail for the conversion (!!!), so indeed quite a while ago. This
>> script moves the MH folder sideways and makes an empty Maildir in its
>> place, then delivers every message from the MH folder into the new
>> maildir.
>
>Thanks very much for this! I will try it out, but it appears to be for individual
folders, is that correct? So, I will maybe write a script to call it.
Yes. I just wrote a for loop on the command line. Something like:
cd ~/mail
for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break; done
So, in this code, there is a command called ismhdir -- where does this come from? Is there
something missing here? (I think it also shows up in your newer code.)
because I feared disaster. Easy as.
>> These days I'd use mutt for the bulk conversion instead of procmail. You
can
>> see an example of that approach in this script:
>>
>> https://hg.sr.ht/~cameron-simpson/css/browse/bin/mboxify?rev=tip
>
>I see, this is for converting to mbox and needs to be modified for
Yep. But the conversion is pretty trivial. Mutt autodetects the folder
type. If a file, mbox. If a dir with tmp,new,cur, a Maildir. Probably MH
otherwise.
So provided you _make_ an empty Maildir (mkdir $d $d/tmp $d/new $d/cur)
mutt will deliver into it correctly - no special modes. You only need to
instruct mutt when _it_ creates the mail folder, by setting:
set mbox_type=maildir
as your preference for new folders.
My apologies, so my plan is to first get the change from MH to Maildir done and then start
fetchmail/procmail to add to those folders after running it through sylfilter (which I
package for Fedora) and then fire mutt up after that for reading/responding, etc. Of
course, I am not sure if it is worthwhile to use sylfilter anymore given that it was
integrated with sylpheed in the training but perhaps I can still keep that. (I will deal
with that later.)
Yes, I can switch to getmail but I honestly don't know how much a learning curve that
would be. Or are you recommending that I use getmail and then mutt on that (instead of
procmail)?
>According to this post here:
https://www.systoolsgroup.com/maildir/ the filename has this
"gator3018.hostgator.com" which I presume comes from the hostname. How do I get
this changed to something else (unique, but not to the hostname)? Is this where mutt or
procmail is to be told to do this, how?
Not sure. The hostname is, IIRC, an optional component for making the
filesnames; I'm not sure mutt offers any control over that. Here're a
couple of names from my main inbox:
1386376685.65689_1131.fleet:2,S
1602208593.#2087M806900P64256:2,S
The former has a hostname in it, the latter does not. [looks...] Hoo,
the one with a hostname is Very Old. Likely delivered by procmail and
spamassassin. These days I collect with getmail and file with my own
mail filer. The code for making a new Maildir folder in the latter is
this:
def newkey(self):
''' Allocate a new key.
'''
now = time.time()
secs = int(now)
subsecs = now-secs
key = '%d.#%dM%dP%d' % (secs, seq(), subsecs * 1e6, self.pid)
assert self.validkey(key), "invalid new key: %s" % (key,)
return key
I see. I do have a lot of conditions set by procmail so would like to try to keep that if
possible. Perhaps I will stick to procmail. But does procmail handle Maildir according to
any recipe that has to be set: for instance, I clearly do not want the hostname in the
filenames.
I believe once made, clients (mutt etc) don't muck with the names except
to change the suffix (",S") which is where message flags are
implemented (means you can set flags without editing the message file to
fiddle a header).
>
>The reason for how I set things up, and that has worked reasonably well, is (you sort
of guessed it) that I read e-mail at work and home, but the work machine is the one that I
consider to be reliably backed up. It is also bigger in terms of disk. So what happens is
that I fetchmail with keep, process e-mail at work using sylpheed and then fire up my home
machine (a laptop) and fetchmail with keep from the POP server and the rsync it down
(including the .sylpheed_mark and .sylpheed_cache). Then I work on the home machine,
continue to fetchmail process e-mails, etc with sylpheed and when I am done (before I
hibernate), I rsync it all up before I go back to work, so that when I go to the other
(work) machine, I have the same status as I left at home/work. It has worked reasonably
well.
Ah, so you've always got an rsync between transitions. That should work
just as well with Maildir.
Yes, thanks, as long as the names are identical between the two (or more) machines.
Otherwise there will be duplication and my carefully-crafted situation will be lost:-(
I'm also aware of a number of people who remote to their main
machine: ssh in
and run mutt there.
Oh, I used to do that, except with pine, around twenty years ago, over a phone line (no
less). Strangely, I rarely lost connection and did not notice much of a slowness. For some
reason, I don't see it succeeding nowadays. That is when I switched to my current
approach (around 15 years ago).
>Thanks, I have currently been using postfix too, but will
probably try mutt's smtp (now that they have enabled office365 oauth2 support).
Postfix has not yet done so, they have done some with gmail, but it is not completely
clear to me yet how to use that with MS Office365. Same with msmtp, it seems.
Fair enough. One advantage of postfix is that local programmes like cron
can also send email. But of course, nothing prevents you having a
working local postfix for the system and using mutt-smtp or one of the
various "send with smtp" standalone tools as well.
If I can figure postfix out to run with MS O365's OAuth tokens (that is mandated by my
employer), I think that will work well for me. Let us see. Problem is that not that many
people in 0365 world with OAuth2 requirements seem to be enamored of local e-mail reading.
So progress in figuring this out is slow.
Thanks again!
Ranjan
7:37 p.m.
On 16Oct2020 18:53, Ranjan Maitra <maitra(a)email.com> wrote:
> >Thanks very much for this! I will try it out, but it appears
to be for individual folders, is that correct? So, I will maybe write a script to call
it.
>
> Yes. I just wrote a for loop on the command line. Something like:
>
> cd ~/mail
> for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break; done
So, in this code, there is a command called ismhdir -- where does this come from? Is there
something missing here? (I think it also shows up in your newer code.)
Oh, there's probably a few of my scripts in that one. They're all
available from the same URL (go up to the bin/ URL). Or I can ship you a
tarball or you can clone the repo, whatever you find easy.
> Yep. But the conversion is pretty trivial. Mutt autodetects the
folder
> type. If a file, mbox. If a dir with tmp,new,cur, a Maildir. Probably MH
> otherwise.
>
> So provided you _make_ an empty Maildir (mkdir $d $d/tmp $d/new $d/cur)
> mutt will deliver into it correctly - no special modes. You only need to
> instruct mutt when _it_ creates the mail folder, by setting:
>
> set mbox_type=maildir
>
> as your preference for new folders.
My apologies, so my plan is to first get the change from MH to Maildir done and then start
fetchmail/procmail to add to those folders after running it through sylfilter (which I
package for Fedora) and then fire mutt up after that for reading/responding, etc. Of
course, I am not sure if it is worthwhile to use sylfilter anymore given that it was
integrated with sylpheed in the training but perhaps I can still keep that. (I will deal
with that later.)
You don't need to "switch to mutt" yet. You can still use it as a
conversion tool :-) Keep the previous MH folder there until you're happy
the Maildir is a faithful reproduction.
Yes, I can switch to getmail but I honestly don't know how much a
learning curve that would be. Or are you recommending that I use getmail and then mutt on
that (instead of procmail)?
No obligation to switch to getmail, I was just describing the tools I
use for content.
I see. I do have a lot of conditions set by procmail so would like to
try to keep that if possible. Perhaps I will stick to procmail. But does procmail handle
Maildir according to any recipe that has to be set: for instance, I clearly do not want
the hostname in the filenames.
Stick with procmail until unhappy with it. It will deliver to Maildirs,
you change the form of the folder name. A trailing slash or something to
indicate the format. [...digs around...] I think "folder" for mbox,
"folder/"
for Maildir, "folder/." for MH.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
8:01 p.m.
On 17Oct2020 11:37, Cameron Simpson <cs(a)cskk.id.au> wrote:
>Yes, I can switch to getmail but I honestly don't know how
much a learning curve that would be. Or are you recommending that I use getmail and then
mutt on that (instead of procmail)?
Oh, and getmail doesn't replace procmail, it replaces fetchmail. You
still need a message filer.
I can't offhand remember why I because unhappy with fetchmail (which I was
using). It may be because it will only deliver to SMTP/LMTP or a delivery
programme like fetchmail; when I left procmail for my own mail filer I
wanted to deliver directly to a Maildir folder ("spool"), which is
monitored by the mailfiler daemon.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
8:40 p.m.
Thanks!
On Sat, 17 Oct 2020 11:37:27 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 16Oct2020 18:53, Ranjan Maitra <maitra(a)email.com> wrote:
>> >Thanks very much for this! I will try it out, but it appears to be for
individual folders, is that correct? So, I will maybe write a script to call it.
>>
>> Yes. I just wrote a for loop on the command line. Something like:
>>
>> cd ~/mail
>> for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break;
done
>
>So, in this code, there is a command called ismhdir -- where does this come from? Is
there something missing here? (I think it also shows up in your newer code.)
Oh, there's probably a few of my scripts in that one. They're all
available from the same URL (go up to the bin/ URL). Or I can ship you a
tarball or you can clone the repo, whatever you find easy.
No, thanks, I got it. Wow, you have a lot of scripts: some more may be useful.
So, I tried your code:
cd ~/mail
for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break; done
But it only did the first folder and quit: I was thinking that it would go through the
whole list of folders and sub-folders.
I did get the following warning:
procmail: Couldn't chdir to ""
However, it appended the hostname. I looked at the script, but I could not figure out
where this comes in from: is it the default in the procmail? Perhaps I should try the
newer version, but am a little confused with how to make it convert the mh to mhdir?
Do I simply set set mbox_type=maildir in my .muttrc?
>> So provided you _make_ an empty Maildir (mkdir $d $d/tmp
$d/new $d/cur)
>> mutt will deliver into it correctly - no special modes. You only need to
>> instruct mutt when _it_ creates the mail folder, by setting:
>>
>> set mbox_type=maildir
>>
>> as your preference for new folders.
>
>My apologies, so my plan is to first get the change from MH to Maildir done and then
start fetchmail/procmail to add to those folders after running it through sylfilter (which
I package for Fedora) and then fire mutt up after that for reading/responding, etc. Of
course, I am not sure if it is worthwhile to use sylfilter anymore given that it was
integrated with sylpheed in the training but perhaps I can still keep that. (I will deal
with that later.)
You don't need to "switch to mutt" yet. You can still use it as a
conversion tool :-) Keep the previous MH folder there until you're happy
the Maildir is a faithful reproduction.
>Yes, I can switch to getmail but I honestly don't know how much a learning curve
that would be. Or are you recommending that I use getmail and then mutt on that (instead
of procmail)?
No obligation to switch to getmail, I was just describing the tools I
use for content.
>I see. I do have a lot of conditions set by procmail so would like to try to keep
that if possible. Perhaps I will stick to procmail. But does procmail handle Maildir
according to any recipe that has to be set: for instance, I clearly do not want the
hostname in the filenames.
Stick with procmail until unhappy with it. It will deliver to Maildirs,
you change the form of the folder name. A trailing slash or something to
indicate the format. [...digs around...] I think "folder" for mbox,
"folder/"
for Maildir, "folder/." for MH.
Yes, that is correct.
Thanks very much again!
Ranjan
9:11 p.m.
On Sat, 17 Oct 2020 12:01:59 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 17Oct2020 11:37, Cameron Simpson <cs(a)cskk.id.au> wrote:
>>Yes, I can switch to getmail but I honestly don't know how much a learning
curve that would be. Or are you recommending that I use getmail and then mutt on that
(instead of procmail)?
Oh, and getmail doesn't replace procmail, it replaces fetchmail. You
still need a message filer.
I can't offhand remember why I because unhappy with fetchmail (which I was
using). It may be because it will only deliver to SMTP/LMTP or a delivery
programme like fetchmail; when I left procmail for my own mail filer I
wanted to deliver directly to a Maildir folder ("spool"), which is
monitored by the mailfiler daemon.
Well, I was unhappy with fetchmail for a while also because it had become excruciatingly
slow and with no development for a while: tends to be an indicator, as is happening with
sylpheed. I considered abandoning it for a while but inertia solved that problem, and a
new team took over. I still think that they have been too slow in moving to 7 (it has been
around for a few years) which is now scheduled to debut only in 2021.
Btw, I also noticed that you use(d) fvwm. I used that for a long time, switching to
openbox (with bouts of pekwm) about 10 years ago.
Ranjan
9:21 p.m.
On 16Oct2020 20:40, Ranjan Maitra <maitra(a)email.com> wrote:
> >So, in this code, there is a command called ismhdir -- where
does this come from? Is there something missing here? (I think it also shows up in your
newer code.)
>
> Oh, there's probably a few of my scripts in that one. They're all
> available from the same URL (go up to the bin/ URL). Or I can ship you a
> tarball or you can clone the repo, whatever you find easy.
No, thanks, I got it. Wow, you have a lot of scripts: some more may be useful.
So, I tried your code:
cd ~/mail
for mhdir in [a-z]*; do (set -x; mh2maildir "$mhdir") || break; done
But it only did the first folder and quit: I was thinking that it would go through the
whole list of folders and sub-folders.
The above for-loop quits if mh2maildir exits nonzero (some error). That
way I get to debug without frying every following folder. Or digging the
error output of a lot of other successful output.
I did get the following warning:
procmail: Couldn't chdir to ""
However, it appended the hostname. I looked at the script, but I could not figure out
where this comes in from: is it the default in the procmail?
Maybe you have no $MAILDIR envvar? Just guessing. That is an old script,
and doesn't abort on unset envvars like my recent ones do.
The script writes a procmailrc file like this:
MAILDIR=$MAILDIR
DEFAULT=$mhdir/
LOGFILE=$LOGDIR/procmail
I presume procmail expects $MAILDIR to exist?
Invoke the script like this:
sh -uex /path/to/mh2maildir your-mh-folder
and see what the output looks like. That will show you what is happening
in more detail.
Perhaps I should try the newer version, but am a little confused with
how to make it convert the mh to mhdir?
Do I simply set set mbox_type=maildir in my .muttrc?
Nah, it isn't using mutt. It sets $DEFAULT to $mhdir/, and that trailing
slash says to treat $mhdir as a Maildir.
The script moves the original MH folder to $mhdir-mh and makes an empty
$mhdir to be the new Maildir. It looks like it expected procmail to make
the tmp/new/cur subdirs.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
4:23 a.m.
On 16Oct2020 21:11, Ranjan Maitra <maitra(a)email.com> wrote:
Btw, I also noticed that you use(d) fvwm. I used that for a long time,
switching to openbox (with bouts of pekwm) about 10 years ago.
I think I'd still be using FVWM - I had a nice setup with no window titles or
borders and keyboard driven layout/positioning, many desktops indexed by name
with supporting scripts to make new ones ("nd TASKNAME" => new desktop and
new menu item), etc.
I've been on a Mac for several years these days, and I'd be missing iterm3
greatly if I were back on an X11 desktop. A keyboard driven window manager
(Divvy for me), autohiding menu bar and dock, and minimalist iTerm3 setup
gets me a setup pretty much as seamless as my FVWM setup.
Example FVWM screenshot:
http://fvwm.sourceforge.net/screenshots/desktops/Cameron_Simpson-desk-256...
http://fvwm.sourceforge.net/screenshots/desktops/index.php?num=50&the...
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
8:29 a.m.
On Sat, 17 Oct 2020 20:23:37 +1100 Cameron Simpson <cs(a)cskk.id.au> wrote:
On 16Oct2020 21:11, Ranjan Maitra <maitra(a)email.com> wrote:
>Btw, I also noticed that you use(d) fvwm. I used that for a long time, switching to
openbox (with bouts of pekwm) about 10 years ago.
I think I'd still be using FVWM - I had a nice setup with no window titles or
borders and keyboard driven layout/positioning, many desktops indexed by name
with supporting scripts to make new ones ("nd TASKNAME" => new desktop and
new menu item), etc.
I've been on a Mac for several years these days, and I'd be missing iterm3
greatly if I were back on an X11 desktop. A keyboard driven window manager
(Divvy for me), autohiding menu bar and dock, and minimalist iTerm3 setup
gets me a setup pretty much as seamless as my FVWM setup.
Example FVWM screenshot:
http://fvwm.sourceforge.net/screenshots/desktops/Cameron_Simpson-desk-256...
http://fvwm.sourceforge.net/screenshots/desktops/index.php?num=50&the...
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
Thanks, Cameron! I just checked at fvwm.org and it appears that fvwm is still back under
active development (with a v3 apparently in the works). However, I have to say that
openbox (and even pekwm) have been pretty good to me.
Btw, can I get your filer? It turns out that procmail does not allow the format of a
Maildir file to be changed. One option is to figure out setting the hostname env for
procmailrc (and mutt) to say localhost.localdomain (that name would be generic enough to
be common across machines) or give up and continue with the MH format (though in that
case, I need to figure out what is in a .mh_sequence in order for a mail-monitoring
program -- originally xbuffy -- I spent days/weeks modified for sylpheed to work).
But I am interested in your filer script for curiosity's sake also. I looked around
your bin pages but could not easily figure out which one it was: of course, it may not
also be there.
Thanks again!
Ranjan
10:15 a.m.
On Sat, 17 Oct 2020 09:20:50 -0400 Tom Horsley <horsley1953(a)gmail.com> wrote:
On Fri, 16 Oct 2020 21:11:27 -0500
Ranjan Maitra wrote:
> as is happening with sylpheed
Didn't sylpheed transform into claws-mail? Claws-mail seems
to get regular updates.
Hi Tom,
Thanks! Originally, sylpheed-claws was a sort of enhancement to sylpheed, but claws-mail
is supposed to be a different product. (For instance, there are differences in how each
handle MH folders. Each have their own different ways.) Regardless, it is very difficult
to incorporate other sending programs with claws-mail or sylpheed, and there is no
documentation on how to do that and no one ever responded to my post enquiring details on
both mailing lists (well, one person responded on claws-mail saying that he saw my similar
post on the sylpheed ML). This creates an issue when you have to incorporate things like
non-google OAUTH2 (MS O365) authentication in the sending and retrieving e-mail. So, I
decided to move to mutt (originally, neomutt) which appears to have a larger community of
users and also has chances of having greater staying power in the long term.
But I am now thinking that I will use mutt with MH folders for now. If the famed race
issues do not crop up for me, then we are good. And then if they crop up and I need to
shift shop, I will not be dealing with learning mutt and Maildir folders at the same
time.
Ranjan
5:02 p.m.
On 17Oct2020 08:29, Ranjan Maitra <maitra(a)email.com> wrote:
On Sat, 17 Oct 2020 20:23:37 +1100 Cameron Simpson
<cs(a)cskk.id.au> wrote:
> I think I'd still be using FVWM - I had a nice setup with no window titles or
> borders and keyboard driven layout/positioning, many desktops indexed by name
> with supporting scripts to make new ones ("nd TASKNAME" => new desktop
and
> new menu item), etc. [...]
Thanks, Cameron! I just checked at fvwm.org and it appears that fvwm is still back under
active development (with a v3 apparently in the works). However, I have to say that
openbox (and even pekwm) have been pretty good to me.
I'll try to keep them in mind for when I next use an X11 desktop.
Btw, can I get your filer? It turns out that procmail does not allow
the format of a Maildir file to be changed. One option is to figure out setting the
hostname env for procmailrc (and mutt) to say localhost.localdomain (that name would be
generic enough to be common across machines) or give up and continue with the MH format
(though in that case, I need to figure out what is in a .mh_sequence in order for a
mail-monitoring program -- originally xbuffy -- I spent days/weeks modified for sylpheed
to work).
But I am interested in your filer script for curiosity's sake also. I looked around
your bin pages but could not easily figure out which one it was: of course, it may not
also be there.
You want the "mailfiler" script in my bin directory, but that is only a
stub which invokes the Python cs.app.mailfiler module. The easiest way
to get that is:
pip install cs.app.mailfiler
which should also get you a "mailfiler" command in the Python bin directory.
The source code is here:
https://hg.sr.ht/~cameron-simpson/css/browse/lib/python/cs/app/mailfiler....
https://hg.sr.ht/~cameron-simpson/css/browse/lib/python/cs/app/mailfiler....
https://hg.sr.ht/~cameron-simpson/css/browse/lib/python/cs/app/mailfiler....
being the code, the command line man page and the rules syntax man page
respectively.
You'll see that the rules syntax is far more succinct and readable than
procmail's.
It does require some setup, and if you bother to try it I'd be interested
to know how that plays out and might be improved, because it doubtless
has an assortment of assumed things about my personal environment.
The basic command is "mailfiler monitor", which watches a set of
Maildirs as incoming mail spools, and applies per spool folder rules to
each message which arrives. So you fetch mail into the spool folder, and
the daemon files from theere, separately from the mail fetcher.
Like you, I have a few mail identities - my main personal one and some
workplace ones - I fetch each into distinct spool folders basic so that
they can get a ittle per-workplace cross filing, then copy them all
additionally into my "spool-in", where my main rules live.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
8:11 p.m.
On Fri, 2020-10-16 at 08:35 -0500, Ranjan Maitra wrote:
According to this post here: https://www.systoolsgroup.com/maildir/
t
he filename has this "gator3018.hostgator.com" which I presume comes
from the hostname. How do I get this changed to something else
(unique, but not to the hostname)? Is this where mutt or procmail is
to be told to do this, how?
I want to ask: Why do you care what the filenames are (that they have
hostnames in them)?
The mail client and server certainly doesn't care, it just needs unique
names. So if you can get mail files from machine A onto machine B,
it'll just use them however they're named. They don't need to have the
same filename as its hostname.
Trying to change maildir filenames requires care, so you don't break
the mail system. And you use a mail client to read your messages, it
gives you an interface where you don't need to know the filenames that
it's making use of.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
8:48 p.m.
On Sun, 18 Oct 2020 11:41:05 +1030 Tim via users <users(a)lists.fedoraproject.org>
wrote:
On Fri, 2020-10-16 at 08:35 -0500, Ranjan Maitra wrote:
> According to this post here: https://www.systoolsgroup.com/maildir/ t
> he filename has this "gator3018.hostgator.com" which I presume comes
> from the hostname. How do I get this changed to something else
> (unique, but not to the hostname)? Is this where mutt or procmail is
> to be told to do this, how?
I want to ask: Why do you care what the filenames are (that they have
hostnames in them)?
The mail client and server certainly doesn't care, it just needs unique
names. So if you can get mail files from machine A onto machine B,
it'll just use them however they're named. They don't need to have the
same filename as its hostname.
Thanks, would it not matter if there is a combination of files in the two folders? I
don't know, that is why I am asking.
Trying to change maildir filenames requires care, so you don't
break
the mail system. And you use a mail client to read your messages, it
gives you an interface where you don't need to know the filenames that
it's making use of.
There is one approach and that is set LD_PRELOAD to set the hostname as
localhost.localdomain from this example in:
https://catonmat.net/simple-ld-preload-tutorial
But if it does not matter, then I do not need to worry at all.
Thanks for clarifying this.
Ranjan
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
_______________________________________________
users mailing list -- users(a)lists.fedoraproject.org
To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
--
Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt.
Please respond to the mailing list if appropriate. For those needing to send personal or
professional e-mail, please use appropriate addresses.
10:06 p.m.
On 17Oct2020 20:48, Ranjan Maitra <maitra(a)email.com> wrote:
On Sun, 18 Oct 2020 11:41:05 +1030 Tim via users
<users(a)lists.fedoraproject.org> wrote:
> On Fri, 2020-10-16 at 08:35 -0500, Ranjan Maitra wrote:
> > According to this post here: https://www.systoolsgroup.com/maildir/ t
> > he filename has this "gator3018.hostgator.com" which I presume comes
> > from the hostname. How do I get this changed to something else
> > (unique, but not to the hostname)? Is this where mutt or procmail is
> > to be told to do this, how?
>
> I want to ask: Why do you care what the filenames are (that they have
> hostnames in them)?
I was wondering this myself. I was imagining some aesthetic issue.
> The mail client and server certainly doesn't care, it just
needs unique
> names. So if you can get mail files from machine A onto machine B,
> it'll just use them however they're named. They don't need to have the
> same filename as its hostname.
Thanks, would it not matter if there is a combination of files in the two folders? I
don't know, that is why I am asking.
Doesn't matter. They just have to be unique and have a legal ,flags
suffix. I've got a mix myself (historical change of the things writing
messages).
I'm having trouble getting at the qmail.org site, but on Wikipedia this
section:
https://en.wikipedia.org/wiki/Maildir#Technical_operation
describes the filename components which may be used to assemble a unique
name.
> Trying to change maildir filenames requires care, so you
don't break
> the mail system. And you use a mail client to read your messages, it
> gives you an interface where you don't need to know the filenames that
> it's making use of.
There is one approach and that is set LD_PRELOAD to set the hostname as
localhost.localdomain from this example in:
https://catonmat.net/simple-ld-preload-tutorial
But if it does not matter, then I do not need to worry at all.
I wouldn't worry, myself.
The hostname component is useful for when mulitple hosts might be
writing into the Maildir (eg over NFS to a shared spool). For a
single host Maildir it isn't necessary.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
6:22 a.m.
On Sat, 2020-10-17 at 20:48 -0500, Ranjan Maitra wrote:
There is one approach and that is set LD_PRELOAD to set the hostname
as localhost.localdomain from this example in:
https://catonmat.net/simple-ld-preload-tutorial
I wouldn't mess with telling software to use a different hostname than
it expects, you might find you break some aspect of how it uses the
network.
But if it does not matter, then I do not need to worry at all.
Doesn't matter to the mail client. It just wants unique filenames per
email, and the different hostnames help. It might help you, too, if
you ever had to debug a mail issue. You'd be able to tell where a
message came from simply by reading the filename.
But, as I said. Generally, you don't see the filenames of emails, you
use an email program that's an interface.
There's a syntax to the email filenames for maildir: There's a unique
id for each message, that's a lengthy part of the filename, it can be
construed from hostnames and datestamps. And there's suffixes added
which are flags for message statuses (e.g. that it's been read).
Horsing around with the filenames just risks doing something that
causes a failure. It's not like the rest of the filename is a
particular sane thing for humans to read, anyway. It's all just a big
long code.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
9:03 a.m.
On Sun, 18 Oct 2020 21:52:52 +1030 Tim via users <users(a)lists.fedoraproject.org>
wrote:
On Sat, 2020-10-17 at 20:48 -0500, Ranjan Maitra wrote:
> There is one approach and that is set LD_PRELOAD to set the hostname
> as localhost.localdomain from this example in:
> https://catonmat.net/simple-ld-preload-tutorial
I wouldn't mess with telling software to use a different hostname than
it expects, you might find you break some aspect of how it uses the
network.
> But if it does not matter, then I do not need to worry at all.
Doesn't matter to the mail client. It just wants unique filenames per
email, and the different hostnames help. It might help you, too, if
you ever had to debug a mail issue. You'd be able to tell where a
message came from simply by reading the filename.
But, as I said. Generally, you don't see the filenames of emails, you
use an email program that's an interface.
There's a syntax to the email filenames for maildir: There's a unique
id for each message, that's a lengthy part of the filename, it can be
construed from hostnames and datestamps. And there's suffixes added
which are flags for message statuses (e.g. that it's been read).
Horsing around with the filenames just risks doing something that
causes a failure. It's not like the rest of the filename is a
particular sane thing for humans to read, anyway. It's all just a big
long code.
Thanks, Tim! I agree with you that that is the safer option. Btw, do you know if Maildir
folders can have a mixture of files and sub-folders. So, let me explain: some of my
sylpheed mh folders have messages in them as well as sub-folders. So, my question is: is
it acceptable to have a Maildir folder that has cur/ new/ and tmp/ in addition to
subfolder inside which is the same format?
Many thanks again and best wishes,
Ranjan
8:04 p.m.
On Sun, 2020-10-18 at 09:03 -0500, Ranjan Maitra wrote:
do you know if Maildir folders can have a mixture of files and sub-
folders. So, let me explain: some of my sylpheed mh folders have
messages in them as well as sub-folders. So, my question is: is it
acceptable to have a Maildir folder that has cur/ new/ and tmp/ in
addition to subfolder inside which is the same format?
Yes and no... I can't answer to the technique of doing what's
permissible, and that's going to depend on the software involved. In
*my* case, I can nest folders, but a folder can contain only mail or
another folder, not both together.
But I do have sub-folders, and I use maildir. However, it's my mail
client handles *how* it does sub-folders, and (in my case) they're done
with a bit of fakery. There aren't folders inside folders stored on my
filesystem. They're all in directly in my ~/.mail directory. They're
named in a way that the mail client can show them in nested folders.
e.g. While my mail client shows a tree like this:
announcements/
/bugzilla
/CentOS
/Fedora
/Google
/Red Hat
lists/
/Fedora
/Musescore
/tech
logs/
/boot
/cron
/logwatch
/printer
With those lists of five, three, and four, folders inside the ones
above them. On-disc, they're all in one place, like this:
~/.mail/announcements.bugzilla
~/.mail/announcements.CentOS
~/.mail/announcements.Fedora
~/.mail/announcements.Google
~/.mail/announcements.Red Hat
~/.mail/lists.Fedora
~/.mail/lists.Musescore
~/.mail/lists.tech
~/.mail/logs.boot
~/.mail/logs.cron
~/.mail/logs.logwatch
~/.mail/logs.printer
It allows you to structure your mail client in a way that's useful to
you, without you having to care about how it's achieved.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
8:22 a.m.
On Mon, 19 Oct 2020 11:34:12 +1030 Tim via users <users(a)lists.fedoraproject.org>
wrote:
On Sun, 2020-10-18 at 09:03 -0500, Ranjan Maitra wrote:
> do you know if Maildir folders can have a mixture of files and sub-
> folders. So, let me explain: some of my sylpheed mh folders have
> messages in them as well as sub-folders. So, my question is: is it
> acceptable to have a Maildir folder that has cur/ new/ and tmp/ in
> addition to subfolder inside which is the same format?
Yes and no... I can't answer to the technique of doing what's
permissible, and that's going to depend on the software involved. In
*my* case, I can nest folders, but a folder can contain only mail or
another folder, not both together.
But I do have sub-folders, and I use maildir. However, it's my mail
client handles *how* it does sub-folders, and (in my case) they're done
with a bit of fakery. There aren't folders inside folders stored on my
filesystem. They're all in directly in my ~/.mail directory. They're
named in a way that the mail client can show them in nested folders.
e.g. While my mail client shows a tree like this:
announcements/
/bugzilla
/CentOS
/Fedora
/Google
/Red Hat
lists/
/Fedora
/Musescore
/tech
logs/
/boot
/cron
/logwatch
/printer
With those lists of five, three, and four, folders inside the ones
above them. On-disc, they're all in one place, like this:
~/.mail/announcements.bugzilla
~/.mail/announcements.CentOS
~/.mail/announcements.Fedora
~/.mail/announcements.Google
~/.mail/announcements.Red Hat
~/.mail/lists.Fedora
~/.mail/lists.Musescore
~/.mail/lists.tech
~/.mail/logs.boot
~/.mail/logs.cron
~/.mail/logs.logwatch
~/.mail/logs.printer
It allows you to structure your mail client in a way that's useful to
you, without you having to care about how it's achieved.
OK, thanks! I have subfolders inside my Mail directory as follows:
Mail/inbox
Mail/Sent
Mail/Drafts
Mail/research
Mail/research/user1
Mail/research/user2
etc, so my real question is with Mail/research which has both mail messages as well as a
sub-folder inside. Of course, I can change it but wanted to also know if that is possible
to have both messages and subfolders with mutt (say) and Maildir.
Best wishes,
Ranjan
7:36 p.m.
On 19Oct2020 08:22, Ranjan Maitra <maitra(a)email.com> wrote:
OK, thanks! I have subfolders inside my Mail directory as follows:
Mail/inbox
Mail/Sent
Mail/Drafts
Mail/research
Mail/research/user1
Mail/research/user2
etc, so my real question is with Mail/research which has both mail messages as well as a
sub-folder inside. Of course, I can change it but wanted to also know if that is possible
to have both messages and subfolders with mutt (say) and Maildir.
Mutt should be fine - it doens't care about your directory structure,
only what format to use for the folder you're looking at right now.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
9:58 p.m.
On Mon, 2020-10-19 at 11:34 +1030, Tim via users wrote:
In *my* case, I can nest folders, but a folder can contain only mail
or another folder, not both together.
But I do have sub-folders, and I use maildir.
A correction: I *can* freely mix mail and folders (putting a folder in
a folder that had messages, putting a message in a folder containing
folders, etc). Previously I recall my mail client (Evolution) refusing
to allow one or the other, but it *is* allowing it *now*.
Of course, my mail server (Dovecot) isn't actually working that way.
It stores all the folders within its root folder, and just generates an
index to make things work with your mail client the way you want them
organised. But that may just be how Dovecot wants to do things.
Anyway, it doesn't really matter how it works behind the scenes, if it
lets you do what you want to do with your mail client.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
12:10 p.m.
On Tue, 20 Oct 2020 13:28:37 +1030 Tim via users <users(a)lists.fedoraproject.org>
wrote:
On Mon, 2020-10-19 at 11:34 +1030, Tim via users wrote:
> In *my* case, I can nest folders, but a folder can contain only mail
> or another folder, not both together.
>
> But I do have sub-folders, and I use maildir.
A correction: I *can* freely mix mail and folders (putting a folder in
a folder that had messages, putting a message in a folder containing
folders, etc). Previously I recall my mail client (Evolution) refusing
to allow one or the other, but it *is* allowing it *now*.
Of course, my mail server (Dovecot) isn't actually working that way.
It stores all the folders within its root folder, and just generates an
index to make things work with your mail client the way you want them
organised. But that may just be how Dovecot wants to do things.
Anyway, it doesn't really matter how it works behind the scenes, if it
lets you do what you want to do with your mail client.
Thanks, very much! Btw, how do I list a directory recursively with the more nested
directories listed first inside a directory? I feel that I may want to do this before I
use my script to convert MH to Maildir.
I tried:
find . -type d
./inbox
./Sent
./Trash
./fedora
./fedora/FAS
./research
./research/user1
./research/user2
./reviewer
./reviewer/can1
./reviewer/can2
and this does the right thing, almost, except that it lists the directories, and then the
sub-directories of each. Is it possible to reverse this listing? If that were possible, I
was considering running my script in order, then removing that directory and then going on
down the list.
Many thanks and best wishes,
Ranjan
12:25 p.m.
On 10/20/20 10:10 AM, Ranjan Maitra wrote:
Thanks, very much! Btw, how do I list a directory recursively with
the more nested directories listed first inside a directory? I feel that I may want to do
this before I use my script to convert MH to Maildir.
I tried:
find . -type d
./inbox
./Sent
./Trash
./fedora
./fedora/FAS
./research
./research/user1
./research/user2
./reviewer
./reviewer/can1
./reviewer/can2
and this does the right thing, almost, except that it lists the directories, and then the
sub-directories of each. Is it possible to reverse this listing? If that were possible, I
was considering running my script in order, then removing that directory and then going on
down the list.
Pipe the output of that to "tac".
12:33 p.m.
On Tue, 20 Oct 2020 10:25:11 -0700 Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/20/20 10:10 AM, Ranjan Maitra wrote:
> Thanks, very much! Btw, how do I list a directory recursively with the more nested
directories listed first inside a directory? I feel that I may want to do this before I
use my script to convert MH to Maildir.
>
> I tried:
>
> find . -type d
>
>
> ./inbox
> ./Sent
> ./Trash
> ./fedora
> ./fedora/FAS
> ./research
> ./research/user1
> ./research/user2
> ./reviewer
> ./reviewer/can1
> ./reviewer/can2
>
> and this does the right thing, almost, except that it lists the directories, and
then the sub-directories of each. Is it possible to reverse this listing? If that were
possible, I was considering running my script in order, then removing that directory and
then going on down the list.
Pipe the output of that to "tac".
Wow! The reverse of cat, who would have guessed? Thanks!
Ranjan
_______________________________________________
users mailing list -- users(a)lists.fedoraproject.org
To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
--
Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt.
Please respond to the mailing list if appropriate. For those needing to send personal or
professional e-mail, please use appropriate addresses.
12:52 p.m.
Ranjan Maitra wrote:
On Tue, 20 Oct 2020 10:25:11 -0700 Samuel Sieb
<samuel(a)sieb.net> wrote:
> Pipe the output of that to "tac".
Wow! The reverse of cat, who would have guessed? Thanks!
While tac is handy (and amusingly named), I don't think it's
what you want. The reason is that the find output order
isn't deterministic. It's in the directory entry order,
which depends on when the directories were created.
Technically, it's a reasonable bet that a sub-directory will
be created after the parent directory. But reasonable bets
still involve more risk than it's worth when operating on
your files. :)
I think you'd want to pipe the find output via something
like sort if you want a stable sort order. And if you're
piping to sort, then you might as well just use -r to
reverse it rather than add another pipe to tac. E.g.:
$ find . -type d | sort -r
(I'd add '-mindepth 1' as well to exclude the top-level "."
in the output. But I'm not sure if that suits your use-case
here or not.)
--
Todd
12:54 p.m.
On 10/20/20 10:33 AM, Ranjan Maitra wrote:
On Tue, 20 Oct 2020 10:25:11 -0700 Samuel Sieb
<samuel(a)sieb.net> wrote:
> Pipe the output of that to "tac".
Wow! The reverse of cat, who would have guessed? Thanks!
Ha, that's great! I was wondering where the name came from and didn't
notice that.
12:58 p.m.
On 10/20/20 10:52 AM, Todd Zullinger wrote:
While tac is handy (and amusingly named), I don't think it's
what you want. The reason is that the find output order
isn't deterministic. It's in the directory entry order,
which depends on when the directories were created.
Technically, it's a reasonable bet that a sub-directory will
be created after the parent directory. But reasonable bets
still involve more risk than it's worth when operating on
your files. :)
I can't imagine how it could work like that. It has to go recursively
so it will *always* find the parent directory first. How could a
subdirectory possibly be created before its parent directory?
2:19 p.m.
On Tue, Oct 20, 2020 at 12:10:13PM -0500, Ranjan Maitra wrote:
On Tue, 20 Oct 2020 13:28:37 +1030 Tim via users
<users(a)lists.fedoraproject.org> wrote:
Thanks, very much! Btw, how do I list a directory recursively with the more nested
directories listed first inside a directory? I feel that I may want to do this before I
use my script to convert MH to Maildir.
I tried:
find . -type d
./inbox
./Sent
./Trash
./fedora
./fedora/FAS
./research
./research/user1
./research/user2
./reviewer
./reviewer/can1
./reviewer/can2
and this does the right thing, almost, except that it lists the directories, and then the
sub-directories of each. Is it possible to reverse this listing? If that were possible, I
was considering running my script in order, then removing that directory and then going on
down the list.
"find" has a "-depth|-d" option. Try:
find . -d -type d
Jon
--
Jon H. LaBadie jonfu(a)jgcomp.com
4:40 p.m.
Samuel Sieb wrote:
On 10/20/20 10:52 AM, Todd Zullinger wrote:
> Technically, it's a reasonable bet that a sub-directory
> will be created after the parent directory. But
> reasonable bets still involve more risk than it's worth
> when operating on your files. :)
I can't imagine how it could work like that. It has to go
recursively so it will *always* find the parent directory
first. How could a subdirectory possibly be created
before its parent directory?
It'd pretty unlikely for that to be the case, but you'd be
relying on the find command, the underlying readdir() system
calls, etc. which just don't guarantee an order.
So when handling needs to be done in an order, ensuring that
order via something like sort is the only sane way to script
such a task. It's along the lines of the old saying "trust,
but verify." :)
--
Todd
4:42 p.m.
On 20Oct2020 10:58, Samuel Sieb <samuel(a)sieb.net> wrote:
On 10/20/20 10:52 AM, Todd Zullinger wrote:
>While tac is handy (and amusingly named), I don't think it's
>what you want. The reason is that the find output order
>isn't deterministic. It's in the directory entry order,
>which depends on when the directories were created.
[...]
I can't imagine how it could work like that. It has to go
recursively
so it will *always* find the parent directory first. How could a
subdirectory possibly be created before its parent directory?
WRT to the recursion side, yes, you're right. But at a particular level
the names won't necessarily come out in lexical order (the come out in
directory entry order), so a sort is often useful anyway.
Cheers,
Cameron Simpson <cs(a)cskk.id.au>
5 p.m.
On 10/20/20 2:40 PM, Todd Zullinger wrote:
Samuel Sieb wrote:
> On 10/20/20 10:52 AM, Todd Zullinger wrote:
>> Technically, it's a reasonable bet that a sub-directory
>> will be created after the parent directory. But
>> reasonable bets still involve more risk than it's worth
>> when operating on your files. :)
>
> I can't imagine how it could work like that. It has to go
> recursively so it will *always* find the parent directory
> first. How could a subdirectory possibly be created
> before its parent directory?
It'd pretty unlikely for that to be the case, but you'd be
relying on the find command, the underlying readdir() system
calls, etc. which just don't guarantee an order.
It's more than unlikely, it's literally impossible. find has to open
the parent directory before it can even see the subdirectories.
6:11 p.m.
Thanks, everybody, I am almost there. I am wondering how do I get to list the files
without the first two depths? So, here is a part of my code:
MAILDIR=$HOME/Maildir
MHDIR=$HOME/mail
FOLDERSPre=$(find $MHDIR -type d -mindepth 1)
So, then I get a list of directories: /home/username/mail/....
How do I get the /home/username out of it so that I can append the rest to my Maildir
directory?
I guess this is now likely a bash question, and well and truly OT. Luckily, every part of
this thread is.
Ranjan
6:26 p.m.
Samuel Sieb wrote:
On 10/20/20 2:40 PM, Todd Zullinger wrote:
> It'd pretty unlikely for that to be the case, but you'd be
> relying on the find command, the underlying readdir() system
> calls, etc. which just don't guarantee an order.
It's more than unlikely, it's literally impossible. find has to open the
parent directory before it can even see the subdirectories.
If anyone is that confident that none of the system calls or
code in find will ever surprise them, they're free to skip
the sort. :)
I always think of this quote when I'm writing some code and
being a bit defensive in its construction:
A good programmer is one who looks both ways before
crossing a one-way street. -- Doug Linder
Being in the habit of ensuring the state/order you want pays
off time and again in my experience.
Cheers,
--
Todd
8:17 p.m.
On Tue, 20 Oct 2020 18:11:23 -0500 Ranjan Maitra <maitra(a)email.com> wrote:
Thanks, everybody, I am almost there. I am wondering how do I get to
list the files without the first two depths? So, here is a part of my code:
MAILDIR=$HOME/Maildir
MHDIR=$HOME/mail
FOLDERSPre=$(find $MHDIR -type d -mindepth 1)
So, then I get a list of directories: /home/username/mail/....
How do I get the /home/username out of it so that I can append the rest to my Maildir
directory?
I guess this is now likely a bash question, and well and truly OT. Luckily, every part of
this thread is.
Sorry to answer my own question, but I managed to do this by using sed.
I used:
for ff in $FOLDERSPre
f=$(echo "$ff" | sed -e 's/mail/Maildir/')
which is easier than what i was trying to do: remove the /home/username/mail out and
replace with /home/username/Maildir.
Ranjan
8:55 p.m.
On Tue, 2020-10-20 at 19:26 -0400, Todd Zullinger wrote:
If anyone is that confident that none of the system calls or
code in find will ever surprise them, they're free to skip
the sort. :)
I always think of this quote when I'm writing some code and
being a bit defensive in its construction:
A good programmer is one who looks both ways before
crossing a one-way street. -- Doug Linder
Being in the habit of ensuring the state/order you want pays
off time and again in my experience.
I can see one way people could easily got out of sorts (pun intended),
where they might sort by modification date rather than creation date.
Folders could be modified in any order.
--
uname -rsvp
Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64
Boilerplate: All unexpected mail to my mailbox is automatically deleted.
I will only get to see the messages that are posted to the mailing list.
9:14 p.m.
Hi Ranjan,
Ranjan Maitra wrote:
On Tue, 20 Oct 2020 18:11:23 -0500 Ranjan Maitra
<maitra(a)email.com> wrote:
> Thanks, everybody, I am almost there. I am wondering how
> do I get to list the files without the first two depths?
> So, here is a part of my code:
>
> MAILDIR=$HOME/Maildir
> MHDIR=$HOME/mail
> FOLDERSPre=$(find $MHDIR -type d -mindepth 1)
>
> So, then I get a list of directories:
> /home/username/mail/....
>
> How do I get the /home/username out of it so that I can
> append the rest to my Maildir directory?
>
> I guess this is now likely a bash question, and well and
> truly OT. Luckily, every part of this thread is.
Sorry to answer my own question, but I managed to do this
by using sed.
Nothing to be sorry about there. You've answered your own
question, but also shared it with the list where others can
learn from it. That's always a plus!
I used:
for ff in $FOLDERSPre
f=$(echo "$ff" | sed -e 's/mail/Maildir/')
which is easier than what i was trying to do: remove the /home/username/mail out and
replace with /home/username/Maildir.
Another way to slice this would be through parameter
expansion. For example:
for ff in $FOLDERSPre; do
f=${ff/$MHDIR/$MAILDIR}
...
done
In the bash manual, search for "Parameter Expansion" for
details on the various types of expansions available. While
the speed of a parameter expansion versus a command
substitution (i.e., the $(echo $ff | sed ... ) in your
example) isn't likely to be noticeable in a script doing a
large MH to Maildir conversion, there are times when it can
improve a script's speed and/or system utilization.
Even without the small performance boost, using a parameter
expansion is often more concise and removes an external
command from the script.
The trade off is that if you're writing a script which needs
to be portable across a wide variety of different operating
systems, you need to consider whether you're using an
expansion that's supported by any POSIX shell (e.g. sh) or
is bash-specific (and ideally adjust the code and/or the
"#!" hashbang appropriately).
Of course, writing such scripts also involves knowing the
same of the utilities like sed, which can vary a good bit
between a GNU/Linux system, BSD-derivatives, Solaris, etc.
--
Todd
8:05 a.m.
Thanks very much for everyone's help!
I have written a script which did exactly what I wanted with regard to my conversions from
sylpheed's MH to Mail dir folders. I borrowed from a perl script online that I had to
modify and then used a bash script to handle folders and sub-folders. (I did not use
Todd's suggestion here because I had already done it by then, so this can be
improved.) I will include both scripts in case it is helpful for someone later on. Some
changes (eg checking if the Maildir directory itself exists should be made to the script,
but I did not need it because it did not, for me.)
Now the move to writing a muttrc which matches all the things I did with sylpheed (and
more), phew!
Thanks,
Ranjan
My bash script:
#!/bin/bash
#
# sylpheedMH2Maildir
#
# Convert a sylpheed MH sequence of folders into a Maildir sequence of folders.
# - Ranjan Maitra <aarem(a)fedoraproject.org>
# October 20, 2020
#
MAILDIR=$HOME/Maildir
MHDIR=$HOME/mail
echo $MAILDIR
echo $MHDIR
FOLDERSPre=$(find $MHDIR -mindepth 1 -type d)
mkdir $MAILDIR
for ff in $FOLDERSPre
do
echo "Processing $ff"
f=$(echo "$ff" | sed -e 's/mail/Maildir/')
mkdir $f
echo "Done with $f"
done
# now make the transfer using each sub-folder individually
#
# the strategy is to convert each sub-folder using MH-to-Maildir and
# then copy to the newly created Maildir sub-folder and then remove the
# original folder, and then go on. Therefore it is important to do it
# in reverse order.
#
FOLDERSPost=$(find $MHDIR -mindepth 1 -type d | sort -r)
for ff in $FOLDERSPost
do
f=$(echo "$ff" | sed -e 's/mail/Maildir/')
echo "Converting $ff"
echo "getting the root of $f"
g="${f%/*}/"
echo "root of $f is $g"
MH-to-Maildir $ff $g
rm -rf $ff
echo "Done converting $ff"
done
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Here is the perl script:
#!/usr/bin/perl
#
# Copyright © 2004 Vincent Untz <vincent vuntz.net>
#
# software and description found from: https://www.vuntz.net/projets/mh2maildir/
# This software is available under the GPL
# http://www.gnu.org/licenses/gpl.txt
#
# Originally based on http://rslomkow.org/Pretender/scripts/mh2Maildir
# which is released under the GPL and is:
# Copyright Robin * Slomkowski, 2003
#
# Edited by Ranjan Maitra, 2020 (edit to remove .folder names and have folder names with
Maildir)
#
# Also changed name to MH-to-Maildir (for greater clarity in what is being done)
#
# The convert_dirname function is based on the imap_utf7_encode_local function
# from Squirrelmail. The encodeBASE64 is based on the function of the same
# name from Squirrelmail.
# Squirrelmail is released under the GPL and is:
# Copyright (C) The SquirrelMail Project Team
#
# Useful reference for maildir:
# http://www.courier-mta.org/maildir.html
use strict;
use File::Copy;
use Date::Parse;
use Sys::Hostname;
my $BASE_DIR = "$ENV{'HOME'}/Maildir";
my $DEBUG = 0;
my $use_sylpheed = 1;
MAIN: {
my($src_dir, $dir_name, $start_dir, $dst_dir);
if ( $#ARGV < 0 ) {
usage();
}
$start_dir = shift @ARGV;
if ( "$ARGV[0]" ) {
$dst_dir = shift @ARGV;
} else {
$dst_dir = $BASE_DIR;
}
if (!($dst_dir =~ /\/$/)) {
$dst_dir .= "/";
}
-d $dst_dir or mkdir $dst_dir,0700 or die("Fatal: Directory $dst_dir doesn't
exist and can't be created.\n");
convert_dir($start_dir, $dst_dir);
exit 0;
}
sub encodeBASE64 {
my $s = shift @_;
my $B64Chars =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+,';
my $p = 0; # phase: 1 / 2 / 3 / 1 / 2 / 3...
my $e = ''; # base64-encoded string
my $t;
my $i;
for ($i = 0; $i < length($s); $i++) {
my $c = substr($s, $i, 1);
if ($p == 0) {
$e .= substr($B64Chars, ((ord($c) & 252) >> 2), 1);
$t = (ord($c) & 3);
$p = 1;
} elsif ($p == 1) {
$e .= substr($B64Chars, (($t << 4) + ((ord($c) & 240)
>> 4)), 1);
$t = (ord($c) & 15);
$p = 2;
} elsif ($p == 2) {
$e .= substr($B64Chars, (($t << 2) + ((ord($c) & 192)
>> 6)), 1);
$e .= substr($B64Chars, ord($c) & 63, 1);
$p = 0;
}
}
# flush buffer
if ($p == 1) {
$e .= substr($B64Chars, $t << 4, 1);
} elsif ($p == 2) {
$e .= substr($B64Chars, $t << 2, 1);
}
return $e;
}
sub convert_dirname {
# input a directory
# output the final part of the name.
my $name = shift @_;
my $b64_s = ''; # buffer for substring to be base64-encoded
my $utf7_s = ''; # imap-utf7-encoded string
my $i;
$name =~ s@/$@@;
$name =~ s@^.*/(a)@;
for ($i = 0; $i < length($name); $i++) {
my $c = substr($name, $i, 1);
my $ord_c = ord($c);
if ((($ord_c >= 0x20) and ($ord_c <= 0x25)) or
(($ord_c >= 0x27) and ($ord_c <= 0x2d)) or
(($ord_c >= 0x30) and ($ord_c <= 0x7e))) {
if ($b64_s) {
$utf7_s = $utf7_s . '&' . encodeBASE64($b64_s)
.'-';
$b64_s = '';
}
$utf7_s = $utf7_s . $c;
} elsif ($ord_c == 0x26) { # ampersand
if ($b64_s) {
$utf7_s = $utf7_s . '&' . encodeBASE64($b64_s) .
'-';
$b64_s = '';
}
$utf7_s = $utf7_s . '&-';
} else {
$b64_s = $b64_s . chr(0) . $c;
}
}
# flush buffer
if ($b64_s) {
$utf7_s = $utf7_s . '&' . encodeBASE64($b64_s) . '-';
$b64_s = '';
}
return $utf7_s;
}
sub find_time {
my $file = shift @_;
my $time = undef;
open DATA, $file;
while (<DATA>) {
# end headers
if ( $_ eq "\n" ) {
last;
}
if ($_ =~ /(for .*; |id .*; |^\s+)(\w+, \d+ \w{3} \d{4} \d+:\d+:\d+ [+\-]\d{4})/)
{
my $timestring = $2;
my $t = str2time($timestring);
print "DEBUG5: $timestring -> $t\n" if $DEBUG > 1;
if ( $t > $time ) {
$time = $t;
}
}
}
close DATA;
if (!defined ($time)) {
# we use mtime
$time = (stat ($file))[9];
warn "WARN: no time found in $file" if $DEBUG > 0;
}
return $time;
}
sub find_all {
my $src_dir = shift @_;
my $file = shift @_;
my $time = undef;
my $flags;
my %info = ();
$flags = 'read';
$info{"new"} = 0;
if ($file =~ /^,/) {
$flags .= 'deleted';
$info{"msgcount"} = $file;
$info{"msgcount"} =~ s/^,//;
$info{"host"} = hostname."del";
} else {
$info{"msgcount"} = $file;
$info{"host"} = hostname;
}
# OK now lets read the file
open DATA, "$src_dir/$file";
while (<DATA>) {
# end headers
if ( $_ eq "\n" ) {
last;
}
# check for replied
if ($_ =~ /^Replied: / && $flags !~ /reply/) {
$flags .= 'reply';
}
if ($_ =~ /(for .*; |id .*; |^\s+)(\w+, \d+ \w{3} \d{4} \d+:\d+:\d+ [+\-]\d{4})/)
{
my $timestring = $2;
my $t = str2time($timestring);
print "DEBUG5: $timestring -> $t\n" if $DEBUG > 1;
if ( $t > $time ) {
$time = $t;
}
}
}
close DATA;
if (!defined ($time)) {
# we use mtime
$time = (stat ("$src_dir/$file"))[9];
warn "WARN: no time found in $src_dir/$file" if $DEBUG > 0;
}
$info{"time"} = $time;
$info{"flags"} = "";
$info{"flags"} .= 'R' if $flags =~ /reply/; # Replied to.
$info{"flags"} .= 'S' if $flags =~ /read/; # Seen or Read.
$info{"flags"} .= 'T' if $flags =~ /deleted/; # Tagged for
deletion.
return %info;
}
sub convert_dir {
my $src_dir = shift @_;
my $dst_dir = shift @_;
my $msg_counter = 0;
my $no_sylpheed_mark;
my $dir_name;
my $folder;
$dir_name = convert_dirname($src_dir);
$dst_dir .= "$dir_name"; #RM: changed from ".$dir_name"
maildirmake ($dst_dir);
$no_sylpheed_mark = (!-f "$src_dir/.sylpheed_mark" || !$use_sylpheed);
$folder = read_sylpheed_mark($src_dir) if (!$no_sylpheed_mark);
$no_sylpheed_mark = 1 if ($no_sylpheed_mark || keys (%$folder) == 0);
opendir DIR, "$src_dir" || die "$!: cannot open $src_dir\n";
foreach (readdir DIR) {
print "DEBUG1: print now working on $src_dir file $_\n" if $DEBUG >
1;
if (-d "$src_dir/$_" && $_ ne "." && $_ ne
"..") {
convert_dir("$src_dir/$_", $dst_dir);
} elsif ( $_ =~ /^,{0,1}\d+$/ && -f "$src_dir/$_" ) {
my $file;
my $messagefn;
my $info;
$file = $_;
if ($no_sylpheed_mark) {
$info = find_all ($src_dir, $file);
} else {
$info = $folder->{$file};
$info->{"time"} = find_time ("$src_dir/$file");
}
if ($info->{"new"}) {
$messagefn = "new/";
} else {
$messagefn = "cur/";
}
# See http://www.courier-mta.org/maildir.html for the format of the
# file name.
$messagefn .= sprintf ("%d.%d_%d.%s:2,%s",
$info->{"time"}, $$,
$info->{"msgcount"},
$info->{"host"}, $info->{"flags"});
copy "$src_dir/$file", "$dst_dir/$messagefn";
chmod 0600, "$dst_dir/$messagefn";
utime $info->{"time"}, $info->{"time"},
"$dst_dir/$messagefn";
$msg_counter++;
print "DEBUG: wrote $dst_dir/$messagefn\n" if $DEBUG > 0;
}
}
closedir DIR;
print "INFO: messages in ${dst_dir}: $msg_counter\n";
}
sub read_sylpheed_mark
{
my $src_dir = shift @_;
my %folder = ();
my $version;
my $version_int;
my $message;
my $message_int;
my $flags;
my $flags_int;
# Useful stuff defined in sylpheed source code:
#
# From: defs.h:
##define MARK_FILE ".sylpheed_mark"
##define MARK_VERSION 2
#
# From procmsg.h:
##define MSG_NEW (1U << 0)
##define MSG_UNREAD (1U << 1)
##define MSG_MARKED (1U << 2)
##define MSG_DELETED (1U << 3)
##define MSG_REPLIED (1U << 4)
##define MSG_FORWARDED (1U << 5)
#
# The mark file is organized like this:
# * one 4 bytes integer containing the version number
# * then a loop:
# * 4 bytes integer for the message number
# * 4 bytes integer for the flags of the message
#
open (MARK, "$src_dir/.sylpheed_mark") || return undef;
binmode MARK;
# read mark version number. We understand version 2.
if (read (MARK, $version, 4) != 4) {
print "$src_dir/.sylpheed_mark: corrupted .sylpheed_mark: no version
number.\n";
return ();
}
$version_int = unpack ('I', $version);
if ($version_int != 2) {
print "$src_dir/.sylpheed_mark: .sylpheed_mark in a version we don't
understand.\n";
return ();
}
while (1) {
if (read (MARK, $message, 4) != 4) {
return \%folder;
}
$message_int = unpack ('I', $message);
if (read (MARK, $flags, 4) != 4) {
print "$src_dir/.sylpheed_mark: corrupted .sylpheed_mark: there are no
flags for message $message\n";
return \%folder;
}
$flags_int = unpack ('I', $flags);
# How to write the info for maildir files, from
# http://www.courier-mta.org/maildir.html:
# Here, info represents the state of the message, and it consists of zero
# or more boolean flags chosen from the following: "D" - this is a
'draft'
# message, "R" - this message has been replied to, "S" - this
message has
# been viewed (seen), "T" - this message has been marked to be deleted
# (trashed), but is not yet removed (messages are removed from maildirs
# simply by deleting their file), "F" - this message has been marked by the
# user, for some purpose. These flags must be stored in alphabetical order.
# New messages contain only the :2, suffix, with no flags, indicating that
# the messages were not seen, replied, marked, or deleted.
$folder{$message_int}{"msgcount"} = $message_int;
$folder{$message_int}{"host"} = hostname;
if ($flags_int & 1) {
$folder{$message_int}{"new"} = 1;
} else {
$folder{$message_int}{"new"} = 0;
}
if (!$folder{$message_int}{"new"} == 1) {
$folder{$message_int}{"flags"} = "";
$folder{$message_int}{"flags"} .= "F" if ($flags_int &
4); # MSG_MARKED
$folder{$message_int}{"flags"} .= "R" if ($flags_int &
16); # MSG_REPLIED
$folder{$message_int}{"flags"} .= "S" if (!($flags_int
& 2)); # MSG_UNREAD
$folder{$message_int}{"flags"} .= "T" if ($flags_int &
8); # MSG_DELETED
}
}
}
# The maildirmake function
# ------------------------
#
# It does the same thing that the maildirmake binary that
# comes with courier-imap distribution
#
sub maildirmake {
foreach(@_) {
-d $_ or mkdir $_,0700 or die("Fatal: Directory $_ doesn't exist and
can't be created.\n");
-d "$_/tmp" or mkdir("$_/tmp",0700) or die("Fatal: Unable
to make $_/tmp/ subdirectory.\n");
-d "$_/new" or mkdir("$_/new",0700) or die("Fatal: Unable
to make $_/new/ subdirectory.\n");
-d "$_/cur" or mkdir("$_/cur",0700) or die("Fatal: Unable
to make $_/cur/ subdirectory.\n");
}
}
sub usage {
print "usage: $0 SourceDirectory/ [DestinationRoot/]\n";
exit 1;
}
1287
days inactive
1294
days old
62 comments
8 participants
participants (8)
-
Cameron Simpson -
Jon LaBadie -
Patrick O'Callaghan -
Ranjan Maitra -
Samuel Sieb -
Tim -
Todd Zullinger -
Tom Horsley