Dovecot corrupted indexes

[RandomUsername]

Posting this here rather than in the RC bug thread as I don't know if it's a problem for anyone else yet.

When migrating from the B2 to B3 I copied all users ~/Mail directories over and since then I've had loads of errors in /var/log/mail.log about corrupted indexes and the like e.g:

Code: Select all

Jul 14 06:15:45 serverb3 dovecot: IMAP(darren): /home/darren/Mail/dovecot.index  log position went backwards (5,25328 < 5,29860)                                  Jul 14 06:15:46 serverb3 dovecot: IMAP(darren): fscking index file /home/darren/ Mail/dovecot.index                                                               Jul 14 06:15:46 serverb3 dovecot: IMAP(darren): fscking index file /home/darren/ Mail/dovecot.index                                                               Jul 14 06:20:45 serverb3 dovecot: IMAP(darren): Transaction log file /home/darre n/Mail/dovecot.index.log: marked corrupted                                       Jul 14 08:59:45 serverb3 dovecot: IMAP(darren): Log synchronization error at seq =8,offset=140 for /home/darren/Mail/.Sent/dovecot.index: Append with UID 1192344 775, but next_uid = 3981498798                                                   Jul 14 08:59:45 serverb3 dovecot: IMAP(darren): Log synchronization error at seq =8,offset=17888 for /home/darren/Mail/.Sent/dovecot.index: Append with UID 2208,  but next_uid = 3981498798                                                       Jul 14 08:59:45 serverb3 dovecot: IMAP(darren): Log synchronization error at seq =8,offset=18192 for /home/darren/Mail/.Sent/dovecot.index: Append with UID 2209,  but next_uid = 3981498798                                                       Jul 14 08:59:45 serverb3 dovecot: IMAP(darren): fscking index file /home/darren/ Mail/.Sent/dovecot.index

A lot of them seem to get fixed automatically but a couple of times it's appeared to my mail clients that I've suddenly got no emails in a particular folder and I've had to delete the relevant index files so they get rebuilt.

Googling around suggests that this might be caused by multiple clients accessing the mailbox simultaneously (and this situation fits - at any one time I've got four or more separate devices polling my mailbox). A suggested fix seems to be to add

Code: Select all

mmap_disable=yes 

to dovecot.conf even though this is really only meant to be for clusters and distributed file systems.

These errors haven't cropped up since I added this setting but I'm not ready to say it was definitely successful yet. Anyone else getting this? I will consider asking Excito to add this setting to the B3's default dovecot.conf if it does appear to work.

Thanks.

Darren.

[Ubi]

you have multiple devices DIRECTLY accessing your mailbox? Or you have multiple devices connecting to your dovecot server onto the same mail account? The distinction is quite important. Also is your mailbox on an NFS mount?

[RandomUsername]

Ha, no. I meant multiple devices (i.e. email clients) connecting to the server via IMAP.

It's not on an NFS mount, but it's just occurred to me that it is on an encrypted LVM but it was on my B2 as well. Never had this problem on the B2 (don't know if I got the errors in mail.log - I don't think so) but certainly my mail never became invisible to my mail clients.

[Ubi]

I've been running dovecot for over 8 years with 100+ mailboxes and I've never seen this issue. My knee-jerk reaction would be to point to the encryption part, which I have no experience with

[RandomUsername]

I've been running dovecot for over 8 years with 100+ mailboxes and I've never seen this issue. My knee-jerk reaction would be to point to the encryption part, which I have no experience with

OK, thanks. I haven't formatted my old B2 yet, I might hook up the disk and see if these errors were present in the mail.log on that machine.

Any possibility this is to do with the migration from B2 to B3? I can't really see how though.

Darren.

[Ubi]

dovecot doesn't care if you upgrade. It should not corrupt the indexes

[RandomUsername]

Two thinks I've checked:

1. These errors never showed up on my B2.

2. I enabled the mmap setting again as a test and so far the error hasn't reoccurred so it might have been a red herring.

Hmm...