New backup,failed error 23

[fstuurman]

Hello,

After running the new backup I got the following error:
Backup job for on local failed
Rsync returned error 23: Partial transfer due to error

First I was at loss where to find more information about this. Is there a 'backup log'?

Then I noticed several entries in the daemon.log, like this:
Sep 17 12:42:57 Gondor ntfs-3g[15129]: Could not convert filename to Unicode: 'Château de Sceaux - Instructions.mht': Invalid or incomplete multibyte or wide character

It appears, files with names containing special characters are not backed up.

Any suggestions how to proceed?

[johannes]

Hmm, you backup to an NTFS file system? Did you install NTFS-3g manually? Looks to be something related to that, and this might be the reason our testing didn't see this error. If this is the case, formatting to ext3 (you can use the format function in the B3, admin -> disk) should help. Could this be it?

[fstuurman]

Yes, I might have installed NTFS-3G manually, although I don't remember.

The USB disk came out of the box preformatted with NTFS. I also use the disk as backup disk for my Windows desktop, so would prefer it not to reformat.

Does your suggestion to reformat imply that the new backup does not support NTFS?

Some googling around I find suggestions to use a locale=en_US.utf8 option with the mount command. I don't know if I should attempt setting this option somewhere. I did read I could specify this option in /etc/fstab. However I do not know how to specify this without interfering with the new backup.

[johannes]

No, NTFS is not supported generally in B3, although I did expect the backup to work properly with ntfs-3g manually installed. Perhaps you could use FAT32?

[Gordon]

A possible fix could be to add`--iconv=utf-8,utf-16` to the rsync parameters (NTFS natively stores in UTF-16). I verified what you said about using a locale option to the mount parameters, but those suggestions appear to be incorrect. The correct mount option should be `nls=utf8`, but it has to be noted that you cannot mount the drive at boot time when adding this to fstab. This is because the `mount -a` command that is issued at boot time will ignore this option. By setting mount option `noauto` you can however push mounting of that drive to rc.local, but being a backup device you'll probably only want it mounted during backup times anyway.

Code: Select all

/dev/sdb1       /mnt/usbdisk    ntfs-3g         noauto,nls=utf8      0 0

It should also be noted that when backing up to a non-linux file system, rsync cannot preserve file rights and you'll need some external method to store (and restore) those rights. IMO you should rethink not wanting to repartition/reformat the drive.

PS Finding out the hard way, if you do intend to automount an external drive at boot time you should add the `nofail` option to fstab. If you don't and the drive is unavailable or damaged the system will not boot (actually it will ask you to press a key, but that's a bit difficult on a headless system).

[fstuurman]

Thanks to Gordon, his solution works, as far as I can test it.
- created the fstab entry
- copied file tëst.txt with cp. This preserves the file name.
- mounted the disk using the B3 network interface in the /home/storage folder & used my Windows computer and Samba network access. file still there.
- used rsync to copy tëst.txt. No error messages. File is readable with correct name using Windows via network.
- unmounted disk & attached the disk to my windows computer. Files still there.

So this looks very promising. Don't know how to manually start the new backup so could not yet test that. I have set the schedule to run it tonight, will post results.

[fstuurman]

Still a Backup Failed.

Viewing the log, the mount command in the new backup does not pick up the fstab entry. I tried altering fstab using UID=... My mount command succeeds, but the backup run last night didn't pick up that entry either.

Summarizing my options
1 ignore the error. I grepped the offending file names. They are not very important if they are lost. I could do a manual backup and wait if bubba development decides to do something about it.
2 reformat to ext3. I found a windows driver called ext2fsd which might be able to mount the disk in WIndows. Unsure of the reliability, I read some warnings. I do not like the idea that the driver might corrupt my backup disk. Alternatively i could buy a second usb disk and dedicate it to backup only.
3 proceed hacking. Need to reverse engineer the new backup to find out how the mount is done. Can anyone point me to the script?

Thanks everyone for your suggestions. If i do not post anything again, assume i chose option 1.

[Gordon]

Figures... The backup procedure will ignore fstab entries, because it wants the backup drive to be mounted somewhere special. Additional mount options will need to be included in the backup script therefore.

Edit: you'll need to hack the backup script /usr/sbin/bubba-run-backupjobs

At line 131 it states:

Code: Select all

        subprocess.call(['mount', dev.strip(), mountdir])

You'll need to change this to:

Code: Select all

        subprocess.call(['mount', '-o', 'nls=utf8', dev.strip(), mountdir])

Alternatively you could make it honour fstab by either removing 'dev.strip()' or 'mountdir' from that line. This will require some additional hacking though, because mountdir as specified here is not a fixed path but a temporary location created uniquely every time and dropped afterwards.