> > > A UDF filesystem image is supposed to bear at its start 32 KiB of zeros. > > B.M. wrote: > > This is indeed the case: > > [...] > > For a readable disk, this look like you said: Only zeros. > > So it looks like at least a part of the problem is decryption.
Agreed > > > If UDF does not work even unencrypted, > > > > Why should UDF not work correctly without encryption? > > It's improbable, i confess. > But for now we are hunting an unexplainable problem. So we have to divide > the situation in order to narrow the set of suspects. > > Verifying that your procdure with two UDF images is not the culprit would > help even if the result is boringly ok, as we expect. (Or we are in for > a surprise ...) I don't have two UDF images. In my script I create a file, put an encrypted UDF filesystem into it and start writing compressed files into it. Unfortunately it can happen (and happened in the past) that the filesystem got filled up completely. Beside that, I use a fully encrypted system with several partitions... Extract from df -h: Filesystem Size Used Avail Use% Mounted on /dev/mapper/sdb2_crypt 28G 23G 3.0G 89% / /dev/sdb1 447M 202M 221M 48% /boot /dev/mapper/var_crypt 27G 18G 8.4G 68% /var /dev/mapper/vraid1-home 1.8T 1.5T 251G 86% /home /dev/mapper/BDbackup 6.5M 6.5M 2.0K 100% /mnt/BDbackup (I create the image file as /home/TMP_BKP/backup.img just because that's where I have enough available space.) > After the boring outcome you have the unencrypted images to make the next > step, namely to create /dev/mapper/BDbackup with a new empty image file > as base, to copy the images into it (e.g. by dd), and to close it. > Then try whether the two encrypted image files can be properly openend > as /dev/mapper/BDbackup ans show mountable UDF filesystems. > > > it's not only the burned disc which is not readable/mountable, it's > > also the image I created before burning. > > So we can exclude growisofs as culprit. > > > Might it be possible, that when my UDF filesystem gets filled completely, > > the encryption get damaged? > > That would be a bad bug in the device-mapper code and also such a mishap > is hard to imagine. The UDF driver is supposed not to write outside its > filesystem data range. That range would be at most as large as the payload > of the device mapping. Doesn't look like that - I tried the following several times: Create (a much smaller) image file, put an encrypted filesystem in it, fill it completely with either cp or dd, unmount it, close and re-open with cryptsetup, than check /dev/mapper/BDbackup: no problems, only hex zeros and it's mountable. > > Multi-disc backups are not > > handled by my script, I have to intervene manually. > > That's always a potential source of problems. > Do i get it right, that your script copies files into the mounted UDF > and gets a "filesystem full" error ? > > What exactly are you doing next ? > (From where to where are you moving the surplus files ? > Does the first /dev/mapper device stay open while you create the encrypted > device for the second UDF filesystem ? Anything i don't think of ... ?) If you want you can have a look at my script, I attached it to this mail... Basically, I use extended attributes (user.xdg.tags) to manage which folders have to get backuped, write the last backup date into user.xdg.comment. By comparing file timestamps with these backup dates this allows for incremental backups. Then for each folder which should be backuped, I use tar and plzip, writing into BKPDIR="/mnt/BDbackup". "Filesystem full" is not handled at all. Typically if this happens it's quite late i.e. most folders are already backuped and I do the following: - remove the last lz-file, I never checked if it is corrupted - burn the image - reset user.xdg.comment for not yet backuped folders manually - execute the script again, burn the so created second image Since this is quite ugly, I try to prevent it by moving very large lz-files from /mnt/BDbackup to a temporary location outside of /mnt/BDbackup while the script is running. When the "create lz-files"-part of my script has finished, I check if there is sufficient space to move the large files back to /mnt/ BDbackup. If yes I do this, if not I leave them outside, burn the first disc, then I create a second image manually, put the large files into the empty filesystem, burn this disc as well. Not perfect at all, I know, but it's working... and I do this about every 3 or 6 months. Beside that, it's just a second kind of backup additionally to bi-weekly backups on external, also encrypted HDDs. (I think with these two kind of backups I'm doing enough to save our precious personal files, images, videos etc., doing much more than most people out there ;-) Honestly I don't see where this process may corrupt the UDF fs or the encryption. And I don't see an error / bug in my script neither. > > Or is my filesystem too large? > > 25 "GB" would rather be too small to swim in the swarm of other cryptsetup > users. > > > ----------------------------------------------------------------------- > > Slightly off topic: A riddle about your UDF image sizes: > > # There is an old comment in my script at this line, saying: > > # let's try that: 24064000K > > # 24438784K according to dvd+rw-mediainfo but creates at > > # least sometimes INVALID ADDRESS FOR WRITE; > > # alternative according to internet research: 23500M > > An unformatted single layer BD-R has 12,219,392 blocks = 23866 MiB = > 24,438,784 KiB. > But growisofs formats it by default to 11,826,176 = 23098 MiB = > 23,652,352 KiB. Thanks for pointing this out, I didn't know that growisofs gives away a few bytes... Do you know why that's the case? > growisofs_mmc.cpp emits a message in function bd_r_format() > fprintf (stderr,"%s: pre-formatting blank BD-R for %.1fGB...\n", > > ioctl_device,(f[0]<<24|f[1]<<16|f[2]<<8|f[3])*2048.0/1e9); Watch your > growisofs run for it. Never noticed this error message though, see my third to last paragraph below. > (Note that it talks of merchant's GB = 1 billion, not of programmer's > GiB = 107.3741,824. 23098 MiB = 24.220008448 GB) > > > IMGSIZE=24064000K > > truncate -s $IMGSIZE $IMGFILE > > The man page of truncate says that it's "K" are 1024, i.e KiB. > So your image has 23500 MiB which is too large for the default format > as normally applied to BD-R by growisofs. > > growisofs has a bug to accept burn jobs which fit into unformatted BD-R > but then to spoil them by applying its default format: > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=699186 > > So how come that your growisofs run does not fail in the end ? > > There is an undocumented growisofs option to suppress BD-R formatting: > -use-the-force-luke=spare=none > There is also > -use-the-force-luke=spare=min > which (i guess) will bring 23610 MiB of payload. I didn't use these options. With all that tried and learned, I'm going to try another full run of my script, closely monitoring what's happening during the different steps. Not today but when I have enough time, maybe in a week or even later. > > (I take the occasion to point out that xorriso does not format BD-R > by default. I.e. default capacity is 23866 MiB.) Never heard about xorriso before - from my understanding I could use it instead of growisofs, but with larger images? General question: Do you think I should completely change my script such that it creates lz- files, encrypts each of them and then writes them on an unencrypted disc? Thank you very much. Best, Bernd
encrypted_backup.sh
Description: application/shellscript