Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Soo, here is an output of sql request: % sqlite3 rep-cache.db '.header on' 'SELECT * FROM rep_cache WHERE revision = 7449' hash|revision|offset|size|expanded_size a684c1201230ed000e8baf11fcd890efebb059db|7449|3|106064003|111204465 And here is 7449 file size % ls -l revs/7/7449 -r--r--r--. 1 apache apache 106067461 Feb 4 2021 revs/7/7449 Now, what about "links" to 7449 in revisions after 7449. There is something in 7450: % strings revs/7/7450 | tail -7 copyroot: 0 / minfo-cnt: 61 _7.0.t7449-67p add-dir false false false /trunk/ProjectB/Art/Characters/Survivors/01/textures/substance_render _9.0.t7449-67p add-file true true false /trunk/ProjectB/Art/Characters/Survivors/01/textures/substance_render/01_render.spp L2P-INDEX P2L-INDEX 51995736 59c66b6d95365e6bdb4be4ec3b2d3a34 51995799 72059006b7c456b03efb7f07e0557795S % strings revs/7/7450 | grep 7449 text: 7450 3 51992815 54791207 b326aa3b7fd0ea02b8e75ac8a8dcc656 1430895ca8250cfb117997d6ee543e7e2c06c265 7449-67p/_b props: 2 757 65 53 113136892f2137aa0116093a524ade0b - 7449-67p/_d DELTA 7449 11 138 pred: 4-7052.0.r7449/12 pred: 3-6161.0.r7449/14 DELTA 7449 15 24 pred: 2-6160.0.r7449/16 DELTA 7449 17 20 pred: 1-6132.0.r7449/18 DELTA 7449 19 25 pred: 3-232.0.r7449/20 DELTA 7449 21 25 pred: 0.0.r7449/2 _7.0.t7449-67p add-dir false false false /trunk/ProjectB/Art/Characters/Survivors/01/textures/substance_render _9.0.t7449-67p add-file true true false /trunk/ProjectB/Art/Characters/Survivors/01/textures/substance_render/01_render.spp And there is no "links" to 7449 in 7451 revision and after it, BUT I still can't dump these revisions. Maybe because of "chain" of "links". Like 7449 <- 7450 <- 7451 etc.? % svnadmin dump /var/repo_serpico -r7450 > ~/sdb/test.dump svnadmin: E160004: Corrupt representation '7449 21 25 159 24ad3bd9d7945c1c7ca3f5e714ea868e - -' svnadmin: E160004: Invalid r7449 footer % svnadmin dump /var/repo_serpico -r7451 > ~/sdb/test.dump svnadmin: E160004: Corrupt representation '7449 21 25 159 8f3d18747d3388ff2b35096cafbd57ab - -' svnadmin: E160004: Invalid r7449 footer -- Dmitry Minsky > On 28.06.2022, at 15:50, Daniel Shahaf wrote: > > Dmitry Minsky wrote on Tue, 28 Jun 2022 13:18 +00:00: >>> What does the "folder with files" contain? >> >> Just a random files on my computer ;) It’s not from working copy or >> repository or anything else meaningful. Let’s assume that it’s just a >> bunch of random files which I want to put in the middle of repo and >> hope that it won’t blow up ;) Is that possible? > > With enough effort, yes. > > Devs: In attempting to recreate db/revs/7/7449, what needs to be > matched? Off the top of my head, it's rep-cache.db references, actual > rep-sharing references in future rev files, and possibly node-rev id's. > Anything else? > > What's the output of «sqlite3 rep-cache.db '.header on' 'SELECT * FROM > rep_cache WHERE revision = 7449'»? > > Does any rev file after 7449 contain " 7449 " on a "text:" or > "props:" line? > > Does any rev file after 7449 contain ".7449/"? > > Daniel > >>> On 28.06.2022, at 15:14, Daniel Shahaf wrote: >>> >>> Dmitry Minsky wrote on Tue, 28 Jun 2022 11:01 +00:00: >>>> Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there >>>> aren’t any signs of any text readable data at the bottom of the file >>>> and the top of file looks similar to 7448, 7450 and to any other >>>> revision. >>>> >>>> So, let’s say I’m 85.23% sure about content of this particular >>>> revision. How can I recreate revision from folder with files? This rev >>>> contains only add-dir and add-file changes. >>> >>> What does the "folder with files" contain? >>> >>> Is it a working copy? A repository? An export? None of the above? >>> >>> Does it contain exactly the files and directories added in r7449 *as >>> they were in that revision*, and nothing else?
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
> What does the "folder with files" contain? Just a random files on my computer ;) It’s not from working copy or repository or anything else meaningful. Let’s assume that it’s just a bunch of random files which I want to put in the middle of repo and hope that it won’t blow up ;) Is that possible? -- Dmitry Minsky > On 28.06.2022, at 15:14, Daniel Shahaf wrote: > > Dmitry Minsky wrote on Tue, 28 Jun 2022 11:01 +00:00: >> Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there >> aren’t any signs of any text readable data at the bottom of the file >> and the top of file looks similar to 7448, 7450 and to any other >> revision. >> >> So, let’s say I’m 85.23% sure about content of this particular >> revision. How can I recreate revision from folder with files? This rev >> contains only add-dir and add-file changes. > > What does the "folder with files" contain? > > Is it a working copy? A repository? An export? None of the above? > > Does it contain exactly the files and directories added in r7449 *as > they were in that revision*, and nothing else?
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there aren’t any signs of any text readable data at the bottom of the file and the top of file looks similar to 7448, 7450 and to any other revision. So, let’s say I’m 85.23% sure about content of this particular revision. How can I recreate revision from folder with files? This rev contains only add-dir and add-file changes. -- Dmitry Minsky > On 21.06.2022, at 09:54, Daniel Shahaf wrote: > > Dmitry Minsky wrote on Sun, 19 Jun 2022 11:53 +00:00: >> There is something in revisions before and after corrupted one: >> >> % strings repo/db/revs/7/7448 | tail -1 >> 2244140591 fa0c1a8229575b0ce27ef0c5a8b898b4 2244140730 >> 7d861109493094a15013c7ea105e33a1W >> >> % strings repo/db/revs/7/7450 | tail -1 >> 51995736 59c66b6d95365e6bdb4be4ec3b2d3a34 51995799 >> 72059006b7c456b03efb7f07e0557795S >> >> But on corrupted (actual number of corrupted revision is 7449) it doesn't >> give anything meaningful >> >> % strings repo/db/revs/7/7449 | tail -1 >> Y$Q8 >> > > OK, so the last line isn't there at all. > > What you now need to do is figure out what happened to the revision > file. For starters, check whether the change-list portion (a list of > lines with "add-file" "modify-dir" and such on them, immediately > before «L2P-INDEX») is still there. > > - If that's the case, the file was likely truncated, but the lost > portion of the file can likely be reconstructed manually, though > perhaps with some effort (what's the size of db/revs/7/7449 in bytes > and the number of lines `svn log -qvr 7449`'s output should have?). > > - If it got truncated earlier, then it's likely that actual deltas > (PLAIN..ENDREP or DELTA..ENDREP) got truncated, i.e., data loss has > occurred, so you'd have to reconstruct the revision manually or > proceed without the lost data. > > - And if it's not merely truncated but otherwise corrupted, well, > it depends. > > Do you have any way to reconstruct the revision or its contents? Older > backups, mirrors, commit mails, a working copy that committed this > revision or updated to it and never got 'svn cleanup' run on it? > (The thinking being to exploit issue #4071; cf. > <https://subversion.apache.org/docs/release-notes/1.7#wc-pristines > <https://subversion.apache.org/docs/release-notes/1.7#wc-pristines>>.) > > Reminder that revision files contain binary data and should not be > opened by editors that auto-fix whitespace and so on. And that > repositories should be backed up before being manually operated on. > >> % < repo/db/revs/7/7449 xxd -s Y$Q8 -l 9 >> : 4445 4c54 410a 5356 4e DELTA.SVN >> > > That's likely because xxd(1) calls atoi(3) on its argv[2], which > (assuming the shell parameter «Q8» is unset) has the value «Y», so the > function call returns 0. > > The output appears sane, though. These bytes are exactly what the > start of a revision file might look like. > > Cheers, > > Daniel > > P.S. For any future FSFS hackers out there, note that it's likely r7449 > added a file or a directory. (Why? Because gung bhgchg fubjf > n /frys-pbzcerffrq/ qrygn.) > >> On Sun, Jun 19, 2022 at 12:19 PM Daniel Shahaf >> wrote: >> >>> Dmitry Minsky wrote on Sat, 18 Jun 2022 17:16 +00:00: >>>> I have a pretty old repository and now going to move it to another >>> machine. >>>> When I start the dump process I stumbled upon this error in one of the >>> old >>>> revisions: >>>> >>>> svnadmin: E16004: Invalid r4422 footer >>>> >>> >>> It's actually E160004. (Just saying this so search engines will find this >>> thread.) >>> >>>> I now workaround with a skip over corrupted revision and continue >>>> --incremental dump. But like every 10-50 revisions I still get this error >>>> because I suppose there are files that depend on something from this old >>>> corrupted revision. The question is, how and where to look in this >>> revision >>>> so I could manually fix the error by changing some files or checksum or >>>> anything? >>> >>> At the very end of the file, like the following but on db/revs/4/4422: >>> >>> % strings db/revs/0/1 | tail -1 >>> 417 b4657c89ff644471b6760fd6389d253c 445 ea755737e485eeb03c0012e5d6bc1b49I >>> % < r/db/revs/0/1 xxd -s 417 -l 9 >>> 01a1: 4c32 502d 494e 4445 58 L2P-INDEX >>> % < r/db/revs/0/1 xxd -s 445 -l 9 >>> 01bd: 5032 4c2d 494e 4445 58 P2L-INDEX >>> % >>> >>> (The first command might pick up some of the P2L/L2P data too? I don't >>> remember whether it's guaranteed that there'll be a non-printable >>> character between that and the last line.)
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
There is something in revisions before and after corrupted one: % strings repo/db/revs/7/7448 | tail -1 2244140591 fa0c1a8229575b0ce27ef0c5a8b898b4 2244140730 7d861109493094a15013c7ea105e33a1W % strings repo/db/revs/7/7450 | tail -1 51995736 59c66b6d95365e6bdb4be4ec3b2d3a34 51995799 72059006b7c456b03efb7f07e0557795S But on corrupted (actual number of corrupted revision is 7449) it doesn't give anything meaningful % strings repo/db/revs/7/7449 | tail -1 Y$Q8 % < repo/db/revs/7/7449 xxd -s Y$Q8 -l 9 : 4445 4c54 410a 5356 4e DELTA.SVN On Sun, Jun 19, 2022 at 12:19 PM Daniel Shahaf wrote: > Dmitry Minsky wrote on Sat, 18 Jun 2022 17:16 +00:00: > > I have a pretty old repository and now going to move it to another > machine. > > When I start the dump process I stumbled upon this error in one of the > old > > revisions: > > > > svnadmin: E16004: Invalid r4422 footer > > > > It's actually E160004. (Just saying this so search engines will find this > thread.) > > > I now workaround with a skip over corrupted revision and continue > > --incremental dump. But like every 10-50 revisions I still get this error > > because I suppose there are files that depend on something from this old > > corrupted revision. The question is, how and where to look in this > revision > > so I could manually fix the error by changing some files or checksum or > > anything? > > At the very end of the file, like the following but on db/revs/4/4422: > > % strings db/revs/0/1 | tail -1 > 417 b4657c89ff644471b6760fd6389d253c 445 ea755737e485eeb03c0012e5d6bc1b49I > % < r/db/revs/0/1 xxd -s 417 -l 9 > 01a1: 4c32 502d 494e 4445 58 L2P-INDEX > % < r/db/revs/0/1 xxd -s 445 -l 9 > 01bd: 5032 4c2d 494e 4445 58 P2L-INDEX > % > > (The first command might pick up some of the P2L/L2P data too? I don't > remember whether it's guaranteed that there'll be a non-printable > character between that and the last line.) > >
svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
I have a pretty old repository and now going to move it to another machine. When I start the dump process I stumbled upon this error in one of the old revisions: svnadmin: E16004: Invalid r4422 footer I now workaround with a skip over corrupted revision and continue --incremental dump. But like every 10-50 revisions I still get this error because I suppose there are files that depend on something from this old corrupted revision. The question is, how and where to look in this revision so I could manually fix the error by changing some files or checksum or anything?
