Re: Decompression of svndiff data failed
Your history is toast. Stop burning cycles. Lock your working server, Take clean working copies of the important branches if you can get them, the clean working copies, import them to a new server, and start with a much lighter working repository. And *stop* putting binaries in the same repository as source code. History is usually much more important for source, binaries harshly burden all source control systems, and even the smallest errors corrupt them irretrievably. If you *must* store binaries, be sure to commit them in entirely distinct commits from the source code, to avoid precisely this problem, Sent from my iPhone
Re: Decompression of svndiff data failed
The decompression errors only seem to happen when we're sending binary data. For a couple of years our marketing team were storing all of their files in subversion and this seems to be the vast majority of the revisions I'm having to fix with fsfsverify.py. So it could possibly be that they were using a TortoiseSVN version that was built with a buggy library. That said the problematic revision that this thread is about was one created by our engineering team and all of our work is done from linux machines so would have been a different client (cli subversion client). So it sounds more likely that it's something server side. We do still get the issue periodically, so there's a chance it could be related to the NAS that the repo is stored on. We did recently move to a new NAS as the old one was getting a bit slow but we hadn't seen any corruption issues with the old NAS, just slowness. I'm not sure if we've had any svndiff issues since we've moved to the new NAS, I'll find out when I've finally got to the point where it's syncing commits from the last month. Also the issue would happen straight away. i.e. if someone tried to do an svn up immediately after a commit had been made they would get the svndiff error. So it seems like it was an error that occurred when the svnserve process received it or wrote it to the file system. So it's not hard disk corruption after the fact (but again doesn't rule out a NAS issue). Sorry for not being able to provide more specific details. If I see it happening again once we've moved over to a modern OS and Subversion server I'll email back again. Mike On Fri, 30 Aug 2019 at 14:22, Branko Čibej wrote: > On 30.08.2019 15:14, Michael Ditum wrote: > > Hi Brane, > > > > Thanks for the reply. Interestingly Daniel's reply had given me the > > idea to try pretty much what you suggested and I gave it a go this > > morning and it seems to be working. > > > > Stopping svnsync in the right place wasn't hard as i dies as soon as > > it tried to get the binary diff but before it's made any changes. > > > > The one bit I didn't do was update the svnsync metadata. When I > > resumed the sync it just automatically carried on with the revision > > after the one I had just committed. Hopefully that won't cause any > > problems, it seems to be working ok as I'm a lot further syncing than > > I've ever managed before (crazy how many times we've had svndiff > > errors, luckily fsfsverify has fixed all of the others so far). > > > > Thanks for everyone's help! > > Great that it works. :) > > I'm curious though ... have you any idea what caused the decompression > errors? The message you posted came from zlib -- not Subversion's code > -- and that has been very, very stable literally for decades. > > Is it possible that you just had the bad luck to have a broken version > of zlib, way back in the dawn of time? If it had been a problem with the > storage, I'm pretty sure you'd have noticed other issues, too. > > -- Brane > >
Re: Decompression of svndiff data failed
On 30.08.2019 15:14, Michael Ditum wrote: > Hi Brane, > > Thanks for the reply. Interestingly Daniel's reply had given me the > idea to try pretty much what you suggested and I gave it a go this > morning and it seems to be working. > > Stopping svnsync in the right place wasn't hard as i dies as soon as > it tried to get the binary diff but before it's made any changes. > > The one bit I didn't do was update the svnsync metadata. When I > resumed the sync it just automatically carried on with the revision > after the one I had just committed. Hopefully that won't cause any > problems, it seems to be working ok as I'm a lot further syncing than > I've ever managed before (crazy how many times we've had svndiff > errors, luckily fsfsverify has fixed all of the others so far). > > Thanks for everyone's help! Great that it works. :) I'm curious though ... have you any idea what caused the decompression errors? The message you posted came from zlib -- not Subversion's code -- and that has been very, very stable literally for decades. Is it possible that you just had the bad luck to have a broken version of zlib, way back in the dawn of time? If it had been a problem with the storage, I'm pretty sure you'd have noticed other issues, too. -- Brane
Re: Decompression of svndiff data failed
Hi Brane, Thanks for the reply. Interestingly Daniel's reply had given me the idea to try pretty much what you suggested and I gave it a go this morning and it seems to be working. Stopping svnsync in the right place wasn't hard as i dies as soon as it tried to get the binary diff but before it's made any changes. The one bit I didn't do was update the svnsync metadata. When I resumed the sync it just automatically carried on with the revision after the one I had just committed. Hopefully that won't cause any problems, it seems to be working ok as I'm a lot further syncing than I've ever managed before (crazy how many times we've had svndiff errors, luckily fsfsverify has fixed all of the others so far). Thanks for everyone's help! Mike On Fri, 30 Aug 2019 at 12:12, Branko Čibej wrote: > On 29.08.2019 20:49, Michael Ditum wrote: > > Apart from using fsfsverify I also tried recreating the diff by > > creating a Fedora 7 VM, running svnsync on it to copy the repo up to > > that point and then manually committing the file and copying the > > revision over to the copy of my original repo. > > Yikes. No, that definitely won't work. > > > Whilst this allows svnsync to get past that revision I then started > > having lots of problems with incorrect byte offsets in later revisions > > and once I (think I correctly) fixed started getting checksum errors. > > And that's why ... binary deltas rely on previously stored data, but > unlike a text diff they have no context. You changed the source of the > delta and that corrupted everything that depends on it in later revisions. > > > > Does anyone have any ideas on how I can fix this revision? As I > > mentioned before, the file gets deleted a couple of revisions later so > > I don't really care about the contents of the revision but I'm > > currently stuck and can't get any further in my svnsync. > > Daniel made the best suggestion, it would work like this: > > * create a new repository > * svnsync up to the revision just before the broken one (stopping > svnsync is the tricky part here) > * commit that one file to the _synced_ repository, and update > svnsync's metadata (in revision properties on r0) to skip the > offending revision on the next run > * svnsync to the end. > > You can do a similar trick with svnadmin dump and (incremental) load; > the benefit of this is that there is no "stopping problem," but it will > be much, much slower than svnsync. You _could_ combine both methods, > i.e., initialize your target repo with a partial dump of the source up > to your offending rX, then commit rX to the target repo, then svnsync > from there. > > -- Brane > >
Re: Decompression of svndiff data failed
On 29.08.2019 20:49, Michael Ditum wrote: > Apart from using fsfsverify I also tried recreating the diff by > creating a Fedora 7 VM, running svnsync on it to copy the repo up to > that point and then manually committing the file and copying the > revision over to the copy of my original repo. Yikes. No, that definitely won't work. > Whilst this allows svnsync to get past that revision I then started > having lots of problems with incorrect byte offsets in later revisions > and once I (think I correctly) fixed started getting checksum errors. And that's why ... binary deltas rely on previously stored data, but unlike a text diff they have no context. You changed the source of the delta and that corrupted everything that depends on it in later revisions. > Does anyone have any ideas on how I can fix this revision? As I > mentioned before, the file gets deleted a couple of revisions later so > I don't really care about the contents of the revision but I'm > currently stuck and can't get any further in my svnsync. Daniel made the best suggestion, it would work like this: * create a new repository * svnsync up to the revision just before the broken one (stopping svnsync is the tricky part here) * commit that one file to the _synced_ repository, and update svnsync's metadata (in revision properties on r0) to skip the offending revision on the next run * svnsync to the end. You can do a similar trick with svnadmin dump and (incremental) load; the benefit of this is that there is no "stopping problem," but it will be much, much slower than svnsync. You _could_ combine both methods, i.e., initialize your target repo with a partial dump of the source up to your offending rX, then commit rX to the target repo, then svnsync from there. -- Brane
Re: Decompression of svndiff data failed
Michael Ditum wrote on Thu, 29 Aug 2019 18:49 +00:00: > Does anyone have any ideas on how I can fix this revision? As I > mentioned before, the file gets deleted a couple of revisions later so > I don't really care about the contents of the revision but I'm > currently stuck and can't get any further in my svnsync. Use authz on the source repository to hide that file from the user svnsync authenticates as. If you re-committed that file a few revisions later and want to preserve _that_ copy, then stop svnsync before the revision that re-creates the file. The easiest way to do this is to ^C svnsync, but you can also install a start-commit hook on the destination repository that aborts if HEAD is r1233, where the file was re-committed in r1234. However, there's a catch. If you re-committed the file by doing 'svn rm' and 'svn add' as two separate commits, or under different names, this approach would work; but if you did the 'rm' and 'add' as a _single_ commit, under the same name, authz alone won't help. If that's the case, let us know. Cheers, Daniel P.S. A word of warning: the trick of using start-commit to prevent r1234 from being committed only relies on a non-obvious property of svnsync mirrors: they never have two in-flight commit attempts simultaneously. The trick won't work on repositories whose use pattern doesn't have this property.
Re: Decompression of svndiff data failed
On Thu, Aug 29, 2019 at 4:21 PM Nathan Hartman wrote: > On Thu, Aug 29, 2019 at 3:15 PM Michael Ditum > wrote: > >> Thanks for the response, I hadn't tried it! I've just given that a go and >> unfortunately the dump command failed with... >> >> [mike@tigger svn]$ svnadmin dump svnroot > svnroot.dump >> * Dumped revision 1. >> ...snip... >> * Dumped revision 24950. >> * Dumped revision 24951. >> * Dumped revision 24952. >> svnadmin: E140001: zlib (uncompress): corrupt data: Decompression of >> svndiff data failed >> > > So it fails on the same revision. > > I need to think about this some more. > Here's an idea, but with the caveat that I never tried this myself, so I don't know whether it works or how well. A google search turns up a script called svn2svn. It looks like this might be the original author, last updated 7 years ago: https://github.com/dblock/svn2svn And it looks like this might be a newer version, forked by a different author and with the last updates dated 2016: https://github.com/tonyduckles/svn2svn The idea is to automatically check out each revision from the old repository in sequence and commit it to a new repository, with the added twist of skipping the revisions that fail to check out properly because of the decompression error. Perhaps there's a way to commit a "placeholder" to the new repository in those cases, so that all of your revision numbers will remain identical after migration to the new server. Assuming this works -- again, I've never done this! -- I like this idea because it avoids doing delicate surgery on dumpfiles and things like that, and because you would retain history and not lose information, even if you might lose the revisions that you've never been able to checkout anyway. One issue I see is that the newer "tonyduckles" version says it requires minimum Subversion 1.6. Maybe try to contact the author(s) of svn2svn and ask some questions?
Re: Decompression of svndiff data failed
On Thu, Aug 29, 2019 at 3:15 PM Michael Ditum wrote: > Thanks for the response, I hadn't tried it! I've just given that a go and > unfortunately the dump command failed with... > > [mike@tigger svn]$ svnadmin dump svnroot > svnroot.dump > * Dumped revision 1. > ...snip... > * Dumped revision 24950. > * Dumped revision 24951. > * Dumped revision 24952. > svnadmin: E140001: zlib (uncompress): corrupt data: Decompression of > svndiff data failed > So it fails on the same revision. I need to think about this some more.
Re: Decompression of svndiff data failed
Thanks for the response, I hadn't tried it! I've just given that a go and unfortunately the dump command failed with... [mike@tigger svn]$ svnadmin dump svnroot > svnroot.dump * Dumped revision 1. ...snip... * Dumped revision 24950. * Dumped revision 24951. * Dumped revision 24952. svnadmin: E140001: zlib (uncompress): corrupt data: Decompression of svndiff data failed Mike On Thu, 29 Aug 2019 at 20:04, Nathan Hartman wrote: > On Thu, Aug 29, 2019 at 2:49 PM Michael Ditum > wrote: > >> As we're running a ridiculously old subversion server (1.4.4) on a >> ridiculously old operating system (Fedora 7) I've decided it's time to >> migrate to a new server. >> > > Have you tried to do a dump from 1.4.4 and load on a newer version of > Subversion? If so, did this process succeed? > >
Re: Decompression of svndiff data failed
On Thu, Aug 29, 2019 at 2:49 PM Michael Ditum wrote: > As we're running a ridiculously old subversion server (1.4.4) on a > ridiculously old operating system (Fedora 7) I've decided it's time to > migrate to a new server. > Have you tried to do a dump from 1.4.4 and load on a newer version of Subversion? If so, did this process succeed?
Decompression of svndiff data failed
Hi, We've been running a subversion server since 2003, over the years we've come across the svndiff decompression error numerous times, it's almost always when we're committing binary files to the repository. Historically we've always noticed it quite quickly and fixed the issue by deleting the offending file and re-committing it. This solved the immediate problem where other people could not checkout or update the repo but it's left the offending revisions in our repo. As we're running a ridiculously old subversion server (1.4.4) on a ridiculously old operating system (Fedora 7) I've decided it's time to migrate to a new server. I'm also looking at making read only mirrors available in our other offices to save time checking out applications over the WAN and to provide geographically separate backups. In order to do this I've been using svnsync to replicate a copy of the repository to a different server. As I've come across the svndiff errors I've been using the fsfsverify.py to fix them and so far it's worked and I've got to revision 24953 of 83080. Unfortunately I have now got to a revision that fsfsverify.py has been unable to fix, it keeps on detecting problems and fixing them but does this with ever changing offsets and amount until it gets to the point where it just constantly cycles between 2 different sets of offfsets and amounts. The error fsfsverify originally comes out with is... Error InvalidCompressedStream: Invalid compressed instr stream at offset 154848839 (Error -3 while decompressing: incorrect header check) Unfortunately the binary file that got corrupted was a copy of an open office install package which makes the revision 150MB and thus is rather tricky to fix by hand. Apart from using fsfsverify I also tried recreating the diff by creating a Fedora 7 VM, running svnsync on it to copy the repo up to that point and then manually committing the file and copying the revision over to the copy of my original repo. Whilst this allows svnsync to get past that revision I then started having lots of problems with incorrect byte offsets in later revisions and once I (think I correctly) fixed started getting checksum errors. Once I (again think I correctly) fixed those I then got another error. At which point I decided I was disappearing down a rabbit hole and gave up. Does anyone have any ideas on how I can fix this revision? As I mentioned before, the file gets deleted a couple of revisions later so I don't really care about the contents of the revision but I'm currently stuck and can't get any further in my svnsync. Thanks in advance for your help! Mike
Decompression of svndiff data failed - Unpacking a repository
Hello, We've recently installed Fisheye to index our Subversion repository. However, it kept looking over an error on SVN. Trying to reproduce the error, I effectively received an error message when performing a diff between two version. svn diff -r 233984:233985 file:///opt/subversion/repository/filename@233985 svn: Decompression of svndiff data failed So far, all my search pointed to a corruption performed by mod_dav_svn and the only solution has been to run a script called fsfsverify.py I cloned my repository, downloaded this script and ran it, only to find out it won't support packed shard. I need to know if there's a way to unpack a single shard so that I can run this script on the faulty revision and repair it, without having to either unpack the whole repository or do a dump / load. Thanks -- *Philippe Busque* , rue St-Charles Ouest, Tour Est, bureau 255 Longueuil (Québec) Canada J4K 5G4 Tél. : 450-449-0102 ext. 3017 Télec. : 450-449-8725 Ce message et les fichiers d'accompagnement transmis avec celui-ci s'adressent expressément au(x) destinataire(s) et peuvent contenir des renseignements confidentiels et privilégiés. Si vous recevez ce message par erreur, veuillez en aviser immédiatement l'expéditeur par courrier électronique. Veuillez également ne pas en prendre connaissance et en supprimer toutes les copies immédiatement. Technologies Interactives Mediagrif Inc. et ses filiales n'acceptent aucune responsabilité à l'égard des opinions exprimées dans le message ou des conséquences de tout virus informatique qui pourrait être transmis avec ce message. Ce message fait également l'objet d'un copyright. Il est interdit d'en reproduire, adapter ou transmettre quelque partie que ce soit sans le consentement écrit du détenteur du copyright.
Re: Decompression of svndiff data failed - Unpacking a repository
Philippe Busque wrote on Mon, Jul 25, 2011 at 15:05:29 -0400: I need to know if there's a way to unpack a single shard so that I can run this script on the faulty revision and repair it, without having to either unpack the whole repository or do a dump / load. Thanks The shards are independent, and fsfsverify.py operates on one revision at a time. It should be possible to extract just one revision from the pack file (use its manifest file), fix it, and then sew the pack file back together (and also fix the manifest file if the fixed rev file has a different length). There is also a script in tools/, called fsfs-reshard.py, that may be useful.
Re: dreaded Decompression of svndiff data failed question
On Thu, 23 Jun 2011 22:34:54 +0200, Stefan Sperling wrote: On Thu, Jun 23, 2011 at 08:50:05AM +0200, martin wrote: Hi all, I've searched the list and could not find an answer to this. I've run into the dreaded Decompression of svndiff data failed problem. Running http://www.szakmeister.net/fsfsverify/utility trying to fix it I end up with a string of errors. So far I've seen: - Error InvalidCompressedStream: Invalid compressed instr stream at offset (Error -3 while decompressing: incorrect header check) - Error InvalidCompressedStream: Invalid compressed data stream at offset (Error -3 while decompressing: incorrect data check) - Error InvalidWindow: The window header at offset xxx appears to be corrupted We have so far never been able to pin down the cause of such corruption problems. It could be a bug in Subversion (and you should be upgrading to avoid the ones that are already known), but it might just as well be a bug in some dependency Subversion is using (such as APR), or a bug in your operating system, or your disk controller or the disk itself having problems. If you'd like to share your corrupt revision file (privately if you like) I'd be happy to take a look at it. If you are lucky it's an instance of this problem: http://subversion.tigris.org/issues/show_bug.cgi?id=3705 fsfsverify can fix some instances of this but not all. I've fixed two such files manually so far which fsfsverify could not fix. And even if it's something else, maybe we can figure out what went wrong. And maybe if we see enough corrupt revision files we'll eventually be able to pin it down to a bug in Subversion and fix it. But the last time I was looking at a corrupted revision file it simply contained corrupted zip data. zlib could not decompress it anymore and nobody knew why. The file was simply broken and we could not reconstruct it within reasonable effort. So keep backups :) Thanks for all the feedback so far, It's an old legacy system I inherited and the earlier guys only did a hotcopy not verify, so on further investigation I found about 20 errors in total in all it's history (fatal ones). It's about half / half with the above error and else the checksum error. The latter I can at least get to a stage where I can 'skip' it and re-commit again. For the former I'm stuck with fsfsverify just looping, and the error revision had a zip file added to it just like yours. I'll have a look into it and see if I can share the corrupt file. Regarding hardware it's on a virtual server which hosts about 20 other guests (and none of those have any data corruption), so I don't think it's hw-related. It's not highly loaded nor are the VM host or any guests. It's running off CentOS and have about 66k revisions. I've tried to see if I can find a pattern in it, and from the about 20 errors one certain user is either the one comitting or the error occur when others merge in his data. I have 2 out of 20 errors which does not follow this pattern. Of course it could just be coincidence, but seems rather suspecious to me. Also, this guy was absent from ~3000 forthgoing revisions and no errors in the meantime, then he started comitting again and at exactly the same time errors started to occur (not his own commit but when someone merged his in later). He's using windows 7 but doesn't seem to have any malware or the like on his system. Also, there is no [seemingly] clear pattern about it having to be a zip-file. Ordinary text-commits seem to trigger it as well, although the majority is because of zip-files. I'm mostly inclined to believe it's something network-related, but the 'evidence' I have point in another direction. Again, I appreciate your time thinking about this, /Martin If I make any progress I'll be sure to let you guys know. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
dreaded Decompression of svndiff data failed question
Hi all, I've searched the list and could not find an answer to this. I've run into the dreaded Decompression of svndiff data failed problem. Running http://www.szakmeister.net/fsfsverify/utility trying to fix it I end up with a string of errors. So far I've seen: - Error InvalidCompressedStream: Invalid compressed instr stream at offset (Error -3 while decompressing: incorrect header check) - Error InvalidCompressedStream: Invalid compressed data stream at offset (Error -3 while decompressing: incorrect data check) - Error InvalidWindow: The window header at offset xxx appears to be corrupted The offset change on [almost] all runs, so for the heck of it I scripted it to run 500 times (with and without the -f flag on each run). After 500 iterations it is still in error and keeps reporting a new offset on each subsequent run. I feel a bit like a goof asking this, but do you guys have any input on this? My guts tell me it's just looping somehow and if I set it to run for another 500 runs it would probably still be doing the same although with different numbers. The script also reports copying in an incresingly larger chunk of data (went from 600k to about 3MB now during those 500 runs). Anyone have any experience with anything taking this many runs or should I just give up already and re-create it saving as much as I can? I'm using svn 1.5.1 and FSFS. Thanks for your time, /Martin -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: dreaded Decompression of svndiff data failed question
On Jun 23, 2011, at 01:50, martin wrote: I'm using svn 1.5.1 You should really upgrade. Many bugs were fixed in later 1.5.x versions, and even more in 1.6.x. The first 1.7.x test release was just published, so you can expect 1.7.0 final to be released before too long, at which time support for Subversion 1.5.x and earlier will end.
Re: Decompression of svndiff data failed
On Tue, May 10, 2011 at 12:20:36PM +0200, vys...@oksystem.cz wrote: Hello. I use svn co to checkout my repo. All worked fine for a while, but today I get error Decompression of svndiff data failed: no size What versions of Subversion are you running on your server and the clients?
Re: Decompression of svndiff data failed
vys...@oksystem.cz wrote on Tue, May 10, 2011 at 12:20:36 +0200: I found Decompression of svndiff data failed message only in one file in sources - /libsvn_delta/svndiff.c - in the zlib_decode function. So, probably data in the repository in the file for this revision somehow corrupted during or after commit? Am I right or there's another reason of these errors (I googled, but what I found is not my case)? svndiff is used both in the backend (for storing revisions of files as binary deltas against previous revisions) and on the wire (if both client and server ≥1.4, for transmitting the delta from what the client has (or from the empty file) to the request revision's contents).
Decompression of svndiff data failed
Hello. I use svn co to checkout my repo. All worked fine for a while, but today I get error Decompression of svndiff data failed: no size every time I try to do svn co. I found out that I got these errors not only from svn co, but also from svnsync, svnadmin dump, etc. This happened after we commited a new version of file in the repo, all operations fail only if they access that subfolder on that commit revision. If I try to access other files, all work fine. I found Decompression of svndiff data failed message only in one file in sources - /libsvn_delta/svndiff.c - in the zlib_decode function. So, probably data in the repository in the file for this revision somehow corrupted during or after commit? Am I right or there's another reason of these errors (I googled, but what I found is not my case)? And the most important thing. How can I make my repo consistent back now? I can't even make svnadmin dump and erase that revision from dump file... Thanks, Petr Vystyd Upozorn?n? spole?nosti OKsystem s.r.o. s ohledem na zaveden? standardy ISO 9001, ISO 27001 a ISO 14001: Tato zpr?va a v?echny p?ipojen? soubory jsou dle obchodn?ho z?kon?ku d?v?rn?. Jestli?e nejste zam??len?m adres?tem, uv?domte pros?m odesilatele a sma?te zpr?vu i p?ilo?en? soubory. Opravdu pot?ebujete vytisknout tento email? Myslete na p??rodu. Disclaimer of OKsystem s.r.o. with respect to implemented standards ISO 9001, ISO 27001 and ISO 14001: This message and all attached files are confidential and legally privileged. If you are not the intended recipient, please notify the sender and delete the message including all attachments. Please consider the environment before printing this email.
{svn | svnadmin}: Decompression of svndiff data failed and svn: Malformed representation header, svn: Corrupt representation messages
Hello,
I recently installed a subversion (version 1.6.11) repository on out RHEL 5
system, that I access using ssh+svn protocol. Already after some very few
commits, I ran into the problem of getting the error svnadmin:
Decompression of svndiff data failed when performing svnadmin dump
/path/to/rep /path/to/backup/repo.dump. I get the same error svn:
Decompression of svndiff data failed when trying to check out the entire
repository using the command line with svn checkout
svn+ssh://localhost/path/to/rep /path/to/co.
I did quite a lot of searches on the internet and tried svnadmin verify
/path/to/rep, also returning the same error.
* Verified revision 1.
* Verified revision 2.
svnadmin: Decompression of svndiff data failed
It seems to be revision 3 which is corrupted, as a revision specific
verifcation with svnadmin verify /path/to/rep -r x succeeds with x=1 and
x=2:
* Verified revision 0.
resp.
* Verified revision 1.
Verifying revision 3 (and any subsequent revision) returns the well-known
error again:
svnadmin: Decompression of svndiff data failed
Strangely, svnadmin recover /path/to/rep successfully succeeds:
svnadmin recover /path/to/rep
Repository lock acquired.
Please wait; recovering the repository may take some time...
Recovery completed.
The latest repos revision is 8.
I came accorss some posts, suggesting to use fsfsverify.py and apply it to
the revisions. Applying ./fsfsverify.py -f /opt/maps/svn/maps/db/revs/0/x
it to revisions x=0 and x=1 successfully terminates but applying it to
revision 2 (and all subsequent ones) terminates with the error:
Traceback (most recent call last):
File ./fsfsverify.py, line 1120, in ?
for noderev in strategy:
File ./fsfsverify.py, line 839, in _nodeWalker
for x in self._nodeWalker():
File ./fsfsverify.py, line 839, in _nodeWalker
for x in self._nodeWalker():
File ./fsfsverify.py, line 832, in _nodeWalker
noderev = NodeRev(self.f, self.currentRev)
File ./fsfsverify.py, line 678, in __init__
(rev, offset, length, size, digest) = value.split(' ')
ValueError: too many values to unpack
I furthermore integrated svn into my IDE (Eclipse Helios) and also tried a
checkout of the project, of which revision 2 (the corrupt one) consisted.
The process aborts, showing the following error:
Get content for 'svn+ssh://server.fqdn.com/path/to/rep/ProjectName/.project'
failed.
svn: Malformed representation header
svn: Corrupt representation '2 325 387 370 d66781ba03f9571d044bf8f257601485
0fd3590a02c19961e78d0ef76efa710d9d6ef194 1-2/_c'
I guess that these errors are related to each, aren't they?
It would be great if someone out there could help me out - there are not
many revisions within the repo currently, but I need to report about the
success of the test.
Thank anyone for reading and helping!
Bastian
Re: {svn | svnadmin}: Decompression of svndiff data failed and svn: Malformed representation header, svn: Corrupt representation messages
Bastian Ugimachi wrote on Thu, Apr 21, 2011 at 12:29:33 +0200: the revisions. Applying ./fsfsverify.py -f /opt/maps/svn/maps/db/revs/0/x You probably know that, but: You should have taken a backup before running this.
Re: {svn | svnadmin}: Decompression of svndiff data failed and svn: Malformed representation header, svn: Corrupt representation messages
2011/4/21 Daniel Shahaf danie...@elego.de
Bastian Ugimachi wrote on Thu, Apr 21, 2011 at 14:17:00 +0200:
2011/4/21 Daniel Shahaf danie...@elego.de
I furthermore integrated svn into my IDE (Eclipse Helios) and also
tried
a
checkout of the project, of which revision 2 (the corrupt one)
consisted.
The process aborts, showing the following error:
Get content for 'svn+ssh://
server.fqdn.com/path/to/rep/ProjectName/.project'
failed.
svn: Malformed representation header
svn: Corrupt representation '2 325 387 370
d66781ba03f9571d044bf8f257601485
0fd3590a02c19961e78d0ef76efa710d9d6ef194 1-2/_c'
That's the interesting part. It tells you that the representation
(either the full plaintext or a binary delta) which starts at byte 325
of the r2 revision file is corrupt. You can continue by looking at
that
offset of that revision and see what bytes you find there. ('xxd -s
325
-l 488 /path/to/revs/0/2' for example.)
OK, I performed xxd -s 325 -l 488 2
145: 7465 6d2e 6f75 742e 7072 696e 746c 6e28 tem.out.println(
155: 2256 6965 7274 6572 2057 7572 6622 293b Vierter Wurf);
165: 0a09 0953 7973 7465 6d2e 6f75 742e 7072 ...System.out.pr
175: 696e 746c 6e28 2246 c3bc 6e66 7465 7220 intln(F..nfter
185: 5775 7266 2229 3b0a 0909 5379 7374 656d Wurf);...System
195: 2e6f 7574 2e70 7269 6e74 6c6e 2822 5365 .out.println(Se
You can imagine - this is just some Java code...
By the way... the source file is UTF8 encoded and where xxd says
F..nfter,
it should be Fünfter - I did some Special-Character-Testing here. Might
this be some problem?
No, that's okay: the bytes there (0xC3 0xB3) are one UTF-8 encoding of ü.
A representation header is either PLAIN\n or DELTA\n or DELTA
$some
$numeric $parameters\n. A representation terminates in the word
ENDREP\n. The representation's length (excluding header and trailer)
should be 387 bytes. Details in the file
subversion/libsvn_fs_fs/structure
in the source tree.
Here's an example:
[[[
DELTA
SVN[9 chars]This is the file 'iota'.
ENDREP
...
text: 1 605 37 25 2d18c5e57e84c5b8a5e9a6e13fa394dc
2c0aa9014a0cd07f01795a333d82485ef6d083e2 0-0/_14
]]]
The intersting thing is that the ID in my error message belongs does not
belong to the same representation as the part extracted by xxd around the
specified bytes...?!
Well, it's corrupted. The correct representation might be somewhere
nearby --- perhaps a few hundred bytes away (in either direction)?
Is there any chance to make this consistent again?
And where could this inconsistence possibly come from?
By the way, I just remarked, that svnadmin verify ...-r x is wrong ONLY
for x=2, not for x=2 as I wrote before. So correcting this one should make
the repository be OK again, I guess?
Re: {svn | svnadmin}: Decompression of svndiff data failed and svn: Malformed representation header, svn: Corrupt representation messages
Bastian Ugimachi wrote on Thu, Apr 21, 2011 at 14:52:33 +0200: 2011/4/21 Daniel Shahaf danie...@elego.de Bastian Ugimachi wrote on Thu, Apr 21, 2011 at 14:17:00 +0200: The intersting thing is that the ID in my error message belongs does not belong to the same representation as the part extracted by xxd around the specified bytes...?! Well, it's corrupted. The correct representation might be somewhere nearby --- perhaps a few hundred bytes away (in either direction)? Is there any chance to make this consistent again? Yes, with a bit of skill and luck. Do you have a working copy of r2? Does the rev file contain all the information (just possibly in the wrong place)? Do you have backups? The solutions I'm getting at are manual revision file surgery/reconstruction. (You may be able to do it yourself if you study the 'structure' file and have some patience.) There may be other classes of solutions. And where could this inconsistence possibly come from? Bug in libsvn_fs_fs. (See issue #3705.) Or bug in any of N other things, including but not limited to hardware bugs. By the way: did you say what version of svn your server was running? By the way, I just remarked, that svnadmin verify ...-r x is wrong ONLY for x=2, not for x=2 as I wrote before. So correcting this one should make the repository be OK again, I guess? Yes. I'd expect verify to fail only for r2, but checkout to fail for (at least some) later revisions as well.
svn checkout/update fails with svn: Decompression of svndiff data failed: size too large error
Hi, Recently we upgraded our svn server/client from 1.6.2 to 1.6.13, and since then, we are encountering the following error when a user tries to checkout/update a svn resource, where he has the required access (both read and write). svn: REPORT of '/svnrepository/!svn/vcc/default': 200 OK (http://10.1.121.199 http://10.1.121.199/ ) - this was with http protocol, and found the below errors in apache error logs: [Tue Nov 16 19:57:07 2010] [error] [client 10.1.128.18] Provider encountered an error while streaming a REPORT response. [500, #0] [Tue Nov 16 19:57:07 2010] [error] [client 10.1.128.18] A failure occurred while driving the update report editor [500, #185005] [Tue Nov 16 19:57:07 2010] [error] [client 10.1.128.18] Decompression of svndiff data failed: size too large [500, #185005] When I tried to do the same activity with svn protocol, I got the below error: svn: Decompression of svndiff data failed: size too large The weirdest part is that, it checks out the folder, but not the contents of the folder, and then throws the error mentioned (for both svn and http protocol). When I tried to checkout all the files in the subjected folder, with the help of svnlook (svnlook -r last_modified_rev_on_that_folder cat repository_path file_path testfile), I got the same error for one of the file. svnlook: Decompression of svndiff data failed: size too large I could re-produce the error with both command line svn client, and tortoise svn and in both windows (xp service pack 3), and linux (RHEL5) environments. As mentioned earlier, both of our client and server are 1.6.13. Please confirm, if this is a bug or not, and suggest a way forward, and let me know if you need any more info. Regards, Rajesh This email is sent for and on behalf of Ivy Comptech Private Limited. Ivy Comptech Private Limited is a limited liability company. This email and any attachments are confidential, and may be legally privileged and protected by copyright. If you are not the intended recipient dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system. Any views or opinions are solely those of the sender. This communication is not intended to form a binding contract on behalf of Ivy Comptech Private Limited unless expressly indicated to the contrary and properly authorised. Any actions taken on the basis of this email are at the recipient's own risk. Registered office: Ivy Comptech Private Limited, Cyber Spazio, Road No. 2, Banjara Hills, Hyderabad 500 033, Andhra Pradesh, India. Registered number: 37994. Registered in India. A list of members' names is available for inspection at the registered office.
