[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Mark A. Hershberger changed: What|Removed |Added CC||duplicate...@googlemail.com --- Comment #21 from Mark A. Hershberger 2012-02-08 22:28:31 UTC --- *** Bug 33221 has been marked as a duplicate of this bug. *** -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Diederik van Liere changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED --- Comment #20 from Diederik van Liere 2012-01-31 21:51:44 UTC --- Yes, I think so. I updated export.php so that it will be exported to the xml files once 1.19 is deployed. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #19 from Rob Lanphier 2012-01-31 21:47:33 UTC --- Diederik, is this work finished? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Bug 25312 depends on bug 21860, which changed state. Bug 21860 Summary: Add checksum field to database table; expose it in API https://bugzilla.wikimedia.org/show_bug.cgi?id=21860 What|Old Value |New Value Status|REOPENED|RESOLVED Resolution||FIXED -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #18 from Diederik van Liere 2011-11-16 19:05:12 UTC --- Hi Ariel, good point! I'll fix it for deleted revisions. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Sumana Harihareswara changed: What|Removed |Added Keywords||need-review, patch -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #17 from Ariel T. Glenn 2011-11-16 08:47:48 UTC --- I guess that the revision row would always contain the field, whether or not it is populated, since the patch to Export.php should go in at the same time as the schema change. I would suggest though that we don't provide the hash when the revision has been deleted; in that case we would want to write an empty tag. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #16 from Diederik van Liere 2011-11-15 23:24:45 UTC --- Created attachment 9461 --> https://bugzilla.wikimedia.org/attachment.cgi?id=9461 Patch adds a new sha1 tag to each revision in XML dump. It will write the sha1 hash if the revision row contains this field, else it will write an empty tag. Not sure if that is the best way to do it and if there are any other edge case that I didn't think of then please let me know. Patch also updates export-0.6.xsd. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #15 from Diederik van Liere 2011-11-09 23:13:52 UTC --- Commit http://www.mediawiki.org/wiki/Special:Code/MediaWiki/101021 adds fields to the tables. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #14 from MZMcBride 2011-08-20 01:43:06 UTC --- (In reply to comment #13) > Bug 2939 did look like something that this blocked. Wouldn't checksum revert > detection be the way to fix that bug? Bug 2939 is about the ability to detect reverts for the purpose of displaying the new messages notification bar. That would rely on the ability to uniquely identify revisions by putting unique identifiers in the database (bug 21860). Putting unique identifiers in the stub dumps (this bug, bug 25312) wouldn't really have anything to do with that. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #13 from Daniel Friesen 2011-08-19 23:12:58 UTC --- Bug 2939 did look like something that this blocked. Wouldn't checksum revert detection be the way to fix that bug? -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 MZMcBride changed: What|Removed |Added Blocks|2939| -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Max Semenik changed: What|Removed |Added Blocks||2939 -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #12 from Ariel T. Glenn 2011-08-15 20:13:37 UTC --- Very correct about the data integrity piece, as I mentioned in comment 1. I use rev_len for now but that is not foolproof. I've seen a number of revisions on other projects that have identical revision lengths (and they are not redirects either but actual content). We've had serious data corruption in the past, and odds are we'll run into it again for one reason or another. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312
--- Comment #11 from Brion Vibber 2011-08-15 19:41:49 UTC
---
(In reply to comment #3)
> I don't see why #21860 is a blocker - if text is being read, calculating
> checksums is cheap enough.
> Storing all that in the database isn't free.
When creating a stub dump, we haven't read the text yet -- the job of fetching
and inserting the text is being deferred to a later process (textDumpPass)
which pulls the text either from a previous dump or from the text table /
external storage etc.
So at that point, only data within the 'page' and 'revision' tables, and
anything else that can be very cheaply fetched, is available.
A rev_sha1 field that's already been pre-filled out would be usable for
creating stub dumps; calculating from text after it's been read would only be
usable on the final dumps (or else a second equivalent pass).
Using a separate field for this also gives greater confidence that there was
not internal data corruption; if the sha1 is generated from the text that's
right next to it in the same file, there's no point -- the client could
calculate it as easily and reliably as the server could have, and in neither
case will it indicate if the data has been corrupted on the backend.
(In reply to comment #7)
> SHA1 *might* make more sense than MD5, if only because it may help us in a
> crazy future where we leverage tools associated with Git or other version
> control systems (for example, Mercurial uses SHA1 as well). Not that there's
> anything planned, but since the choice of hash is somewhat arbitrary
> otherwise,
> SHA1 might be slightly preferable.
I don't think there'd be much chance at integration here really; git's object
references are based on SHA-1 checksums, but of the entire object including a
header indicating type ('blob' for files) and size prepended.
--
Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are on the CC list for the bug.
___
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 MZMcBride changed: What|Removed |Added CC||b...@mzmcbride.com --- Comment #10 from MZMcBride 2011-08-15 19:16:09 UTC --- r94289 and subsequent revisions reverted by Brion in r94541. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 --- Comment #9 from Diederik van Liere 2011-08-11 21:57:49 UTC --- Thanks Aaron! This is a very welcome feature. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Aaron Schulz changed: What|Removed |Added CC||aschulz4...@gmail.com --- Comment #8 from Aaron Schulz 2011-08-11 21:55:20 UTC --- Fields added to tables in r94289. -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 25312] MD5 or SHA1 checksum in stub dumps
https://bugzilla.wikimedia.org/show_bug.cgi?id=25312 Rob Lanphier changed: What|Removed |Added Summary|MD5 in stub dumps |MD5 or SHA1 checksum in ||stub dumps -- Configure bugmail: https://bugzilla.wikimedia.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
