Failed pipeline for branch/default | mercurial-devel | b41554f4
Pipeline #22459 has failed! Project: mercurial-devel ( https://foss.heptapod.net/octobus/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/octobus/mercurial-devel/-/commits/branch/default ) Commit: b41554f4 ( https://foss.heptapod.net/octobus/mercurial-devel/-/commit/b41554f4d89958fd4d8c74b46aa30461c4f793d5 ) Commit Message: revlog: close the index file handle after the d... Commit Author: Pierre-Yves David ( https://foss.heptapod.net/marmoute ) Pipeline #22459 ( https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines/22459 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 2 failed builds. Job #200994 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/200994/raw ) Stage: tests Name: checks-py2 Job #200995 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/200995/raw ) Stage: tests Name: checks-py3 -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Failed pipeline for branch/default | mercurial-devel | 9f571d55
Pipeline #22455 has failed! Project: mercurial-devel ( https://foss.heptapod.net/octobus/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/octobus/mercurial-devel/-/commits/branch/default ) Commit: 9f571d55 ( https://foss.heptapod.net/octobus/mercurial-devel/-/commit/9f571d55afed89d28b14b2b61b6bfbb4105a445d ) Commit Message: rhg: A missing .hg/dirstate file is not an erro... Commit Author: Simon Sapin ( https://foss.heptapod.net/SimonSapin ) Pipeline #22455 ( https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines/22455 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 2 failed builds. Job #200934 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/200934/raw ) Stage: tests Name: checks-py2 Job #200935 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/200935/raw ) Stage: tests Name: checks-py3 -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[Bug 6527] New: Incorrect error message shown when aborting a merge with an ongoing bisect operation
https://bz.mercurial-scm.org/show_bug.cgi?id=6527 Bug ID: 6527 Summary: Incorrect error message shown when aborting a merge with an ongoing bisect operation Product: Mercurial Version: stable branch Hardware: Other OS: Linux Status: UNCONFIRMED Severity: bug Priority: wish Component: Mercurial Assignee: bugzi...@mercurial-scm.org Reporter: tom.ingelbi...@nokia.com CC: mercurial-devel@mercurial-scm.org Python Version: --- Since version 5.1.2-1, Mercurial gives an incorrect error message when aborting a merge while a bisect is still ongoing (admittedly quite a corner case). I'm working on CentOS, where I can easily reproduce it with following minimized scenario: # Set up a small repo hg init echo "lorem ipsum" > textfile; hg add textfile; hg commit -m "First commit" echo "dolor sit amet" >> textfile; hg commit -m "Second commit" # Create second head hg up -r 0 echo "foo" > bar; hg add bar hg commit -m "Additional head for merging" # Run a bisect (without explicitly resetting at the end) hg bisect --reset hg bisect --bad 1 hg bisect --good 0 # Initiate a merge, and abort it hg merge hg merge --abort # Result: following incorrect error message # abort: cannot abort merge with bisect in progress # (use 'hg bisect --continue' or 'hg bisect --abort') The bisect command doesn't recognize either of those two suggested options, though. Note that I only target this incorrect help text in this bug ticket. The decision to refuse aborting a merge with a bisect operation in progress is not what I'm challenging here. I didn't see the issue on versions older than 5.1.2-1, but _only_ because those happily aborted the merge, despite the bisect still being in progress. Since there's no error message, there's also no incorrect hint. I've verified this on 4.5-1, 4.6-1, 4.7.1-1 and 4.9-1. As I could resolve the situation by just running "hg bisect --reset" again, it seems that this is actually the hint that should be displayed (rather than the --continue and --abort options). -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10777: fuzz: add hg to sys.path when constructing mpatch corpus
durin42 created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10777 AFFECTED FILES contrib/fuzz/mpatch_corpus.py CHANGE DETAILS diff --git a/contrib/fuzz/mpatch_corpus.py b/contrib/fuzz/mpatch_corpus.py --- a/contrib/fuzz/mpatch_corpus.py +++ b/contrib/fuzz/mpatch_corpus.py @@ -1,10 +1,15 @@ from __future__ import absolute_import, print_function import argparse +import os import struct import sys import zipfile +# Add ../.. to sys.path as an absolute path so we can import hg modules +hgloc = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..')) +sys.path[0:0] = [hgloc] + from mercurial import ( hg, ui as uimod, To: durin42, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10776: revlog: close the index file handle after the data one
marmoute created this revision. Herald added a reviewer: indygreg. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This make sure the data file is flushed before the index. preventing the index to reference unflushed data. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10776 AFFECTED FILES mercurial/revlog.py CHANGE DETAILS diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -2267,10 +2267,12 @@ self._write_docket(transaction) finally: self._writinghandles = None +if dfh is not None: +dfh.close() +# closing the index file last to avoid exposing referent to +# potential unflushed data content. if ifh is not None: ifh.close() -if dfh is not None: -dfh.close() def _write_docket(self, transaction): """write the current docket on disk To: marmoute, indygreg, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10772: revlogv2: use a unique filename for data
marmoute created this revision. Herald added a reviewer: indygreg. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Having a unique data will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10772 AFFECTED FILES mercurial/revlog.py mercurial/revlogutils/docket.py mercurial/store.py tests/test-revlog-v2.t CHANGE DETAILS diff --git a/tests/test-revlog-v2.t b/tests/test-revlog-v2.t --- a/tests/test-revlog-v2.t +++ b/tests/test-revlog-v2.t @@ -58,6 +58,7 @@ date:Thu Jan 01 00:00:00 1970 + summary: initial + Header written as expected $ f --hexdump --bytes 4 .hg/store/00changelog.i @@ -77,9 +78,9 @@ - a data file $ ls .hg/store/00changelog* .hg/store/00manifest* - .hg/store/00changelog-b870a51b.idx - .hg/store/00changelog.d + .hg/store/00changelog-6b8ab34b.dat + .hg/store/00changelog-88698448.idx .hg/store/00changelog.i - .hg/store/00manifest-88698448.idx - .hg/store/00manifest.d + .hg/store/00manifest-1335303a.dat + .hg/store/00manifest-b875dfc5.idx .hg/store/00manifest.i diff --git a/mercurial/store.py b/mercurial/store.py --- a/mercurial/store.py +++ b/mercurial/store.py @@ -389,7 +389,14 @@ ] REVLOG_FILES_MAIN_EXT = (b'.i', b'i.tmpcensored') -REVLOG_FILES_OTHER_EXT = (b'.idx', b'.d', b'.n', b'.nd', b'd.tmpcensored') +REVLOG_FILES_OTHER_EXT = ( +b'.idx', +b'.d', +b'.dat', +b'.n', +b'.nd', +b'd.tmpcensored', +) # files that are "volatile" and might change between listing and streaming # # note: the ".nd" file are nodemap data and won't "change" but they might be @@ -414,6 +421,7 @@ if f.endswith(REVLOG_FILES_VOLATILE_EXT): t |= FILEFLAGS_VOLATILE return t +return None # the file is part of changelog data @@ -753,6 +761,7 @@ ef = self.encode(f) try: t = revlog_type(f) +assert t is not None, f t |= FILEFLAGS_FILELOG yield t, f, ef, self.getsize(ef) except OSError as err: diff --git a/mercurial/revlogutils/docket.py b/mercurial/revlogutils/docket.py --- a/mercurial/revlogutils/docket.py +++ b/mercurial/revlogutils/docket.py @@ -89,12 +89,13 @@ # | This is mandatory as docket must be compatible with the previous # | revlog index header. # * 1 bytes: size of index uuid +# * 1 bytes: size of data uuid # * 8 bytes: size of index-data # * 8 bytes: pending size of index-data # * 8 bytes: size of data # * 8 bytes: pending size of data # * 1 bytes: default compression header -S_HEADER = struct.Struct(constants.INDEX_HEADER.format + 'Bc') +S_HEADER = struct.Struct(constants.INDEX_HEADER.format + 'BBc') class RevlogDocket(object): @@ -106,6 +107,7 @@ use_pending=False, version_header=None, index_uuid=None, +data_uuid=None, index_end=0, pending_index_end=0, data_end=0, @@ -119,6 +121,7 @@ self._path = revlog._docket_file self._opener = revlog.opener self._index_uuid = index_uuid +self._data_uuid = data_uuid # thes asserts should be True as long as we have a single index filename assert index_end <= pending_index_end assert data_end <= pending_data_end @@ -141,6 +144,13 @@ self._index_uuid = make_uid() return b"%s-%s.idx" % (self._radix, self._index_uuid) +def data_filepath(self): +"""file path to the current index file associated to this docket""" +# very simplistic version at first +if self._data_uuid is None: +self._data_uuid = make_uid() +return b"%s-%s.dat" % (self._radix, self._data_uuid) + @property def index_end(self): return self._index_end @@ -195,6 +205,7 @@ data = ( self._version_header, len(self._index_uuid), +len(self._data_uuid), official_index_end, self._index_end, official_data_end, @@ -204,6 +215,7 @@ s = [] s.append(S_HEADER.pack(*data)) s.append(self._index_uuid) +s.append(self._data_uuid) return b''.join(s) @@ -230,16 +242,20 @@ index_uuid_size = header[1] index_uuid = data[offset : offset + index_uuid_size] offset += index_uuid_size -index_size = header[2] -pending_index_size = header[3] -data_size = header[4] -pending_data_size = header[5] -default_compression_header = header[6] +data_uuid_size = header[2] +data_uuid = data[offset : offset + data_uuid_size] +offset += data_uuid_size +index_size = header[3] +pending_index_size = header[4] +data_size = header[5] +
D10775: revlog: simplify the try nesting in the `_writing` context
marmoute created this revision. Herald added a reviewer: indygreg. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Lets use a single try, with conditional cleanup. This make is easier to add a file handle dedicated to sidedata. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10775 AFFECTED FILES mercurial/revlog.py CHANGE DETAILS diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -2232,41 +2232,43 @@ if self._writinghandles is not None: yield else: -r = len(self) -dsize = 0 -if r: -dsize = self.end(r - 1) -dfh = None -if not self._inline: -try: -dfh = self._datafp(b"r+") -if self._docket is None: -dfh.seek(0, os.SEEK_END) -else: -dfh.seek(self._docket.data_end, os.SEEK_SET) -except IOError as inst: -if inst.errno != errno.ENOENT: -raise -dfh = self._datafp(b"w+") -transaction.add(self._datafile, dsize) +ifh = dfh = None try: +r = len(self) +# opening the data file. +dsize = 0 +if r: +dsize = self.end(r - 1) +dfh = None +if not self._inline: +try: +dfh = self._datafp(b"r+") +if self._docket is None: +dfh.seek(0, os.SEEK_END) +else: +dfh.seek(self._docket.data_end, os.SEEK_SET) +except IOError as inst: +if inst.errno != errno.ENOENT: +raise +dfh = self._datafp(b"w+") +transaction.add(self._datafile, dsize) + +# opening the index file. isize = r * self.index.entry_size ifh = self.__index_write_fp() if self._inline: transaction.add(self._indexfile, dsize + isize) else: transaction.add(self._indexfile, isize) -try: -self._writinghandles = (ifh, dfh) -try: -yield -if self._docket is not None: -self._write_docket(transaction) -finally: -self._writinghandles = None -finally: +# exposing all file handle for writing. +self._writinghandles = (ifh, dfh) +yield +if self._docket is not None: +self._write_docket(transaction) +finally: +self._writinghandles = None +if ifh is not None: ifh.close() -finally: if dfh is not None: dfh.close() To: marmoute, indygreg, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10774: revlogv2: add a `get_data` helper to grab the next piece of docket
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This make the processing more compact but abstracting repetitive processing away. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10774 AFFECTED FILES mercurial/revlogutils/docket.py CHANGE DETAILS diff --git a/mercurial/revlogutils/docket.py b/mercurial/revlogutils/docket.py --- a/mercurial/revlogutils/docket.py +++ b/mercurial/revlogutils/docket.py @@ -237,19 +237,30 @@ def parse_docket(revlog, data, use_pending=False): """given some docket data return a docket object for the given revlog""" header = S_HEADER.unpack(data[: S_HEADER.size]) -offset = S_HEADER.size + +# this is a mutable closure capture used in `get_data` +offset = [S_HEADER.size] + +def get_data(size): +"""utility closure to access the `size` next bytes""" +if offset[0] + size > len(data): +# XXX better class +msg = b"docket is too short, expected %d got %d" +msg %= (offset[0] + size, len(data)) +raise error.Abort(msg) +raw = data[offset[0] : offset[0] + size] +offset[0] += size +return raw iheader = iter(header) version_header = next(iheader) index_uuid_size = next(iheader) -index_uuid = data[offset : offset + index_uuid_size] -offset += index_uuid_size +index_uuid = get_data(index_uuid_size) data_uuid_size = next(iheader) -data_uuid = data[offset : offset + data_uuid_size] -offset += data_uuid_size +data_uuid = get_data(data_uuid_size) index_size = next(iheader) To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10771: revlogv2: use a unique filename for index
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Having a unique index will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10771 AFFECTED FILES mercurial/revlogutils/docket.py tests/test-revlog-v2.t CHANGE DETAILS diff --git a/tests/test-revlog-v2.t b/tests/test-revlog-v2.t --- a/tests/test-revlog-v2.t +++ b/tests/test-revlog-v2.t @@ -67,3 +67,19 @@ $ f --hexdump --bytes 4 .hg/store/data/foo.i .hg/store/data/foo.i: : 00 00 de ad || + +The expected files are generated + + +We should have have: +- a docket +- a index file with a unique name +- a data file + + $ ls .hg/store/00changelog* .hg/store/00manifest* + .hg/store/00changelog-b870a51b.idx + .hg/store/00changelog.d + .hg/store/00changelog.i + .hg/store/00manifest-88698448.idx + .hg/store/00manifest.d + .hg/store/00manifest.i diff --git a/mercurial/revlogutils/docket.py b/mercurial/revlogutils/docket.py --- a/mercurial/revlogutils/docket.py +++ b/mercurial/revlogutils/docket.py @@ -88,12 +88,13 @@ # * 4 bytes: revlog version # | This is mandatory as docket must be compatible with the previous # | revlog index header. +# * 1 bytes: size of index uuid # * 8 bytes: size of index-data # * 8 bytes: pending size of index-data # * 8 bytes: size of data # * 8 bytes: pending size of data # * 1 bytes: default compression header -S_HEADER = struct.Struct(constants.INDEX_HEADER.format + 'c') +S_HEADER = struct.Struct(constants.INDEX_HEADER.format + 'Bc') class RevlogDocket(object): @@ -104,6 +105,7 @@ revlog, use_pending=False, version_header=None, +index_uuid=None, index_end=0, pending_index_end=0, data_end=0, @@ -116,6 +118,7 @@ self._radix = revlog.radix self._path = revlog._docket_file self._opener = revlog.opener +self._index_uuid = index_uuid # thes asserts should be True as long as we have a single index filename assert index_end <= pending_index_end assert data_end <= pending_data_end @@ -134,7 +137,9 @@ def index_filepath(self): """file path to the current index file associated to this docket""" # very simplistic version at first -return b"%s.idx" % self._radix +if self._index_uuid is None: +self._index_uuid = make_uid() +return b"%s-%s.idx" % (self._radix, self._index_uuid) @property def index_end(self): @@ -189,13 +194,17 @@ assert official_data_end <= self._data_end data = ( self._version_header, +len(self._index_uuid), official_index_end, self._index_end, official_data_end, self._data_end, self.default_compression_header, ) -return S_HEADER.pack(*data) +s = [] +s.append(S_HEADER.pack(*data)) +s.append(self._index_uuid) +return b''.join(s) def default_docket(revlog, version_header): @@ -216,16 +225,21 @@ def parse_docket(revlog, data, use_pending=False): """given some docket data return a docket object for the given revlog""" header = S_HEADER.unpack(data[: S_HEADER.size]) +offset = S_HEADER.size version_header = header[0] -index_size = header[1] -pending_index_size = header[2] -data_size = header[3] -pending_data_size = header[4] -default_compression_header = header[5] +index_uuid_size = header[1] +index_uuid = data[offset : offset + index_uuid_size] +offset += index_uuid_size +index_size = header[2] +pending_index_size = header[3] +data_size = header[4] +pending_data_size = header[5] +default_compression_header = header[6] docket = RevlogDocket( revlog, use_pending=use_pending, version_header=version_header, +index_uuid=index_uuid, index_end=index_size, pending_index_end=pending_index_size, data_end=data_size, To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D10773: revlogv2: simplify and clarify the processing of each entry
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY As we add more entries and some of them has non trivial processing it seems useful to make the processing leaner and clearly separated to simplify futures patches. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D10773 AFFECTED FILES mercurial/revlogutils/docket.py CHANGE DETAILS diff --git a/mercurial/revlogutils/docket.py b/mercurial/revlogutils/docket.py --- a/mercurial/revlogutils/docket.py +++ b/mercurial/revlogutils/docket.py @@ -238,18 +238,29 @@ """given some docket data return a docket object for the given revlog""" header = S_HEADER.unpack(data[: S_HEADER.size]) offset = S_HEADER.size -version_header = header[0] -index_uuid_size = header[1] + +iheader = iter(header) + +version_header = next(iheader) + +index_uuid_size = next(iheader) index_uuid = data[offset : offset + index_uuid_size] offset += index_uuid_size -data_uuid_size = header[2] + +data_uuid_size = next(iheader) data_uuid = data[offset : offset + data_uuid_size] offset += data_uuid_size -index_size = header[3] -pending_index_size = header[4] -data_size = header[5] -pending_data_size = header[6] -default_compression_header = header[7] + +index_size = next(iheader) + +pending_index_size = next(iheader) + +data_size = next(iheader) + +pending_data_size = next(iheader) + +default_compression_header = next(iheader) + docket = RevlogDocket( revlog, use_pending=use_pending, To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
