Re: [PATCH 2 of 4 website] base: add a requirements.txt
On Wed, May 10, 2017 at 7:55 PM, Kevin Bullock < kbullock+mercur...@ringworld.org> wrote: > > On May 10, 2017, at 19:16, Gregory Szorc> wrote: > > > >> On Wed, May 10, 2017 at 10:22 AM, Kevin Bullock < > kbullock+mercur...@ringworld.org> wrote: > >> # HG changeset patch > >> # User Kevin Bullock > >> # Date 1494433855 18000 > >> # Wed May 10 11:30:55 2017 -0500 > >> # Node ID b99251cd874a4a559ceadbd6fd33e6f411b1e776 > >> # Parent 3faffcf56082846bd385f6594d9729bceceaf83b > >> base: add a requirements.txt > >> > >> diff --git a/requirements.txt b/requirements.txt > >> new file mode 100644 > >> --- /dev/null > >> +++ b/requirements.txt > >> @@ -0,0 +1,1 @@ > >> +Flask>=0.12.1,<0.13 > >> > > Modern versions of pip support pinning hashes in requirements files. It > is quite nice as it not only buffers you against MitM attacks, compromised > servers, and corruption, but also forces you to have hashes pinned for > *all* dependencies. In effect, it requires you to list *all* dependencies > and gives you peace of mind that your virtualenv is reproducible. I'd > highly recommend implementing that as a follow-up to this series and using > that practice in any other production pip requirements files you maintain. > > Does anyone in the Python world put that sort of thing in a separate file? > It seems perfectly reasonable to do something like: > > $ pip install -r requirements.txt > $ pip freeze -r requirements.txt > requirements.frozen > > and then to install the precise hash-pinned versions somewhere else: > > $ pip install -r requirements.frozen > > In fact this is precisely equivalent to what Bundler (for Ruby) and Yarn > (for Node.js) do. I'd much rather have one file that declares the > dependencies, and a _separate_ one that locks down the precise versions of > those and all transitive dependencies. > The Python Packaging Guide seems to suggest that dependencies should be declared with install_requires in setup.py and full environments should be declared with pip requirements files. https://packaging.python.org/requirements/#install-requires-vs-requirements-files I've heard of people parsing files (possibly requirements files) in setup.py in order to populate install_requires to avoid the duplication. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 4 website] base: add a requirements.txt
> On May 10, 2017, at 19:16, Gregory Szorcwrote: > >> On Wed, May 10, 2017 at 10:22 AM, Kevin Bullock >> wrote: >> # HG changeset patch >> # User Kevin Bullock >> # Date 1494433855 18000 >> # Wed May 10 11:30:55 2017 -0500 >> # Node ID b99251cd874a4a559ceadbd6fd33e6f411b1e776 >> # Parent 3faffcf56082846bd385f6594d9729bceceaf83b >> base: add a requirements.txt >> >> diff --git a/requirements.txt b/requirements.txt >> new file mode 100644 >> --- /dev/null >> +++ b/requirements.txt >> @@ -0,0 +1,1 @@ >> +Flask>=0.12.1,<0.13 >> > Modern versions of pip support pinning hashes in requirements files. It is > quite nice as it not only buffers you against MitM attacks, compromised > servers, and corruption, but also forces you to have hashes pinned for *all* > dependencies. In effect, it requires you to list *all* dependencies and gives > you peace of mind that your virtualenv is reproducible. I'd highly recommend > implementing that as a follow-up to this series and using that practice in > any other production pip requirements files you maintain. Does anyone in the Python world put that sort of thing in a separate file? It seems perfectly reasonable to do something like: $ pip install -r requirements.txt $ pip freeze -r requirements.txt > requirements.frozen and then to install the precise hash-pinned versions somewhere else: $ pip install -r requirements.frozen In fact this is precisely equivalent to what Bundler (for Ruby) and Yarn (for Node.js) do. I'd much rather have one file that declares the dependencies, and a _separate_ one that locks down the precise versions of those and all transitive dependencies. pacem in terris / мир / शान्ति / سَلاَم / 平和 Kevin R. Bullock ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 3] flagprocessor: migrate addflagprocessor to registerflagprocessor
# HG changeset patch # User Jun Wu# Date 1494457921 25200 # Wed May 10 16:12:01 2017 -0700 # Node ID a5a22d616f981e8c220f1c2fd4eae098e104a11c # Parent 1b4a17fefa2e67b6bf9294c9fbce586a6646bdaa # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r a5a22d616f98 flagprocessor: migrate addflagprocessor to registerflagprocessor This patch migrates all users of addflagprocessor to registerflagprocessor. So addflagprocessor is not used in the code base and could be removed when external code completes migration. diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -76,5 +76,5 @@ CensoredNodeError = error.CensoredNodeEr ProgrammingError = error.ProgrammingError -# Store flag processors (cf. 'addflagprocessor()' to register) +# Store flag processors (cf. 'registerflagprocessor()' to register) _flagprocessors = { REVIDX_ISCENSORED: None, diff --git a/mercurial/verify.py b/mercurial/verify.py --- a/mercurial/verify.py +++ b/mercurial/verify.py @@ -408,5 +408,5 @@ class verifier(object): # L1 should be equal to L2. L3 could be different from them. # "text" may or may not affect commit hash depending on flag -# processors (see revlog.addflagprocessor). +# processors (see revlog.registerflagprocessor). # # | common | rename | meta | ext diff --git a/tests/flagprocessorext.py b/tests/flagprocessorext.py --- a/tests/flagprocessorext.py +++ b/tests/flagprocessorext.py @@ -23,24 +23,18 @@ REVIDX_GZIP = (1 << 1) REVIDX_FAIL = 1 -def validatehash(self, text): -return True - -def bypass(self, text): -return False - def noopdonothing(self, text): -return (text, True) +return text def b64encode(self, text): -return (base64.b64encode(text), False) +return base64.b64encode(text) def b64decode(self, text): -return (base64.b64decode(text), True) +return base64.b64decode(text) def gzipcompress(self, text): -return (zlib.compress(text), False) +return zlib.compress(text) def gzipdecompress(self, text): -return (zlib.decompress(text), True) +return zlib.decompress(text) def supportedoutgoingversions(orig, repo): @@ -117,26 +111,17 @@ def extsetup(ui): # Register flag processors for each extension -revlog.addflagprocessor( +revlog.registerflagprocessor( REVIDX_NOOP, -( -noopdonothing, -noopdonothing, -validatehash, -) +noopdonothing, +noopdonothing, ) -revlog.addflagprocessor( +revlog.registerflagprocessor( REVIDX_BASE64, -( -b64decode, -b64encode, -bypass, -), +b64decode, +b64encode, ) -revlog.addflagprocessor( +revlog.registerflagprocessor( REVIDX_GZIP, -( -gzipdecompress, -gzipcompress, -bypass -) +gzipdecompress, +gzipcompress, ) diff --git a/tests/test-revlog-raw.py b/tests/test-revlog-raw.py --- a/tests/test-revlog-raw.py +++ b/tests/test-revlog-raw.py @@ -38,18 +38,13 @@ def readprocessor(self, rawtext): # True: the returned text could be used to verify hash text = rawtext[len(_extheader):].replace(b'i', b'1') -return text, True +return text def writeprocessor(self, text): # False: the returned rawtext shouldn't be used to verify hash rawtext = _extheader + text.replace(b'1', b'i') -return rawtext, False +return rawtext -def rawprocessor(self, rawtext): -# False: do not verify hash. Only the content returned by "readprocessor" -# can be used to verify hash. -return False - -revlog.addflagprocessor(revlog.REVIDX_EXTSTORED, -(readprocessor, writeprocessor, rawprocessor)) +revlog.registerflagprocessor(revlog.REVIDX_EXTSTORED, readprocessor, + writeprocessor) # Utilities about reading and appending revlog @@ -229,5 +224,5 @@ def writecases(rlog, tr): # Verify text, rawtext, and rawsize if isext: -rawtext = writeprocessor(None, text)[0] +rawtext = writeprocessor(None, text) else: rawtext = text ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 3] flagprocessor: simplify flag processor interface
# HG changeset patch # User Jun Wu# Date 1494455614 25200 # Wed May 10 15:33:34 2017 -0700 # Node ID 1b4a17fefa2e67b6bf9294c9fbce586a6646bdaa # Parent 1ada3d18e7fbc9069910f2c036992d2f2b28e058 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 1b4a17fefa2e flagprocessor: simplify flag processor interface This is a series of cleaning up flag processor to make it more solid and easier to use. The current flag processor API allows different "validatehash" result per revision, per read, write, raw "operation". That looks flexible but could cause confusing - read and write operations should not both return True for validatehash, if rawtext != text. "validatehash" is conceptually a static setting - pick one from ['text', 'rawtext'] and let mercurial hash that. Practically, there are no real users wanting to hash "rawtext" for now. And we don't really support hashing "rawtext" at present - it breaks "cmp" test which assumes "text" is used in hash. Instead of supporting the choice of hashing "rawtext" and adding overhead to "cmp" for non-existed yet usecase, I'd prefer dropping "rawtext" hashing for now - i.e. assume all flag processors want to hash "text" for now. If it's really useful, we can add it back as an optional argument in the future. This patch deprecates "addflagprocessor" and adds a new, simplified "registerflagprocessor" method in revlog.py. diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -81,27 +81,13 @@ ProgrammingError = error.ProgrammingErro } -def addflagprocessor(flag, processor): +def registerflagprocessor(flag, read, write): """Register a flag processor on a revision data flag. -Invariant: -- Flags need to be defined in REVIDX_KNOWN_FLAGS and REVIDX_FLAGS_ORDER. -- Only one flag processor can be registered on a specific flag. -- flagprocessors must be 3-tuples of functions (read, write, raw) with the - following signatures: - - (read) f(self, rawtext) -> text, bool - - (write) f(self, text) -> rawtext, bool - - (raw) f(self, rawtext) -> bool - "text" is presented to the user. "rawtext" is stored in revlog data, not - directly visible to the user. - The boolean returned by these transforms is used to determine whether - the returned text can be used for hash integrity checking. For example, - if "write" returns False, then "text" is used to generate hash. If - "write" returns True, that basically means "rawtext" returned by "write" - should be used to generate hash. Usually, "write" and "read" return - different booleans. And "raw" returns a same boolean as "write". +flag: int, should be defined in REVIDX_KNOWN_FLAGS, and REVIDX_FLAGS_ORDER +read: function, (revlog, rawtext) -> text +write: function, (revlog, text) -> rawtext - Note: The 'raw' transform is used for changegroup generation and in some - debug commands. In this case the transform only indicates whether the - contents can be used for hash integrity checks. +"text" is presented to the user (ex. working copy) and decides hash. +"rawtext" is stored in revlog data, used for exchange and bundle. """ if not flag & REVIDX_KNOWN_FLAGS: @@ -114,5 +100,12 @@ def addflagprocessor(flag, processor): msg = _("cannot register multiple processors on flag '%#x'.") % (flag) raise error.Abort(msg) -_flagprocessors[flag] = processor +_flagprocessors[flag] = (read, write) + +def addflagprocessor(flag, processor): +"""deprecated. please use registerflagprocessor instead""" +readtransform, writetransform, rawtransform = processor +read = lambda rlog, rawtext: readtransform(rlog, rawtext)[0] +write = lambda rlog, text: writetransform(rlog, text)[0] +registerflagprocessor(flag, read, write) def getoffset(q): @@ -1377,6 +1370,4 @@ class revlog(object): # related operation transform and update result tuple. if flag & flags: -vhash = True - if flag not in _flagprocessors: message = _("missing processor for flag '%#x'") % (flag) @@ -1385,13 +1376,13 @@ class revlog(object): processor = _flagprocessors[flag] if processor is not None: -readtransform, writetransform, rawtransform = processor +readtransform, writetransform = processor if raw: -vhash = rawtransform(self, text) +validatehash = False elif operation == 'read': -text, vhash = readtransform(self, text) +text = readtransform(self, text) else: # write operation -text, vhash = writetransform(self, text) -
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On Tue, 09 May 2017 19:55:57 -0400, Martin von Zweigbergk via Mercurial-develwrote: On Tue, May 9, 2017 at 4:34 PM, Durham Goode wrote: # HG changeset patch # User Durham Goode # Date 1494372571 25200 # Tue May 09 16:29:31 2017 -0700 # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f obsolete: add operation metadata to rebase/amend/histedit obsmarkers By recording what operation created the obsmarker, we can show very intuitive messages to the user in various UIs. For instance, log output could have messages like "Amended as XXX" to show why a commit is old and has an 'x' on it. Nice! I had previously assumed that that information was already in the obsmarker, but then I didn't find it in `hg debugobsolete`. I didn't realize it would be so easy to add. Seems like an obvious improvement. So obvious that I'm hesitating to queue it only because it seems like it should have been proposed before. Any idea why that hasn't happened? It's not obsolete specific, so maybe not relevant, but operation metadata made me recall this thread: https://www.mercurial-scm.org/pipermail/mercurial-devel/2015-April/068692.html @ ac28e3 durham / First commit | | o d4afe7 durham | | Second commit | | | x 8e9a5d (Amended as ac28e3) durham |/ First commit | diff --git a/hgext/histedit.py b/hgext/histedit.py --- a/hgext/histedit.py +++ b/hgext/histedit.py @@ -1631,7 +1631,7 @@ def safecleanupnode(ui, repo, name, node key=repo.changelog.rev) markers = [getmarker(t) for t in sortednodes] if markers: -obsolete.createmarkers(repo, markers) +obsolete.createmarkers(repo, markers, operation='histedit') else: return cleanupnode(ui, repo, name, nodes) diff --git a/hgext/rebase.py b/hgext/rebase.py --- a/hgext/rebase.py +++ b/hgext/rebase.py @@ -1341,7 +1341,7 @@ def clearrebased(ui, repo, state, skippe succs = (repo[newrev],) markers.append((repo[rev], succs)) if markers: -obsolete.createmarkers(repo, markers) +obsolete.createmarkers(repo, markers, operation='rebase') else: rebased = [rev for rev in state if state[rev] > nullmerge] if rebased: diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -2740,7 +2740,7 @@ def amend(ui, repo, commitfunc, old, ext if node: obs.append((ctx, ())) -obsolete.createmarkers(repo, obs) +obsolete.createmarkers(repo, obs, operation='amend') if not createmarkers and newid != old.node(): # Strip the intermediate commit (if there was one) and the amended # commit diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py --- a/mercurial/obsolete.py +++ b/mercurial/obsolete.py @@ -1204,7 +1204,8 @@ def _computedivergentset(repo): return divergent -def createmarkers(repo, relations, flag=0, date=None, metadata=None): +def createmarkers(repo, relations, flag=0, date=None, metadata=None, + operation=None): """Add obsolete markers between changesets in a repo must be an iterable of (, (, ...)[,{metadata}]) @@ -1225,6 +1226,8 @@ def createmarkers(repo, relations, flag= metadata = {} if 'user' not in metadata: metadata['user'] = repo.ui.username() +if operation: +metadata['operation'] = operation tr = repo.transaction('add-obsolescence-marker') try: markerargs = [] diff --git a/tests/test-histedit-obsolete.t b/tests/test-histedit-obsolete.t --- a/tests/test-histedit-obsolete.t +++ b/tests/test-histedit-obsolete.t @@ -50,10 +50,10 @@ Test that histedit learns about obsolesc o 0:cb9a9f314b8b a $ hg debugobsolete - e72d22b19f8ecf4150ab4f91d0973fd9955d3ddf 49d44ab2be1b67a79127568a67c9c99430633b48 0 (*) {'user': 'test'} (glob) - 3e30a45cf2f719e96ab3922dfe039cfd047956ce 0 {e72d22b19f8ecf4150ab4f91d0973fd9955d3ddf} (*) {'user': 'test'} (glob) - 1b2d564fad96311b45362f17c2aa855150efb35f 46abc7c4d8738e8563e577f7889e1b6db3da4199 0 (*) {'user': 'test'} (glob) - 114f4176969ef342759a8a57e6bccefc4234829b 49d44ab2be1b67a79127568a67c9c99430633b48 0 (*) {'user': 'test'} (glob) + e72d22b19f8ecf4150ab4f91d0973fd9955d3ddf 49d44ab2be1b67a79127568a67c9c99430633b48 0 (*) {'operation': 'amend', 'user': 'test'} (glob) + 3e30a45cf2f719e96ab3922dfe039cfd047956ce 0 {e72d22b19f8ecf4150ab4f91d0973fd9955d3ddf} (*) {'operation': 'amend', 'user': 'test'} (glob) + 1b2d564fad96311b45362f17c2aa855150efb35f 46abc7c4d8738e8563e577f7889e1b6db3da4199 0 (*) {'operation': 'histedit', 'user': 'test'} (glob) + 114f4176969ef342759a8a57e6bccefc4234829b
Re: [PATCH 2 of 4 website] base: add a requirements.txt
On Wed, May 10, 2017 at 10:22 AM, Kevin Bullock < kbullock+mercur...@ringworld.org> wrote: > # HG changeset patch > # User Kevin Bullock> # Date 1494433855 18000 > # Wed May 10 11:30:55 2017 -0500 > # Node ID b99251cd874a4a559ceadbd6fd33e6f411b1e776 > # Parent 3faffcf56082846bd385f6594d9729bceceaf83b > base: add a requirements.txt > > diff --git a/requirements.txt b/requirements.txt > new file mode 100644 > --- /dev/null > +++ b/requirements.txt > @@ -0,0 +1,1 @@ > +Flask>=0.12.1,<0.13 > Modern versions of pip support pinning hashes in requirements files. It is quite nice as it not only buffers you against MitM attacks, compromised servers, and corruption, but also forces you to have hashes pinned for *all* dependencies. In effect, it requires you to list *all* dependencies and gives you peace of mind that your virtualenv is reproducible. I'd highly recommend implementing that as a follow-up to this series and using that practice in any other production pip requirements files you maintain. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On 05/10/2017 05:45 PM, Durham Goode wrote: On 5/10/17 3:40 AM, Pierre-Yves David wrote: On 05/10/2017 01:34 AM, Durham Goode wrote: # HG changeset patch # User Durham Goode# Date 1494372571 25200 # Tue May 09 16:29:31 2017 -0700 # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f obsolete: add operation metadata to rebase/amend/histedit obsmarkers TL;DR; Storing more information is useful, but the pre-existing plans aims at storing slightly different data in a different way. We should stick to it. The idea is to use the obsmarker bitfields to store the data and record a pre-defined set of "effects" in each markers. (eg: "content change, message change, parent change") For examples: 1) a `hg amend` that update the diff will create a markers recording the "content changes". 2) a `hg amend -e` that only update he commit message will record the "message change". 3) a `hg rebase` will record the "parent change". 4) a `hg amend` that update the message, the content and user will record these three things (content, message and meta changes). These data can then be used to have nice template as shown by Durham (eg: is X is a successors of Y and the markers as some "content change" bit, we can display "amended as Y"; if their are some parent change, we can display "rebased as Y", if both are present "amended and rebased as Y"). While having the above data seems nice, I don't know if it justifies not going with the current simple approach today. Otherwise we have to wait for this large obsstore refactor, which seems like a rather big dependency to block an important UX feature we could deliver today. There seems to be a major misunderstanding here. Obsmarkers -already- has a bitfield (for many years) so are -already- able to use them to record actions. There are no delay or blocker here everything is already in place for it. There are basically no reason to not just do it. When the bit field is added in the future we can always change our templates and messaging to use that data as well. And if it's good enough, we can drop the 'operation' metadata entirely, with little compatibility issue. The above approach as multiple advantages: Compact: Since we use the (existing) bitfield, the storage cost is free (or minimal if we need to adds an extra byte of bitfield). Accessing the data is also very cheap: Compoundable: We'll often needs to combine multiple markers to reports changesets between two revisions (eg: we do not have the intermediate revisions locally). the bit field approach makes it trivial to compound the information. We can display display the same final information when the same result if obtained from different path (eg: from two markers [content change; message change] or from one markers [content change | message change]). This part is important because simply using operation name makes it quite hard to combine the information. Yes, if there were multiple operations we'd have to union them just as operation='modified' or something. Not ideal, but should be an outlier case. Having multiple markers between local changesets is the common case once people start exchanging draft changesets. We can not dismiss this as an outliers, behaving properly here is an actual request from the current users. UI Abstraction: since we record the "effect" information, we abstract the command names. This makes use more agile about the actual UI. User can use their own "hg myownrewritetool" command and still record information useful for the other users. I think recording the actual command the user ran is important. The goal of showing this in the UI is to let the user know what past action caused this commit to become a new commit, so if they run 'hg flabbernate' it's much more useful to show them 'fblabbernated to be X' than it is to show them 'amended to be X'. So potentially, even if we have the bitfield data, it could still be important to record the actual command that caused this obsmarker to be created (ex: "amended to be X by hg flabbernate") Having more information is always useful, but this looks like something to add as a second step. The action-bit approach has clear advantages (no size concerns, clear behavior with marker compositions), so this appears as a clear good first step here. In addition, if that first step turn out to not be enough and we want more precise data we might want to look at leveraging the "hg journal" local data for this. I would recommend the following step forward: 1) add a simple "rewritten as" template, 2) record action-bit, use it in the template (and other related output), 3) gather data and look at the best way to issue more details. By chance step (1) and (2) are on the list of UI improvement founded by Unity (1) is about to be done (read: next week) and (2) is likely to be scheduled in the next batch (read, this month). If you want to work
[PATCH] chgserver: more explicit about sensitive environ variables
# HG changeset patch # User Jun Wu# Date 1494442522 25200 # Wed May 10 11:55:22 2017 -0700 # Node ID f275d21ea713de70a8cae521d3afc1fbe497de25 # Parent 1ada3d18e7fbc9069910f2c036992d2f2b28e058 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r f275d21ea713 chgserver: more explicit about sensitive environ variables Environment variables like HGUSER, HGEDITOR, HGEDITFROM should not trigger a new chgserver. This patch uses a whitelist for environ variables starting with "HG" to reduce the number of servers. I have went through `grep -o "[\"']HG[A-Z_0-9]*['\"]" -hR . | sort -u` so the list should be up-to-date. diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py --- a/mercurial/chgserver.py +++ b/mercurial/chgserver.py @@ -76,5 +76,6 @@ def _hashlist(items): _envre = re.compile(r'''\A(?: CHGHG -|HG(?:[A-Z].*)? +|HG(?:DEMANDIMPORT|EMITWARNINGS|MODULEPOLICY|PROF|RCPATH)? +|HG(?:ENCODING|PLAIN).* |LANG(?:UAGE)? |LC_.* ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Augie Facklerwrites: >> On May 10, 2017, at 13:59, Sean Farley wrote: >> >> Should this include aliases too? My initial gut reaction is 'yes' since >> that would be what the user actually types and can help debug easier. > > Maybe. Typically (not at G and F, but otherwise) users will have a vague > sense of what their alias is configured to do. My thought is that users (myself included) have aliases for flags that change behavior to the command, e.g. amend_with_grandparent = amend -r .^^:: Seeing "amend_with_grandparent" would vastly help convey what I did as a user. Though, if this is too complicated, I'm totally ok with ditching it. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4 website] downloads: replace inline scripts with an onload function
Kevin Bullockwrites: >> On May 10, 2017, at 12:22, Kevin Bullock >> wrote: >> >> # HG changeset patch >> # User Kevin Bullock >> # Date 1494436785 18000 >> # Wed May 10 12:19:45 2017 -0500 >> # Node ID f194cff8d65af231bf2cb1780daaecef2b0ed3d3 >> # Parent 130a8e2a946d71eaf031420fa43e9a0d0f7a6e8d >> downloads: replace inline scripts with an onload function > > I neglected to mention in the commit message that this allows us to improve > our Content-Security-Policy usage (we should be able to remove > 'unsafe-inline'). I figured as much since I got notified from the bug tracker :-) signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4 website] downloads: replace inline scripts with an onload function
Kevin Bullockwrites: > # HG changeset patch > # User Kevin Bullock > # Date 1494436785 18000 > # Wed May 10 12:19:45 2017 -0500 > # Node ID f194cff8d65af231bf2cb1780daaecef2b0ed3d3 > # Parent 130a8e2a946d71eaf031420fa43e9a0d0f7a6e8d > downloads: replace inline scripts with an onload function > > diff --git a/static/js/download.js b/static/js/download.js > --- a/static/js/download.js > +++ b/static/js/download.js > @@ -132,3 +132,21 @@ var Downloader = { > return out; > } > }; > + > +(function (document, window) { > + var load = function () { > +Downloader.init(sources); > +var dl = Downloader.select(); > +var versElement = document.getElementById('download-version') > + , descElement = document.getElementById('download-description'); Man, I hate javascript. Anyway, queued! signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
> On May 10, 2017, at 13:59, Sean Farleywrote: > > Should this include aliases too? My initial gut reaction is 'yes' since > that would be what the user actually types and can help debug easier. Maybe. Typically (not at G and F, but otherwise) users will have a vague sense of what their alias is configured to do.___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 02 of 11] filectx: make renamed a property cache
Jun Wuwrites: > Excerpts from Sean Farley's message of 2017-05-10 10:09:39 -0700: >> Am I reading this right that this is _copied and not 'renamed'? Or is >> the diff hunk misaligned? > > They are the same. > > memfilectx uses the word "copied" so this minimizes churn. If we use > "_renamed", then memfilectx and its callers need update. Aha, fair enough. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 12 of 12 RFC] policy: add helper to import cext/pure module
Yuya Nishiharawrites: > On Tue, 9 May 2017 11:08:03 -0700, Jun Wu wrote: >> > > Patch 12 is where I disagree. _dualmod still requires careful renaming >> > > and >> > > could cause subtle problems that does not get warned by any code checking >> > > tools. They also require extra effort in writing and reviewing related >> > > code. >> > > I'd like to go through the explicit versioning way so even the renaming >> > > churn could be avoided. >> > > >> > > [1]: >> > > https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-May/097329.html >> > > >> > >> > I'm okay for the explicit versioning. I can adjust the last patch for that. >> > >> > I don't have strong preference since managing API versions would be as >> > simple as managing explicit version constants unless we have internal >> > dependency in C layer. >> >> We do have complex cases: module-level exception (mpatch.mpatchError), and >> parsers.c with complex Python objects. Think about if someone wants to catch >> mpatchError, or change the return value of lmiter_iterkeysnext in manifest.c >> for example. > > Yeah, I know. My opinion is that I won't care much about that as we don't > right now. FWIW, mpatchError isn't exported by the C module. > > That said, there seems no objection to switching to the per-module versioning, > so maybe we can go for it. Maybe would could have a discussion of all the pros and cons? If that's overkill, then I trust Yuya to make a good call here. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: San Francisco Meetup
Danek Duvallwrites: > Sean Farley wrote: > >> Just a quick followup to my suggestion of a mini hackathon in San >> Francisco (hopefully in a somewhat frequent manner). I was thinking of >> testing it out this Saturday for coffee / brunch: >> >> https://www.meetup.com/Bay-Area-Mercurial-Meetup/events/239870170/ >> >> It's a free-form thing. Suggestions on locations / time welcomed. If no >> one is interested, that's cool too. I'll be around the city hacking >> anyways :-) > > I'm in Boston for the OpenStack conference, so not this time. Alas! I was thinking of going to Arlequin Cafe since they have wifi and an outdoor patio. I'll try to do this type of thing once or twice a month (probably will schedule the next one for early June). signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
Augie Facklerwrites: >> On May 10, 2017, at 11:45, Durham Goode wrote: >> >> On 5/10/17 3:40 AM, Pierre-Yves David wrote: >>> On 05/10/2017 01:34 AM, Durham Goode wrote: # HG changeset patch # User Durham Goode # Date 1494372571 25200 # Tue May 09 16:29:31 2017 -0700 # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f obsolete: add operation metadata to rebase/amend/histedit obsmarkers >>> >>> TL;DR; Storing more information is useful, but the pre-existing plans >>> aims at storing slightly different data in a different way. We should >>> stick to it. >>> >>> The idea is to use the obsmarker bitfields to store the data and record >>> a pre-defined set of "effects" in each markers. (eg: "content change, >>> message change, parent change") >>> >>> For examples: >>> >>> 1) a `hg amend` that update the diff will create a markers recording the >>> "content changes". >>> >>> 2) a `hg amend -e` that only update he commit message will record the >>> "message change". >>> >>> 3) a `hg rebase` will record the "parent change". >>> >>> 4) a `hg amend` that update the message, the content and user will >>> record these three things (content, message and meta changes). >>> >>> These data can then be used to have nice template as shown by Durham >>> (eg: is X is a successors of Y and the markers as some "content change" >>> bit, we can display "amended as Y"; if their are some parent change, we >>> can display "rebased as Y", if both are present "amended and rebased as >>> Y"). >>> >> >> While having the above data seems nice, I don't know if it justifies not >> going with the current simple approach today. Otherwise we have to wait for >> this large obsstore refactor, which seems like a rather big dependency to >> block an important UX feature we could deliver today. > > I agree. This is something that we've observed would be a big help to our git > refugees at Google. I'd like to move forward with putting it in extra as a > string. > >> When the bit field is added in the future we can always change our templates >> and messaging to use that data as well. And if it's good enough, we can >> drop the 'operation' metadata entirely, with little compatibility issue. > > Yep. Sounds good to me. > >> >>> The above approach as multiple advantages: >>> >>> Compact: Since we use the (existing) bitfield, the storage cost is >>> free (or minimal if we need to adds an extra byte of bitfield). >>> Accessing the data is also very cheap: >>> >>> Compoundable: We'll often needs to combine multiple markers to reports >>> changesets between two revisions (eg: we do not have the intermediate >>> revisions locally). the bit field approach makes it trivial to compound >>> the information. We can display display the same final information when >>> the same result if obtained from different path (eg: from two markers >>> [content change; message change] or from one markers [content change | >>> message change]). >>> This part is important because simply using operation name makes it >>> quite hard to combine the information. >> >> Yes, if there were multiple operations we'd have to union them just as >> operation='modified' or something. Not ideal, but should be an outlier case. >> >>> UI Abstraction: since we record the "effect" information, we abstract >>> the command names. This makes use more agile about the actual UI. User >>> can use their own "hg myownrewritetool" command and still record >>> information useful for the other users. >> >> I think recording the actual command the user ran is important. The goal of >> showing this in the UI is to let the user know what past action caused this >> commit to become a new commit, so if they run 'hg flabbernate' it's much >> more useful to show them 'fblabbernated to be X' than it is to show them >> 'amended to be X'. So potentially, even if we have the bitfield data, it >> could still be important to record the actual command that caused this >> obsmarker to be created (ex: "amended to be X by hg flabbernate") > > I 100% agree here. The command verbs should map obviously onto the verbs that > are saved in the obsmarkers. Should this include aliases too? My initial gut reaction is 'yes' since that would be what the user actually types and can help debug easier. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 06 of 11] filectx: add an overlayfilectx class
Excerpts from Sean Farley's message of 2017-05-10 10:39:01 -0700: > I think I can live with this class for now. Eventually, though, I'd like > to consolidate (if possible) our use of metadataonly, workingcommitctx, > and this new overlayctx. Anyways, that can be done later. For changectx, we do plan to merge them. Phil could talk about more details with you tomorrow. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 11 of 11] histedit: use overlayfilectx
Jun Wuwrites: > # HG changeset patch > # User Jun Wu > # Date 1494399497 25200 > # Tue May 09 23:58:17 2017 -0700 > # Node ID 4eee1c8299c0c4cd48cd99e2bcc6dfcfcb755c9d > # Parent c8bef30e0a55b7f71add722b4f584bd813140eca > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 4eee1c8299c0 > histedit: use overlayfilectx > > Like the previous patch, this simplifies the code and could have perf win if > an involved flag processor is expensive. Good series to move this forward. I didn't see any big gotchas so it looks fine to me. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 06 of 11] filectx: add an overlayfilectx class
Jun Wuwrites: > # HG changeset patch > # User Jun Wu > # Date 1494386601 25200 > # Tue May 09 20:23:21 2017 -0700 > # Node ID 963d6fa49835ce66db6961ca31b02241c979cb99 > # Parent f736ea580edbf4ec199b55af586b31f2798cbee4 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 963d6fa49835 > filectx: add an overlayfilectx class > > The end goal is to make it possible to avoid potential expensive fctx.data() > when unnecessary. > > While memctx is useful for creating new file contexts, there are many cases > where we could reuse an existing raw file revision (amend, histedit, rebase, > process a revision constructed by a remote peer, etc). The overlayfilectx > class is made to support such reuse cases. Together with a later patch, hash > calculation and expensive flag processor could be avoided. I think I can live with this class for now. Eventually, though, I'd like to consolidate (if possible) our use of metadataonly, workingcommitctx, and this new overlayctx. Anyways, that can be done later. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4 website] downloads: replace inline scripts with an onload function
> On May 10, 2017, at 12:22, Kevin Bullock> wrote: > > # HG changeset patch > # User Kevin Bullock > # Date 1494436785 18000 > # Wed May 10 12:19:45 2017 -0500 > # Node ID f194cff8d65af231bf2cb1780daaecef2b0ed3d3 > # Parent 130a8e2a946d71eaf031420fa43e9a0d0f7a6e8d > downloads: replace inline scripts with an onload function I neglected to mention in the commit message that this allows us to improve our Content-Security-Policy usage (we should be able to remove 'unsafe-inline'). pacem in terris / мир / शान्ति / سَلاَم / 平和 Kevin R. Bullock ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 4 website] fix indentation of download button content
# HG changeset patch # User Kevin Bullock# Date 1494434808 18000 # Wed May 10 11:46:48 2017 -0500 # Node ID 130a8e2a946d71eaf031420fa43e9a0d0f7a6e8d # Parent b99251cd874a4a559ceadbd6fd33e6f411b1e776 fix indentation of download button content diff --git a/templates/base.html b/templates/base.html --- a/templates/base.html +++ b/templates/base.html @@ -50,9 +50,9 @@ {% block sidebar %} -Download now -Mercurial dl.write('version'); -dl.write('desc'); +Download now +Mercurial dl.write('version'); +dl.write('desc'); Another OS?Get Mercurial for: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 4 website] downloads: replace inline scripts with an onload function
# HG changeset patch # User Kevin Bullock# Date 1494436785 18000 # Wed May 10 12:19:45 2017 -0500 # Node ID f194cff8d65af231bf2cb1780daaecef2b0ed3d3 # Parent 130a8e2a946d71eaf031420fa43e9a0d0f7a6e8d downloads: replace inline scripts with an onload function diff --git a/static/js/download.js b/static/js/download.js --- a/static/js/download.js +++ b/static/js/download.js @@ -132,3 +132,21 @@ var Downloader = { return out; } }; + +(function (document, window) { + var load = function () { +Downloader.init(sources); +var dl = Downloader.select(); +var versElement = document.getElementById('download-version') + , descElement = document.getElementById('download-description'); +versElement.appendChild(document.createTextNode(dl.attr('version'))); +descElement.appendChild(document.createTextNode(dl.attr('desc'))); + } + + if (document.readyState !== 'complete') { +document.addEventListener('DOMContentLoaded', load); +return; + } + + load(); +})(document, window); diff --git a/templates/base.html b/templates/base.html --- a/templates/base.html +++ b/templates/base.html @@ -7,10 +7,6 @@ - - Downloader.init(sources); - var dl = Downloader.select(); - Mercurial SCM @@ -49,10 +45,10 @@ {% block sidebar %} - + Download now -Mercurial dl.write('version'); -dl.write('desc'); +Mercurial + Another OS?Get Mercurial for: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 4 website] downloads: ignore auto-generated sources.js
# HG changeset patch # User Kevin Bullock# Date 1494433322 18000 # Wed May 10 11:22:02 2017 -0500 # Node ID 3faffcf56082846bd385f6594d9729bceceaf83b # Parent 68ea82e8ff7cae15bae6eb9eb17c3572bffa8277 downloads: ignore auto-generated sources.js diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -4,3 +4,4 @@ syntax: glob .env out/** +sources.js ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 4 website] base: add a requirements.txt
# HG changeset patch # User Kevin Bullock# Date 1494433855 18000 # Wed May 10 11:30:55 2017 -0500 # Node ID b99251cd874a4a559ceadbd6fd33e6f411b1e776 # Parent 3faffcf56082846bd385f6594d9729bceceaf83b base: add a requirements.txt diff --git a/requirements.txt b/requirements.txt new file mode 100644 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,1 @@ +Flask>=0.12.1,<0.13 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 05 of 11] filectx: remove __new__
Jun Wuwrites: > # HG changeset patch > # User Jun Wu > # Date 1494382608 25200 > # Tue May 09 19:16:48 2017 -0700 > # Node ID f736ea580edbf4ec199b55af586b31f2798cbee4 > # Parent b022d125236c47e07db6209e4de648bb70a8de65 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > f736ea580edb > filectx: remove __new__ > > It does not seem to be used anywhere, and breaks a later patch. > > diff --git a/mercurial/context.py b/mercurial/context.py > --- a/mercurial/context.py > +++ b/mercurial/context.py > @@ -689,7 +689,4 @@ class basefilectx(object): > directory, > memfilectx: a filecontext that represents files in-memory.""" > -def __new__(cls, repo, path, *args, **kwargs): > -return super(basefilectx, cls).__new__(cls) > - I remember writing __new__ for basectx due to mpm and I talking about fast-pathing reusing a ctx without duplicating it. As for filectx, I seem to recall wanting to do something similar for self.data but ran into issues of cyclic dependencies later down the line. So, yes, this patch seems fine to me. signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 02 of 11] filectx: make renamed a property cache
Excerpts from Sean Farley's message of 2017-05-10 10:09:39 -0700: > Am I reading this right that this is _copied and not 'renamed'? Or is > the diff hunk misaligned? They are the same. memfilectx uses the word "copied" so this minimizes churn. If we use "_renamed", then memfilectx and its callers need update. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 02 of 11] filectx: make renamed a property cache
Jun Wuwrites: > # HG changeset patch > # User Jun Wu > # Date 1494384537 25200 > # Tue May 09 19:48:57 2017 -0700 > # Node ID 8fe9241cb76e0e1b91d6deaa9af81de3e97f696e > # Parent e4403b22312ef8d7a0d4cded7f23f5fc0753d50c > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 8fe9241cb76e > filectx: make renamed a property cache > > See previous patch for context - mainly to avoid code duplication. > > diff --git a/mercurial/context.py b/mercurial/context.py > --- a/mercurial/context.py > +++ b/mercurial/context.py > @@ -798,4 +798,6 @@ class basefilectx(object): > def changectx(self): > return self._changectx > +def renamed(self): > +return self._copied > def repo(self): > return self._repo > @@ -1150,5 +1152,6 @@ class filectx(basefilectx): > return self._filelog.size(self._filerev) > > -def renamed(self): > +@propertycache > +def _copied(self): Am I reading this right that this is _copied and not 'renamed'? Or is the diff hunk misaligned? signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On Wed, May 10, 2017 at 8:57 AM, Augie Facklerwrote: > >> On May 10, 2017, at 11:52, Martin von Zweigbergk >> wrote: >> Compoundable: We'll often needs to combine multiple markers to reports changesets between two revisions (eg: we do not have the intermediate revisions locally). the bit field approach makes it trivial to compound the information. We can display display the same final information when the same result if obtained from different path (eg: from two markers [content change; message change] or from one markers [content change | message change]). This part is important because simply using operation name makes it quite hard to combine the information. >>> >>> >>> Yes, if there were multiple operations we'd have to union them just as >>> operation='modified' or something. Not ideal, but should be an outlier case. >> >> With our setup internally, it's not really an outlier case. We tag >> revisions that are sent for code review and leave the tags until next >> time the change is sent for review. There can thus be many amends and >> rebases between the revision sent for review and the current latest >> revision. I'm not sure what a good way of presenting that to the user >> would be. > > I think the common case is rebased and amended - couldn't we observe the list > of actions in the list and emit something like > > deadbeef rebased, amended to beefdead > > in `hg log` output? > > I'd rather not let the perfect be the enemy of the good here - we can at > least get the data recorded and start experimenting with ways of operating on > it. It's already missing from countless thousands of markers, so we'll never > be able to assume it exists, so I feel like we might as well start recording > it and see if we need to iterate on how we're recording it from there. I agree. I didn't mean to use it as an argument against Durham's patch, just something to think about. And I think agree with you about how to do it. My initial thought was also to keep a set of all distinct operations along the chain. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On 5/10/17 8:52 AM, Martin von Zweigbergk wrote: On Wed, May 10, 2017 at 8:45 AM, Durham Goodewrote: On 5/10/17 3:40 AM, Pierre-Yves David wrote: Compoundable: We'll often needs to combine multiple markers to reports changesets between two revisions (eg: we do not have the intermediate revisions locally). the bit field approach makes it trivial to compound the information. We can display display the same final information when the same result if obtained from different path (eg: from two markers [content change; message change] or from one markers [content change | message change]). This part is important because simply using operation name makes it quite hard to combine the information. Yes, if there were multiple operations we'd have to union them just as operation='modified' or something. Not ideal, but should be an outlier case. With our setup internally, it's not really an outlier case. We tag revisions that are sent for code review and leave the tags until next time the change is sent for review. There can thus be many amends and rebases between the revision sent for review and the current latest revision. I'm not sure what a good way of presenting that to the user would be. Ah, yes. I was thinking about multiple markers between the same nodes, but the chain of commits that are not all visible is a common case (and is obviously what Pierre-Yves was talking about when I read closer). Yes, this case wouldn't be well handled by the operation string solution. One solution we've had internally is to have a smartlog output that shows all the successors to a node. So you can see "X was amended to be Y", "Y was rebased to be Z", "Z was landed as V". It makes it very easy to visually traverse the history of a node and could allow a user to dig in to a generic "X was modified to be V" unioned-message. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
> On May 10, 2017, at 11:52, Martin von Zweigbergk> wrote: > >>> Compoundable: We'll often needs to combine multiple markers to reports >>> changesets between two revisions (eg: we do not have the intermediate >>> revisions locally). the bit field approach makes it trivial to compound >>> the information. We can display display the same final information when >>> the same result if obtained from different path (eg: from two markers >>> [content change; message change] or from one markers [content change | >>> message change]). >>> This part is important because simply using operation name makes it >>> quite hard to combine the information. >> >> >> Yes, if there were multiple operations we'd have to union them just as >> operation='modified' or something. Not ideal, but should be an outlier case. > > With our setup internally, it's not really an outlier case. We tag > revisions that are sent for code review and leave the tags until next > time the change is sent for review. There can thus be many amends and > rebases between the revision sent for review and the current latest > revision. I'm not sure what a good way of presenting that to the user > would be. I think the common case is rebased and amended - couldn't we observe the list of actions in the list and emit something like deadbeef rebased, amended to beefdead in `hg log` output? I'd rather not let the perfect be the enemy of the good here - we can at least get the data recorded and start experimenting with ways of operating on it. It's already missing from countless thousands of markers, so we'll never be able to assume it exists, so I feel like we might as well start recording it and see if we need to iterate on how we're recording it from there. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On Wed, May 10, 2017 at 8:45 AM, Durham Goodewrote: > On 5/10/17 3:40 AM, Pierre-Yves David wrote: >> >> On 05/10/2017 01:34 AM, Durham Goode wrote: >>> >>> # HG changeset patch >>> # User Durham Goode >>> # Date 1494372571 25200 >>> # Tue May 09 16:29:31 2017 -0700 >>> # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 >>> # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f >>> obsolete: add operation metadata to rebase/amend/histedit obsmarkers >> >> >> TL;DR; Storing more information is useful, but the pre-existing plans >> aims at storing slightly different data in a different way. We should >> stick to it. >> >> The idea is to use the obsmarker bitfields to store the data and record >> a pre-defined set of "effects" in each markers. (eg: "content change, >> message change, parent change") >> >> For examples: >> >> 1) a `hg amend` that update the diff will create a markers recording the >> "content changes". >> >> 2) a `hg amend -e` that only update he commit message will record the >> "message change". >> >> 3) a `hg rebase` will record the "parent change". >> >> 4) a `hg amend` that update the message, the content and user will >> record these three things (content, message and meta changes). >> >> These data can then be used to have nice template as shown by Durham >> (eg: is X is a successors of Y and the markers as some "content change" >> bit, we can display "amended as Y"; if their are some parent change, we >> can display "rebased as Y", if both are present "amended and rebased as >> Y"). >> > > While having the above data seems nice, I don't know if it justifies not > going with the current simple approach today. Otherwise we have to wait for > this large obsstore refactor, which seems like a rather big dependency to > block an important UX feature we could deliver today. > > When the bit field is added in the future we can always change our templates > and messaging to use that data as well. And if it's good enough, we can > drop the 'operation' metadata entirely, with little compatibility issue. > >> The above approach as multiple advantages: >> >> Compact: Since we use the (existing) bitfield, the storage cost is >> free (or minimal if we need to adds an extra byte of bitfield). >> Accessing the data is also very cheap: >> >> Compoundable: We'll often needs to combine multiple markers to reports >> changesets between two revisions (eg: we do not have the intermediate >> revisions locally). the bit field approach makes it trivial to compound >> the information. We can display display the same final information when >> the same result if obtained from different path (eg: from two markers >> [content change; message change] or from one markers [content change | >> message change]). >> This part is important because simply using operation name makes it >> quite hard to combine the information. > > > Yes, if there were multiple operations we'd have to union them just as > operation='modified' or something. Not ideal, but should be an outlier case. With our setup internally, it's not really an outlier case. We tag revisions that are sent for code review and leave the tags until next time the change is sent for review. There can thus be many amends and rebases between the revision sent for review and the current latest revision. I'm not sure what a good way of presenting that to the user would be. > >> UI Abstraction: since we record the "effect" information, we abstract >> the command names. This makes use more agile about the actual UI. User >> can use their own "hg myownrewritetool" command and still record >> information useful for the other users. > > > I think recording the actual command the user ran is important. The goal of > showing this in the UI is to let the user know what past action caused this > commit to become a new commit, so if they run 'hg flabbernate' it's much > more useful to show them 'fblabbernated to be X' than it is to show them > 'amended to be X'. So potentially, even if we have the bitfield data, it > could still be important to record the actual command that caused this > obsmarker to be created (ex: "amended to be X by hg flabbernate") > > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
> On May 10, 2017, at 11:45, Durham Goodewrote: > > On 5/10/17 3:40 AM, Pierre-Yves David wrote: >> On 05/10/2017 01:34 AM, Durham Goode wrote: >>> # HG changeset patch >>> # User Durham Goode >>> # Date 1494372571 25200 >>> # Tue May 09 16:29:31 2017 -0700 >>> # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 >>> # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f >>> obsolete: add operation metadata to rebase/amend/histedit obsmarkers >> >> TL;DR; Storing more information is useful, but the pre-existing plans >> aims at storing slightly different data in a different way. We should >> stick to it. >> >> The idea is to use the obsmarker bitfields to store the data and record >> a pre-defined set of "effects" in each markers. (eg: "content change, >> message change, parent change") >> >> For examples: >> >> 1) a `hg amend` that update the diff will create a markers recording the >> "content changes". >> >> 2) a `hg amend -e` that only update he commit message will record the >> "message change". >> >> 3) a `hg rebase` will record the "parent change". >> >> 4) a `hg amend` that update the message, the content and user will >> record these three things (content, message and meta changes). >> >> These data can then be used to have nice template as shown by Durham >> (eg: is X is a successors of Y and the markers as some "content change" >> bit, we can display "amended as Y"; if their are some parent change, we >> can display "rebased as Y", if both are present "amended and rebased as >> Y"). >> > > While having the above data seems nice, I don't know if it justifies not > going with the current simple approach today. Otherwise we have to wait for > this large obsstore refactor, which seems like a rather big dependency to > block an important UX feature we could deliver today. I agree. This is something that we've observed would be a big help to our git refugees at Google. I'd like to move forward with putting it in extra as a string. > When the bit field is added in the future we can always change our templates > and messaging to use that data as well. And if it's good enough, we can drop > the 'operation' metadata entirely, with little compatibility issue. Yep. Sounds good to me. > >> The above approach as multiple advantages: >> >> Compact: Since we use the (existing) bitfield, the storage cost is >> free (or minimal if we need to adds an extra byte of bitfield). >> Accessing the data is also very cheap: >> >> Compoundable: We'll often needs to combine multiple markers to reports >> changesets between two revisions (eg: we do not have the intermediate >> revisions locally). the bit field approach makes it trivial to compound >> the information. We can display display the same final information when >> the same result if obtained from different path (eg: from two markers >> [content change; message change] or from one markers [content change | >> message change]). >> This part is important because simply using operation name makes it >> quite hard to combine the information. > > Yes, if there were multiple operations we'd have to union them just as > operation='modified' or something. Not ideal, but should be an outlier case. > >> UI Abstraction: since we record the "effect" information, we abstract >> the command names. This makes use more agile about the actual UI. User >> can use their own "hg myownrewritetool" command and still record >> information useful for the other users. > > I think recording the actual command the user ran is important. The goal of > showing this in the UI is to let the user know what past action caused this > commit to become a new commit, so if they run 'hg flabbernate' it's much more > useful to show them 'fblabbernated to be X' than it is to show them 'amended > to be X'. So potentially, even if we have the bitfield data, it could still > be important to record the actual command that caused this obsmarker to be > created (ex: "amended to be X by hg flabbernate") I 100% agree here. The command verbs should map obviously onto the verbs that are saved in the obsmarkers. AF ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On 5/10/17 3:40 AM, Pierre-Yves David wrote: On 05/10/2017 01:34 AM, Durham Goode wrote: # HG changeset patch # User Durham Goode# Date 1494372571 25200 # Tue May 09 16:29:31 2017 -0700 # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f obsolete: add operation metadata to rebase/amend/histedit obsmarkers TL;DR; Storing more information is useful, but the pre-existing plans aims at storing slightly different data in a different way. We should stick to it. The idea is to use the obsmarker bitfields to store the data and record a pre-defined set of "effects" in each markers. (eg: "content change, message change, parent change") For examples: 1) a `hg amend` that update the diff will create a markers recording the "content changes". 2) a `hg amend -e` that only update he commit message will record the "message change". 3) a `hg rebase` will record the "parent change". 4) a `hg amend` that update the message, the content and user will record these three things (content, message and meta changes). These data can then be used to have nice template as shown by Durham (eg: is X is a successors of Y and the markers as some "content change" bit, we can display "amended as Y"; if their are some parent change, we can display "rebased as Y", if both are present "amended and rebased as Y"). While having the above data seems nice, I don't know if it justifies not going with the current simple approach today. Otherwise we have to wait for this large obsstore refactor, which seems like a rather big dependency to block an important UX feature we could deliver today. When the bit field is added in the future we can always change our templates and messaging to use that data as well. And if it's good enough, we can drop the 'operation' metadata entirely, with little compatibility issue. The above approach as multiple advantages: Compact: Since we use the (existing) bitfield, the storage cost is free (or minimal if we need to adds an extra byte of bitfield). Accessing the data is also very cheap: Compoundable: We'll often needs to combine multiple markers to reports changesets between two revisions (eg: we do not have the intermediate revisions locally). the bit field approach makes it trivial to compound the information. We can display display the same final information when the same result if obtained from different path (eg: from two markers [content change; message change] or from one markers [content change | message change]). This part is important because simply using operation name makes it quite hard to combine the information. Yes, if there were multiple operations we'd have to union them just as operation='modified' or something. Not ideal, but should be an outlier case. UI Abstraction: since we record the "effect" information, we abstract the command names. This makes use more agile about the actual UI. User can use their own "hg myownrewritetool" command and still record information useful for the other users. I think recording the actual command the user ran is important. The goal of showing this in the UI is to let the user know what past action caused this commit to become a new commit, so if they run 'hg flabbernate' it's much more useful to show them 'fblabbernated to be X' than it is to show them 'amended to be X'. So potentially, even if we have the bitfield data, it could still be important to record the actual command that caused this obsmarker to be created (ex: "amended to be X by hg flabbernate") ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH V2 STABLE] serve: move hg-ssh readonly logic into hg serve
On Tue, 9 May 2017 15:48:50 -0700, Durham Goode wrote: > On 4/28/17 6:27 AM, Yuya Nishihara wrote: > > On Thu, 27 Apr 2017 10:13:21 -0400, Augie Fackler wrote: > >> On Wed, Apr 26, 2017 at 06:24:52PM -0700, Durham Goode wrote: > >>> # HG changeset patch > >>> # User Durham Goode> >>> # Date 1493255976 25200 > >>> # Wed Apr 26 18:19:36 2017 -0700 > >>> # Branch stable > >>> # Node ID b1964bbc387fb53b4152f19d6e929309e3f21ac6 > >>> # Parent 6e0368b6e0bb2aa5210daec091c0200583553a78 > >>> serve: move hg-ssh readonly logic into hg serve > >> > >> This looks good, but I'm very hesitant to add a new feature like this > >> 4 days before a release. Does anyone feel strongly that this should be > >> in 4.2? > > > > Agreed. The feature seems good, but no need to hurry to pick it up. > > > >> +if realcmd == 'serve' and '--read-only' in req.args: > >> +req.args.remove('--read-only') > > > > This increases the risk of wrong command parsing. We can't say --read-only > > is always a flag. > > > >> +req.ui.setconfig('hooks', 'pretxnopen.readonlyrejectpush', > >> + rejectpush, 'dispatch') > >> +req.ui.setconfig('hooks', 'prepushkey.readonlyrejectpush', > >> + rejectpush, 'dispatch') > > > > And --read-only won't work as expected in command server since there are > > write > > operations other than push. > > Are there write operations that wouldn't trigger the pretxnopen hook? Perhaps, "hg rollback" ? > Alternatively, I could rename this flag to be > --add-hgssh-push-blocking-hooks, remove it from the hg server command > definition entirely (just watch for it at the dispatch level and strip > it like I currently do). That way we can maintain the current pattern > while letting it be a special path for hg-ssh only. > > And if someone passes --add-hgssh-push-blocking-hooks to hg serve in a > position that isn't a flag, well it'll just break for them. Another idea is adding --unsafe-stdio option so the frontend script may do anything on the 'hg serve --stdio' process. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4 stateful-chg] commandserver: allow service to process handler's heartbeat result
On Tue, 9 May 2017 10:37:25 -0700, Jun Wu wrote: > Excerpts from Yuya Nishihara's message of 2017-05-09 22:57:23 +0900: > > Does it mean we can't avoid sub-classing the service class? > > If that's a goal, a possible solution is to move "_sock" state from service > to handler. The goal was to split application logic of chg from the service class so that it would be reusable in prefork model (if any.) I have no idea how and for what _handleheartbeat() will be used. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 5] perf: don't convert rev to node before calling revlog.revision()
On Wed, 10 May 2017 01:07:50 -0700, Jun Wu wrote: > Excerpts from Yuya Nishihara's message of 2017-05-09 22:23:03 +0900: > > > diff --git a/contrib/perf.py b/contrib/perf.py > > > --- a/contrib/perf.py > > > +++ b/contrib/perf.py > > > @@ -865,7 +865,7 @@ def perfrevlog(ui, repo, file_=None, sta > > > dist = -1 * dist > > > > > > for x in xrange(beginrev, endrev, dist): > > > -r.revision(r.node(x)) > > > +r.revision(x) > > > > IIRC, this was kept unchanged for some reason. Jun? > > Yes. tl;dr hg earlier than 9117c6561b0b does not support nodeorrev. > > See https://patchwork.mercurial-scm.org/patch/19836/ Thanks. Gregory, can you send a follow-up patch? I don't want to drop the other patches because of this. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 12 of 12 RFC] policy: add helper to import cext/pure module
On Tue, 9 May 2017 11:08:03 -0700, Jun Wu wrote: > > > Patch 12 is where I disagree. _dualmod still requires careful renaming and > > > could cause subtle problems that does not get warned by any code checking > > > tools. They also require extra effort in writing and reviewing related > > > code. > > > I'd like to go through the explicit versioning way so even the renaming > > > churn could be avoided. > > > > > > [1]: > > > https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-May/097329.html > > > > > > > I'm okay for the explicit versioning. I can adjust the last patch for that. > > > > I don't have strong preference since managing API versions would be as > > simple as managing explicit version constants unless we have internal > > dependency in C layer. > > We do have complex cases: module-level exception (mpatch.mpatchError), and > parsers.c with complex Python objects. Think about if someone wants to catch > mpatchError, or change the return value of lmiter_iterkeysnext in manifest.c > for example. Yeah, I know. My opinion is that I won't care much about that as we don't right now. FWIW, mpatchError isn't exported by the C module. That said, there seems no objection to switching to the per-module versioning, so maybe we can go for it. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: San Francisco Meetup
Sean Farley wrote: > Just a quick followup to my suggestion of a mini hackathon in San > Francisco (hopefully in a somewhat frequent manner). I was thinking of > testing it out this Saturday for coffee / brunch: > > https://www.meetup.com/Bay-Area-Mercurial-Meetup/events/239870170/ > > It's a free-form thing. Suggestions on locations / time welcomed. If no > one is interested, that's cool too. I'll be around the city hacking > anyways :-) I'm in Boston for the OpenStack conference, so not this time. Danek ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 stable] graft: fix graft across merges of duplicates of grafted changes
On Tue, 09 May 2017 00:45:02 +0200, Mads Kiilerich wrote: > # HG changeset patch > # User Mads Kiilerich> # Date 1494283376 -7200 > # Tue May 09 00:42:56 2017 +0200 > # Branch stable > # Node ID dd256de590dfd363fa5d497d566d456f471b8d52 > # Parent 6710017995b4e8b361d6ad5b897ff7d0cc658285 > graft: fix graft across merges of duplicates of grafted changes Looks good to me. A couple of nits inline. > Graft used findmissingrevs to find the candidates for graft duplicates in the > destination. That function operates with the constraint: > > 2. N is not an ancestor of any node in 'common' > > For our purpose, we do however need: > > 2. There are nodes in 'common' that doesn't have N as ancestor > > The right candiates for graft duplicates could be computed with a revset: > > only(destination,transitiveroots(graftset)) > > where transitiveroots would be a revset function for 'transitive root' and > return changesets in set with no ancestor changeset in set. There doesn't seem > to be any such function readily available, and we thus use the approximation > of > just using the smallest revision in the graft set. Can you copy this message as a code comment? It will help future readers. > This change will graft more correctly, but it will also in some cases make > graft slower by making it search through a bigger and unnecessary large sets > of > changes to find duplicates. Suppose revisions to be grafted are linear in general, I think this is acceptable. > @@ -2295,7 +2295,8 @@ def _dograft(ui, repo, *revs, **opts): > # check ancestors for earlier grafts > ui.debug('scanning for duplicate grafts\n') > > -for rev in repo.changelog.findmissingrevs(revs, [crev]): > +expr = revsetlang.formatspec('only(%d,min(%ld))', crev, revs) > +for rev in scmutil.revrange(repo, [expr]): scmutil.revrange() may expand user aliases. Please use repo.revs() instead. Alternatively, maybe we could use findmissingrevs(min(revs), ...) to minimize the change? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] obsolete: add operation metadata to rebase/amend/histedit obsmarkers
On 05/10/2017 01:34 AM, Durham Goode wrote: # HG changeset patch # User Durham Goode# Date 1494372571 25200 # Tue May 09 16:29:31 2017 -0700 # Node ID 22051b0924bcbc5cde6d25d700dcce6afcaafd98 # Parent 8f1a2b848b52ea7bf3fe2404e3b62924c7aae93f obsolete: add operation metadata to rebase/amend/histedit obsmarkers TL;DR; Storing more information is useful, but the pre-existing plans aims at storing slightly different data in a different way. We should stick to it. The idea is to use the obsmarker bitfields to store the data and record a pre-defined set of "effects" in each markers. (eg: "content change, message change, parent change") For examples: 1) a `hg amend` that update the diff will create a markers recording the "content changes". 2) a `hg amend -e` that only update he commit message will record the "message change". 3) a `hg rebase` will record the "parent change". 4) a `hg amend` that update the message, the content and user will record these three things (content, message and meta changes). These data can then be used to have nice template as shown by Durham (eg: is X is a successors of Y and the markers as some "content change" bit, we can display "amended as Y"; if their are some parent change, we can display "rebased as Y", if both are present "amended and rebased as Y"). The above approach as multiple advantages: Compact: Since we use the (existing) bitfield, the storage cost is free (or minimal if we need to adds an extra byte of bitfield). Accessing the data is also very cheap: Compoundable: We'll often needs to combine multiple markers to reports changesets between two revisions (eg: we do not have the intermediate revisions locally). the bit field approach makes it trivial to compound the information. We can display display the same final information when the same result if obtained from different path (eg: from two markers [content change; message change] or from one markers [content change | message change]). This part is important because simply using operation name makes it quite hard to combine the information. UI Abstraction: since we record the "effect" information, we abstract the command names. This makes use more agile about the actual UI. User can use their own "hg myownrewritetool" command and still record information useful for the other users. I've updated the wiki[1] with the above and and updated the existing entry in the Roadmap to point to it. [1] https://www.mercurial-scm.org/wiki/ChangesetEvolutionDevel#Record_types_of_operation Beside the details of what data we store and how, there are useful intermediate steps we can take here. The "amended as 134506abcdef" message is nice but we won't always have the data to precisely determine the web here (eg: old client not collecting data, old markers, etc). So starting with a neutral "rewritten as 134506abcdef" would be useful. We could get that first, independently of the data collection. Such change will belong well to the evolve extension first. There are multiple test case that can be reused, we can ship it to more user to get feedback sooner and we'll likely adjust it regularly at the beginning. So please submit it there. Good starting point would be the 'test-evolve-obshistory.t' file and the 'hg debugobshistory' command. By recording what operation created the obsmarker, we can show very intuitive messages to the user in various UIs. For instance, log output could have messages like "Amended asXXX" to show why a commit is old and has an 'x' on it. @ ac28e3 durham / First commit | | o d4afe7 durham | | Second commit | | | x 8e9a5d (Amended as ac28e3) durham |/ First commit | diff --git a/hgext/histedit.py b/hgext/histedit.py --- a/hgext/histedit.py +++ b/hgext/histedit.py @@ -1631,7 +1631,7 @@ def safecleanupnode(ui, repo, name, node key=repo.changelog.rev) markers = [getmarker(t) for t in sortednodes] if markers: -obsolete.createmarkers(repo, markers) +obsolete.createmarkers(repo, markers, operation='histedit') else: return cleanupnode(ui, repo, name, nodes) diff --git a/hgext/rebase.py b/hgext/rebase.py --- a/hgext/rebase.py +++ b/hgext/rebase.py @@ -1341,7 +1341,7 @@ def clearrebased(ui, repo, state, skippe succs = (repo[newrev],) markers.append((repo[rev], succs)) if markers: -obsolete.createmarkers(repo, markers) +obsolete.createmarkers(repo, markers, operation='rebase') else: rebased = [rev for rev in state if state[rev] > nullmerge] if rebased: diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -2740,7 +2740,7 @@ def amend(ui, repo, commitfunc, old, ext if node: obs.append((ctx, ())) -
[Bug 5563] New: Too many bounce on mercurial devel mailing list
https://bz.mercurial-scm.org/show_bug.cgi?id=5563 Bug ID: 5563 Summary: Too many bounce on mercurial devel mailing list Product: Mercurial Version: unspecified Hardware: PC OS: Linux Status: UNCONFIRMED Severity: feature Priority: wish Component: infrastructure Assignee: bugzi...@mercurial-scm.org Reporter: lothiral...@gmail.com CC: kbullock+mercur...@ringworld.org, mercurial-devel@mercurial-scm.org I've registered with the email 'boris.f...@octobus.net' to the mercurial mailing-list and I saw today that I was unsubscripted because of too many bounces. Do you have more informations about the bounces so I can fix the problem, thanks. -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 10 of 11] amend: use overlayfilectx
# HG changeset patch # User Jun Wu# Date 1494397814 25200 # Tue May 09 23:30:14 2017 -0700 # Node ID c8bef30e0a55b7f71add722b4f584bd813140eca # Parent 9d4e0dc68196d113a7dd153748eea1cc731734d3 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r c8bef30e0a55 amend: use overlayfilectx This simplifies the code and could speed things up, especially when expensive flag processors are involved. diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -2658,13 +2658,9 @@ def amend(ui, repo, commitfunc, old, ext try: fctx = ctx[path] -flags = fctx.flags() -mctx = context.memfilectx(repo, - fctx.path(), fctx.data(), - islink='l' in flags, - isexec='x' in flags, - copied=copied.get(path)) -return mctx except KeyError: return None +else: +c = copied.get(path, False) +return context.overlayfilectx(fctx, copied=c) else: ui.note(_('copying changeset %s to %s\n') % (old, base)) diff --git a/tests/test-commit-amend.t b/tests/test-commit-amend.t --- a/tests/test-commit-amend.t +++ b/tests/test-commit-amend.t @@ -1201,6 +1201,4 @@ File node could be reused committed changeset 2:92bc7a9d76f010337ece134e095054c094d44760 -(but is not smart enough to reuse manifest-only changes) - $ chmod +x b $ hg ci --debug --amend -m 'without content change' @@ -1214,5 +1212,7 @@ File node could be reused committing files: a + reusing a filelog node (exact match) b + reusing b filelog node (parent match) committing manifest committing changelog ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 08 of 11] filecommit: reuse filenode matching fparents regardless of type(fctx)
# HG changeset patch # User Jun Wu# Date 1494387835 25200 # Tue May 09 20:43:55 2017 -0700 # Node ID e4ad86c15b06986e51d4754f3c11bd24cbae30bb # Parent eb7b96bc42d08670983d5d75635198f62c3459a8 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r e4ad86c15b06 filecommit: reuse filenode matching fparents regardless of type(fctx) context.filectx is not the only class providing reusable filenode. overlayfilectx may provide reusable filenode too. So drop the isinstance check. Other filectx classes (memfilectx and workingfilectx) are subclasses of committablefilectx, which has set filenode to None. So filenode won't be reused as expected. The debug message is changed slightly to be distinguish from other kind of reuses introduced in a future patch. diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py --- a/mercurial/localrepo.py +++ b/mercurial/localrepo.py @@ -1423,11 +1423,10 @@ class localrepository(object): fparent1 = manifest1.get(fname, nullid) fparent2 = manifest2.get(fname, nullid) -if isinstance(fctx, context.filectx): -node = fctx.filenode() -if node in [fparent1, fparent2]: -self.ui.debug('reusing %s filelog entry\n' % fname) -if manifest1.flags(fname) != fctx.flags(): -changelist.append(fname) -return node +node = getattr(fctx, 'filenode', lambda: None)() +if node is not None and node in [fparent1, fparent2]: +self.ui.debug('reusing %s filelog node (parent match)\n' % fname) +if manifest1.flags(fname) != fctx.flags(): +changelist.append(fname) +return node flog = self.file(fname) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 06 of 11] filectx: add an overlayfilectx class
# HG changeset patch # User Jun Wu# Date 1494386601 25200 # Tue May 09 20:23:21 2017 -0700 # Node ID 963d6fa49835ce66db6961ca31b02241c979cb99 # Parent f736ea580edbf4ec199b55af586b31f2798cbee4 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 963d6fa49835 filectx: add an overlayfilectx class The end goal is to make it possible to avoid potential expensive fctx.data() when unnecessary. While memctx is useful for creating new file contexts, there are many cases where we could reuse an existing raw file revision (amend, histedit, rebase, process a revision constructed by a remote peer, etc). The overlayfilectx class is made to support such reuse cases. Together with a later patch, hash calculation and expensive flag processor could be avoided. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -688,5 +688,7 @@ class basefilectx(object): workingfilectx: a filecontext that represents files from the working directory, -memfilectx: a filecontext that represents files in-memory.""" +memfilectx: a filecontext that represents files in-memory, +overlayfilectx: duplicate another filecontext with some fields overridden. +""" @propertycache def _filelog(self): @@ -2079,4 +2081,75 @@ class memfilectx(committablefilectx): self._data = data +class overlayfilectx(committablefilectx): +"""Like memfilectx but take an original filectx and optional parameters to +override parts of it. This is useful when fctx.data() is expensive (i.e. +flag processor is expensive) and raw data, flags, and filenode could be +reused (ex. rebase or mode-only amend a REVIDX_EXTSTORED file). +""" + +def __init__(self, originalfctx, datafunc=None, path=None, flags=None, + copied=None, ctx=None): +"""originalfctx: filecontext to duplicate + +datafunc: None or a function to override data (file content). It is a +function to be lazy. path, flags, copied, ctx: None or overridden value + +copied could be (path, rev), or False. copied could also be just path, +and will be converted to (path, nullid). This simplifies some callers. +""" + +if path is None: +path = originalfctx.path() +if ctx is None: +ctx = originalfctx.changectx() +ctxmatch = lambda: True +else: +ctxmatch = lambda: ctx == originalfctx.changectx() + +repo = originalfctx.repo() +flog = originalfctx.filelog() +super(overlayfilectx, self).__init__(repo, path, flog, ctx) + +if copied is None: +copied = originalfctx.renamed() +copiedmatch = lambda: True +else: +if copied and not isinstance(copied, tuple): +# repo._filecommit will recalculate copyrev so nullid is okay +copied = (copied, nullid) +copiedmatch = lambda: copied == originalfctx.renamed() + +# When data, copied (could affect data), ctx (could affect filelog +# parents) are not overridden, rawdata, rawflags, and filenode may be +# reused (repo._filecommit should double check filelog parents). +# +# path, flags are not hashed in filelog (but in manifestlog) so they do +# not affect reusable here. +# +# If ctx or copied is overridden to a same value with originalfctx, +# still consider it's reusable. originalfctx.renamed() may be a bit +# expensive so it's not called unless necessary. Assuming datafunc is +# always expensive, do not call it for this "reusable" test. +reusable = datafunc is None and ctxmatch() and copiedmatch() + +if datafunc is None: +datafunc = originalfctx.data +if flags is None: +flags = originalfctx.flags() + +self._datafunc = datafunc +self._flags = flags +self._copied = copied + +if reusable: +# copy extra fields from originalfctx +attrs = ['rawdata', 'rawflags', '_filenode', '_filerev'] +for attr in attrs: +if util.safehasattr(originalfctx, attr): +setattr(self, attr, getattr(originalfctx, attr)) + +def data(self): +return self._datafunc() + class metadataonlyctx(committablectx): """Like memctx but it's reusing the manifest of different commit. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 09 of 11] filecommit: add a fast path to reuse raw revision data
# HG changeset patch # User Jun Wu# Date 1494402182 25200 # Wed May 10 00:43:02 2017 -0700 # Node ID 9d4e0dc68196d113a7dd153748eea1cc731734d3 # Parent e4ad86c15b06986e51d4754f3c11bd24cbae30bb # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 9d4e0dc68196 filecommit: add a fast path to reuse raw revision data The high-level idea is similar to metadataonlyctx. If filelog parents and metadata match, then raw revision data (rawtext, rawflags, hash) could be reused. This saves time calculating hash or going through flag processors. diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py --- a/mercurial/localrepo.py +++ b/mercurial/localrepo.py @@ -1433,4 +1433,5 @@ class localrepository(object): meta = {} copy = fctx.renamed() +metamatched = not bool(copy) if copy and copy[0] != fname: # Mark the new revision of this file as a copy of another @@ -1476,4 +1477,5 @@ class localrepository(object): meta["copy"] = cfname meta["copyrev"] = hex(crev) +metamatched = (crev == copy[1]) fparent1, fparent2 = nullid, newfparent else: @@ -1491,4 +1493,22 @@ class localrepository(object): fparent2 = nullid +# reuse rawdata? (skip flagprocessors and hash calculation) +if (metamatched and node is not None +# some filectxs do not support rawdata or flags +and util.safehasattr(fctx, 'rawdata') +and util.safehasattr(fctx, 'rawflags') +# some (external) filelogs do not have addrawrevision +and util.safehasattr(flog, 'addrawrevision') +# parents must match to be able to reuse rawdata +and fctx.filelog().parents(node) == (fparent1, fparent2)): +# node is different from fparents, no need to check manifest flag +changelist.append(fname) +if node in getattr(flog, 'nodemap', ()): +self.ui.debug('reusing %s filelog node (exact match)\n' % fname) +return node +self.ui.debug('reusing %s filelog rawdata\n' % fname) +return flog.addrawrevision(fctx.rawdata(), tr, linkrev, fparent1, + fparent2, node, fctx.rawflags()) + # is the file changed? text = fctx.data() diff --git a/tests/test-commit-amend.t b/tests/test-commit-amend.t --- a/tests/test-commit-amend.t +++ b/tests/test-commit-amend.t @@ -1179,2 +1179,41 @@ Test if amend preserves executable bit c #endif + +File node could be reused + + $ cd $TESTTMP + $ hg init reuse-test + $ cd reuse-test + $ echo 1 > a + $ echo 2 > b + $ hg commit -m 12 -A a b + $ echo 3 >> a + $ hg commit -m 3 + + $ hg ci --debug --amend -m 'without content change' + amending changeset 0bd823dca296 + copying changeset 0bd823dca296 to dd3d87f356df + committing files: + a + reusing a filelog node (exact match) + committing manifest + committing changelog + committed changeset 2:92bc7a9d76f010337ece134e095054c094d44760 + +(but is not smart enough to reuse manifest-only changes) + + $ chmod +x b + $ hg ci --debug --amend -m 'without content change' + invalid branchheads * (glob) (?) + amending changeset 92bc7a9d76f0 + committing files: + b + committing manifest + committing changelog + copying changeset 2bfd5c45b3f5 to dd3d87f356df + committing files: + a + b + committing manifest + committing changelog + committed changeset 4:ba954a28eb454eb63e7348349f8e87e7b1be3601 diff --git a/tests/test-context.py b/tests/test-context.py --- a/tests/test-context.py +++ b/tests/test-context.py @@ -147,2 +147,23 @@ print(actx2.status(other=wcctx, listclean=True)) print('wcctx._status=%s' % (str(wcctx._status))) + +# copy filectx from repo to repo2 using overlayfilectx and memctx +print('=== filelog rawdata reuse ===') +os.chdir('..') +u.setconfig('ui', 'debug', '1') +repo2 = hg.repository(u, 'test2', create=1) + +def copyctx(newrepo, ctx): +cl = newrepo.changelog +p1 = cl.node(len(cl) - 1) +files = ctx.files() +desc = 'copied: %s' % ctx.description() +def getfctx(repo, memctx, path): +if path not in ctx: +return None +return context.overlayfilectx(ctx[path]) +return context.memctx(newrepo, [p1, None], desc, files, getfctx) + +for rev in repo: +print('copying rev %d from test1 to test2' % rev) +copyctx(repo2, repo[rev]).commit() diff --git a/tests/test-context.py.out b/tests/test-context.py.out --- a/tests/test-context.py.out +++ b/tests/test-context.py.out @@ -45,2 +45,23 @@ wcctx._status= wcctx._status= +=== filelog rawdata reuse === +copying rev 0 from test1 to test2 +committing files: +foo +reusing foo filelog rawdata +committing manifest +committing changelog +copying rev 1 from test1 to test2 +committing files: +foo +reusing
[PATCH 01 of 11] filectx: make flags a property cache
# HG changeset patch # User Jun Wu# Date 1494383008 25200 # Tue May 09 19:23:28 2017 -0700 # Node ID e4403b22312ef8d7a0d4cded7f23f5fc0753d50c # Parent 1ada3d18e7fbc9069910f2c036992d2f2b28e058 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r e4403b22312e filectx: make flags a property cache Make basefilectx._flags a property cache, so basefilectx.flags() could be reasonably reused. This avoids code duplication between memfilectx and a class added in a later patch. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -763,6 +763,9 @@ class basefilectx(object): def filenode(self): return self._filenode +@propertycache +def _flags(self): +return self._changectx.flags(self._path) def flags(self): -return self._changectx.flags(self._path) +return self._flags def filelog(self): return self._filelog @@ -2062,6 +2065,4 @@ class memfilectx(committablefilectx): def size(self): return len(self.data()) -def flags(self): -return self._flags def renamed(self): return self._copied ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 04 of 11] filectx: add a rawflags method
# HG changeset patch # User Jun Wu# Date 1494372852 25200 # Tue May 09 16:34:12 2017 -0700 # Node ID b022d125236c47e07db6209e4de648bb70a8de65 # Parent 86e994114f0e5971007b92af1aa7322c91c4cd16 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r b022d125236c filectx: add a rawflags method The new method returns the low-level revlog flag. We already have "rawdata" so a "rawflags" makes sense. Both "rawflags" and "rawdata" will be used in a later patch. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -1142,4 +1142,8 @@ class filectx(basefilectx): return self._filelog.revision(self._filenode, raw=True) +def rawflags(self): +"""low-level revlog flags""" +return self._filelog.flags(self._filerev) + def data(self): try: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 05 of 11] filectx: remove __new__
# HG changeset patch # User Jun Wu# Date 1494382608 25200 # Tue May 09 19:16:48 2017 -0700 # Node ID f736ea580edbf4ec199b55af586b31f2798cbee4 # Parent b022d125236c47e07db6209e4de648bb70a8de65 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r f736ea580edb filectx: remove __new__ It does not seem to be used anywhere, and breaks a later patch. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -689,7 +689,4 @@ class basefilectx(object): directory, memfilectx: a filecontext that represents files in-memory.""" -def __new__(cls, repo, path, *args, **kwargs): -return super(basefilectx, cls).__new__(cls) - @propertycache def _filelog(self): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 02 of 11] filectx: make renamed a property cache
# HG changeset patch # User Jun Wu# Date 1494384537 25200 # Tue May 09 19:48:57 2017 -0700 # Node ID 8fe9241cb76e0e1b91d6deaa9af81de3e97f696e # Parent e4403b22312ef8d7a0d4cded7f23f5fc0753d50c # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 8fe9241cb76e filectx: make renamed a property cache See previous patch for context - mainly to avoid code duplication. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -798,4 +798,6 @@ class basefilectx(object): def changectx(self): return self._changectx +def renamed(self): +return self._copied def repo(self): return self._repo @@ -1150,5 +1152,6 @@ class filectx(basefilectx): return self._filelog.size(self._filerev) -def renamed(self): +@propertycache +def _copied(self): """check if file was actually renamed in this changeset revision @@ -2065,6 +2068,4 @@ class memfilectx(committablefilectx): def size(self): return len(self.data()) -def renamed(self): -return self._copied def remove(self, ignoremissing=False): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 03 of 11] filectx: move size to basefilectx
# HG changeset patch # User Jun Wu# Date 1494384811 25200 # Tue May 09 19:53:31 2017 -0700 # Node ID 86e994114f0e5971007b92af1aa7322c91c4cd16 # Parent 8fe9241cb76e0e1b91d6deaa9af81de3e97f696e # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 86e994114f0e filectx: move size to basefilectx See previous patch for context - avoid code duplication. diff --git a/mercurial/context.py b/mercurial/context.py --- a/mercurial/context.py +++ b/mercurial/context.py @@ -802,4 +802,6 @@ class basefilectx(object): def repo(self): return self._repo +def size(self): +return len(self.data()) def path(self): @@ -2066,6 +2068,4 @@ class memfilectx(committablefilectx): def data(self): return self._data -def size(self): -return len(self.data()) def remove(self, ignoremissing=False): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 5] perf: don't convert rev to node before calling revlog.revision()
Excerpts from Yuya Nishihara's message of 2017-05-09 22:23:03 +0900: > > diff --git a/contrib/perf.py b/contrib/perf.py > > --- a/contrib/perf.py > > +++ b/contrib/perf.py > > @@ -865,7 +865,7 @@ def perfrevlog(ui, repo, file_=None, sta > > dist = -1 * dist > > > > for x in xrange(beginrev, endrev, dist): > > -r.revision(r.node(x)) > > +r.revision(x) > > IIRC, this was kept unchanged for some reason. Jun? Yes. tl;dr hg earlier than 9117c6561b0b does not support nodeorrev. See https://patchwork.mercurial-scm.org/patch/19836/ ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
San Francisco Meetup
Just a quick followup to my suggestion of a mini hackathon in San Francisco (hopefully in a somewhat frequent manner). I was thinking of testing it out this Saturday for coffee / brunch: https://www.meetup.com/Bay-Area-Mercurial-Meetup/events/239870170/ It's a free-form thing. Suggestions on locations / time welcomed. If no one is interested, that's cool too. I'll be around the city hacking anyways :-) signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel