Google Summer of Code 2017.
Hello everyone, Google Summer of Code 2016 was a good experience for both me and Piotr. We learned a lot. With GSoC 2017 near, students have started asking me about Mercurial taking part in GSoC 2017. Taking part in GSoC is good as we get new contributors (I am one of them). Are we going to take part in GSoC 2017? Regards Pulkit Goyal ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2 stable] debian: update mailing list address
# HG changeset patch # User Anton Shestakov # Date 1486211353 -28800 # Sat Feb 04 20:29:13 2017 +0800 # Branch stable # Node ID dc22729b4ebd0a8b4d9a747c1b346a71b55bdd0f # Parent b1c9f9d366f1037dcbb2a9468ee46e738a722c26 debian: update mailing list address diff --git a/contrib/debian/changelog b/contrib/debian/changelog --- a/contrib/debian/changelog +++ b/contrib/debian/changelog @@ -2,4 +2,4 @@ mercurial (__VERSION__) __CODENAME__; ur * Automated build performed by upstream. - -- Mercurial Devel __DATE__ + -- Mercurial Devel __DATE__ diff --git a/contrib/debian/control b/contrib/debian/control --- a/contrib/debian/control +++ b/contrib/debian/control @@ -1,7 +1,7 @@ Source: mercurial Section: vcs Priority: optional -Maintainer: Mercurial Developers +Maintainer: Mercurial Developers Build-Depends: debhelper (>= 9), dh-python, ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 stable] debian: update copyright years
# HG changeset patch # User Anton Shestakov # Date 1486211374 -28800 # Sat Feb 04 20:29:34 2017 +0800 # Branch stable # Node ID af3b5aa61fc05a124697809bf472a5592f38489c # Parent dc22729b4ebd0a8b4d9a747c1b346a71b55bdd0f debian: update copyright years diff --git a/contrib/debian/copyright b/contrib/debian/copyright --- a/contrib/debian/copyright +++ b/contrib/debian/copyright @@ -3,7 +3,7 @@ Upstream-Name: mercurial Source: https://www.mercurial-scm.org/ Files: * -Copyright: 2005-2015, Matt Mackall and others. +Copyright: 2005-2017, Matt Mackall and others. License: GPL-2+ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 stable] debian: update copyright years
Anton Shestakov writes: > # HG changeset patch > # User Anton Shestakov > # Date 1486211374 -28800 > # Sat Feb 04 20:29:34 2017 +0800 > # Branch stable > # Node ID af3b5aa61fc05a124697809bf472a5592f38489c > # Parent dc22729b4ebd0a8b4d9a747c1b346a71b55bdd0f > debian: update copyright years Looks good to me. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Making chg stateful
On Fri, 3 Feb 2017 20:03:18 +, Jun Wu wrote:
> Excerpts from Yuya Nishihara's message of 2017-02-04 00:11:22 +0900:
> > On Thu, 2 Feb 2017 16:56:11 +, Jun Wu wrote:
> > > Excerpts from Yuya Nishihara's message of 2017-02-03 00:45:22 +0900:
> > > > On Thu, 2 Feb 2017 09:34:47 +, Jun Wu wrote:
> > > > > So what state do we store?
> > > > >
> > > > > {repopath: {name: (hash, content)}}. For example:
> > > > >
> > > > > cache = {'/home/foo/repo1': {'index': ('hash', changelogindex),
> > > > > 'bookmarks': ('hash', bookmarks),
> > > > > },
> > > > > '/home/foo/repo2': { }, }
> > > > >
> > > > > The main ideas here are:
> > > > > 1) Store the lowest level objects, like the C changelog index.
> > > > >Because higher level objects could be changed by extensions in
> > > > >unpredictable ways. (this is not true in my hacky prototype
> > > > > though)
> > > > > 2) Hash everything. For changelog, it's like the file stat of
> > > > >changelog.i. There must be a strong guarantee that the hash
> > > > > matches
> > > > >the content, which could be challenging, but not impossible.
> > > > > I'll
> > > > >cover more details below.
> > > > >
> > > > > The cache is scoped by repo to make the API simpler/easy to use. It
> > > > > may
> > > > > be interesting to have some global state (like passing back the
> > > > > extension
> > > > > path to import them at runtime).
> > > >
> > > > Regarding this and "2) Side-effect-free repo", can or should we design
> > > > the API
> > > > as something like a low-level storage interface? That will allow us to
> > > > not
> > > > make repo/revlog know too much about chg.
> > > >
> > > > I don't have any concrete idea, but that would work as follows:
> > > >
> > > > 1. chg injects an object to select storage backends
> > > > e.g. repo.storage = chgpreloadable(repo.storage, cache)
> > > > 2. repo passes it to revlog, etc.
> > > > 3. revlog uses it to read indexfile, where in-memory cache may be
> > > > returned
> > > > e.g. storage.parserevlog(indexfile)
> > > >
> > > > Perhaps, this 'storage' object is similar to the one you call
> > > > 'baserepository'.
> > >
> > > I'm not sure if I get the idea (probably not). How does the implementation
> > > in the master server look like?
> >
> > I was just thinking about how to hack the real repo object without
> > introducing
> > much mess. Perhaps the master server wouldn't be that different from your
> > idea.
> >
> > > It feels more like "repo.chgcache" to me and the difference is that the
> > > vanilla hg will be changed to access objects via it (so the interface
> > > looks
> > > more consistent).
> >
> > Yeah, it might be like repo.chgcache.
> >
> > Since we shouldn't pass repo to revlog (it's layering violation), I think
> > we'll need a thin wrapper for chgcache anyway.
>
> I mentioned this in the second mail, "4) Where to get preloaded results (in
> worker)", we could just expose some kind of global state, like a
> "globalcache" module.
Does it mean any low-level objects will directly access to the global cache?
That seems ugly (but maybe I'm biased as I'm really allergic to global data.)
> > > Things to consider:
> > >
> > > a) Objects being preloaded have dependency - ex. the obsstore depends on
> > > changelog and other things. The preload functions run in a defined
> > > order.
> >
> > Maybe dependencies could be passed as arguments?
>
> Ideally, these expensive calculating (ex. obsstore) could be moved to the
> index object. In the reality, that requires too much work, and obsstore
> preloading requires a subset of "repo", including "repo.revs".
>
> It's possible to decouple obsstore preloading from the repo object, but that
> could be a lot of work too.
My opinion for obsstore is that the most costly part would be populating 100k+
objects from file, and the other costly parts could be mitigated by some higher-
level cache in repoview.py.
But I think this topic was discussed thoroughly between you and pyd before.
I'm not intended to bring it up again.
> > > b) The index file is not always a single file, depending on "vfs".
> >
> > Yes. vfs could be owned by storage/chgcache class.
> >
> > > c) The user may want to control what to preload. For example, if they
> > > have
> > > an incompatible manifest, they could make changelog preloaded, but
> > > not
> > > manifest.
> >
> > No idea about (c).
> >
> > > d) Users can add other preloading items easily, not only just the
> > > predefined ones.
> >
> > So probably we'll need an extensible table of preloadable items.
>
> If you check my prototype code, it's using a registrar to collect all
> @preload functions.
Yes. I wanted to say we would need this kind of abstraction anyway.
___
Mercurial-deve
Re: Making chg stateful
On Fri, 3 Feb 2017 19:50:48 +, Jun Wu wrote:
> Excerpts from Yuya Nishihara's message of 2017-02-04 00:31:45 +0900:
> > On Fri, 3 Feb 2017 09:33:32 +, Jun Wu wrote:
> > > 1) What does the worker talk to the master via IPC?
> > >
> > > Current: worker sends "possible_dirty_repo: $REPO_PATH" after runcommand
> > >
> > > It's coupled with the (pseudo) repo object. But the master could
> > > preload
> > > things in correct order - first changelog, then phase, obsstore, etc.
> > >
> > > What if: worker sends "possible_dirty_index: $INDEX_PATH",
> > > "possible_dirty_obsstore $OBSSTORE_PATH" etc.
> > >
> > > It looks more flexible at first sight. However the dependency issue
> > > could be tricky to solve - for example, the obsstore requires
> > > changelog
> > > (and revset). It's much harder to preload obsstore without a pseudo
> > > repo object with repo.changelog, repo.revs available.
> > >
> > > Therefore, probably just send the repo path.
> >
> > Given this, I think pipes would be simpler than using shared memory. The
> > master
> > IO can be multiplexed by select().
>
> If it's one pipe per worker, the code change (I tried) is too much. I think
> a single non-blocking pipe could be an option.
Yep. As we'll rely on atomic write(), we won't have to create pipe per worker.
> > > 2) Preload function signature (used by master's background thread).
> > >
> > > This function is to hash and load things. It runs in the master, and is
> > > basically a generator (so the "loading" part could be skipped on cache
> > > hit, and it's easier to make content and hash_with_content consistent,
> > > if
> > > they share states):
> > >
> > > @preloadregistar(something)
> > > def preloadfunc(...):
> > > yield quick_hash
> > > yield content, hash_with_content
> >
> > Isn't it simpler if preloadfunc() returns quick_hash and a delayed function
> > to build content? I feel this generator is kinda abuse and makes it less
> > clear
> > where the costly part starts.
>
> I have thought about this. "Generator" is used as a lightweight process that
> could be terminated cheaply. It shares states (local variables) for hashing
> and loading, making certain things easier.
>
> Compare:
>
> @preload('index')
> def preloadindex(repo):
> f = repo.svfs('changelog.i') # "f" can be shared naturally
> hash = fstat(f).st_size
> yield hash
>
> indexdata = f.read()
> hash = len(indexdata)
> yield parseindex(indexdata), hash
>
> vs.
>
> def loadindex(f):
> indexdata = f.read()
> hash = len(indexdata)
> return parseindex(indexdata), hash
>
> @preload('index')
> def preloadindex(repo):
> f = repo.svfs('changelog.i')
> hash = repo.svfs('changelog.i').st_size
> loadfunc = lambda: loadindex(f) # pass "f" explicitly
> return hash, loadfunc
>
> Generator seems to be the shortest way to express the logic.
> I think it's easy to say that things after the first "yield" is costly.
>
> For readibility, I think improved form mentioned at the end of the previous
> mail "yield preload.foo(...)" makes things obvious.
Perhaps that is a matter of taste, I like the explicit one. But I heard
Python 3 has a generator-based coroutine, so using "yield" for that purpose
is more common in Python 3?
@preload('index')
def preloadindex(repo):
f = repo.svfs('changelog.i') # "f" can be shared naturally
hash = fstat(f).st_size
def loadindex():
indexdata = f.read()
hash = len(indexdata)
return parseindex(indexdata), hash
return hash, loadindex
BTW, when will 'f' be closed if we take the generator-based abstraction?
def preloadindex(repo):
f = repo.svfs('changelog.i')
try:
yield ...
yield ...
finally:
f.close()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH v3] util: always force line buffered stdout when stdout is a tty (BC)
On Fri, 3 Feb 2017 15:15:06 -0800, Simon Farnsworth wrote: > # HG changeset patch > # User Simon Farnsworth > # Date 1486163427 28800 > # Fri Feb 03 15:10:27 2017 -0800 > # Node ID 08b6e1bb20ca6bd1465a3d22bab49debfe776bae > # Parent 1f51b4658f21bbb797e922d155c1046eddccf91d > util: always force line buffered stdout when stdout is a tty (BC) Queued, thanks. > diff --git a/mercurial/util.py b/mercurial/util.py > --- a/mercurial/util.py > +++ b/mercurial/util.py > @@ -63,6 +63,18 @@ > urlreq = pycompat.urlreq > xmlrpclib = pycompat.xmlrpclib > > +def isatty(fp): > +try: > +return fp.isatty() > +except AttributeError: > +return False > + > +# glibc determines buffering on first write to stdout - if we replace a TTY > +# destined stdout with a pipe destined stdout (e.g. pager), we want line > +# buffering > +if isatty(stdout): > +stdout = os.fdopen(stdout.fileno(), 'wb', 1) > + > if pycompat.osname == 'nt': > from . import windows as platform > stdout = platform.winstdout(pycompat.stdout) Fixed this in flight (otherwise stdout would be closed by GC): if pycompat.osname == 'nt': from . import windows as platform -stdout = platform.winstdout(pycompat.stdout) +stdout = platform.winstdout(stdout) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] cmdutil: remove forwarding methods per deprecation policy
On Fri, 03 Feb 2017 16:03:07 -0500, Augie Fackler wrote: > # HG changeset patch > # User Augie Fackler > # Date 1486155679 18000 > # Fri Feb 03 16:01:19 2017 -0500 > # Node ID 88a67501322af79870758125007ecd1a71c6fc89 > # Parent 1791be8a95c5d59cb8dd96b1479e24ad4470a9b6 > cmdutil: remove forwarding methods per deprecation policy Queued this, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2 stable] debian: update mailing list address
On Sat, 04 Feb 2017 20:34:21 +0800, Anton Shestakov wrote: > # HG changeset patch > # User Anton Shestakov > # Date 1486211353 -28800 > # Sat Feb 04 20:29:13 2017 +0800 > # Branch stable > # Node ID dc22729b4ebd0a8b4d9a747c1b346a71b55bdd0f > # Parent b1c9f9d366f1037dcbb2a9468ee46e738a722c26 > debian: update mailing list address Queued these, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
