Re: Making chg stateful

2017-02-06 Thread Jun Wu
Excerpts from Yuya Nishihara's message of 2017-02-05 13:59:32 +0900: > Perhaps that is a matter of taste, I like the explicit one. But I heard > Python 3 has a generator-based coroutine, so using "yield" for that purpose > is more common in Python 3? > > @preload('index') > def preloadindex

Re: Making chg stateful

2017-02-05 Thread Jun Wu
Excerpts from Yuya Nishihara's message of 2017-02-05 13:31:29 +0900: > > > Since we shouldn't pass repo to revlog (it's layering violation), I think > > > we'll need a thin wrapper for chgcache anyway. > > > > I mentioned this in the second mail, "4) Where to get preloaded results (in > > worker)"

Re: Making chg stateful

2017-02-04 Thread Yuya Nishihara
On Fri, 3 Feb 2017 19:50:48 +, Jun Wu wrote: > Excerpts from Yuya Nishihara's message of 2017-02-04 00:31:45 +0900: > > On Fri, 3 Feb 2017 09:33:32 +, Jun Wu wrote: > > > 1) What does the worker talk to the master via IPC? > > > > > > Current: worker sends "possible_dirty_repo: $REPO_PAT

Re: Making chg stateful

2017-02-04 Thread Yuya Nishihara
On Fri, 3 Feb 2017 20:03:18 +, Jun Wu wrote: > Excerpts from Yuya Nishihara's message of 2017-02-04 00:11:22 +0900: > > On Thu, 2 Feb 2017 16:56:11 +, Jun Wu wrote: > > > Excerpts from Yuya Nishihara's message of 2017-02-03 00:45:22 +0900: > > > > On Thu, 2 Feb 2017 09:34:47 +, Jun Wu w

Re: Making chg stateful

2017-02-03 Thread Jun Wu
Excerpts from Yuya Nishihara's message of 2017-02-04 00:11:22 +0900: > On Thu, 2 Feb 2017 16:56:11 +, Jun Wu wrote: > > Excerpts from Yuya Nishihara's message of 2017-02-03 00:45:22 +0900: > > > On Thu, 2 Feb 2017 09:34:47 +, Jun Wu wrote: > > > > So what state do we store? > > > > > > > >

Re: Making chg stateful

2017-02-03 Thread Jun Wu
Excerpts from Yuya Nishihara's message of 2017-02-04 00:31:45 +0900: > On Fri, 3 Feb 2017 09:33:32 +, Jun Wu wrote: > > 1) What does the worker talk to the master via IPC? > > > > Current: worker sends "possible_dirty_repo: $REPO_PATH" after runcommand > > > > It's coupled with the (pse

Re: Making chg stateful

2017-02-03 Thread Yuya Nishihara
On Fri, 3 Feb 2017 09:33:32 +, Jun Wu wrote: > 1) What does the worker talk to the master via IPC? > > Current: worker sends "possible_dirty_repo: $REPO_PATH" after runcommand > > It's coupled with the (pseudo) repo object. But the master could preload > things in correct order - fi

Re: Making chg stateful

2017-02-03 Thread Yuya Nishihara
On Thu, 2 Feb 2017 16:56:11 +, Jun Wu wrote: > Excerpts from Yuya Nishihara's message of 2017-02-03 00:45:22 +0900: > > On Thu, 2 Feb 2017 09:34:47 +, Jun Wu wrote: > > > So what state do we store? > > > > > > {repopath: {name: (hash, content)}}. For example: > > > > > > cache = {'/

Re: Making chg stateful

2017-02-03 Thread Jun Wu
To be clear, the whole preloading framework needs design decisions made in different places. I'll go through them one by one: 1) What does the worker talk to the master via IPC? Current: worker sends "possible_dirty_repo: $REPO_PATH" after runcommand It's coupled with the (pseudo) repo obj

Re: Making chg stateful

2017-02-02 Thread Jun Wu
Excerpts from Yuya Nishihara's message of 2017-02-03 00:45:22 +0900: > On Thu, 2 Feb 2017 09:34:47 +, Jun Wu wrote: > > Perf Numbers > > > > I wrote a hacky prototype [2] that shows significant improvements on > > various commands in our repo: > > > >Befo

Re: Making chg stateful

2017-02-02 Thread Jun Wu
Excerpts from Simon Farnsworth's message of 2017-02-02 15:25:45 +: > On 02/02/2017 09:34, Jun Wu wrote: > > > > What's the API? > > > > (This is an existing implementation detail. I'm open to any ideas) > > > > For example, let's say we want to preload the changelog index (code > > simp

Re: Making chg stateful

2017-02-02 Thread Yuya Nishihara
On Thu, 2 Feb 2017 09:34:47 +, Jun Wu wrote: > Perf Numbers > > I wrote a hacky prototype [2] that shows significant improvements on > various commands in our repo: > >Before After (in seconds) > chg bookmark 0.40 0.08 > chg

Re: Making chg stateful

2017-02-02 Thread Simon Farnsworth
On 02/02/2017 09:34, Jun Wu wrote: What's the API? (This is an existing implementation detail. I'm open to any ideas) For example, let's say we want to preload the changelog index (code simplified so it does not take care of all corner cases). First, tell chg how to hash and load som

Re: Making chg stateful

2017-02-02 Thread Jun Wu
Fixes a mysterious typo that affects reading. Excerpts from Jun Wu's message of 2017-02-02 09:34:47 +: > This is mainly to discuss more details about chg repo preloading idea [1]. > > Perf Numbers > > I wrote a hacky prototype [2] that shows significant improvements on > various commands

Making chg stateful

2017-02-02 Thread Jun Wu
This is mainly to discuss more details about chg repo preloading idea [1]. Perf Numbers I wrote a hacky prototype [2] that shows significant improvements on various commands in our repo: Before After (in seconds) chg bookmark 0.40 0.08