On Fri, Aug 18, 2017 at 10:40 AM, RjOllos <[email protected]> wrote:
>
> On Thursday, August 17, 2017 at 3:38:40 PM UTC-4, Jun Omae wrote:
>>
>> Enabling [git] persistent_cache, it checks whether it needs to refresh
>> the cache when Repository.sync() is invoked via explicit
>> synchronization. If explicit synchronization is not set up, new
>> commits in the repository wouldn't be shown.
>>
>> Disabling [git] persistent_cache, it checks whether it needs to
>> refresh the cache each web request. That is `git show-ref` command is
>> invoked each web request. However, new commits in the repository would
>> be shown even if no explicit synchronization.
>
>
> We have a sync_per_request attribute that determines when the db cache is
> synchronized. If you've setup explicit synchronization then I assume you'd
> want persistent_cache = True and sync_per_request = False. If you haven't
> setup explicit synchronization, you'd want persistent_cache = False and
> sync_per_request = True.
>
> Would it make sense to just remove the persistent_cache option and use
> sync_per_request to determine whether the Git revision cache is
> reconstructed on every web request?
I consider that is not good.
GitCachedRepository.sync() checks whether each revision of the
repository is cached in revision table to execute a SELECT query. That
would lead performance down if too many revisions.
However, I think we could improve the sync() for already synchronized
cached git repository, to check each ref is cached before checking
whether all revisions are cached. I agree removing persistent_cache
option if improving sync() by the following patch.
This patch is for 1.0-stable:
diff --git a/tracopt/versioncontrol/git/PyGIT.py
b/tracopt/versioncontrol/git/PyGIT.py
index 47f397389..32aac9184 100644
--- a/tracopt/versioncontrol/git/PyGIT.py
+++ b/tracopt/versioncontrol/git/PyGIT.py
@@ -596,6 +596,11 @@ class Storage(object):
key=lambda (name, rev, head): (not head, name))
return [(name, rev) for name, rev, head in branches]
+ def get_refs(self):
+ for refname, rev in self.rev_cache.refs_dict.iteritems():
+ if refname != 'HEAD':
+ yield refname, rev
+
def get_commits(self):
return self.rev_cache.rev_dict
diff --git a/tracopt/versioncontrol/git/git_fs.py
b/tracopt/versioncontrol/git/git_fs.py
index 031f68c2b..c33eb7a5d 100644
--- a/tracopt/versioncontrol/git/git_fs.py
+++ b/tracopt/versioncontrol/git/git_fs.py
@@ -100,6 +100,21 @@ class GitCachedRepository(CachedRepository):
return count > 0
return False
+ def needs_sync():
+ max_holders = 999
+ revs = sorted(set(rev for refname, rev in repos.git.get_refs()))
+ for idx in xrange(0, len(revs), max_holders):
+ revs_ = revs[idx:idx + max_holders]
+ holders = ','.join(('%s',) * len(revs_))
+ args = [self.id]
+ args.extend(revs_)
+ query = 'SELECT COUNT(*) FROM revision ' \
+ 'WHERE repos=%s AND rev IN (' + holders + ')'
+ for count, in self.env.db_query(query, args):
+ if count < len(revs_):
+ return True
+ return False
+
def traverse(rev, seen):
revs = []
merge_revs = []
@@ -121,9 +136,7 @@ class GitCachedRepository(CachedRepository):
revs[idx:idx] = traverse(rev, seen)
return revs
- while True:
- repos.sync()
- repos_youngest = repos.youngest_rev or ''
+ def sync_revs():
updated = False
seen = set()
@@ -148,9 +161,13 @@ class GitCachedRepository(CachedRepository):
if feedback:
feedback(rev)
- if updated:
- continue # sync again
+ return updated
+ while True:
+ repos.sync()
+ repos_youngest = repos.youngest_rev or ''
+ if needs_sync() and sync_revs():
+ continue # sync again
if meta_youngest != repos_youngest:
with self.env.db_transaction as db:
db("""
--
Jun Omae <[email protected]> (大前 潤)
--
You received this message because you are subscribed to the Google Groups "Trac
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/trac-dev.
For more options, visit https://groups.google.com/d/optout.
