On Tuesday, April 30, 2019 2:02:32 PM MDT Jeff King wrote:
> On Tue, Apr 23, 2019 at 02:09:31PM -0600, Martin Fick wrote:
> > I think that if there were no default limit during a clone it could have
> > disastrous effects on people using the repo tool from the android project,
&
On Tuesday, April 23, 2019 5:08:40 PM MDT Duy Nguyen wrote:
> On Tue, Apr 23, 2019 at 11:45 AM Jeff King wrote:
> > On Mon, Apr 22, 2019 at 09:55:38PM -0400, Jeff King wrote:
> > > Here are my p5302 numbers on linux.git, by the way.
> > >
> > > Test jk/
On Monday, April 22, 2019 4:56:54 PM MDT Jeff King wrote:
> On Mon, Apr 22, 2019 at 02:21:40PM -0600, Martin Fick wrote:
> > > Try this (with a recent version of git; your v1.8.2.1 won't have
> > >
> > > --batch-all-objects):
> > > # count the on-d
On Friday, April 19, 2019 11:58:25 PM MDT Jeff King wrote:
> On Fri, Apr 19, 2019 at 03:47:22PM -0600, Martin Fick wrote:
> > I have been thinking about this problem, and I suspect that this compute
> > time is actually spent doing SHA1 calculations, is that possible? Some
> &
We have a serious performance problem with one of our large repos. The repo is
our internal version of the android platform/manifest project. Our repo after
running a clean "repack -A -d -F" is close to 8G in size, has over 700K refs,
and it has over 8M objects. The repo takes around 40min to cl
On Tuesday, June 12, 2018 09:12:19 PM Peter Backes wrote:
> So? If a thousand lawyers claim 1+1=3, it becomes a
> mathematical truth?
No, but probably a legal "truth". :)
--
The Qualcomm Innovation Center, Inc. is a member of Code
Aurora Forum, hosted by The Linux Foundation
On Wednesday, May 16, 2018 01:06:59 PM Jeff King wrote:
> On Wed, May 16, 2018 at 01:40:56PM -0600, Martin Fick
wrote:
> > > In theory the fetch means that it's safe to actually
> > > prune in the mother repo, but in practice there are
> > > still races. T
On Wednesday, May 16, 2018 12:37:45 PM Jeff King wrote:
> On Wed, May 16, 2018 at 03:29:42PM -0400, Konstantin
Ryabitsev wrote:
> Yes, that's pretty close to what we do at GitHub. Before
> doing any repacking in the mother repo, we actually do
> the equivalent of:
>
> git fetch --prune ../$id.g
On Wednesday, May 16, 2018 03:11:47 PM Konstantin Ryabitsev
wrote:
> On 05/16/18 15:03, Martin Fick wrote:
> >> I'm undecided about that. On the one hand this does
> >> create lots of small files and inevitably causes
> >> (some) performance degradation. On t
On Wednesday, May 16, 2018 03:01:13 PM Konstantin Ryabitsev
wrote:
> On 05/16/18 14:26, Martin Fick wrote:
> > If you are going to keep the unreferenced objects around
> > forever, it might be better to keep them around in
> > packed
> > form?
>
> I'm unde
On Wednesday, May 16, 2018 02:12:24 PM Konstantin Ryabitsev
wrote:
> The loose objects I'm thinking of are those that are
> generated when we do "git repack -Ad" -- this takes all
> unreachable objects and loosens them (see man git-repack
> for more info). Normally, these would be pruned after a
>
On Wednesday, May 16, 2018 10:58:19 AM Konstantin Ryabitsev
wrote:
>
> 1. Find every repo mentioning the parent repository in
> their alternates 2. Repack them without the -l switch
> (which copies all the borrowed objects into those repos)
> 3. Once all child repos have been repacked this way, p
On Monday, May 14, 2018 05:32:35 PM Barodia, Anjali wrote:
> I was trying to push local git to another git on gerrit,
> but stuck with an hook error. This is a very large repo
> and while running the command "git push origin --all" I
> am getting this errors:
>
> remote: (W) 92e19d4: too many comm
On Wednesday, January 10, 2018 02:39:13 PM Derrick Stolee
wrote:
> On 1/10/2018 1:25 PM, Martin Fick wrote:
> > On Sunday, January 07, 2018 01:14:41 PM Derrick Stolee
> >
> > wrote:
> >> This RFC includes a new way to index the objects in
> >> multiple pack
On Sunday, January 07, 2018 01:14:41 PM Derrick Stolee
wrote:
> This RFC includes a new way to index the objects in
> multiple packs using one file, called the multi-pack
> index (MIDX).
...
> The main goals of this RFC are:
>
> * Determine interest in this feature.
>
> * Find other use cases fo
> On Jan 4, 2018 11:19 AM, "Martin Fick"
wrote:
> > On Tuesday, December 26, 2017 12:40:26 AM Jacob Keller
> >
> > wrote:
> > > On Mon, Dec 25, 2017 at 10:02 PM, Carl Baldwin
> >
> > wrote:
> > > >> On Mon, Dec 25, 2017 at
On Tuesday, December 26, 2017 01:31:55 PM Carl Baldwin
wrote:
...
> What I propose is that gerrit and github could end up more
> robust, featureful, and interoperable if they had this
> feature to build from.
I agree (assuming we come up with a well defined feature)
> With gerrit specifically, a
On Monday, December 25, 2017 06:16:40 PM Carl Baldwin wrote:
> On Sun, Dec 24, 2017 at 10:52:15PM -0500, Theodore Ts'o
wrote:
> Look at what happens in a rebase type workflow in any of
> the following scenarios. All of these came up regularly
> in my time with Gerrit.
>
> 1. Make a quick edit
On Sunday, December 24, 2017 12:01:38 AM Johannes Schindelin
wrote:
> Hi Carl,
>
> On Sat, 23 Dec 2017, Carl Baldwin wrote:
> > I imagine that a "git commit --amend" would also insert
> > a "replaces" reference to the original commit but I
> > failed to mention that in my original post.
>
> And
On Tuesday, December 26, 2017 12:40:26 AM Jacob Keller
wrote:
> On Mon, Dec 25, 2017 at 10:02 PM, Carl Baldwin
wrote:
> >> On Mon, Dec 25, 2017 at 5:16 PM, Carl Baldwin
wrote:
> >> A bit of a tangent here, but a thought I didn't wanna
> >> lose: In the general case where a patch was rebased
>
On Wednesday, May 10, 2017 11:20:49 AM Jonathan Nieder
wrote:
> Hi,
>
> Ævar Arnfjörð Bjarmason wrote:
> > Just a side question, what are the people who use this
> > feature using it for? The only thing I can think of
> > myself is some out of band ref advertisement because
> > you've got squilli
On Thursday, April 13, 2017 02:28:07 PM David Turner wrote:
> On Thu, 2017-04-13 at 12:08 -0600, Martin Fick wrote:
> > On Thursday, April 13, 2017 11:03:14 AM Jacob Keller
wrote:
> > > On Thu, Apr 13, 2017 at 10:31 AM, David Turner
> >
> > wrote:
> > >
On Thursday, April 13, 2017 11:03:14 AM Jacob Keller wrote:
> On Thu, Apr 13, 2017 at 10:31 AM, David Turner
wrote:
> > Git gc locks the repository (using a gc.pid file) so
> > that other gcs don't run concurrently. But git repack
> > doesn't respect this lock, so it's possible to have a
> > repa
On Sunday, March 12, 2017 11:03:44 AM Junio C Hamano wrote:
> Jeff King writes:
> > I can think of one downside of a time-based solution,
> > though: if you run multiple gc's during the time
> > period, you may end up using a lot of disk space (one
> > repo's worth per gc). But that's a fundamenta
On Thursday, March 09, 2017 10:50:21 AM
jmel...@codeaurora.org wrote:
> On 2017-03-07 13:33, Junio C Hamano wrote:
> > James Melvin writes:
> >> These options are designed to prevent stale file handle
> >> exceptions during git operations which can happen on
> >> users of NFS repos when repacking
On Tuesday, January 17, 2017 04:50:13 PM Ben Peart wrote:
> While large files can be a real problem, our biggest issue
> today is having a lot (millions!) of source files when
> any individual developer only needs a small percentage of
> them. Git with 3+ million local files just doesn't
> perform
On Monday, January 09, 2017 01:21:37 AM Jeff King wrote:
> On Wed, Jan 04, 2017 at 09:11:55AM -0700, Martin Fick
wrote:
> > I am replying to this email across lists because I
> > wanted to highlight to the git community this jgit
> > change to repacking that we have up for r
On Monday, January 09, 2017 05:55:45 AM Jeff King wrote:
> On Mon, Jan 09, 2017 at 04:01:19PM +0900, Mike Hommey
wrote:
> > > That's _way_ more complicated than your problem, and
> > > as I said, I do not have a finished solution. But it
> > > seems like they touch on a similar concept (a
> > > po
I am replying to this email across lists because I wanted to
highlight to the git community this jgit change to repacking
that we have up for review
https://git.eclipse.org/r/#/c/87969/
This change introduces a new convention for how to preserve
old pack files in a staging area
(.git/objects
On Friday, August 05, 2016 08:39:58 AM you wrote:
> * A new topic, when you merge it to the "lit" branch, you
> describe the cover as the merge commit message.
>
> * When you updated an existing topic, you tell a tool
> like "rebase -i -p" to recreate "lit" branch on top of
> the mainline. This
Brigning this back on list so that someone else can help...
On Thursday, June 23, 2016 05:01:18 PM John Ajah wrote:
> I'm on a private git, installed on a work server. Now the
> guy who set it up is not available and I want to give
> access to someone working for me, but I don't know how to
> do t
On Tuesday, December 22, 2015 06:17:28 PM you wrote:
> On Tue, Dec 22, 2015 at 7:41 AM, Michael Haggerty
wrote:
>
> At a deeper level, the "refs/" part of reference names is
> actually pretty useless in general. I suppose it
> originated in the practice of storing loose references
> under "refs/"
+repo-disc...@googlegroups.com (to hit Gerrit developers
also)
On Thursday, September 10, 2015 09:28:52 AM Jacob Keller
wrote:
> does anyone know of any tricks for storing a cover letter
> for a patch series inside of git somehow? I'd guess the
> only obvious way currently is to store it at the
> The current protocol has the following problems that limit
> us:
>
> - It is not easy to make it resumable, because we
> recompute every time. This is especially problematic for
> the initial fetch aka "clone" as we will be talking about
> a large transfer. Redirection to a bundle hosted on CD
On Friday, February 20, 2015 01:29:12 PM David Turner wrote:
>...
> For a more general solution, perhaps a log of ref updates
> could be used. Every time a ref is updated on the server,
> that ref would be written into an append-only log. Every
> time a client pulls, their pull data includes an in
On Feb 19, 2015 5:42 PM, David Turner wrote:
>
> On Fri, 2015-02-20 at 06:38 +0700, Duy Nguyen wrote:
> > > * 'git push'?
> >
> > This one is not affected by how deep your repo's history is, or how
> > wide your tree is, so should be quick..
> >
> > Ah the number of refs may affect both g
There currently is a thread on the Gerrit list about how much faster cloning
can be when using Gerrit/jgit GCed packs with bitmaps versus C git GCed packs
with bitmaps.
Some differences outlined are that jgit seems to have more bitmaps, it creates
one for every refs/heads, is C git doing that?
Perhaps the receiving process is dying hard and leaving
stuff behind? Out-of-memory, out of disk space?
-Martin
On Tuesday, January 14, 2014 10:10:31 am Martin Langhoff
wrote:
> On Tue, Jan 14, 2014 at 9:54 AM, Martin Langhoff
>
> wrote:
> > Is there a handy way to list the blobs in a pack,
> Martin Fick writes:
> > * Setup 1:
> > Do a full repack. All loose and packed objects are
> > added
...
> > * Scenario 1:
> > Start with Setup 1. Nothing has changed on the repo
> > contents (no new object/packs, refs all the same), but
> &g
I wanted to explore the idea of exploiting knowledge about
previous repacks to help speed up future repacks.
I had various ideas that seemed like they might be good
places to start, but things quickly got away from me.
Mainly I wanted to focus on reducing and even sometimes
eliminating reac
On Monday, October 21, 2013 12:40:58 pm
james.mo...@gitblit.com wrote:
> On Mon, Oct 21, 2013, at 02:29 PM, Thomas Koch wrote:
> > As I understand, a UUID could also be used for the same
> > purbose as the change-
> > id. How is the change-id generated by the way? Would it
> > be a good english na
On Wednesday, October 16, 2013 02:34:01 am Jeff King wrote:
> I was recently presented with a repository with a
> corrupted packfile, and was asked if the data was
> recoverable. This post-mortem describes the steps I took
> to investigate and fix the problem. I thought others
> might find the proc
On Friday, September 06, 2013 11:19:02 am Junio C Hamano
wrote:
> mf...@codeaurora.org writes:
> > Object lookups should likely not get any slower than if
> > repack were not run, and the extra new pack might
> > actually help find some objects quicker.
>
> In general, having an extra pack, only
On Thursday, August 29, 2013 08:11:48 am Brad King wrote:
>
>fatal: Unable to create 'lock': File exists.
>If no other git process is currently running, this
> probably means a git process crashed in this repository
> earlier. Make sure no other git process is running and
> remove the
On Thursday, August 15, 2013 01:46:02 am Stefan Beller
wrote:
> On 08/15/2013 01:25 AM, Martin Fick wrote:
> > On Wednesday, August 14, 2013 04:51:14 pm Matthieu Moy
> >
> > wrote:
> >> Antoine Pelisse writes:
> >>> On Wed, Aug 14, 2013 at
On Wednesday, August 14, 2013 05:25:42 pm Martin Fick wrote:
> On Wednesday, August 14, 2013 04:51:14 pm Matthieu Moy
>
> wrote:
> > Antoine Pelisse writes:
> > > On Wed, Aug 14, 2013 at 6:27 PM, Stefan Beller
> > >
> > > wrote:
&g
On Wednesday, August 14, 2013 04:53:36 pm Junio C Hamano
wrote:
> Martin Fick writes:
> > One suggestion would be to change the repack code to
> > create pack filenames based on the sha1 of the
> > contents of the pack file instead of on the sha1 of
> > the objects in
On Wednesday, August 14, 2013 04:51:14 pm Matthieu Moy
wrote:
> Antoine Pelisse writes:
> > On Wed, Aug 14, 2013 at 6:27 PM, Stefan Beller
> >
> > wrote:
> >> builtin/repack.c | 410
> >> +
> >> contrib/examples/git-repack.sh | 194
> >> +
On Wednesday, August 14, 2013 04:16:35 pm Stefan Beller
wrote:
> On 08/14/2013 07:25 PM, Martin Fick wrote:
> > I have been holding off a bit on expressing this
> > opinion too because I don't want to squash someone's
> > energy to improve things, and because I am n
On Wednesday, August 14, 2013 10:49:58 am Antoine Pelisse
wrote:
> On Wed, Aug 14, 2013 at 6:27 PM, Stefan Beller
>
> wrote:
> > builtin/repack.c | 410
> > +
> > contrib/examples/git-repack.sh | 194
> > +++ git-repack.sh
On Thursday, August 08, 2013 10:56:38 am Junio C Hamano
wrote:
> I thought the discussion was about making the local gc
> cheaper, and the "Imagine we have a cheap way" was to
> address it by assuming that the daily "pack young
> objects into a single pack" can be sped up if we did not
> have to t
these big packs we just generated". In
> subsequent runs, the little packfiles from the fetch are
> absorbed into a pack that is immune to gc. You're also
> using a size heuristic, to consolidate similarly sized
> packfiles. You also have a --ratio to tweak the ratio
>
On Tuesday, August 06, 2013 06:24:50 am Duy Nguyen wrote:
> On Tue, Aug 6, 2013 at 9:38 AM, Ramkumar Ramachandra
wrote:
> > + Garbage collect using a pseudo
> > logarithmic packfile maintenance +
> > approach. This approach attempts to minimize packfile
> > churn +
On Monday, August 05, 2013 11:34:24 am Ramkumar Ramachandra
wrote:
> Martin Fick wrote:
> > https://gerrit-review.googlesource.com/#/c/35215/
>
> Very cool. Of what I understood:
>
> So, the problem is that my .git/objects/pack is polluted
> with little packs everyt
On Tuesday, July 02, 2013 12:11:49 am Jeff King wrote:
> Here are my patches to deal with Martin's pathological
> case, split out for easy reading. I took a few timings
> to show that the results of the 3rd patch are noticeable
> even with 50,000 unique refs (which is still a lot, but
> something t
On Tuesday, July 02, 2013 03:24:14 am Michael Haggerty
wrote:
> > git rev-list HEAD | for nn in $(seq 0 100) ; do for c
> > in $(seq 0 1) ; do read sha ; echo $sha
> > refs/c/$nn/$c$nn ; done ; done > .git/packed-refs
>
> I believe this generates a packed-refs file that is not
> sorted lexic
I have often reported problems with git fetch when there are
many refs in a repo, and I have been pleasantly surprised
how many problems I reported were so quickly fixed. :) With
time, others have created various synthetic test cases to
ensure that git can handle many many refs. A simple
synt
I have been trying to think of ways to fix git-repack so
that it no longer momentarily makes the objects in a repo
inaccessible to all processes when it replaces packfiles
with the same objects in them as an already existing pack
file. To be more explicit, I am talking about the way it
moves
On Thursday, May 23, 2013 07:01:43 am you wrote:
>
> I'm running a rather special configuration, basically i
> have a gerrit server pushing
...
> I have found "git receive-pack"s that has been running
> for days/weeks without terminating
>
...
> Anyone that has any clues about what could be
On Sunday, February 10, 2013 12:03:00 pm Robert Zeh wrote:
> On Sat, Feb 9, 2013 at 1:35 PM, Junio C Hamano
wrote:
> > Ramkumar Ramachandra writes:
> >> This is much better than Junio's suggestion to study
> >> possible implementations on all platforms and
> >> designing a generic daemon/ commun
Jeff King wrote:
>On Sat, Jan 26, 2013 at 10:32:42PM -0800, Junio C Hamano wrote:
>
>> Both makes sense to me.
>>
>> I also wonder if we would be helped by another "repack" mode that
>> coalesces small packs into a single one with minimum overhead, and
>> run that often from "gc --auto", so th
...[Sorry about the previous HTML reposts]
Jeff King wrote:
>On Mon, Dec 31, 2012 at 03:30:53AM -0700, Martin Fick
wrote:
>
>> The general approach is to setup a transaction and
>> either commit or abort it. A transaction can be setup
>> by renaming an appropriatel
On Friday, January 04, 2013 10:52:43 am Pyeron, Jason J
CTR (US) wrote:
> > From: Martin Fick
> > Sent: Thursday, January 03, 2013 6:53 PM
> >
> > Any thoughts on this idea? Is it flawed? I am
trying
> > to write it up in a more formal generalized manner
and
Any thoughts on this idea? Is it flawed? I am trying to
write it up in a more formal generalized manner and was
hoping to get at least one "it seems sane" before I do.
Thanks,
-Martin
On Monday, December 31, 2012 03:30:53 am Martin Fick wrote:
> On Thursday, December 27, 201
On Friday, December 21, 2012 06:41:43 pm Earl Gresh wrote:
> Hi-
>
> I have observed that after running GC, one particular git
> repository ended up with some missing refs in the
> refs/changes/* namespace the Gerrit uses for storing
> patch sets. The refs were valid and should not have been
> pru
On Thursday, December 27, 2012 04:11:51 pm Martin Fick wrote:
> It concerns me that git uses any locking at all, even for
> refs since it has the potential to leave around stale
> locks.
> ...
> [a previous not so great attempt to fix this]
> ...
I may have finally figured out a
On Saturday, December 29, 2012 03:18:49 pm Martin Fick wrote:
> Jeff King wrote:
> >On Thu, Dec 27, 2012 at 04:11:51PM -0700, Martin Fick
wrote:
> >> My idea is based on using filenames to store sha1s
> >> instead of file contents. To do this, the sha1 one of
>
Jeff King wrote:
>On Thu, Dec 27, 2012 at 04:11:51PM -0700, Martin Fick wrote:
>> My idea is based on using filenames to store sha1s instead of
>> file contents. To do this, the sha1 one of a ref would be
>> stored in a file in a directory named after the loose ref. I
Jeff King wrote:
>On Fri, Dec 28, 2012 at 07:50:14AM -0700, Martin Fick wrote:
>
>> Hmm, actually I believe that with a small modification to the
>> semantics described here it would be possible to make multi
>> repo/branch commits work. Simply allow the ref filena
Jeff King wrote:
>On Fri, Dec 28, 2012 at 09:15:52AM -0800, Junio C Hamano wrote:
>
>> Martin Fick writes:
>>
>> > Hmm, actually I believe that with a small modification to the
>> > semantics described here it would be possible to make
On Friday, December 28, 2012 09:58:36 am Junio C Hamano
wrote:
> Martin Fick writes:
> > 3) To create a ref, it must be renamed from the null
> > file (sha ...) to the new value just as if it were
> > being updated from any other value, but there is one
> > extra
On Thursday, December 27, 2012 04:11:51 pm Martin Fick wrote:
> On Wednesday, December 26, 2012 01:24:39 am Michael
> Haggerty
>
> wrote:
> > ... lots of discussion about ref locking...
>
> It concerns me that git uses any locking at all, even for
> refs since it
On Wednesday, December 26, 2012 01:24:39 am Michael Haggerty
wrote:
> ... lots of discussion about ref locking...
It concerns me that git uses any locking at all, even for
refs since it has the potential to leave around stale locks.
For a single user repo this is not a big deal, the lock can
I have been reading the git-repack.sh script and I have
found a piece that I am concerned with. It looks like after
repacking there is a place when packfiles could be
temporarily unaccessible making the objects within
temporarily unaccessible. If my evaluation is true, it
would seem like git
74 matches
Mail list logo
