Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Renato Golin via lldb-dev]

On 5 Jul 2016 10:45 p.m., "Mehdi Amini"  wrote:
>
>
> > On Jul 5, 2016, at 3:44 AM, Renato Golin via llvm-dev <
llvm-...@lists.llvm.org> wrote:
> >
> > Quick re-cap.
> >
> > After a few rounds, not only the "external server" proposal got
> > obliterated as totally unnecessary, but the idea that we may even need
> > a hook at all is now challenged.
>
> This is not clear to me.
> How is the umbrella repository updated?

Sorry, I meant no hooks for updating sequential ids. We still need a hook
to update the umbrella project.

Cheers,
Renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Mehdi Amini via lldb-dev]

> On Jul 5, 2016, at 3:44 AM, Renato Golin via llvm-dev 
>  wrote:
> 
> Quick re-cap.
> 
> After a few rounds, not only the "external server" proposal got
> obliterated as totally unnecessary, but the idea that we may even need
> a hook at all is now challenged.

This is not clear to me. 
How is the umbrella repository updated?

— 
Mehdi



> 
> Jared's idea to use "git describe" is in line with previous proposals
> to use rev-list --count and to do so only up to the previous tag, but
> all in one nice and standard little feature.
> 
> There were concerns by applying one tag per commit, but most of them
> offered weak evidence. However, if "describe" can cover all our needs,
> there is no point in even discussing tags.
> 
> Just for reference, GitHub *does* have an SVN interface [1], and you
> can already checkout a specific revision with "svn checkout -r NNN
> repo", which *is already* using "git rev-list --count".
> 
> This means that, for SVN based bisects, using GitHub will make it
> *completely transparent* for SVN users. You can also base your
> releases off an SVN view of the Git repo.
> 
> So, to clear up this discussion and finish my proposal to move to
> GitHub, my final questions, only to those that *want* SVN
> compatibility:
> 
> 1. Is there anything in the SVN view of GitHub that *doesn't* work for
> you? (ie. same as using "rev-list --count")
> 
> 2. If so, can "git describe" solve the problem?
> 
> 3. If not, please describe, in details, why < solution>> would be the *only* way forward.
> 
> I'll let this sit for a few days, and if no one has any serious issue,
> I'll write up the final proposal and start the voting process with the
> Foundation.
> 
> cheers,
> --renato
> 
> [1] https://github.com/blog/626-announcing-svn-support
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Matthias Braun via lldb-dev]

I assumed we want ids for the umbrella repository to ease bisection and having 
something to print as a version identifier, but do we really need them for the 
other repositories?

I also still don't see why `git rev-list --count --all` does not work. Sure the 
count is only per branch, but why would ever need to have continuous numbering 
between say a 3.8.XX revision and a 3.9.XX branch...

- Matthias

> On Jun 30, 2016, at 9:33 AM, Reid Kleckner via llvm-dev 
>  wrote:
> 
> On Thu, Jun 30, 2016 at 9:16 AM, James Y Knight via lldb-dev 
> > wrote:
> I don't think we should do any of that. It's too complicated -- and I don't 
> see the reason to even do it.
> 
> There's a need for the "llvm-project" repository -- that's been discussed 
> plenty -- but where does the need for a separate "id" that must be pushed 
> into all of the sub-projects come from? This is the first I've heard of that 
> as a thing that needs to be done.
> 
> There was a previous discussion about putting an sequential ID in the 
> "llvm-project" repo commit messages (although, even that I'd say is 
> unnecessary), but not anywhere else.
> 
> Agreed, the llvm-project repository can completely take on the role of the 
> SQL database in Renato's proposal.
> 
> Chromium created a "git-number" extension that assigns sequential ids to 
> commits in the obvious way, and that provided some continuity with the 
> "git-svn-id:" footers in commit messages. I'm not sure their extension is 
> particularly reusable, though:
> https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/master/git_number.py
>  
> 
> 
> I think for LLVM, whatever process updates the umbrella repo should add the 
> sequential IDs to the commit message, and that will help provide continuity 
> across the git/svn bridge.
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Renato Golin via lldb-dev]

On 30 June 2016 at 17:33, Reid Kleckner  wrote:
> Agreed, the llvm-project repository can completely take on the role of the
> SQL database in Renato's proposal.

Hum, doing it in a separate server was suggested by the GitHub folks,
so I just assumed they can't do that in the umbrella project for some
reason.

I'm all for using the umbrella if we can, I just though we couldn't... :(

Can someone try the suggested tag style? Are we sure we can guarantee
atomicity in there? I know SQL can. :)

I know that changing the commit message works because of Gerrit and
our current SVN integration, I don't know how much adding one tag for
each push will work in Git over time.

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Reid Kleckner via lldb-dev]

On Thu, Jun 30, 2016 at 9:16 AM, James Y Knight via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> I don't think we should do any of that. It's too complicated -- and I
> don't see the reason to even do it.
>
> There's a need for the "llvm-project" repository -- that's been discussed
> plenty -- but where does the need for a separate "id" that must be pushed
> into all of the sub-projects come from? This is the first I've heard of
> that as a thing that needs to be done.
>
> There was a previous discussion about putting an sequential ID in the
> "llvm-project" repo commit messages (although, even that I'd say is
> unnecessary), but not anywhere else.
>

Agreed, the llvm-project repository can completely take on the role of the
SQL database in Renato's proposal.

Chromium created a "git-number" extension that assigns sequential ids to
commits in the obvious way, and that provided some continuity with the
"git-svn-id:" footers in commit messages. I'm not sure their extension is
particularly reusable, though:
https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/master/git_number.py

I think for LLVM, whatever process updates the umbrella repo should add the
sequential IDs to the commit message, and that will help provide continuity
across the git/svn bridge.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [llvm-dev] Sequential ID Git hook

[Tom Honermann via lldb-dev]

On 6/30/2016 7:43 AM, Renato Golin via llvm-dev wrote:
> Given the nature of our project's repository structure, triggers in
> each repository can't just update their own sequential ID (like
> Gerrit) because we want a sequence in order for the whole project, not
> just each component. But it's clear to me that we have to do something
> similar to Gerrit, as this has been proven to work on a larger
> infrastructure.

I'm assuming that pushes to submodules will result in a (nearly) 
immediate commit/push to the umbrella repo to update it with the new 
submodule head.  Otherwise, checking out the umbrella repo won't get you 
the latest submodule updates.

Since updates to the umbrella project are needed to synchronize it for 
updates to sub-modules, it seems to me that if you want an ID that 
applies to all projects, that it would have to be coordinated relative 
to the umbrella project.

>   Design decisions
>
> This could be a pre/post-commit trigger on each repository that
> receives an ID from somewhere (TBD) and updates the commit message.
> When the umbrella project synchronises, it'll already have the
> sequential number in. In this case, the umbrella project is not
> necessary for anything other than bisect, buildbots and releases.

I recommend using git tag rather than updating the commit message 
itself.  Tags are more versatile.

> I personally believe that having the trigger in the umbrella project
> will be harder to implement and more error prone.

Relative to a SQL database and a server, I think managing the ID from 
the umbrella repository would be much simpler and more reliable.

Managing IDs from a repo using git meta data is pretty simple.  Here's 
an example script that creates a repo and allocates a push tag in 
conjunction with a sequence of commits (here I'm simulating pushes of 
individual commits rather than using git hooks for simplicity).  I'm not 
a git expert, so there may be better ways of doing this, but I don't 
know of any problems with this approach.

#!/bin/sh

rm -rf repo

# Create a repo
mkdir repo
cd repo
git init

# Create a well known object.
PUSH_OBJ=$(echo "push ID" | git hash-object -w --stdin)
echo "PUSH_OBJ: $PUSH_OBJ"

# Initialize the push ID to 0.
git notes add -m 0 $PUSH_OBJ

# Simulate some commits and pushes.
for i in 1 2 3; do
   echo $i > file$i
   git add file$i
   git commit -m "Added file$i" file$i
   PUSH_TAG=$(git notes show $PUSH_OBJ)
   PUSH_TAG=$((PUSH_TAG+1))
   git notes add -f -m $PUSH_TAG $PUSH_OBJ
   git tag -m "push-$PUSH_TAG" push-$PUSH_TAG
done

# list commits with push tags
git log --decorate=full


Running the above shows a git log with the tags:

commit a4ca4a0b54d5fb61a2dacbab5732d00cf8216029 (HEAD, tag: 
refs/tags/push-3, refs/heads/master)
...
 Added file3

commit e98e2669569d5cfb15bf4cd1f268507873bcd63f (tag: refs/tags/push-2)
...
 Added file2

commit 5c7f29107838b4af91fe6fa5c2fc5e3769b87bef (tag: refs/tags/push-1)
...
 Added file1


The above script is not transaction safe because it runs commands 
individually.  In a real deployment, git hooks would be used and would 
rely on push locks to synchronize updates.  Those hooks could also 
distribute ID updates to the submodules to keep them synchronized.

Tom.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev