Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-19 Thread David Woodhouse 

> On 03/15/16 18:20, Kinney, Michael D wrote:
> I don't object to pulling, if the submitter explicitly requests it, and
> if we're making this option official now.

Let's say "at the discretion of the person doing the merge" rather than
writing down that it should need an explicit request.

Over time, as people get used to using the tools properly, I would expect
that pulling will become the default behaviour. By choosing to rebase
instead, *you* are choosing to discard the correct history of the original
development vs. the merge, and to take the responsibility for any
resulting problems.

> Regarding Reviewed-by tags, I like to review series per-patch, and I
> also like the tags to be present in the commit history per-patch. For
> example, when a series modifies several top-level packages, I don't /
> can't always review all patches in full depth; so I might give an
> Acked-by for some patches, or even nothing.
>
> In this case I'm fine with the submitter rebasing one last time, adding
> the tags, and then I can do the pull. (I'll have to remember to verify
> the individual patches on the branch being pulled, and the labels on the
> commit messages, against the mailing list traffic.)

Of course you had to do that verification anyway, as the patch in your
inbox might not actually match the "same" message on the list. So no
change there.

It's also worth noting that we use "rebase" for two different things. In
normal usage it's considered bad form just to recommit the same patch on
the *same* parent. We will probably continue to do that freely here, and
as long as you don't move it to a new parent, that's OK. But people
generally still call that "rebasing".


-- 
dwmw2

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-18 Thread Laszlo Ersek 
On 03/15/16 18:20, Kinney, Michael D wrote:
> David,
> 
> Thanks for the suggestion.  Today the git dev process requires a rebase.
> Given that this operation is transferring content from one community 
> maintained feature branch in edk2-staging into edk2/master, I agree the
> option to use 'git pull' should be available.  Not sure it should be 
> required in all cases.  I would prefer let the feature owner(s) decide 
> which method makes the most sense.
> 
> Does anyone have any objections to supporting either the current 
> rebase work flow or a 'git pull' work flow in step (6)?

I don't object to pulling, if the submitter explicitly requests it, and
if we're making this option official now.

Regarding Reviewed-by tags, I like to review series per-patch, and I
also like the tags to be present in the commit history per-patch. For
example, when a series modifies several top-level packages, I don't /
can't always review all patches in full depth; so I might give an
Acked-by for some patches, or even nothing.

In this case I'm fine with the submitter rebasing one last time, adding
the tags, and then I can do the pull. (I'll have to remember to verify
the individual patches on the branch being pulled, and the labels on the
commit messages, against the mailing list traffic.)

Thanks
Laszlo

> 
> 
> I think there are several methods we can support for development.
> 
> 1) Simple bug fixes/features sent directly to edk2-devel for PRs.
> 2) A larger or more complex bug fix/feature can optionally post a link
>to a branch on personal github fork to help simplify the review process 
>for those reviewers that prefer to use that method.  This type of bug 
>fix or feature is usually owned by a single subject matter expert.
> 3) A larger or more complex feature that requires design/dev/test by more
>than one subject matter expert.
> 
> We already support (1) and (2) today.  Feature branches on edk2-staging are 
> intended for (3).
> 
> I can think of a couple ways we end up in (3). The first is a feature we know
> requires multiple subject matter experts and the request is made to add
> to edk2-staging from the beginning.  The second is a PR that is sent to 
> edk2-devel, and the community believes it needs more design/dev/test work
> that requires cooperation of multiple subject matter experts and the PR 
> is re-directed to a feature branch in edk2-staging.
> 
> Thanks,
> 
> Mike
> 
> 
>> -Original Message-
>> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of David 
>> Woodhouse
>> Sent: Tuesday, March 15, 2016 3:50 AM
>> To: Kinney, Michael D <michael.d.kin...@intel.com>; Mangefeste, Tony
>> <tony.mangefe...@intel.com>; edk2-devel@lists.01.org <edk2-de...@ml01.01.org>
>> Subject: Re: [edk2] EDK2 Staging Repository 2nd Draft
>>
>> On Tue, 2016-03-15 at 00:16 +, Kinney, Michael D wrote:
>>>
>>>
>>> Can you provide some revised text you would like to see in step 6.
>>>
>>> I agree that we need to use the tools in ways that help make this easy, 
>>> prevent
>>> errors, and preserve history.  Given that step 6 describes promoting a 
>>> feature
>>> from edk2-staging repo to edk2 repo, what set of git operations you would
>>> recommend?
>>
>> Literally, 'git pull'. So my change in patch form would be:
>>
>> @@
>>  6) Process to promote an edk2-staging branch to edk2 trunk
>>  a) Request sent to edk2-devel that describes the feature, design, 
>> testing,
>> etc.
>>  b) Stewards evaluate request and determine if the feature meets 
>> edk2 criteria.
>> -c) If approved, use edk2 patch review/commit process on edk2-devel 
>> mailing
>> list
>> +c) If approved, use 'git pull' to merge the branch into edk2 
>> master, adding
>> +   any final 'Reviewed-by:' tags to the merge commit as appropriate.
>>  d) Remove feature branch from edk2-staging (maybe archived 
>> elsewhere?).
>>
>>
>> I'm still interested in what benefit we gain from a centralised
>> edk2-staging repository though, over the normal process of contributors
>> just pushing their work to github repositories and submitting PRs.
>>
>> --
>> dwmw2
> 
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> 

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-15 Thread Kinney, Michael D 
David,

Thanks for the suggestion.  Today the git dev process requires a rebase.
Given that this operation is transferring content from one community 
maintained feature branch in edk2-staging into edk2/master, I agree the
option to use 'git pull' should be available.  Not sure it should be 
required in all cases.  I would prefer let the feature owner(s) decide 
which method makes the most sense.

Does anyone have any objections to supporting either the current 
rebase work flow or a 'git pull' work flow in step (6)?



I think there are several methods we can support for development.

1) Simple bug fixes/features sent directly to edk2-devel for PRs.
2) A larger or more complex bug fix/feature can optionally post a link
   to a branch on personal github fork to help simplify the review process 
   for those reviewers that prefer to use that method.  This type of bug 
   fix or feature is usually owned by a single subject matter expert.
3) A larger or more complex feature that requires design/dev/test by more
   than one subject matter expert.

We already support (1) and (2) today.  Feature branches on edk2-staging are 
intended for (3).

I can think of a couple ways we end up in (3). The first is a feature we know
requires multiple subject matter experts and the request is made to add
to edk2-staging from the beginning.  The second is a PR that is sent to 
edk2-devel, and the community believes it needs more design/dev/test work
that requires cooperation of multiple subject matter experts and the PR 
is re-directed to a feature branch in edk2-staging.

Thanks,

Mike


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of David 
> Woodhouse
> Sent: Tuesday, March 15, 2016 3:50 AM
> To: Kinney, Michael D <michael.d.kin...@intel.com>; Mangefeste, Tony
> <tony.mangefe...@intel.com>; edk2-devel@lists.01.org <edk2-de...@ml01.01.org>
> Subject: Re: [edk2] EDK2 Staging Repository 2nd Draft
> 
> On Tue, 2016-03-15 at 00:16 +, Kinney, Michael D wrote:
> >
> >
> > Can you provide some revised text you would like to see in step 6.
> >
> > I agree that we need to use the tools in ways that help make this easy, 
> > prevent
> > errors, and preserve history.  Given that step 6 describes promoting a 
> > feature
> > from edk2-staging repo to edk2 repo, what set of git operations you would
> > recommend?
> 
> Literally, 'git pull'. So my change in patch form would be:
> 
> @@
>  6) Process to promote an edk2-staging branch to edk2 trunk
>  a) Request sent to edk2-devel that describes the feature, design, 
> testing,
> etc.
>  b) Stewards evaluate request and determine if the feature meets edk2 
> criteria.
> -c) If approved, use edk2 patch review/commit process on edk2-devel 
> mailing
> list
> +    c) If approved, use 'git pull' to merge the branch into edk2 master, 
> adding
> +   any final 'Reviewed-by:' tags to the merge commit as appropriate.
>  d) Remove feature branch from edk2-staging (maybe archived 
> elsewhere?).
> 
> 
> I'm still interested in what benefit we gain from a centralised
> edk2-staging repository though, over the normal process of contributors
> just pushing their work to github repositories and submitting PRs.
> 
> --
> dwmw2

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-15 Thread David Woodhouse 
On Tue, 2016-03-15 at 00:16 +, Kinney, Michael D wrote:
> 
> 
> Can you provide some revised text you would like to see in step 6.
> 
> I agree that we need to use the tools in ways that help make this easy, 
> prevent
> errors, and preserve history.  Given that step 6 describes promoting a 
> feature 
> from edk2-staging repo to edk2 repo, what set of git operations you would 
> recommend?

Literally, 'git pull'. So my change in patch form would be:

@@
 6) Process to promote an edk2-staging branch to edk2 trunk
 a) Request sent to edk2-devel that describes the feature, design, 
testing, etc.
 b) Stewards evaluate request and determine if the feature meets edk2 
criteria.
-c) If approved, use edk2 patch review/commit process on edk2-devel 
mailing list
+    c) If approved, use 'git pull' to merge the branch into edk2 master, 
adding
+   any final 'Reviewed-by:' tags to the merge commit as appropriate.
 d) Remove feature branch from edk2-staging (maybe archived 
elsewhere?). 


I'm still interested in what benefit we gain from a centralised
edk2-staging repository though, over the normal process of contributors
just pushing their work to github repositories and submitting PRs.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread Kinney, Michael D 
David,

Can you provide some revised text you would like to see in step 6.

I agree that we need to use the tools in ways that help make this easy, prevent
errors, and preserve history.  Given that step 6 describes promoting a feature 
from edk2-staging repo to edk2 repo, what set of git operations you would 
recommend?

Thanks,

Mike

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of David 
> Woodhouse
> Sent: Monday, March 14, 2016 3:48 PM
> To: Mangefeste, Tony <tony.mangefe...@intel.com>; edk2-devel@lists.01.org 
>  de...@ml01.01.org>
> Subject: Re: [edk2] EDK2 Staging Repository 2nd Draft
> 
> On Mon, 2016-03-14 at 22:38 +, Mangefeste, Tony wrote:
> > Yes there is work that is substantial enough.
> 
> I'm not sure what that's an answer to; you didn't leave a specific part
> of my email above it as context.
> 
> I was saying that *when* work is substantial enough to warrant such
> staging, that is *precisely* the kind of work where we want to be using
> the tools properly and *pull* it in rather than rebasing it.
> 
> --
> David WoodhouseOpen Source Technology Centre
> david.woodho...@intel.com  Intel Corporation

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread David Woodhouse 
On Mon, 2016-03-14 at 22:38 +, Mangefeste, Tony wrote:
> Yes there is work that is substantial enough.

I'm not sure what that's an answer to; you didn't leave a specific part
of my email above it as context.

I was saying that *when* work is substantial enough to warrant such
staging, that is *precisely* the kind of work where we want to be using
the tools properly and *pull* it in rather than rebasing it.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation



smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread Mangefeste, Tony 
Yes there is work that is substantial enough.

-- Original message--
From: David Woodhouse
Date: Mon, Mar 14, 2016 3:19 PM
To: Mangefeste, Tony;edk2-devel@lists.01.org;
Cc:
Subject:Re: [edk2] EDK2 Staging Repository 2nd Draft


On Sat, 2016-03-12 at 00:25 +, Mangefeste, Tony wrote:> > 
6) Process to promote an edk2-staging branch to edk2 trunk> a) Request 
sent to edk2-devel that describes the feature, design, testing, etc.> 
b) Stewards evaluate request and determine if the feature meets edk2 criteria.> 
c) If approved, use edk2 patch review/commit process on edk2-devel 
mailing list> d) Remove feature branch from edk2-staging (maybe 
archived elsewhere?). Anything which is substantial enough to warrant this kind 
of 'staging'definitely wants to be *pulled* into the main tree, rather 
thanrebasing it onto the current master, losing accurate history 
andinvalidating the testing that's been done.The 'Reviewed-by' 
and similar tags could be collected by the submitterand added in their own tree 
(leaving them in control of whether theyrebase, and responsible for subsequent 
retesting everything).Perhaps better still, the Reviewed-by: could be included 
in 
 the *merge*commit.Either way, let's not further entrench incorrect rebase 
behaviour.-- David WoodhouseOpen Source Technology 
centredavid.woodho...@intel.com  
Intel Corporation
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread David Woodhouse 
On Sat, 2016-03-12 at 00:25 +, Mangefeste, Tony wrote:
> 
> 6) Process to promote an edk2-staging branch to edk2 trunk
> a) Request sent to edk2-devel that describes the feature, design, 
> testing, etc.
> b) Stewards evaluate request and determine if the feature meets edk2 
> criteria.
> c) If approved, use edk2 patch review/commit process on edk2-devel 
> mailing list
> d) Remove feature branch from edk2-staging (maybe archived 
> elsewhere?). 

Anything which is substantial enough to warrant this kind of 'staging'
definitely wants to be *pulled* into the main tree, rather than
rebasing it onto the current master, losing accurate history and
invalidating the testing that's been done.

The 'Reviewed-by' and similar tags could be collected by the submitter
and added in their own tree (leaving them in control of whether they
rebase, and responsible for subsequent retesting everything).

Perhaps better still, the Reviewed-by: could be included in the *merge*
commit.

Either way, let's not further entrench incorrect rebase behaviour.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation



smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread David Woodhouse 
On Sat, 2016-03-12 at 00:25 +, Mangefeste, Tony wrote:
> 
> Problem statement
> =
> Need place on tianocore.org where new features that are not ready for
> product integration can be checked in for evaluation by the EDK II
> community prior to adding to the edk2 trunk.  This serves several
> purposes:
> 
> * Encourage source code to be shared earlier in the development
> process
> * Allow source code to be shared that does not yet meet all edk2
> required quality criteria
> * Allow source code to be shared so the EDK II community can help
> finish and validate new features
> * Provide a location to hold new features until they are deemed ready
> for product integration
> * Provide a location to hold new features until there is a natural
> point in edk2 release cycle to fully validate the new feature.
> * Not intended to be used for bug fixes.
> * Not intended to be used for small, simple, or low risk features.

Surely all of this is covered by the normal process of contributors
publishing their work in github repositories of their own?

Why do we need to do something different?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation



smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-14 Thread Gao, Liming 
On edk2-staging, do we need to regularly sync edk2 change into edk2-staging and 
keep edk2-staging as the mirror of ed2 project? Or align it to edk2 project 
only when new udk release is made? I think it will be fine to let edk2-staging 
align to edk2 udk release, not as edk2 mirror. New feature can be developed 
based on edk2 udk branch, not edk2 master. 

Thanks
Liming
-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
Mangefeste, Tony
Sent: Saturday, March 12, 2016 8:26 AM
To: edk2-devel@lists.01.org 
Subject: [edk2] EDK2 Staging Repository 2nd Draft
Importance: High

After collecting numerous feedback, here's a clean 2nd proposal for review.  
* Message format clean
* Approval process updates for stanging -> EDK2 trunk
* The intention of the staging branch is _not_ to work on features that grow to 
unreasonable sizes.  It is to manage features in an isolated environment that 
requires collaboration and testing by the community.  The stewards will ensure 
that any features that are developed, tested within the staging area are 
managed closely, and prevent unmanageable merges.
* Other minor fixes, changes...

Let's drive for closure on this by Tuesday, March 15, 2016.  If anyone else 
speaks up and needs more time, please speak up, right away.



=
edk2-staging Proposal V2
=

Problem statement
=
Need place on tianocore.org where new features that are not ready for product 
integration can be checked in for evaluation by the EDK II community prior to 
adding to the edk2 trunk.  This serves several purposes:

* Encourage source code to be shared earlier in the development process
* Allow source code to be shared that does not yet meet all edk2 required 
quality criteria
* Allow source code to be shared so the EDK II community can help finish and 
validate new features
* Provide a location to hold new features until they are deemed ready for 
product integration
* Provide a location to hold new features until there is a natural point in 
edk2 release cycle to fully validate the new feature.
* Not intended to be used for bug fixes.
* Not intended to be used for small, simple, or low risk features.

Proposal

1) Create a new repo called edk2-staging
a) edk2-staging is a fork of edk
b) edk2-staging/master tracks edk2/master

2) All edk2-staging discussions use the existing edk2-devel mailing list for 
design/patch/test.
Use the following style for discussion of a specific feature branch in 
edk2-staging repo.

[staging/branch]: Subject

3) All commits to edk2-staging must follow same edk2 rules (e.g. Tiano 
Contributor's Agreement)

4) Process to add a new feature to edk2-staging
a) Request to create a new edk2-staging feature branch sent to 
edk2-devel
Request should include feature summary, owners, timeline, and quality 
criteria to add to edk2.
If Request is a platform or package specific feature, pre-approval for 
edk2 trunk promotion may be requested here.
b) Branch request and branch name must be approved by stewards
c) Branch maintainer creates edk2-staging feature branch
d) Branch maintainer creates Readme.MD in root of feature branch with 
summary, owners, timeline, links to related materials.
e) Branch maintainer is responsible for making sure feature is 
frequently synced to edk2/master

 NOTE: Feature branch may initially use a stable edk2 tag.  As feature 
stabilizes, syncs to edk2/master can begin.

5) Process to update sources in feature branch
a) Patch email send to edk2-devel
b) Commit message subject format

[staging/branch PATCH]: Package/Module: Subject

c) Use same edk2-devel review process 
d) If pass review, then maintainer commits change to edk2-staging 
feature branch

NOTE: win32 binaries are not automatically generated if a feature 
branch includes BaseTools source changes.

6) Process to promote an edk2-staging branch to edk2 trunk
a) Request sent to edk2-devel that describes the feature, design, 
testing, etc.
b) Stewards evaluate request and determine if the feature meets edk2 
criteria.
c) If approved, use edk2 patch review/commit process on edk2-devel 
mailing list
d) Remove feature branch from edk2-staging (maybe archived elsewhere?). 

7) Process to remove an edk2-staging branch
a) Stewards periodically review of feature branches in edk2-staging 
(once a quarter?)
b) If no activity for extended period of time and feature is no longer 
deemed a candidate for edk2 then stewards send email to edk2-devel to request 
deletion of feature branch.  
c) If no objections from EDK II community, then feature branch is 
deleted (maybe archived elsewhere?).

8) Process to evaluate a feature in edk2-staging
a) Clone edk2
b) Create local

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-11 Thread Mangefeste, Tony 
That's a good idea.

-- Original message--
From: Justen, Jordan L
Date: Fri, Mar 11, 2016 5:34 PM
To: Mangefeste, Tony;edk2-devel@lists.01.org;
Cc:
Subject:Re: [edk2] EDK2 Staging Repository 2nd Draft

On 2016-03-11 16:25:39, Mangefeste, Tony wrote:
>
> 6) Process to promote an edk2-staging branch to edk2 trunk
> a) Request sent to edk2-devel that describes the feature, design, 
> testing, etc.
> b) Stewards evaluate request and determine if the feature meets edk2 
> criteria.
> c) If approved, use edk2 patch review/commit process on edk2-devel 
> mailing list
> d) Remove feature branch from edk2-staging (maybe archived 
> elsewhere?).
>
> 7) Process to remove an edk2-staging branch
> a) Stewards periodically review of feature branches in edk2-staging 
> (once a quarter?)
> b) If no activity for extended period of time and feature is no 
> longer deemed a candidate for edk2 then stewards send email to edk2-devel to 
> request deletion of feature branch.
> c) If no objections from EDK II community, then feature branch is 
> deleted (maybe archived elsewhere?).

I recommend we move all committed and abandoned staging branches to:

https://github.com/tianocore/edk2-archive

Archiving the committed patchsets might not be useful though.

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository 2nd Draft

2016-03-11 Thread Jordan Justen 
On 2016-03-11 16:25:39, Mangefeste, Tony wrote:
> 
> 6) Process to promote an edk2-staging branch to edk2 trunk
> a) Request sent to edk2-devel that describes the feature, design, 
> testing, etc.
> b) Stewards evaluate request and determine if the feature meets edk2 
> criteria.
> c) If approved, use edk2 patch review/commit process on edk2-devel 
> mailing list
> d) Remove feature branch from edk2-staging (maybe archived 
> elsewhere?). 
> 
> 7) Process to remove an edk2-staging branch
> a) Stewards periodically review of feature branches in edk2-staging 
> (once a quarter?)
> b) If no activity for extended period of time and feature is no 
> longer deemed a candidate for edk2 then stewards send email to edk2-devel to 
> request deletion of feature branch.  
> c) If no objections from EDK II community, then feature branch is 
> deleted (maybe archived elsewhere?).

I recommend we move all committed and abandoned staging branches to:

https://github.com/tianocore/edk2-archive

Archiving the committed patchsets might not be useful though.

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Laszlo Ersek 
On 03/10/16 19:29, Kinney, Michael D wrote:
>> -Original Message-
>> From: Laszlo Ersek [mailto:ler...@redhat.com]

>> Keeping work-in-progress modules in a separate repo has a critical
>> drawback: those modules will not share the commit history of the main
>> edk2 repo.
> 
> The issue with a single staging directory in edk2 is that we can not work on
> different features that touch the same module/lib.  This is where branching
> provides helps.  One purpose of edk2-staging is complex features that touch
> several modules, likely across multiple packages.
> 
> Maybe there are 2 concepts here.  One is feature branches for active 
> Design/dev/test.  Another is content that is functional and stable and 
> synced with the edk2 trunk, but has some number of "known" issues.
> 
>>
>> Consider the following example: a new release or erratum of the UEFI or
>> PI spec is published, and a developer contributes the implementation of
>> a new feature (or bugfix) to one of the core packages: MdePkg,
>> MdeModulePkg, NetworkPkg, etc. Then a platform developer would like to
>> rely on this UEFI or PI change in his or her work-in-progress module. If
>> that module lives outside of the main edk2 repo, it will cause a
>> dependency nightmare.
> 
> I the proposal, step (4e) says the feature branch maintainer needs to 
> keep the feature branch synced with the edk2 trunk.  The proposal uses
> the term "rebase", but that may not be the right operation to track
> changes from edk2 into an edk2-staging branch.  Maybe it is a "merge" 
> operation.
> 
> Also, for some new features, we may want to use a stable tag from edk2,
> so the feature development work can progress without having to track
> edk2.  Then as the feature matures and preparation for commit back to 
> edk2 begins, sync with edk2 is required.

Thank you for the clarification. I re-read the proposal. It seems to me
that the staging repo (with its feature branches) would enable several
people to collaborate on a longer term development, without disturbing
the master branch in edk2, and also without creating those feature
branches in edk2.

If several people (collaborators) intend to fetch those topic branches
periodically, then the "sync" operations will indeed have to be pulls
from edk2's master branch (otherwise everyone who fetches will have to
rebase locally).

However, I don't know what a good strategy is for deciding *when* to
pull. In another discussion, David linked

yesterday. In that message the suggestion seems to be: pull from master
only when master sees an important milestone; for example it satisfies a
new dependency of the feature branch, or reaches a release, or something
like that.

Also, in this case, when the feature is finally accepted, it will also
have to be a pull (in the opposite direction). Clearly, this will make
David's recent proposal (about accepting pull requests) unavoidable.

> I agree that not every development task should use a feature branch.  We 
> need clear criteria for using feature branches.  Please feel free to 
> provide additional guidelines on when you think a feature branch is 
> required or not required.

In QEMU and Linux, subsystems (and their maintainers) are the units of
separation, I think.

I also think that the main motivation for such separation is primarily
scalability -- the patch influx is just too high for a single (set) of
maintainers to handle. (Occasionally they have separate mailing lists
too.) For a counter-example, libvirt has a mostly linear history,
perhaps enabled by a lower patch load.

Edk2 doesn't have a huge patch influx. I guess the only guideline I can
come up with is: request a branch if you have to take turns with other
people at developing a large feature, and the feature really cannot be
committed in smaller waves to the main repo.

However, this also hides dangers. What if the feature grows too large
and too risky to pull at once? In Linux and QEMU this is mitigated by
time-based development cycles and frequent releases, and everyone does
their best to fit into such a cycle with their next feature -- or at
least plan and break up the feature into such milestones. Edk2 doesn't
have frequent time-based development cycles and releases. And conflict
resolution is decidedly un-fun.

... I'd like to defer to people who have experience with multi-level
distributed development from the maintainer side.

Laszlo

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Kinney, Michael D 
> -Original Message-
> From: Laszlo Ersek [mailto:ler...@redhat.com]
> Sent: Thursday, March 10, 2016 2:03 AM
> To: Kinney, Michael D <michael.d.kin...@intel.com>; Justen, Jordan L
> <jordan.l.jus...@intel.com>; Mangefeste, Tony <tony.mangefe...@intel.com>; 
> edk2-
> de...@lists.01.org <edk2-de...@ml01.01.org>
> Subject: Re: [edk2] EDK2 Staging Repository
> 
> On 03/10/16 00:02, Kinney, Michael D wrote:
> > Jordan,
> >
> > Responses included below.
> >
> > Mike
> >
> >
> >
> >> -Original Message-
> >> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> >> Jordan Justen
> >> Sent: Wednesday, March 9, 2016 1:47 PM
> >> To: Mangefeste, Tony <tony.mangefe...@intel.com>; edk2-devel@lists.01.org
> >> Subject: Re: [edk2] EDK2 Staging Repository
> >>
> >> On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
> >>> Below are the details of the proposal for a staging branch, please review 
> >>> and
> >> comment.
> >>>
> >>> 
> >>>
> >>> Problem statement
> >>> =
> >>> Need place on tianocore.org where new features that are not ready for 
> >>> product
> >>> integration can be checked in for evaluation by the EDK II community 
> >>> prior to
> >>> adding to the edk2 trunk.  This serves several purposes:
> >>>
> >>> * Encourage source code to be shared earlier in the development process
> >>> * Allow source code to be shared that does not yet meet all edk2 required 
> >>> quality
> >> criteria
> >>> * Allow source code to be shared so the EDK II community can help finish 
> >>> and
> validate
> >> new features
> >>> * Provide a location to hold new features until they are deemed ready for 
> >>> product
> >> integration
> >>> * Provide a location to hold new features until there is a natural point 
> >>> in edk2
> >> release cycle to fully validate the new feature.
> >>> * Not intended to be used for bug fixes.
> >>> * Not intended to be used for small, simple, or low risk features.
> >>>
> >>
> >> I think the parts about making a central place for big feature
> >> branches seems good. I think Laszlo might have appreciated having a
> >> well defined central location for his OVMF SMM enabling branch.
> >>
> >> I think the processes around allowing the code to be merged seems too
> >> heavyweight. In the cases of big patchsets, it can be burdensome
> >> enough just to get code review. To insert another process step (and
> >> set of approvers) to actually merge the feature seems perhaps
> >> unneccesary.
> >>
> >>> Proposal
> >>> 
> >>> 1) Create a new repo called edk2-staging
> >>> a) edk2-staging is a fork of edk
> >>> b) edk2-staging/master tracks edk2/master
> >>>
> >>
> >> For branches maintained by package maintainers, could we have them
> >> under the edk2 repo and named staging/*?
> >
> > Do you mean a directory in the edk2 repo called staging?  I did think about 
> > this
> option,
> > but I am concerned about code that is not ready for product integration 
> > being in the
> edk2
> > repo and I am also concerned about the size of the edk2 repo.  Creating a 
> > new repo
> that is
> > a fork of edk2 addressed both of those issues.
> 
> I disagree.
> 
> I have now re-read our private discussion from last year, when we were
> working on SMM (the core infrastructure and the OVMF platform enablement
> in parallel). The message ID of the root of the sub-thread is
> <564f50b9.6040...@redhat.com>, if someone on CC at that time would like
> to re-read it now.
> 
> At that time, the consensus seemed to be to create a top-level Staging/
> directory, with packages under it (with their DEC files and modules).
> So, we would have:
> 
> ArmVirtPkg/...
> EmulatorPkg/...
> OvmfPkg/...
> Staging/ArmVirtPkg/...
> Staging/EmulatorPkg/...
> Staging/OvmfPkg/...
> 
> This matches how Linux works, and the naming makes it very clear that
> modules (drivers, applications, library instances) under Staging are
> work in progress. It is not possible to include a module from under
> Staging as an oversight.
> 
> Keeping work-in-progress modules in a separate repo has a critical
> drawback: those modules will not

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Mangefeste, Tony 
Yep, I've heard that loud and clear.  It must not be only web UI, but can be 
over email _if_ we move with Gerrit or something similar.



-Original Message-
From: Laszlo Ersek [mailto:ler...@redhat.com] 
Sent: Thursday, March 10, 2016 9:34 AM
To: Mangefeste, Tony <tony.mangefe...@intel.com>; Gerd Hoffmann 
<kra...@redhat.com>; Bruce Cran <br...@cran.org.uk>
Cc: Justen, Jordan L <jordan.l.jus...@intel.com>; edk2-devel@lists.01.org 
<edk2-de...@ml01.01.org>
Subject: Re: [edk2] EDK2 Staging Repository

On 03/10/16 17:55, Mangefeste, Tony wrote:
> This is fantastic, because the other two things I would like for us all to 
> consider is Gerrit+Jenkins support.

Sorry for repeating myself :(, but:

Please make sure that all reviews done on the web are forwarded to the mailing 
list, and that I can chime in using only my MUA.

Thanks
Laszlo

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Laszlo Ersek 
On 03/10/16 17:55, Mangefeste, Tony wrote:
> This is fantastic, because the other two things I would like for us all to 
> consider is Gerrit+Jenkins support.

Sorry for repeating myself :(, but:

Please make sure that all reviews done on the web are forwarded to the
mailing list, and that I can chime in using only my MUA.

Thanks
Laszlo

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Bruce Cran 

On 3/10/16 9:55 AM, Mangefeste, Tony wrote:

This is fantastic, because the other two things I would like for us all to 
consider is Gerrit+Jenkins support.


Why Gerrit?  Everyone I've seen talk about it hate its UI.

FWIW I seem to be leading a bit of a rebellion at $work, since I setup a 
demo Phabricator instance (for the repo browser and code review) and 
most people love it, to the extent that they're saying they wish we'd 
used it from the start and wondering if any engineers were involved in 
the decision to use the tool that we're supposed to be using.


--
Bruce
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Mangefeste, Tony 
This is fantastic, because the other two things I would like for us all to 
consider is Gerrit+Jenkins support.

I'll be talking to you to get more details. 

-Original Message-
From: Gerd Hoffmann [mailto:kra...@redhat.com] 
Sent: Thursday, March 10, 2016 12:39 AM
To: Bruce Cran <br...@cran.org.uk>
Cc: Justen, Jordan L <jordan.l.jus...@intel.com>; Mangefeste, Tony 
<tony.mangefe...@intel.com>; edk2-devel@lists.01.org <edk2-de...@ml01.01.org>
Subject: Re: [edk2] EDK2 Staging Repository

On Mi, 2016-03-09 at 16:57 -0700, Bruce Cran wrote:
> On 3/9/16 2:59 PM, Jordan Justen wrote:
> 
> > So far I have a setup that can test building BaseTools and OVMF on 
> > Linux. The Windows setup is a little more complicated, but I don't 
> > think it should be too difficult.
> 
> I guess that's Linux x86_64? Any solution we come up with should also 
> be extendable to AARCH64.

I do have jenkins running, doing the builds for https://www.kraxel.org/repos/, 
including aarch64 and arm builds (using a cross compiler).

If there is interest I configure jenkins to send build failures to the list 
too.  There are some drawbacks though:

  (1) There are false positives now and then for various reasons:
  * Now and then manual invention is needed to keep things going
(for example on openssl updates).
  * The rpm package has patches which need a rebase now and then.
  * Occasionally infrastructure hickups.
  (2) Sometimes it takes me a few days to deal with those issues,
  jenkins will keep on reporting the build failures until I did.
  (3) The system runs on a private network behind a NAT router, so you
  wouldn't be able to click on the links for build logs etc, all you
  get is the mail reporting the failure.
  (4) It's designed to provide rpms for the master branch, that it
  test-builds the master branch is more a side-effect.  I most
  likely wouldn't extend this to also build staging branches.

If you ever wondered how Laszlo notices quickly if someone breaks the build, he 
gets the jenkins reports ;)

cheers,
  Gerd

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Laszlo Ersek 
On 03/10/16 00:02, Kinney, Michael D wrote:
> Jordan,
> 
> Responses included below.
> 
> Mike
> 
> 
> 
>> -Original Message-
>> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
>> Jordan Justen
>> Sent: Wednesday, March 9, 2016 1:47 PM
>> To: Mangefeste, Tony <tony.mangefe...@intel.com>; edk2-devel@lists.01.org
>> Subject: Re: [edk2] EDK2 Staging Repository
>>
>> On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
>>> Below are the details of the proposal for a staging branch, please review 
>>> and
>> comment.
>>>
>>> 
>>>
>>> Problem statement
>>> =
>>> Need place on tianocore.org where new features that are not ready for 
>>> product
>>> integration can be checked in for evaluation by the EDK II community prior 
>>> to
>>> adding to the edk2 trunk.  This serves several purposes:
>>>
>>> * Encourage source code to be shared earlier in the development process
>>> * Allow source code to be shared that does not yet meet all edk2 required 
>>> quality
>> criteria
>>> * Allow source code to be shared so the EDK II community can help finish 
>>> and validate
>> new features
>>> * Provide a location to hold new features until they are deemed ready for 
>>> product
>> integration
>>> * Provide a location to hold new features until there is a natural point in 
>>> edk2
>> release cycle to fully validate the new feature.
>>> * Not intended to be used for bug fixes.
>>> * Not intended to be used for small, simple, or low risk features.
>>>
>>
>> I think the parts about making a central place for big feature
>> branches seems good. I think Laszlo might have appreciated having a
>> well defined central location for his OVMF SMM enabling branch.
>>
>> I think the processes around allowing the code to be merged seems too
>> heavyweight. In the cases of big patchsets, it can be burdensome
>> enough just to get code review. To insert another process step (and
>> set of approvers) to actually merge the feature seems perhaps
>> unneccesary.
>>
>>> Proposal
>>> 
>>> 1) Create a new repo called edk2-staging
>>> a) edk2-staging is a fork of edk
>>> b) edk2-staging/master tracks edk2/master
>>>
>>
>> For branches maintained by package maintainers, could we have them
>> under the edk2 repo and named staging/*?
> 
> Do you mean a directory in the edk2 repo called staging?  I did think about 
> this option,
> but I am concerned about code that is not ready for product integration being 
> in the edk2
> repo and I am also concerned about the size of the edk2 repo.  Creating a new 
> repo that is
> a fork of edk2 addressed both of those issues.

I disagree.

I have now re-read our private discussion from last year, when we were
working on SMM (the core infrastructure and the OVMF platform enablement
in parallel). The message ID of the root of the sub-thread is
<564f50b9.6040...@redhat.com>, if someone on CC at that time would like
to re-read it now.

At that time, the consensus seemed to be to create a top-level Staging/
directory, with packages under it (with their DEC files and modules).
So, we would have:

ArmVirtPkg/...
EmulatorPkg/...
OvmfPkg/...
Staging/ArmVirtPkg/...
Staging/EmulatorPkg/...
Staging/OvmfPkg/...

This matches how Linux works, and the naming makes it very clear that
modules (drivers, applications, library instances) under Staging are
work in progress. It is not possible to include a module from under
Staging as an oversight.

Keeping work-in-progress modules in a separate repo has a critical
drawback: those modules will not share the commit history of the main
edk2 repo.

Consider the following example: a new release or erratum of the UEFI or
PI spec is published, and a developer contributes the implementation of
a new feature (or bugfix) to one of the core packages: MdePkg,
MdeModulePkg, NetworkPkg, etc. Then a platform developer would like to
rely on this UEFI or PI change in his or her work-in-progress module. If
that module lives outside of the main edk2 repo, it will cause a
dependency nightmare.

Proprietary downstreams of edk2 already manage this, but let that remain
only their problem.

For OvmfPkg and ArmVirtPkg minimally, a Staging/ directory only makes
sense to me if it lives in the main edk2 repository, and shares its
commit history. Staging/ doesn't stand for separate product, separate
maintainership, separate life cycle, etc -- the sole difference with
staging modules is that we publicly admit, "yeah these are not fully
polished yet

Re: [edk2] EDK2 Staging Repository

2016-03-10 Thread Gerd Hoffmann 
On Mi, 2016-03-09 at 16:57 -0700, Bruce Cran wrote:
> On 3/9/16 2:59 PM, Jordan Justen wrote:
> 
> > So far I have a setup that can test building BaseTools and OVMF on
> > Linux. The Windows setup is a little more complicated, but I don't
> > think it should be too difficult.
> 
> I guess that's Linux x86_64? Any solution we come up with should also be 
> extendable to AARCH64.

I do have jenkins running, doing the builds for
https://www.kraxel.org/repos/, including aarch64 and arm builds (using a
cross compiler).

If there is interest I configure jenkins to send build failures to the
list too.  There are some drawbacks though:

  (1) There are false positives now and then for various reasons:
  * Now and then manual invention is needed to keep things going
(for example on openssl updates).
  * The rpm package has patches which need a rebase now and then.
  * Occasionally infrastructure hickups.
  (2) Sometimes it takes me a few days to deal with those issues,
  jenkins will keep on reporting the build failures until I did.
  (3) The system runs on a private network behind a NAT router, so you
  wouldn't be able to click on the links for build logs etc, all you
  get is the mail reporting the failure.
  (4) It's designed to provide rpms for the master branch, that it
  test-builds the master branch is more a side-effect.  I most
  likely wouldn't extend this to also build staging branches.

If you ever wondered how Laszlo notices quickly if someone breaks the
build, he gets the jenkins reports ;)

cheers,
  Gerd

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Jordan Justen 
On 2016-03-09 16:36:04, Kinney, Michael D wrote:
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> > Jordan Justen
> > 
> > On 2016-03-09 15:02:41, Kinney, Michael D wrote:
> > >
> > > Package maintainers are involved in the code review of all patches in the 
> > > edk2-staging
> > > repo, so they have already looked at the code.  When the maintainer 
> > > thinks it is ready
> > > for the edk2 trunk, then the idea is that submission of major features 
> > > may need to be
> > > scheduled based on other feature integration, current stability of edk2 
> > > trunk, and the
> > > current point in the edk2 release cycle.  So some features may be held 
> > > temporarily even
> > > if they are considered 100% ready.  These are some of the reasons for the 
> > > additional
> > > approval steps.  Do you have suggestions on how to make it lighter weight?
> > >
> > 
> > I can think of two concerns.
> > 
> > 1. Remember all the effort Laszlo went through to rework various
> >packages for OVMF SMM support? It took quite a while. At the end of
> >that would we be adding another several week delay/hurdle?
> > 
> > 2. If someone is working on a big feature for OVMF, then I might want
> >to have it be a staging branch. Especially if we can enable
> >Continuous Integration for staging branches. But, once the code
> >review is complete, it would be nice if we could merge this code to
> >OvmfPkg without going through this extra step.
> > 
> >Maybe we'd just need a separate process to accommodate something
> >like this?
> 
> I would hope there is no significant delays to get a response to a request.
> Given time zones, should be max of 1 day.

Ok. That sounds reasonable.

> I think it depends on the scope of the change.  If the change is only to 
> source code in the scope of a single platform, then maybe a feature branch
> in edk2-staging is not required.  Though I agree there are platform 
> specific features that are very complex and using a staging branch helps.
> However, if it is a complex feature that touches a combination of core 
> packages and platform packages, then coordination of the commit back to 
> the edk2 trunk is required because of the core package changes.
> 
> Maybe the process used to request a new feature branch (step 4a) should 
> include a discussion of the criteria that must be met to commit back to 
> edk2.  In the example of an OVMF only feature, the feature branch creation
> process can request pre-approval of commit back to edk2 once the OVMF 
> maintainer(s) agree the criteria have been met.
> 

It sounds like this could work for platform specific patchsets.

Thanks,

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Kinney, Michael D 


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Jordan 
> Justen
> Sent: Wednesday, March 9, 2016 4:01 PM
> To: Kinney, Michael D <michael.d.kin...@intel.com>; Mangefeste, Tony
> <tony.mangefe...@intel.com>; edk2-devel@lists.01.org; Kinney, Michael D
> <michael.d.kin...@intel.com>
> Subject: Re: [edk2] EDK2 Staging Repository
> 
> On 2016-03-09 15:02:41, Kinney, Michael D wrote:
> >
> > > -Original Message-
> > > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> > > Jordan
> Justen
> > >
> > > On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
> > > >
> > > > Proposal
> > > > 
> > > > 1) Create a new repo called edk2-staging
> > > > a) edk2-staging is a fork of edk
> > > > b) edk2-staging/master tracks edk2/master
> > > >
> > >
> > > For branches maintained by package maintainers, could we have them
> > > under the edk2 repo and named staging/*?
> >
> > Do you mean a directory in the edk2 repo called staging?  I did think about 
> > this
> option,
> > but I am concerned about code that is not ready for product integration 
> > being in the
> edk2
> > repo and I am also concerned about the size of the edk2 repo.  Creating a 
> > new repo
> that is
> > a fork of edk2 addressed both of those issues.
> >
> 
> No, I just meant a branch named staging/foo directly in the main edk2
> repo.
> 
> Regarding the repo size, isn't it expected that the staging branches
> will be removed once merged?
> 
> They sound similar to a 'feature' branches which commonly would be
> added to the main repo.
> 
> Maybe it would be to much noise in the main repo that most people
> would prefer to not see.
> 
> > >
> > > > c) Branch maintainer creates edk2-staging feature branch
> > >
> > > Who would be the maintainer? The submitter, or perhaps a package
> > > maintainer that is closely related to the new series. (For example, if
> > > it was a series closely related to OVMF, then Laszlo and I would be
> > > responsible for the branch.)
> >
> > Yes.  Every new feature that is accepted requires one of the Package 
> > Maintainers
> > to be the sponsor for the feature and perform the branch level maintenance 
> > actions.
> >
> 
> This makes it feasible to have all the staging branches in the main
> repo under staging/* branches ... if we decided that was a good idea.
> 
> > > >
> > > > c) Use same edk2-devel review process
> > > > d) If pass review, then maintainer commits change to 
> > > > edk2-staging feature
> > > branch
> > > >
> > > > NOTE: win32 binaries are not automatically generated if a 
> > > > feature branch
> > > includes BaseTools source changes.
> > > >
> > > > 5) Process to promote an edk2-staging branch to edk2 trunk
> > > > a) Request sent to edk2-devel that describes the feature, 
> > > > design,
> testing,
> > > etc.
> > > > b) Stewards evaluate request and determine if the feature meets 
> > > > edk2
> > > criteria.
> > > > c) If approved, use edk2 patch review/commit process on 
> > > > edk2-devel
> mailing
> > > list
> > > > d) Remove feature branch from edk2-staging (maybe archived 
> > > > elsewhere?).
> > > >
> > >
> > > This part seems a bit heavy weight. I think once code has passed
> > > review by the package maintainers it should be eligible to be
> > > committed to the trunk.
> > >
> > > Or, is this a process by which the package maintainer may not review
> > > the code until this step?
> > >
> >
> > Package maintainers are involved in the code review of all patches in the 
> > edk2-
> staging
> > repo, so they have already looked at the code.  When the maintainer thinks 
> > it is
> ready
> > for the edk2 trunk, then the idea is that submission of major features may 
> > need to be
> > scheduled based on other feature integration, current stability of edk2 
> > trunk, and
> the
> > current point in the edk2 release cycle.  So some features may be held 
> > temporarily
> even
> > if they are considered 100% ready.  These are some of the reasons for the 
> > additional
> > approv

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Jordan Justen 
On 2016-03-09 15:02:41, Kinney, Michael D wrote:
> 
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
> > Jordan Justen
> > 
> > On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
> > >
> > > Proposal
> > > 
> > > 1) Create a new repo called edk2-staging
> > > a) edk2-staging is a fork of edk
> > > b) edk2-staging/master tracks edk2/master
> > >
> > 
> > For branches maintained by package maintainers, could we have them
> > under the edk2 repo and named staging/*?
> 
> Do you mean a directory in the edk2 repo called staging?  I did think about 
> this option,
> but I am concerned about code that is not ready for product integration being 
> in the edk2
> repo and I am also concerned about the size of the edk2 repo.  Creating a new 
> repo that is
> a fork of edk2 addressed both of those issues.
> 

No, I just meant a branch named staging/foo directly in the main edk2
repo.

Regarding the repo size, isn't it expected that the staging branches
will be removed once merged?

They sound similar to a 'feature' branches which commonly would be
added to the main repo.

Maybe it would be to much noise in the main repo that most people
would prefer to not see.

> > 
> > > c) Branch maintainer creates edk2-staging feature branch
> > 
> > Who would be the maintainer? The submitter, or perhaps a package
> > maintainer that is closely related to the new series. (For example, if
> > it was a series closely related to OVMF, then Laszlo and I would be
> > responsible for the branch.)
> 
> Yes.  Every new feature that is accepted requires one of the Package 
> Maintainers
> to be the sponsor for the feature and perform the branch level maintenance 
> actions.
>

This makes it feasible to have all the staging branches in the main
repo under staging/* branches ... if we decided that was a good idea.

> > >
> > > c) Use same edk2-devel review process
> > > d) If pass review, then maintainer commits change to edk2-staging 
> > > feature
> > branch
> > >
> > > NOTE: win32 binaries are not automatically generated if a feature 
> > > branch
> > includes BaseTools source changes.
> > >
> > > 5) Process to promote an edk2-staging branch to edk2 trunk
> > > a) Request sent to edk2-devel that describes the feature, design, 
> > > testing,
> > etc.
> > > b) Stewards evaluate request and determine if the feature meets 
> > > edk2
> > criteria.
> > > c) If approved, use edk2 patch review/commit process on 
> > > edk2-devel mailing
> > list
> > > d) Remove feature branch from edk2-staging (maybe archived 
> > > elsewhere?).
> > >
> > 
> > This part seems a bit heavy weight. I think once code has passed
> > review by the package maintainers it should be eligible to be
> > committed to the trunk.
> > 
> > Or, is this a process by which the package maintainer may not review
> > the code until this step?
> > 
> 
> Package maintainers are involved in the code review of all patches in the 
> edk2-staging
> repo, so they have already looked at the code.  When the maintainer thinks it 
> is ready
> for the edk2 trunk, then the idea is that submission of major features may 
> need to be
> scheduled based on other feature integration, current stability of edk2 
> trunk, and the
> current point in the edk2 release cycle.  So some features may be held 
> temporarily even
> if they are considered 100% ready.  These are some of the reasons for the 
> additional 
> approval steps.  Do you have suggestions on how to make it lighter weight?
> 

I can think of two concerns.

1. Remember all the effort Laszlo went through to rework various
   packages for OVMF SMM support? It took quite a while. At the end of
   that would we be adding another several week delay/hurdle?

2. If someone is working on a big feature for OVMF, then I might want
   to have it be a staging branch. Especially if we can enable
   Continuous Integration for staging branches. But, once the code
   review is complete, it would be nice if we could merge this code to
   OvmfPkg without going through this extra step.

   Maybe we'd just need a separate process to accommodate something
   like this?

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Bruce Cran 

On 3/9/16 2:59 PM, Jordan Justen wrote:


So far I have a setup that can test building BaseTools and OVMF on
Linux. The Windows setup is a little more complicated, but I don't
think it should be too difficult.


I guess that's Linux x86_64? Any solution we come up with should also be 
extendable to AARCH64.


--
Bruce

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Andrew Fish 

> On Mar 9, 2016, at 3:02 PM, Kinney, Michael D <michael.d.kin...@intel.com> 
> wrote:
> 
> Jordan,
> 
> Responses included below.
> 
> Mike
> 
> 
> 
>> -Original Message-
>> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of 
>> Jordan Justen
>> Sent: Wednesday, March 9, 2016 1:47 PM
>> To: Mangefeste, Tony <tony.mangefe...@intel.com>; edk2-devel@lists.01.org
>> Subject: Re: [edk2] EDK2 Staging Repository
>> 
>> On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
>>> Below are the details of the proposal for a staging branch, please review 
>>> and
>> comment.
>>> 
>>> 
>>> 
>>> Problem statement
>>> =
>>> Need place on tianocore.org where new features that are not ready for 
>>> product
>>> integration can be checked in for evaluation by the EDK II community prior 
>>> to
>>> adding to the edk2 trunk.  This serves several purposes:
>>> 
>>> * Encourage source code to be shared earlier in the development process
>>> * Allow source code to be shared that does not yet meet all edk2 required 
>>> quality
>> criteria
>>> * Allow source code to be shared so the EDK II community can help finish 
>>> and validate
>> new features
>>> * Provide a location to hold new features until they are deemed ready for 
>>> product
>> integration
>>> * Provide a location to hold new features until there is a natural point in 
>>> edk2
>> release cycle to fully validate the new feature.
>>> * Not intended to be used for bug fixes.
>>> * Not intended to be used for small, simple, or low risk features.
>>> 
>> 
>> I think the parts about making a central place for big feature
>> branches seems good. I think Laszlo might have appreciated having a
>> well defined central location for his OVMF SMM enabling branch.
>> 
>> I think the processes around allowing the code to be merged seems too
>> heavyweight. In the cases of big patchsets, it can be burdensome
>> enough just to get code review. To insert another process step (and
>> set of approvers) to actually merge the feature seems perhaps
>> unneccesary.
>> 
>>> Proposal
>>> 
>>> 1) Create a new repo called edk2-staging
>>>a) edk2-staging is a fork of edk
>>>b) edk2-staging/master tracks edk2/master
>>> 
>> 
>> For branches maintained by package maintainers, could we have them
>> under the edk2 repo and named staging/*?
> 
> Do you mean a directory in the edk2 repo called staging?

Mike,

I was confused too when I first read that sentence, but I think it means a 
naming convention for the branch. So the branch name would be 
staging/ExperimentX. Branch names can contain / as long as they don't end in /. 

Thanks,

Andrew Fish

>  I did think about this option,
> but I am concerned about code that is not ready for product integration being 
> in the edk2
> repo and I am also concerned about the size of the edk2 repo.  Creating a new 
> repo that is
> a fork of edk2 addressed both of those issues.
> 
>> 
>>> 2) All edk2-staging discussions use the existing edk2-devel mailing list for
>> design/patch/test.
>>>Use the following style for discussion of a specific feature branch 
>>> in edk2-
>> staging repo.
>>> 
>>>[staging][branch name]: Subject
>> 
>> What about [staging/branch]?
> 
> This is a good suggestion and aligned with your comment below on patch email 
> format that works with git.
> 
>> 
>>> 
>>> 3) All commits to edk2-staging must follow same edk2 rules (e.g. Tiano 
>>> Contributor's
>> Agreement)
>>> 
>>> 4) Process to add a new feature to edk2-staging
>>>a) Request to create a new edk2-staging feature branch sent to 
>>> edk2-devel
>>>b) Branch request and branch name must be approved by stewards
>> 
>> Who?
> 
> See other email from Tony.
> 
>> 
>>>c) Branch maintainer creates edk2-staging feature branch
>> 
>> Who would be the maintainer? The submitter, or perhaps a package
>> maintainer that is closely related to the new series. (For example, if
>> it was a series closely related to OVMF, then Laszlo and I would be
>> responsible for the branch.)
> 
> Yes.  Every new feature that is accepted requires one of the Package 
> Maintainers
> to be the sponsor for the feature and perform the branch level maintenance 
>

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Jordan Justen 
On 2016-03-09 11:15:21, Bruce Cran wrote:
> On 3/9/16 11:53 AM, Mangefeste, Tony wrote:
> > Below are the details of the proposal for a staging branch, please review 
> > and comment.
> >
> > 
> >
> > Problem statement
> > =
> > Need place on tianocore.org where new features that are not ready for 
> > product
> > integration can be checked in for evaluation by the EDK II community prior 
> > to
> > adding to the edk2 trunk.  This serves several purposes:
> >
> > * Encourage source code to be shared earlier in the development process
> > * Allow source code to be shared that does not yet meet all edk2 required 
> > quality criteria
> > * Allow source code to be shared so the EDK II community can help finish 
> > and validate new features
> > * Provide a location to hold new features until they are deemed ready for 
> > product integration
> > * Provide a location to hold new features until there is a natural point in 
> > edk2 release cycle to fully validate the new feature.
> > * Not intended to be used for bug fixes.
> > * Not intended to be used for small, simple, or low risk features.
> 
> Possibly related but for future work, I played around with setting up a 
> Jenkins server so people could build-test their patches on various 
> platforms/architectures but it turned out to be too expensive for me to 
> run on AWS so I took it down. I wonder if this is something we could 
> consider setting up for the staging area at some point?
> 

I've been working on configuring travis-ci (Linux) and appveyor
(Windows) for the edk2 master.

So far I have a setup that can test building BaseTools and OVMF on
Linux. The Windows setup is a little more complicated, but I don't
think it should be too difficult.

So, yeah, we could probably apply that to the staging branches as
well.

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] EDK2 Staging Repository

2016-03-09 Thread Jordan Justen 
On 2016-03-09 10:53:38, Mangefeste, Tony wrote:
> Below are the details of the proposal for a staging branch, please review and 
> comment.
> 
> 
> 
> Problem statement
> =
> Need place on tianocore.org where new features that are not ready for product 
> integration can be checked in for evaluation by the EDK II community prior to 
> adding to the edk2 trunk.  This serves several purposes:
> 
> * Encourage source code to be shared earlier in the development process
> * Allow source code to be shared that does not yet meet all edk2 required 
> quality criteria
> * Allow source code to be shared so the EDK II community can help finish and 
> validate new features
> * Provide a location to hold new features until they are deemed ready for 
> product integration
> * Provide a location to hold new features until there is a natural point in 
> edk2 release cycle to fully validate the new feature.
> * Not intended to be used for bug fixes.
> * Not intended to be used for small, simple, or low risk features.
> 

I think the parts about making a central place for big feature
branches seems good. I think Laszlo might have appreciated having a
well defined central location for his OVMF SMM enabling branch.

I think the processes around allowing the code to be merged seems too
heavyweight. In the cases of big patchsets, it can be burdensome
enough just to get code review. To insert another process step (and
set of approvers) to actually merge the feature seems perhaps
unneccesary.

> Proposal
> 
> 1) Create a new repo called edk2-staging
> a) edk2-staging is a fork of edk
> b) edk2-staging/master tracks edk2/master
>

For branches maintained by package maintainers, could we have them
under the edk2 repo and named staging/*?

> 2) All edk2-staging discussions use the existing edk2-devel mailing list for 
> design/patch/test.
> Use the following style for discussion of a specific feature branch 
> in edk2-staging repo.
> 
> [staging][branch name]: Subject

What about [staging/branch]?

> 
> 3) All commits to edk2-staging must follow same edk2 rules (e.g. Tiano 
> Contributor's Agreement)
> 
> 4) Process to add a new feature to edk2-staging
> a) Request to create a new edk2-staging feature branch sent to 
> edk2-devel
> b) Branch request and branch name must be approved by stewards

Who?

> c) Branch maintainer creates edk2-staging feature branch

Who would be the maintainer? The submitter, or perhaps a package
maintainer that is closely related to the new series. (For example, if
it was a series closely related to OVMF, then Laszlo and I would be
responsible for the branch.)

> d) Branch maintainer creates Readme.MD in root of feature branch with 
> summary, owners, timeline, links to related materials.
> e) Branch maintainer is responsible for making sure feature is 
> frequently rebased to edk2/master
> 
> 5) Process to update sources in feature branch
> a) Patch email send to edk2-devel
> b) Commit message subject format
> 
> [staging][branch name][PATCH]: Package/Module: Subject

What about [staging/branch PATCH]? (It would work better with git
format-patch.)

> 
> c) Use same edk2-devel review process 
> d) If pass review, then maintainer commits change to edk2-staging 
> feature branch
> 
> NOTE: win32 binaries are not automatically generated if a feature 
> branch includes BaseTools source changes.
> 
> 5) Process to promote an edk2-staging branch to edk2 trunk
> a) Request sent to edk2-devel that describes the feature, design, 
> testing, etc.
> b) Stewards evaluate request and determine if the feature meets edk2 
> criteria.
> c) If approved, use edk2 patch review/commit process on edk2-devel 
> mailing list
> d) Remove feature branch from edk2-staging (maybe archived 
> elsewhere?). 
>

This part seems a bit heavy weight. I think once code has passed
review by the package maintainers it should be eligible to be
committed to the trunk.

Or, is this a process by which the package maintainer may not review
the code until this step?

-Jordan

> 6) Process to remove an edk2-staging branch
> a) Stewards periodically review of feature branches in edk2-staging 
> (once a quarter?)
> b) If no activity for extended period of time and feature is no 
> longer deemed a candidate for edk2 then stewards send email to edk2-devel to 
> request deletion of feature branch.  
> c) If no objections from EDK II community, then feature branch is 
> deleted (maybe archived elsewhere?).
> 
> 7) Process to evaluate a feature in edk2-staging
> a) Clone edk2
> b) Create local branch with optional platform packages
> c) Build platform in local branch and verify it is stable
> d) Clone edk2-staging/[branch name]
> e) Create local branch with optional platform packages
> f) Build platform in