Re: Pre-Master checks (Was: Nightlies)
On 04/02/2014 01:41, Joachim Breitner wrote: Proposal Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch validating/some id. One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/some id, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. The system should try to rebase and re-validate automatically in this case. I think this is pretty important - otherwise pushes can fail for random reasons that are no fault of the developer. Actually I'd even go so far as to say that if the rebase succeeds without conflicts then the push should go ahead. Yes this can introduce bugs, but since a validate takes ~30 minutes at a certain rate of commits it becomes impossible to push anything if the tree must be validated and fast-forwarded to push. Also, bear in mind that since this stands between a developer and getting their changes in, it really must work, all the time. Be careful what you're signing up for :-) My feeling is that as a first step this system should be treated as a validation service: we don't force people to use it if they don't want to, so that we can see how well it works over time and developers can gradually migrate to using it. On that basis I'd be really happy to see it, and possibly to rely on it more over time. Cheers, Simon Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch master-unchecked will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x ./validate is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use git merge --squash to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
Hi, Am Samstag, den 08.02.2014, 08:51 -0800 schrieb Simon Marlow: The system should try to rebase and re-validate automatically in this case. I think this is pretty important - otherwise pushes can fail for random reasons that are no fault of the developer. Actually I'd even go so far as to say that if the rebase succeeds without conflicts then the push should go ahead. Yes this can introduce bugs, but since a validate takes ~30 minutes at a certain rate of commits it becomes impossible to push anything if the tree must be validated and fast-forwarded to push. I’d actually prefer that behavior; I just didn’t want to put it in the proposal because people might be much too scared by the prospect of patches entering master that no human eye has seen so far – but after someone suggested the same to me in private, and you now on the list, I guess it would be a viable. Also, bear in mind that since this stands between a developer and getting their changes in, it really must work, all the time. Be careful what you're signing up for :-) My feeling is that as a first step this system should be treated as a validation service: we don't force people to use it if they don't want to, so that we can see how well it works over time and developers can gradually migrate to using it. On that basis I'd be really happy to see it, and possibly to rely on it more over time. Correct: If I were to implement this (which has to wait for http://hackage.haskell.org/trac/ghc/ticket/8545 to be reasonably reliable) I’d set it up first as _alternative_ way to push, inviting validate-lazy developers to use it instead of master. And if it works great, we can consider making it mandatory (or at least the default). So there seems to be a consensus here that this is a worthwhile feature. Great! Is there a roadmap for #8545? Greetings, Joachim -- Joachim “nomeata” Breitner m...@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nome...@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
Hi, Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person I believe we can do better, so that Austin’s big red message does not have to be written in the first place. Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions: Proposal Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch validating/some id. One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/some id, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch master-unchecked will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x ./validate is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use git merge --squash to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. -- Joachim Breitner e-Mail: m...@joachim-breitner.de Homepage: http://www.joachim-breitner.de Jabber-ID: nome...@joachim-breitner.de signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
Yay! I, for one, would probably submit more bugfix patches with this structure in place. I take validating seriously, but my daily schedule often involves switching machines and tasks, and once I press validate, it's often hours before I actually look at the results... by which time, some other commit may have come through. My need (real or perceived) for an uninterrupted chunk of time to make a patch stops me from doing them, sometimes. With Joachim's structure, I could spend a half hour doing a quick patch (especially comments!) and push and move on. Please do it! :) Thanks! Richard PS: Transparency in the git redirects would be nice, of course, but is in no way necessary. We could I suppose add a push script to the repo that handles some of the overhead. On Feb 4, 2014, at 4:41 AM, Joachim Breitner m...@joachim-breitner.de wrote: Hi, Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person I believe we can do better, so that Austin’s big red message does not have to be written in the first place. Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions: Proposal Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch validating/some id. One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/some id, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch master-unchecked will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x ./validate is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use git merge --squash to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. -- Joachim Breitner e-Mail: m...@joachim-breitner.de Homepage: http://www.joachim-breitner.de Jabber-ID: nome...@joachim-breitner.de ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
On 04/02/14 09:41, Joachim Breitner wrote: Hi, Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person I believe we can do better, so that Austin’s big red message does not have to be written in the first place. Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions: Proposal Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch validating/some id. One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/some id, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch master-unchecked will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x ./validate is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use git merge --squash to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs You mention that it's enough for one machine to validate the patch. What if the change was for example to fix something on ARM (where builds take a long time) and has no effect on x86? The x86 machine is likely to come through first and give you the OK, pushing to master something that's effectively not checked. In fact, the fastest slave is likely to come through first every time and if it validates there (for example, I hear that 64-bit Linux is the golden platform for validate), it will be pushed to master. I think the proposal needs an enhancement: ability to specify which platform the commit needs to validate on (i.e. even if i686 comes through first, we'll wait for ARM anyway) before being pushed in. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
This actually raises an important question: we don't have any good policy for making sure validate / etc stays clean on all the platforms. There's enough variations that it might take an implausible amount of resources to test all the targets we want to support on every commit. (Eg given how long builds apparently take on certain platforms like arm, we'd need quite a few wee arm machines to test arm at every commit. Or maybe not?) Point being: I agree that support tooling to make sure builds don't stay broken and don't get broken aren't where it should be. And I agree that something like you describe should be done. I'm just wondering if we have the resources to do that correctly. On Tuesday, February 4, 2014, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 04/02/14 09:41, Joachim Breitner wrote: Hi, Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: I want just to say thank you for thinking about this. We badly need better nightly-builds for GHC, on a variety of platforms a) to identify regressions, preferably to the actually commit that caused it, so Austin's big red message can go to the right person I believe we can do better, so that Austin’s big red message does not have to be written in the first place. Here is what I have in mind, and I’ll volunteer to implement it if people think it is a good idea and I get the resources/permissions: Proposal Nobody gets to push to master directly. Instead, every push to master is diverted¹ to a temporary branch validating/some id. One of our servers detects the appearance of such a branch and will * check it out, * validate it, * if ok: check if master can still be fast-forward’ed to it, * if yes: push to master. If it does not validate, or if master has changed in between, the branch will be moved to failed/some id, and a message is sent to the pushing developer², including a tail of the log and a link to the full log. Systems can fail, and maybe nothing validates anymore for reasons not easily fixable. For that case, a backdoor is available: Pushes to the branch master-unchecked will be moved to master, well, unchecked. Benefits: * It is guaranteed that master has validated at least once somewhere. I.e. no need to ask on the mailing list “does master validate for you right now?” * It is now ok to do changes that are “obviously correct” (comment changes, removing dead code, code reformatting) without having to validate manually, which _saves developer time_ (our most precious resource). Downsides: * When two commits are racing for master, one will be rejected for being a non-fast-forward commit. The user will then have to merge or rebase and try again. But: The same would be true if he was validating locally (unless he does not validate the merge/rebase, in which case we are again where we don’t want to be: Unvalidated versions in master.) Is this something you would what, or could live with? If it is technically feasible (given our hardware constraints, repository structure and git’s feature) is a different question, which needs to be discussed afterwards. Greetings, Joachim ¹ Might not be possible transparently (http://stackoverflow.com/questions/21362833), but for the sake of argument and workflow design, assume it was. ² As an approximation: The committer of the latest patch. PS: I’m also considering (but not pushing hard) for a stronger variant as follows. We do not need to discuss that now and should, if at all, start the with the proposal above. I’m just adding it to show where this is going ... Stronger proposal ~ Every commit in master needs to be validated! I tend to make sure that all patches on my branch validate individually (git rebase -i -x ./validate is a great tool here, you should use it! ). Contributors who do not want to go through that trouble should then use git merge --squash to produce a single commit from their branch. This would make the git history more useful for things like bitsecting. ___ ghc-devs mailing list ghc-devs@haskell.org javascript:; http://www.haskell.org/mailman/listinfo/ghc-devs You mention that it's enough for one machine to validate the patch. What if the change was for example to fix something on ARM (where builds take a long time) and has no effect on x86? The x86 machine is likely to come through first and give you the OK, pushing to master something that's effectively not checked. In fact, the fastest slave is likely to come through first every time and if it validates there (for example, I hear that 64-bit Linux is the golden platform for validate), it will be pushed to master. I think the proposal needs an enhancement: ability to specify which platform the commit needs to
RE: Pre-Master checks (Was: Nightlies)
I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head. Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list? Would need careful documenting, so that new people know exactly what to do. Simon | -Original Message- | From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: | I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | a) to identify regressions, preferably to the actually commit that | caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin’s big red message does not | have to be written in the first place. | | Here is what I have in mind, and I’ll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch validating/some id. One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward’ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/some id, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch master-unchecked will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. |I.e. no need to ask on the mailing list “does master validate for | you |right now?” | * It is now ok to do changes that are “obviously correct” (comment |changes, removing dead code, code reformatting) without having |to validate manually, which _saves developer time_ (our most | precious |resource). | | Downsides: | * When two commits are racing for master, one will be rejected for |being a non-fast-forward commit. The user will then have to merge |or rebase and try again. |But: The same would be true if he was validating locally (unless he |does not validate the merge/rebase, in which case we are again where |we don’t want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git’s feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I’m also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I’m just adding it to show where | this | is going ... | | Stronger proposal | ~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x ./validate is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use git merge --squash to produce a single commit from their branch. | | This would make the git history more useful for things like bitsecting. | | | | -- | Joachim Breitner | e-Mail: m...@joachim-breitner.de | Homepage: http://www.joachim-breitner.de | Jabber-ID: nome...@joachim-breitner.de ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Pre-Master checks (Was: Nightlies)
one point in the design space (mind you one i'm not going to advocate) is the testing / merging workflow that the Mozilla folks are doing for rust. Every patch / pull request gets tested by their build bot before merging. Does make the commits on master look pretty mess / mergy though. mind you its a bit tightly coupled to github, but some of the ideas *might* be a good strawman. Likewise, how does eg LLVM/Clang manage its testing infrastructure? How does GCC manage it? On Tue, Feb 4, 2014 at 12:56 PM, Simon Peyton Jones simo...@microsoft.comwrote: I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head. Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list? Would need careful documenting, so that new people know exactly what to do. Simon | -Original Message- | From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: | I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | a) to identify regressions, preferably to the actually commit that | caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin’s big red message does not | have to be written in the first place. | | Here is what I have in mind, and I’ll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch validating/some id. One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward’ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/some id, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch master-unchecked will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. |I.e. no need to ask on the mailing list “does master validate for | you |right now?” | * It is now ok to do changes that are “obviously correct” (comment |changes, removing dead code, code reformatting) without having |to validate manually, which _saves developer time_ (our most | precious |resource). | | Downsides: | * When two commits are racing for master, one will be rejected for |being a non-fast-forward commit. The user will then have to merge |or rebase and try again. |But: The same would be true if he was validating locally (unless he |does not validate the merge/rebase, in which case we are again where |we don’t want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git’s feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I’m also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I’m just adding it to show where | this | is going ... | | Stronger proposal | ~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x ./validate is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use git merge --squash to produce a single commit from their branch. | | This would make the git history more useful for things like bitsecting. | | | | -- | Joachim Breitner | e-Mail: m...@joachim-breitner.de | Homepage: http://www.joachim-breitner.de | Jabber-ID: nome...@joachim-breitner.de ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc
Re: Pre-Master checks (Was: Nightlies)
On 2/4/14, Carter Schonwald carter.schonw...@gmail.com wrote: one point in the design space (mind you one i'm not going to advocate) is the testing / merging workflow that the Mozilla folks are doing for rust. Every patch / pull request gets tested by their build bot before merging. Does make the commits on master look pretty mess / mergy though. mind you its a bit tightly coupled to github, but some of the ideas *might* be a good strawman. Likewise, how does eg LLVM/Clang manage its testing infrastructure? How does GCC manage it? LLVM devs need to run 'make check' before committing to the repo. This runs 8000+ tests in about 30 seconds. Then there is the bot infrastructure at http://lab.llvm.org:8011/console , and who breaks a target is alerted by mail. But this applies to all committers whose changes entered into (slow) builds. It is not commit-exact, though better on the fast bots. Cheers, Gabor On Tue, Feb 4, 2014 at 12:56 PM, Simon Peyton Jones simo...@microsoft.comwrote: I can see advantages here, similar to Richard. I'm just a bit worried about keeping all those branches straight in my head. Who names all these temporary branches? Does someone delete them so they don't accumulate and dominate the branch list? Would need careful documenting, so that new people know exactly what to do. Simon | -Original Message- | From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of | Joachim Breitner | Sent: 04 February 2014 09:42 | To: ghc-devs@haskell.org | Subject: Re: Pre-Master checks (Was: Nightlies) | | Hi, | | Am Dienstag, den 04.02.2014, 08:04 + schrieb Simon Peyton Jones: | I want just to say thank you for thinking about this. We badly need | better nightly-builds for GHC, on a variety of platforms | a) to identify regressions, preferably to the actually commit that | caused it, so Austin's big red message can go to the right person | | I believe we can do better, so that Austin's big red message does not | have to be written in the first place. | | Here is what I have in mind, and I'll volunteer to implement it if | people think it is a good idea and I get the resources/permissions: | | Proposal | | | Nobody gets to push to master directly. Instead, every push to master | is | diverted¹ to a temporary branch validating/some id. One of our | servers detects the appearance of such a branch and will | * check it out, | * validate it, | * if ok: check if master can still be fast-forward'ed to it, | * if yes: push to master. | | If it does not validate, or if master has changed in between, the | branch | will be moved to failed/some id, and a message is sent to the pushing | developer², including a tail of the log and a link to the full log. | | Systems can fail, and maybe nothing validates anymore for reasons not | easily fixable. For that case, a backdoor is available: Pushes to the | branch master-unchecked will be moved to master, well, unchecked. | | Benefits: | * It is guaranteed that master has validated at least once somewhere. |I.e. no need to ask on the mailing list does master validate for | you |right now? | * It is now ok to do changes that are obviously correct (comment |changes, removing dead code, code reformatting) without having |to validate manually, which _saves developer time_ (our most | precious |resource). | | Downsides: | * When two commits are racing for master, one will be rejected for |being a non-fast-forward commit. The user will then have to merge |or rebase and try again. |But: The same would be true if he was validating locally (unless he |does not validate the merge/rebase, in which case we are again where |we don't want to be: Unvalidated versions in master.) | | | Is this something you would what, or could live with? | | If it is technically feasible (given our hardware constraints, | repository structure and git's feature) is a different question, which | needs to be discussed afterwards. | | Greetings, | Joachim | | | | ¹ Might not be possible transparently | (http://stackoverflow.com/questions/21362833), but for the sake of | argument and workflow design, assume it was. | ² As an approximation: The committer of the latest patch. | | PS: I'm also considering (but not pushing hard) for a stronger variant | as follows. We do not need to discuss that now and should, if at all, | start the with the proposal above. I'm just adding it to show where | this | is going ... | | Stronger proposal | ~ | | Every commit in master needs to be validated! | I tend to make sure that all patches on my branch validate individually | (git rebase -i -x ./validate is a great tool here, you should use it! | ). Contributors who do not want to go through that trouble should then | use git merge --squash to produce a single commit from their branch
