Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Fri, Apr 28, 2017 at 10:42:10AM +0200, Lars Schneider wrote: > >> A separate mailing list sounds like a very good idea to me! > >> Maybe "git-bui...@vger.kernel.org" or something? > >> What would it take to set something up like this? > > > > I suspect that emailing the vger admins is the right place (or that they > > can point us in the right direction, or tell us to get lost). The best > > address is probably postmas...@vger.kernel.org. > > > > (I resisted just cc-ing them here to see if other people had opinions on > > just sending the output to the regular list). > > OK, I'll send them an email. I just realized a strong reason for a > separate mailing list: I haven't found a knob to make TravisCI > send plain text emails. I think that's going to be a problem for any list on vger. I suspect the spam filtering all happens at an early stage. You might consider just setting up a google group or some other list as an initial experiment. It wouldn't be that hard to switch the list later (people will need to resubscribe, but you can notify them by sending an email to...the list). -Peff
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
> On 25 Apr 2017, at 12:01, Jeff King wrote: > > On Tue, Apr 25, 2017 at 10:51:20AM +0200, Lars Schneider wrote: > Off topic, is it possible to receive mail notifications from Travis when a fault is found in either 'pu', 'next' or 'master'? I know how to do it in Jenkins, but I'm not familiar with Travis and there's no obvious button from the web page.. >>> >>> I looked into this a bit for my personal builds. Notification config has >>> to go into the .travis.yml file[1]. So I think the best we could do is >>> send a notification email to some mailing list, and then let people >>> subscribe to that (or it could go to git@vger; I don't know how noisy it >>> would be). >> >> A separate mailing list sounds like a very good idea to me! >> Maybe "git-bui...@vger.kernel.org" or something? >> What would it take to set something up like this? > > I suspect that emailing the vger admins is the right place (or that they > can point us in the right direction, or tell us to get lost). The best > address is probably postmas...@vger.kernel.org. > > (I resisted just cc-ing them here to see if other people had opinions on > just sending the output to the regular list). OK, I'll send them an email. I just realized a strong reason for a separate mailing list: I haven't found a knob to make TravisCI send plain text emails. - Lars
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Tue, Apr 25, 2017 at 10:51:20AM +0200, Lars Schneider wrote: > >> Off topic, is it possible to receive mail notifications from Travis > >> when a fault is found in either 'pu', 'next' or 'master'? I know how > >> to do it in Jenkins, but I'm not familiar with Travis and there's no > >> obvious button from the web page.. > > > > I looked into this a bit for my personal builds. Notification config has > > to go into the .travis.yml file[1]. So I think the best we could do is > > send a notification email to some mailing list, and then let people > > subscribe to that (or it could go to git@vger; I don't know how noisy it > > would be). > > A separate mailing list sounds like a very good idea to me! > Maybe "git-bui...@vger.kernel.org" or something? > What would it take to set something up like this? I suspect that emailing the vger admins is the right place (or that they can point us in the right direction, or tell us to get lost). The best address is probably postmas...@vger.kernel.org. (I resisted just cc-ing them here to see if other people had opinions on just sending the output to the regular list). -Peff
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
> On 24 Apr 2017, at 22:37, Jeff King wrote: > > On Mon, Apr 24, 2017 at 08:22:36PM +0700, Duy Nguyen wrote: > >> Off topic, is it possible to receive mail notifications from Travis >> when a fault is found in either 'pu', 'next' or 'master'? I know how >> to do it in Jenkins, but I'm not familiar with Travis and there's no >> obvious button from the web page.. > > I looked into this a bit for my personal builds. Notification config has > to go into the .travis.yml file[1]. So I think the best we could do is > send a notification email to some mailing list, and then let people > subscribe to that (or it could go to git@vger; I don't know how noisy it > would be). A separate mailing list sounds like a very good idea to me! Maybe "git-bui...@vger.kernel.org" or something? What would it take to set something up like this? - Lars
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Mon, Apr 24, 2017 at 08:22:36PM +0700, Duy Nguyen wrote: > Off topic, is it possible to receive mail notifications from Travis > when a fault is found in either 'pu', 'next' or 'master'? I know how > to do it in Jenkins, but I'm not familiar with Travis and there's no > obvious button from the web page.. I looked into this a bit for my personal builds. Notification config has to go into the .travis.yml file[1]. So I think the best we could do is send a notification email to some mailing list, and then let people subscribe to that (or it could go to git@vger; I don't know how noisy it would be). -Peff [1] I wanted to set up an email just to me for my personal fork, but I couldn't make it work. AFAICT there's no way to override upstream's config short of adding another commit, which would mean I'd have to base all my branches on a "fix up .travis.yml" commit.
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Fri, Apr 21, 2017 at 9:54 PM, Lars Schneider wrote: > >> Am 20.04.2017 um 23:58 schrieb Ævar Arnfjörð Bjarmason : >> >> As a refresh of everyone's memory (because mine needed it). This is a >> feature I added back in 2011 when the i18n support was initially >> added. >> >> There was concern at the time that we would inadvertently mark >> plumbing messages for translation, particularly something in a shared >> code path, and this was a way to hopefully smoke out those issues with >> the test suite. >> >> However compiling with it breaks a couple of dozen tests, I stopped >> digging when I saw some broke back in 2014. >> >> What should be done about this? I think if we're going to keep them >> they need to be run regularly by something like Travis (Lars CC'd), >> however empirical evidence suggests that not running them is just fine >> too, so should we just remove support for this test mode? > > Right now we are building and testing Git in the following configurations: > > 1. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) > 2. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) * > 3. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) > 4. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) * > 5. Linux32, gcc, no git-p4 tests > 6. Windows, gcc, no git-p4 tests > > 1-4 run the same tests right now. This was especially useful in the beginning > to identify flaky tests (t0025 is still flaky!). > > We could easily run the tests in 1-4 with different configurations. E.g. > enable GETTEXT_POISON in 2. If CI stays idle some time, I suggest adding a test run with valgrind on 'pu' or 'next'. If there's still spare capacity I would add another test run with GIT_TEST_SPLIT_INDEX=1 (but not right now, it's broken) Off topic, is it possible to receive mail notifications from Travis when a fault is found in either 'pu', 'next' or 'master'? I know how to do it in Jenkins, but I'm not familiar with Travis and there's no obvious button from the web page.. -- Duy
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
Michael J Gruber writes: > Ævar Arnfjörð Bjarmason venit, vidit, dixit 20.04.2017 23:58: >> As a refresh of everyone's memory (because mine needed it). This is a >> feature I added back in 2011 when the i18n support was initially >> added. >> >> There was concern at the time that we would inadvertently mark >> plumbing messages for translation, particularly something in a shared >> code path, and this was a way to hopefully smoke out those issues with >> the test suite. >> >> However compiling with it breaks a couple of dozen tests, I stopped >> digging when I saw some broke back in 2014. >> >> What should be done about this? I think if we're going to keep them >> they need to be run regularly by something like Travis (Lars CC'd), >> however empirical evidence suggests that not running them is just fine >> too, so should we just remove support for this test mode? >> >> I don't care, but I can come up with the patch either way, but would >> only be motivated to write the one-time fix for it if some CI system >> is actually running them regularly, otherwise they'll just be subject >> to bitrotting again. > > I use that switch when I change something that involves l10n, but > usually I run specific tests only. To be honest: I have to make sure not > to get confused by (nor forget one of) the build flag GETTEXT_POISON and > the environment variable GIT_GETTEXT_POISON. I'm not sure I always > tested what I meant to test... To be quite honest, I have always felt that we are just as likely inadvertently use test_i18ncmp when we should use test_cmp (and vice versa) as we would mark plumbing messages with _() by mistake with this approach, and even with constant monitoring by something like Travis, GETTEXT_POISON may be able to catch mistakes only some of the time (i.e. when we do not make mistakes in writing our tests). Without constant monitoring, I agree that the mechanism does not work well to catch our mistakes.
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Fri, Apr 21, 2017 at 4:47 PM, Michael J Gruber wrote: > Ævar Arnfjörð Bjarmason venit, vidit, dixit 20.04.2017 23:58: >> As a refresh of everyone's memory (because mine needed it). This is a >> feature I added back in 2011 when the i18n support was initially >> added. >> >> There was concern at the time that we would inadvertently mark >> plumbing messages for translation, particularly something in a shared >> code path, and this was a way to hopefully smoke out those issues with >> the test suite. >> >> However compiling with it breaks a couple of dozen tests, I stopped >> digging when I saw some broke back in 2014. >> >> What should be done about this? I think if we're going to keep them >> they need to be run regularly by something like Travis (Lars CC'd), >> however empirical evidence suggests that not running them is just fine >> too, so should we just remove support for this test mode? >> >> I don't care, but I can come up with the patch either way, but would >> only be motivated to write the one-time fix for it if some CI system >> is actually running them regularly, otherwise they'll just be subject >> to bitrotting again. >> > > I use that switch when I change something that involves l10n, but > usually I run specific tests only. To be honest: I have to make sure not > to get confused by (nor forget one of) the build flag GETTEXT_POISON and > the environment variable GIT_GETTEXT_POISON. I'm not sure I always > tested what I meant to test... For any of the built-in tests, you just need to compile git with GETTEXT_POISON=YesPlease, the env var is set by test-lib.sh for you, you only need to set GIT_GETTEXT_POISON=1 if you're ad-hoc running some git command yourself, e.g.: $ ./git status On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working tree clean $ GIT_GETTEXT_POISON=1 ./git status # GETTEXT POISON #master # GETTEXT POISON #
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
On Fri, Apr 21, 2017 at 4:54 PM, Lars Schneider wrote: > >> Am 20.04.2017 um 23:58 schrieb Ævar Arnfjörð Bjarmason : >> >> As a refresh of everyone's memory (because mine needed it). This is a >> feature I added back in 2011 when the i18n support was initially >> added. >> >> There was concern at the time that we would inadvertently mark >> plumbing messages for translation, particularly something in a shared >> code path, and this was a way to hopefully smoke out those issues with >> the test suite. >> >> However compiling with it breaks a couple of dozen tests, I stopped >> digging when I saw some broke back in 2014. >> >> What should be done about this? I think if we're going to keep them >> they need to be run regularly by something like Travis (Lars CC'd), >> however empirical evidence suggests that not running them is just fine >> too, so should we just remove support for this test mode? > > Right now we are building and testing Git in the following configurations: > > 1. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) > 2. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) * > 3. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) > 4. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) * > 5. Linux32, gcc, no git-p4 tests > 6. Windows, gcc, no git-p4 tests > > 1-4 run the same tests right now. This was especially useful in the beginning > to identify flaky tests (t0025 is still flaky!). > > We could easily run the tests in 1-4 with different configurations. E.g. > enable GETTEXT_POISON in 2. > > Cheers, > Lars > > *) 2 and 4 use the wrong compiler right now. 2 should use clang on Linux and > 4 should use gcc. A patch is on my todo list. Great, thanks. I'll fixup the poison tests then.
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
Ævar Arnfjörð Bjarmason venit, vidit, dixit 20.04.2017 23:58: > As a refresh of everyone's memory (because mine needed it). This is a > feature I added back in 2011 when the i18n support was initially > added. > > There was concern at the time that we would inadvertently mark > plumbing messages for translation, particularly something in a shared > code path, and this was a way to hopefully smoke out those issues with > the test suite. > > However compiling with it breaks a couple of dozen tests, I stopped > digging when I saw some broke back in 2014. > > What should be done about this? I think if we're going to keep them > they need to be run regularly by something like Travis (Lars CC'd), > however empirical evidence suggests that not running them is just fine > too, so should we just remove support for this test mode? > > I don't care, but I can come up with the patch either way, but would > only be motivated to write the one-time fix for it if some CI system > is actually running them regularly, otherwise they'll just be subject > to bitrotting again. > I use that switch when I change something that involves l10n, but usually I run specific tests only. To be honest: I have to make sure not to get confused by (nor forget one of) the build flag GETTEXT_POISON and the environment variable GIT_GETTEXT_POISON. I'm not sure I always tested what I meant to test... Michael
Re: [BUG] test suite broken with GETTEXT_POISON=YesPlease
> Am 20.04.2017 um 23:58 schrieb Ævar Arnfjörð Bjarmason : > > As a refresh of everyone's memory (because mine needed it). This is a > feature I added back in 2011 when the i18n support was initially > added. > > There was concern at the time that we would inadvertently mark > plumbing messages for translation, particularly something in a shared > code path, and this was a way to hopefully smoke out those issues with > the test suite. > > However compiling with it breaks a couple of dozen tests, I stopped > digging when I saw some broke back in 2014. > > What should be done about this? I think if we're going to keep them > they need to be run regularly by something like Travis (Lars CC'd), > however empirical evidence suggests that not running them is just fine > too, so should we just remove support for this test mode? Right now we are building and testing Git in the following configurations: 1. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) 2. Linux, gcc, stable Perforce an GitLFS version (used by git-p4 tests) * 3. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) 4. OSX, clang, latest Perforce an GitLFS version (used by git-p4 tests) * 5. Linux32, gcc, no git-p4 tests 6. Windows, gcc, no git-p4 tests 1-4 run the same tests right now. This was especially useful in the beginning to identify flaky tests (t0025 is still flaky!). We could easily run the tests in 1-4 with different configurations. E.g. enable GETTEXT_POISON in 2. Cheers, Lars *) 2 and 4 use the wrong compiler right now. 2 should use clang on Linux and 4 should use gcc. A patch is on my todo list.
[BUG] test suite broken with GETTEXT_POISON=YesPlease
As a refresh of everyone's memory (because mine needed it). This is a feature I added back in 2011 when the i18n support was initially added. There was concern at the time that we would inadvertently mark plumbing messages for translation, particularly something in a shared code path, and this was a way to hopefully smoke out those issues with the test suite. However compiling with it breaks a couple of dozen tests, I stopped digging when I saw some broke back in 2014. What should be done about this? I think if we're going to keep them they need to be run regularly by something like Travis (Lars CC'd), however empirical evidence suggests that not running them is just fine too, so should we just remove support for this test mode? I don't care, but I can come up with the patch either way, but would only be motivated to write the one-time fix for it if some CI system is actually running them regularly, otherwise they'll just be subject to bitrotting again.
