Re: Go error when building containers

Yup that is and always has been the right answer. I can't look up the JIRA right now but once I've finished schemas, the plan is to get Go Modules working which will be a much more reliable solution. On Fri, Jul 24, 2020, 5:19 PM Brian Hulette wrote: > Spoke with Daniel about this offline and h

Re: Go error when building containers

Spoke with Daniel about this offline and he suggested deleting all the vendor directories. I ran `find . -name vendor | xargs -I{} rm -rf {}` from my Beam clone and all is well now. On Fri, Jul 24, 2020 at 11:13 AM Brian Hulette wrote: > I tried removing every .gogradle directory in Beam, as wel

Re: Python2.7 Beam End-of-Life Date

Regardless of the outcome, it would be good to have some more details here. Can you share links for people to find out more about Python 3 support in those products and their timeline? I did some quick searching out of curiosity but I do not believe I found the authoritative information. Separatel

Making reporting bugs/feature request easier

Hi folks, I recently made a few Jira boards [1][2] to help triage and organize Beam's backlog. Something that came to my mind is that reporting bugs and feature requests using Jira might be imposing a high barrier for users who don't have an account. This process might also make it difficult to

Re: Chronically flaky tests

Adding https://testautonation.com/analyse-test-results-deflake-flaky-tests/ to the list which seems a more powerful test history tool. On Fri, Jul 24, 2020 at 1:51 PM Kenneth Knowles wrote: > Had some off-list chats to brainstorm and I wanted to bring ideas back to > the dev@ list for considerat

Re: Chronically flaky tests

Had some off-list chats to brainstorm and I wanted to bring ideas back to the dev@ list for consideration. A lot can be combined. I would really like to have a section in the release notes. I like the idea of banishing flakes from pre-commit (since you can't tell easily if it was a real failure cau

Re: [VOTE] Make Apache Beam 2.24.0 the final release supporting Python 2.

+1 On Fri, Jul 24, 2020 at 10:47 AM Maximilian Michels wrote: > +1 > > On 24.07.20 18:54, Pablo Estrada wrote: > > +1 - thank you Valentyn! > > -P. > > > > On Thu, Jul 23, 2020 at 1:29 PM Chamikara Jayalath > > wrote: > > > > +1 > > > > On Thu, Jul 23, 2020

Re: Jenkins trigger phrase "run seed job" not working?

I made my membership public and now the phrase works. (page: https://github.com/orgs/apache/people?query=udim) On Fri, Jul 24, 2020 at 1:04 PM Kenneth Knowles wrote: > > > On Thu, Jul 23, 2020 at 1:11 PM Damian Gadomski < > damian.gadom...@polidea.com> wrote: > >> Yes, I thought that whitelistin

Re: No space left on device - beam-jenkins 1 and 7

Tests may not be doing docker cleanup. Inventory job runs a docker prune every 12 hours for images older than 24 hrs [1]. Randomly looking at one of the recent runs [2], it cleaned up a long list of containers consuming 30+GB space. That should be just 12 hours worth of containers. [1] https://git

Re: No space left on device - beam-jenkins 1 and 7

Yes, these are on the same volume in the /var/lib/docker directory. I'm unsure if they clean up leftover images. On Fri, Jul 24, 2020 at 12:52 PM Udi Meiri wrote: > I forgot Docker images: > > ehudm@apache-ci-beam-jenkins-3:~$ sudo docker system df > TYPETOTAL ACTIV

Re: Jenkins trigger phrase "run seed job" not working?

On Thu, Jul 23, 2020 at 1:11 PM Damian Gadomski wrote: > Yes, I thought that whitelisting apache organization will do the trick, > but apparently, it doesn't. Actually, it makes sense as we want to allow > only beam committers and not all apache committers. I don't know the > implications of memb

Re: No space left on device - beam-jenkins 1 and 7

I forgot Docker images: ehudm@apache-ci-beam-jenkins-3:~$ sudo docker system df TYPETOTAL ACTIVE SIZE RECLAIMABLE Images 88 9 125.4GB 124.2GB (99%) Containers 40 4

Re: No space left on device - beam-jenkins 1 and 7

The additional slots (@3 directories) take up even more space now than before. I'm testing out https://github.com/apache/beam/pull/12326 which could help by cleaning up workspaces after a run (just started a seed job). On Fri, Jul 24, 2020 at 12:13 PM Tyson Hamilton wrote: > 664Mbeam_PreCom

Re: No space left on device - beam-jenkins 1 and 7

664Mbeam_PreCommit_JavaPortabilityApi_Commit 656Mbeam_PreCommit_JavaPortabilityApi_Commit@2 611Mbeam_PreCommit_JavaPortabilityApi_Cron 616Mbeam_PreCommit_JavaPortabilityApiJava11_Commit 598Mbeam_PreCommit_JavaPortabilityApiJava11_Commit@2 662Mbeam_PreCommit_JavaPortabilityAp

Re: No space left on device - beam-jenkins 1 and 7

Ya looks like something in the workspaces is taking up room: @apache-ci-beam-jenkins-8:/home/jenkins$ sudo du -shc . 191G. 191Gtotal On Fri, Jul 24, 2020 at 11:44 AM Tyson Hamilton wrote: > Node 8 is also full. The partition that /tmp is on is here: > > Filesystem Size Used Avail

Re: No space left on device - beam-jenkins 1 and 7

Node 8 is also full. The partition that /tmp is on is here: Filesystem Size Used Avail Use% Mounted on /dev/sda1 485G 482G 2.9G 100% / however after cleaning up tmp with the crontab command, there is only 8G usage yet it still remains 100% full: @apache-ci-beam-jenkins-8:/tmp$ sudo

Re: No space left on device - beam-jenkins 1 and 7

In the "jenkins" user home directory. On Fri, Jul 24, 2020, 11:19 Tyson Hamilton wrote: > Everything I've been looking at is in the /tmp dir. Where are the > workspaces, or what are the named? > > > > > On Fri, Jul 24, 2020 at 11:03 AM Udi Meiri wrote: > >> I'm curious to what you find. Was it

Re: [BROKEN] Please add "Fix Version" when resolving or closing Jiras

I've noticed that when status == Resolved and Resolution == Unresolved, subtasks are not given the green checkmark of doneness. Just noting another quality-of-life issue caused here. Kenn On Thu, Jul 23, 2020 at 4:09 PM Brian Hulette wrote: > +1 - I think I saw somewhere Resolved is intended fo

Re: No space left on device - beam-jenkins 1 and 7

Everything I've been looking at is in the /tmp dir. Where are the workspaces, or what are the named? On Fri, Jul 24, 2020 at 11:03 AM Udi Meiri wrote: > I'm curious to what you find. Was it /tmp or the workspaces using up the > space? > > On Fri, Jul 24, 2020 at 10:57 AM Tyson Hamilton > wro

Re: Go error when building containers

I tried removing every .gogradle directory in Beam, as well as ~/.cache/go-build and ~/.gradle/go/repo but it's still happening. Maybe there's some other cache I'm missing? I also synced past https://github.com/apache/beam/pull/12364 in case it was related. I thought maybe others were having the s

Re: No space left on device - beam-jenkins 1 and 7

I'm curious to what you find. Was it /tmp or the workspaces using up the space? On Fri, Jul 24, 2020 at 10:57 AM Tyson Hamilton wrote: > Bleck. I just realized that it is 'offline' so that won't work. I'll clean > up manually on the machine using the cron command. > > On Fri, Jul 24, 2020 at 10:

Use concrete instances of ExternalTransformBuilder in ExternalTransformRegistrar?

Hi all, I've been working with +Scott Lukas on using the new schema io interfaces [1] in cross-language. This means adding a general-purpose ExternalTransformRegistrar [2,3] that will register all SchemaIOProvider implementations via ServiceLoader. We've run into an issue though - ExternalTransfo

Re: No space left on device - beam-jenkins 1 and 7

Bleck. I just realized that it is 'offline' so that won't work. I'll clean up manually on the machine using the cron command. On Fri, Jul 24, 2020 at 10:56 AM Tyson Hamilton wrote: > Something isn't working with the current set up because node 15 appears to > be out of space and is currently 'of

Re: No space left on device - beam-jenkins 1 and 7

Something isn't working with the current set up because node 15 appears to be out of space and is currently 'offline' according to Jenkins. Can someone run the cleanup job? The machine is full, @apache-ci-beam-jenkins-15:/tmp$ df -h Filesystem Size Used Avail Use% Mounted on udev

Re: [VOTE] Make Apache Beam 2.24.0 the final release supporting Python 2.

+1 On 24.07.20 18:54, Pablo Estrada wrote: +1 - thank you Valentyn! -P. On Thu, Jul 23, 2020 at 1:29 PM Chamikara Jayalath > wrote: +1 On Thu, Jul 23, 2020 at 1:15 PM Brian Hulette mailto:bhule...@google.com>> wrote: +1 On Thu, Jul 23, 2

Re: [VOTE] Make Apache Beam 2.24.0 the final release supporting Python 2.

+1 - thank you Valentyn! -P. On Thu, Jul 23, 2020 at 1:29 PM Chamikara Jayalath wrote: > +1 > > On Thu, Jul 23, 2020 at 1:15 PM Brian Hulette wrote: > >> +1 >> >> On Thu, Jul 23, 2020 at 1:05 PM Robert Bradshaw >> wrote: >> >>> [X] +1: Remove Python 2 support in Apache Beam 2.25.0. >>> >>> Acc

Re: Contributor permission for Beam Jira tickets

Hi Varun, I added you to Contributors list. Thanks for contribution and welcome! Regards, Alexey > On 23 Jul 2020, at 20:40, Varun Sharma wrote: > > Hi, > > This is Varun from THG. I have created a couple of tickets related to > improvements in Beam Java SDK. Could someone add me as a contr