I suppose it's ok to lock builds to Ubuntu agents only, since we don't officially support building or running on Windows. I just made that change.
Do we need hadoop-lzo? I am guessing no - so maybe the right call is to exclude it. It seems that its purpose is to allow reading LZO-compressed Thrift files. But it could be provided in other ways, including by the Druid admin (by putting it into the extension directory) or even by the remote Hadoop cluster (it might already be there, and it should be okay if it's only on the Hadoop side, since it wouldn't be needed on the Druid side). On Tue, Feb 26, 2019 at 5:33 PM Jonathan Wei <jon...@apache.org> wrote: > The Windows agents don't work now since we have non-Windows executables in > the build process: > > ``` > Cannot run program > "Z:\buildAgent\work\9f653ecb5b5406e8\web-console\script\build" (in > directory "Z:\buildAgent\work\9f653ecb5b5406e8\web-console"): CreateProcess > error=193, %1 is not a valid Win32 application > ``` > > The ubuntu agents do get past that, but Twitter's maven repo is having > issues right now so the build fails. > ``` > > [*ERROR*] Failed to execute goal on project druid-thrift-extensions: *Could > not resolve dependencies for project > > org.apache.druid.extensions.contrib:druid-thrift-extensions:jar:0.14.0-incubating-SNAPSHOT: > Failed to collect dependencies at > com.twitter.elephantbird:elephant-bird-core:jar:4.8 -> > com.hadoop.gplcompression:hadoop-lzo:jar:0.4.19*: Failed to read artifact > descriptor for com.hadoop.gplcompression:hadoop-lzo:jar:0.4.19: Could not > transfer artifact com.hadoop.gplcompression:hadoop-lzo:pom:0.4.19 from/to > twitter (http://maven.twttr.com): Failed to transfer file: > > http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.19/hadoop-lzo-0.4.19.pom > . > Return code is: 503 , ReasonPhrase:Service Temporarily Unavailable. -> > *[Help > 1]* > ``` > > > > On Tue, Feb 26, 2019 at 1:28 PM Roman Leventov <leventov...@gmail.com> > wrote: > > > I've added Maven compilation build steps as per > > > > > https://youtrack.jetbrains.com/issue/TW-21010#focus=streamItem-27-3304712.0-0 > > and disabled the Agent Requirement. If TeamCity keeps being flaky the > Agent > > Requirement needs to be re-enabled here: > > > > > https://teamcity.jetbrains.com/admin/editRequirements.html?id=buildType:OpenSourceProjects_Druid_Inspections > > and > > added here: > > > > > https://teamcity.jetbrains.com/admin/editRequirements.html?id=buildType:OpenSourceProjects_Druid_InspectionsPullRequests > > > > On Mon, 25 Feb 2019 at 16:43, Gian Merlino <g...@apache.org> wrote: > > > > > In the meantime, since Windows builds seem to be ok, I've pinned our > > > TeamCity builds to Windows agents only as an effort to work around the > > > problem. It might slow them down a bit -- there's only 4 Windows agents > > and > > > 12 Ubuntu agents -- but should hopefully stem the tide of false > failures. > > > If this seems to cause more trouble than it's worth it can be reverted > > (the > > > change is in "Agent Requirements"). > > > > > > On Fri, Feb 22, 2019 at 2:00 PM Roman Leventov <leventov...@gmail.com> > > > wrote: > > > > > > > There is a file ci/README_TeamCity.md in Druid tree regarding > TeamCity: > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/blob/afb239b17a1111c98c73825828d3f273998c165e/ci/README_TeamCity.md > > > > > > > > Feel free to amend it with more info. > > > > > > > > I've CC'ed this e-mail to teamcity-supp...@jetbrains.com. > > > > > > > > On Fri, 22 Feb 2019 at 18:46, David Glasser < > glas...@apollographql.com > > > > > > > wrote: > > > > > > > > > It would be good to document that — for my first few PRs I just > > assumed > > > > > that I couldn't get a login as a random contributor. > > > > > > > > > > On Fri, Feb 22, 2019 at 12:56 PM Justin Borromeo < > > > > justin.borro...@imply.io > > > > > > > > > > > wrote: > > > > > > > > > > > You can click the “Log in as Guest” button to access the results > > > > without > > > > > > registering a TeamCity account. > > > > > > > > > > > > Justin > > > > > > > > > > > > > On Feb 22, 2019, at 12:55 PM, Julian Hyde <jh...@apache.org> > > > wrote: > > > > > > > > > > > > > > The page [1] requires a login. So, a Druid contributor has to > > get a > > > > > > TeamCity account? > > > > > > > > > > > > > > That would be OK, if TeamCity provides a lot of value. But it > > > raises > > > > > the > > > > > > bar for occasional and first-time contributors. > > > > > > > > > > > > > > Julian > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > https://hub.jetbrains.com/auth/login?response_type=code&client_id=7b63a7f6-a4ca-4ad2-99d6-ecede5a769a5&redirect_uri=https:%2F%2Fteamcity.jetbrains.com%2FhubPlugin%2Flogin.html&scope=0-0-0-0-0%207b63a7f6-a4ca-4ad2-99d6-ecede5a769a5&state=%2FviewLog.html%3FbuildId%3D1998017%26buildTypeId%3DOpenSourceProjects_Druid_InspectionsPullRequests&access_type=online > > > > > > > > > > > > > >> On Feb 22, 2019, at 10:59 AM, Gian Merlino <g...@apache.org> > > > wrote: > > > > > > >> > > > > > > >> It's one of the two integrations we have on GitHub that run to > > > > > validate > > > > > > >> pull requests. We have TeamCity doing static analysis; and > > Travis > > > > > > running > > > > > > >> unit tests, integration tests, and checkstyle. You can see an > > > > example > > > > > on > > > > > > >> this pull request here: > > > > > > https://github.com/apache/incubator-druid/pull/7118, > > > > > > >> under "Inspections: pull requests (Druid)". > > > > > > >> > > > > > > >> On Fri, Feb 22, 2019 at 10:17 AM Julian Hyde < > jh...@apache.org> > > > > > wrote: > > > > > > >> > > > > > > >>> What is TeamCity? How does Druid use it? Is it accessible to > > all > > > > > Druid > > > > > > >>> contributors? Are all Druid contributors required to use it? > Is > > > its > > > > > use > > > > > > >>> documented somewhere? (I searched > > > > > > >>> https://www.google.com/search?q=apache+druid+teamcity < > > > > > > >>> https://www.google.com/search?q=apache+druid+teamcity> but > > > > couldn’t > > > > > > find > > > > > > >>> any mention.) > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>>> On Feb 22, 2019, at 8:26 AM, Furkan KAMACI < > > > > furkankam...@gmail.com> > > > > > > >>> wrote: > > > > > > >>>> > > > > > > >>>> Hi Gian, > > > > > > >>>> > > > > > > >>>> There maybe a problem with permissions after upgrading > Apache > > > > parent > > > > > > POM > > > > > > >>>> version to 21. This maybe a related issue: > > > > > > >>>> https://jira.apache.org/jira/browse/INFRA-10286 > > > > > > >>>> > > > > > > >>>> Kind Regards, > > > > > > >>>> Furkan KAMACI > > > > > > >>>> > > > > > > >>>> On Fri, Feb 22, 2019 at 7:13 PM Gian Merlino < > g...@apache.org > > > > > > > > wrote: > > > > > > >>>> > > > > > > >>>>> It looks like TeamCity is having problems since a few days > > ago > > > - > > > > > > builds > > > > > > >>> of > > > > > > >>>>> master and PRs are flaky. Here's the link to recent builds > of > > > > > master: > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > https://teamcity.jetbrains.com/viewType.html?buildTypeId=OpenSourceProjects_Druid_Inspections&tab=buildTypeHistoryList&branch_OpenSourceProjects_Druid=__all_branches__ > > > > > > >>>>> > > > > > > >>>>> It looks like the Windows agents generally work fine and > the > > > > Ubuntu > > > > > > >>> agents > > > > > > >>>>> are flaky. I can't see any recent commits that seem likely > to > > > > have > > > > > > >>> caused > > > > > > >>>>> this so am assuming something is going wrong on the > TeamCity > > > > server > > > > > > >>> side. > > > > > > >>>>> > > > > > > >>>>> I looked at the logs of a few successful and a few failed > > > builds > > > > > and > > > > > > >>>>> noticed the failed ones have a lot of these messages, but > the > > > > > > successful > > > > > > >>>>> ones didn't: > > > > > > >>>>> > > > > > > >>>>> [Step 1/1] [ 43336] WARN - > ution.rmi.RemoteProcessSupport > > - > > > > [RMI > > > > > > TCP > > > > > > >>>>> Connection(3)-127.0.0.1] WARN > > > > > > >>>>> org.eclipse.aether.internal.impl.DefaultUpdateCheckManager > - > > > > Failed > > > > > > to > > > > > > >>>>> create parent directories for tracking file > > > > > > >>>>> > > > > > > >>>>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > /home/teamcity/.m2/repository/org/apache/apache/21/apache-21.pom.lastUpdated > > > > > > >>>>> > > > > > > >>>>> I'm not sure what that might indicate or what to do about > it. > > > > Maybe > > > > > > we > > > > > > >>> need > > > > > > >>>>> to… clear… some caches? Anyone have any ideas? > > > > > > >>>>> > > > > > > >>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > > > > > > For additional commands, e-mail: dev-h...@druid.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > > > > > For additional commands, e-mail: dev-h...@druid.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > >
