Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

Update: Implemented and deployed.

On 02/08/2019 12:11, Jark Wu wrote:

Wow. That's great! Thanks Chesnay.

On Fri, 2 Aug 2019 at 17:50, Chesnay Schepler > wrote:


I'm currently modifying the cibot to do this automatically; should be
finished until Monday.

On 02/08/2019 07:41, Jark Wu wrote:
> Hi Chesnay,
>
> Can we assign Flink Committers the permission of flink-ci/flink
repo?
> Several times, when I pushed some new commits, the old build
jobs are still
> in pending and not canceled.
> Before we fix that, we can manually cancel some old jobs to save
build
> resource.
>
> Best,
> Jark
>
>
> On Wed, 10 Jul 2019 at 16:17, Chesnay Schepler
mailto:ches...@apache.org>> wrote:
>
>> Your best bet would be to check the first commit in the PR and
check the
>> parent commit.
>>
>> To re-run things, you will have to rebase the PR on the latest
master.
>>
>> On 10/07/2019 03:32, Kurt Young wrote:
>>> Thanks for all your efforts Chesnay, it indeed improves a lot
for our
>>> develop experience. BTW, do you know how to find the master branch
>>> information which the CI runs with?
>>>
>>> For example, like this one:
>>> https://travis-ci.com/flink-ci/flink/jobs/214542568
>>> It shows pass with the commits, which rebased on the master
when the CI
>>> is triggered. But it's both possible that the master branch CI
runs on is
>>> the
>>> same or different with current master. If it's the same, I can
simply
>> rely
>>> on the
>>> passed information to push commits, but if it's not, I think i
should
>> find
>>> another
>>> way to re-trigger tests based on the newest master.
>>>
>>> Do you know where can I get such information?
>>>
>>> Best,
>>> Kurt
>>>
>>>
>>> On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler
mailto:ches...@apache.org>>
>> wrote:
 The kinks have been worked out; the bot is running again and
pr builds
 are yet again no longer running on ASF resources.

 PRs are mirrored to: https://github.com/flink-ci/flink
 Bot source: https://github.com/flink-ci/ci-bot

 On 08/07/2019 17:14, Chesnay Schepler wrote:
> I have temporarily re-enabled running PR builds on the ASF
account;
> migrating to the Travis subscription caused some issues in
the bot
> that I have to fix first.
>
> On 07/07/2019 23:01, Chesnay Schepler wrote:
>> The vote has passed unanimously in favor of migrating to a
separate
>> Travis account.
>>
>> I will now set things up such that no PullRequest is no
longer run on
>> the ASF servers.
>> This is a major setup in reducing our usage of ASF resources.
>> For the time being we'll use free Travis plan for flink-ci
(i.e. 5
>> workers, which is the same the ASF gives us). Over the
course of the
>> next week we'll setup the Ververica subscription to
increase this
>> limit.
>>   From now now, a bot will mirror all new and updated
PullRequests to a
>> mirror repository (https://github.com/flink-ci/flink-ci)
and write an
>> update into the PR once the build is complete.
>> I have ran the bots for the past 3 days in parallel to our
existing
>> Travis and it was working without major issues.
>>
>> The biggest change that contributors will see is that
there's no
>> longer a icon next to each commit. We may revisit this in
the future.
>>
>> I'll setup a repo with the source of the bot later.
>>
>> On 04/07/2019 10:46, Chesnay Schepler wrote:
>>> I've raised a JIRA
>>> with
INFRA to
>>> inquire whether it would be possible to switch to a
different Travis
>>> account, and if so what steps would need to be taken.
>>> We need a proper confirmation from INFRA since we are not
in full
>>> control of the flink repository (for example, we cannot
access the
>>> settings page).
>>>
>>> If this is indeed possible, Ververica is willing sponsor a
Travis
>>> account for the Flink project.
>>> This would provide us with more than enough resources than
we need.
>>>
>>> Since this makes the project more reliant on resources
provided by
>>> external companies I would like to vote on this.
>>>
>>> Please vote on this proposal, as follows:
>>> [ ] +1, Approve the migration to a Ververica-sponsored Travis
>>> account, provided that INFRA approves
>>> [ ] -1, Do not approach the migration to a Ververica-sponsored
>>> Travis a

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Jark Wu]

Wow. That's great! Thanks Chesnay.

On Fri, 2 Aug 2019 at 17:50, Chesnay Schepler  wrote:

> I'm currently modifying the cibot to do this automatically; should be
> finished until Monday.
>
> On 02/08/2019 07:41, Jark Wu wrote:
> > Hi Chesnay,
> >
> > Can we assign Flink Committers the permission of flink-ci/flink repo?
> > Several times, when I pushed some new commits, the old build jobs are
> still
> > in pending and not canceled.
> > Before we fix that, we can manually cancel some old jobs to save build
> > resource.
> >
> > Best,
> > Jark
> >
> >
> > On Wed, 10 Jul 2019 at 16:17, Chesnay Schepler 
> wrote:
> >
> >> Your best bet would be to check the first commit in the PR and check the
> >> parent commit.
> >>
> >> To re-run things, you will have to rebase the PR on the latest master.
> >>
> >> On 10/07/2019 03:32, Kurt Young wrote:
> >>> Thanks for all your efforts Chesnay, it indeed improves a lot for our
> >>> develop experience. BTW, do you know how to find the master branch
> >>> information which the CI runs with?
> >>>
> >>> For example, like this one:
> >>> https://travis-ci.com/flink-ci/flink/jobs/214542568
> >>> It shows pass with the commits, which rebased on the master when the CI
> >>> is triggered. But it's both possible that the master branch CI runs on
> is
> >>> the
> >>> same or different with current master. If it's the same, I can simply
> >> rely
> >>> on the
> >>> passed information to push commits, but if it's not, I think i should
> >> find
> >>> another
> >>> way to re-trigger tests based on the newest master.
> >>>
> >>> Do you know where can I get such information?
> >>>
> >>> Best,
> >>> Kurt
> >>>
> >>>
> >>> On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler 
> >> wrote:
>  The kinks have been worked out; the bot is running again and pr builds
>  are yet again no longer running on ASF resources.
> 
>  PRs are mirrored to: https://github.com/flink-ci/flink
>  Bot source: https://github.com/flink-ci/ci-bot
> 
>  On 08/07/2019 17:14, Chesnay Schepler wrote:
> > I have temporarily re-enabled running PR builds on the ASF account;
> > migrating to the Travis subscription caused some issues in the bot
> > that I have to fix first.
> >
> > On 07/07/2019 23:01, Chesnay Schepler wrote:
> >> The vote has passed unanimously in favor of migrating to a separate
> >> Travis account.
> >>
> >> I will now set things up such that no PullRequest is no longer run
> on
> >> the ASF servers.
> >> This is a major setup in reducing our usage of ASF resources.
> >> For the time being we'll use free Travis plan for flink-ci (i.e. 5
> >> workers, which is the same the ASF gives us). Over the course of the
> >> next week we'll setup the Ververica subscription to increase this
> >> limit.
> >>   From now now, a bot will mirror all new and updated PullRequests
> to a
> >> mirror repository (https://github.com/flink-ci/flink-ci) and write
> an
> >> update into the PR once the build is complete.
> >> I have ran the bots for the past 3 days in parallel to our existing
> >> Travis and it was working without major issues.
> >>
> >> The biggest change that contributors will see is that there's no
> >> longer a icon next to each commit. We may revisit this in the
> future.
> >>
> >> I'll setup a repo with the source of the bot later.
> >>
> >> On 04/07/2019 10:46, Chesnay Schepler wrote:
> >>> I've raised a JIRA
> >>> with INFRA to
> >>> inquire whether it would be possible to switch to a different
> Travis
> >>> account, and if so what steps would need to be taken.
> >>> We need a proper confirmation from INFRA since we are not in full
> >>> control of the flink repository (for example, we cannot access the
> >>> settings page).
> >>>
> >>> If this is indeed possible, Ververica is willing sponsor a Travis
> >>> account for the Flink project.
> >>> This would provide us with more than enough resources than we need.
> >>>
> >>> Since this makes the project more reliant on resources provided by
> >>> external companies I would like to vote on this.
> >>>
> >>> Please vote on this proposal, as follows:
> >>> [ ] +1, Approve the migration to a Ververica-sponsored Travis
> >>> account, provided that INFRA approves
> >>> [ ] -1, Do not approach the migration to a Ververica-sponsored
> >>> Travis account
> >>>
> >>> The vote will be open for at least 24h, and until we have
> >>> confirmation from INFRA. The voting period may be shorter than the
> >>> usual 3 days since our current is effectively not working.
> >>>
> >>> On 04/07/2019 06:51, Bowen Li wrote:
>  Re: > Are they using their own Travis CI pool, or did the switch
> to
>  an entirely different CI service?
> 
>  I reached out to Wes and Krisztián 

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

I'm currently modifying the cibot to do this automatically; should be 
finished until Monday.


On 02/08/2019 07:41, Jark Wu wrote:

Hi Chesnay,

Can we assign Flink Committers the permission of flink-ci/flink repo?
Several times, when I pushed some new commits, the old build jobs are still
in pending and not canceled.
Before we fix that, we can manually cancel some old jobs to save build
resource.

Best,
Jark


On Wed, 10 Jul 2019 at 16:17, Chesnay Schepler  wrote:


Your best bet would be to check the first commit in the PR and check the
parent commit.

To re-run things, you will have to rebase the PR on the latest master.

On 10/07/2019 03:32, Kurt Young wrote:

Thanks for all your efforts Chesnay, it indeed improves a lot for our
develop experience. BTW, do you know how to find the master branch
information which the CI runs with?

For example, like this one:
https://travis-ci.com/flink-ci/flink/jobs/214542568
It shows pass with the commits, which rebased on the master when the CI
is triggered. But it's both possible that the master branch CI runs on is
the
same or different with current master. If it's the same, I can simply

rely

on the
passed information to push commits, but if it's not, I think i should

find

another
way to re-trigger tests based on the newest master.

Do you know where can I get such information?

Best,
Kurt


On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler 

wrote:

The kinks have been worked out; the bot is running again and pr builds
are yet again no longer running on ASF resources.

PRs are mirrored to: https://github.com/flink-ci/flink
Bot source: https://github.com/flink-ci/ci-bot

On 08/07/2019 17:14, Chesnay Schepler wrote:

I have temporarily re-enabled running PR builds on the ASF account;
migrating to the Travis subscription caused some issues in the bot
that I have to fix first.

On 07/07/2019 23:01, Chesnay Schepler wrote:

The vote has passed unanimously in favor of migrating to a separate
Travis account.

I will now set things up such that no PullRequest is no longer run on
the ASF servers.
This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5
workers, which is the same the ASF gives us). Over the course of the
next week we'll setup the Ververica subscription to increase this

limit.

  From now now, a bot will mirror all new and updated PullRequests to a
mirror repository (https://github.com/flink-ci/flink-ci) and write an
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing
Travis and it was working without major issues.

The biggest change that contributors will see is that there's no
longer a icon next to each commit. We may revisit this in the future.

I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:

I've raised a JIRA
with INFRA to
inquire whether it would be possible to switch to a different Travis
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full
control of the flink repository (for example, we cannot access the
settings page).

If this is indeed possible, Ververica is willing sponsor a Travis
account for the Flink project.
This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by
external companies I would like to vote on this.

Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis
account, provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored
Travis account

The vote will be open for at least 24h, and until we have
confirmation from INFRA. The voting period may be shorter than the
usual 3 days since our current is effectively not working.

On 04/07/2019 06:51, Bowen Li wrote:

Re: > Are they using their own Travis CI pool, or did the switch to
an entirely different CI service?

I reached out to Wes and Krisztián from Apache Arrow PMC. They are
currently moving away from ASF's Travis to their own in-house metal
machines at [1] with custom CI application at [2]. They've seen
significant improvement w.r.t both much higher performance and
basically no resource waiting time, "night-and-day" difference
quoting Wes.

Re: > If we can just switch to our own Travis pool, just for our
project, then this might be something we can do fairly quickly?

I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3]


https://docs.travis-ci.com/user/migrate/open-source-repository-migration

[4]


https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com



On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler
mailto:ches...@apache.org>> wrote:

  Are they using their own Travis CI pool, or did the switch to

an

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Jark Wu]

Hi Chesnay,

Can we assign Flink Committers the permission of flink-ci/flink repo?
Several times, when I pushed some new commits, the old build jobs are still
in pending and not canceled.
Before we fix that, we can manually cancel some old jobs to save build
resource.

Best,
Jark


On Wed, 10 Jul 2019 at 16:17, Chesnay Schepler  wrote:

> Your best bet would be to check the first commit in the PR and check the
> parent commit.
>
> To re-run things, you will have to rebase the PR on the latest master.
>
> On 10/07/2019 03:32, Kurt Young wrote:
> > Thanks for all your efforts Chesnay, it indeed improves a lot for our
> > develop experience. BTW, do you know how to find the master branch
> > information which the CI runs with?
> >
> > For example, like this one:
> > https://travis-ci.com/flink-ci/flink/jobs/214542568
> > It shows pass with the commits, which rebased on the master when the CI
> > is triggered. But it's both possible that the master branch CI runs on is
> > the
> > same or different with current master. If it's the same, I can simply
> rely
> > on the
> > passed information to push commits, but if it's not, I think i should
> find
> > another
> > way to re-trigger tests based on the newest master.
> >
> > Do you know where can I get such information?
> >
> > Best,
> > Kurt
> >
> >
> > On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler 
> wrote:
> >
> >> The kinks have been worked out; the bot is running again and pr builds
> >> are yet again no longer running on ASF resources.
> >>
> >> PRs are mirrored to: https://github.com/flink-ci/flink
> >> Bot source: https://github.com/flink-ci/ci-bot
> >>
> >> On 08/07/2019 17:14, Chesnay Schepler wrote:
> >>> I have temporarily re-enabled running PR builds on the ASF account;
> >>> migrating to the Travis subscription caused some issues in the bot
> >>> that I have to fix first.
> >>>
> >>> On 07/07/2019 23:01, Chesnay Schepler wrote:
>  The vote has passed unanimously in favor of migrating to a separate
>  Travis account.
> 
>  I will now set things up such that no PullRequest is no longer run on
>  the ASF servers.
>  This is a major setup in reducing our usage of ASF resources.
>  For the time being we'll use free Travis plan for flink-ci (i.e. 5
>  workers, which is the same the ASF gives us). Over the course of the
>  next week we'll setup the Ververica subscription to increase this
> limit.
> 
>   From now now, a bot will mirror all new and updated PullRequests to a
>  mirror repository (https://github.com/flink-ci/flink-ci) and write an
>  update into the PR once the build is complete.
>  I have ran the bots for the past 3 days in parallel to our existing
>  Travis and it was working without major issues.
> 
>  The biggest change that contributors will see is that there's no
>  longer a icon next to each commit. We may revisit this in the future.
> 
>  I'll setup a repo with the source of the bot later.
> 
>  On 04/07/2019 10:46, Chesnay Schepler wrote:
> > I've raised a JIRA
> > with INFRA to
> > inquire whether it would be possible to switch to a different Travis
> > account, and if so what steps would need to be taken.
> > We need a proper confirmation from INFRA since we are not in full
> > control of the flink repository (for example, we cannot access the
> > settings page).
> >
> > If this is indeed possible, Ververica is willing sponsor a Travis
> > account for the Flink project.
> > This would provide us with more than enough resources than we need.
> >
> > Since this makes the project more reliant on resources provided by
> > external companies I would like to vote on this.
> >
> > Please vote on this proposal, as follows:
> > [ ] +1, Approve the migration to a Ververica-sponsored Travis
> > account, provided that INFRA approves
> > [ ] -1, Do not approach the migration to a Ververica-sponsored
> > Travis account
> >
> > The vote will be open for at least 24h, and until we have
> > confirmation from INFRA. The voting period may be shorter than the
> > usual 3 days since our current is effectively not working.
> >
> > On 04/07/2019 06:51, Bowen Li wrote:
> >> Re: > Are they using their own Travis CI pool, or did the switch to
> >> an entirely different CI service?
> >>
> >> I reached out to Wes and Krisztián from Apache Arrow PMC. They are
> >> currently moving away from ASF's Travis to their own in-house metal
> >> machines at [1] with custom CI application at [2]. They've seen
> >> significant improvement w.r.t both much higher performance and
> >> basically no resource waiting time, "night-and-day" difference
> >> quoting Wes.
> >>
> >> Re: > If we can just switch to our own Travis pool, just for our
> >> project, then this might be something we can do

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

Your best bet would be to check the first commit in the PR and check the 
parent commit.


To re-run things, you will have to rebase the PR on the latest master.

On 10/07/2019 03:32, Kurt Young wrote:

Thanks for all your efforts Chesnay, it indeed improves a lot for our
develop experience. BTW, do you know how to find the master branch
information which the CI runs with?

For example, like this one:
https://travis-ci.com/flink-ci/flink/jobs/214542568
It shows pass with the commits, which rebased on the master when the CI
is triggered. But it's both possible that the master branch CI runs on is
the
same or different with current master. If it's the same, I can simply rely
on the
passed information to push commits, but if it's not, I think i should find
another
way to re-trigger tests based on the newest master.

Do you know where can I get such information?

Best,
Kurt


On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler  wrote:


The kinks have been worked out; the bot is running again and pr builds
are yet again no longer running on ASF resources.

PRs are mirrored to: https://github.com/flink-ci/flink
Bot source: https://github.com/flink-ci/ci-bot

On 08/07/2019 17:14, Chesnay Schepler wrote:

I have temporarily re-enabled running PR builds on the ASF account;
migrating to the Travis subscription caused some issues in the bot
that I have to fix first.

On 07/07/2019 23:01, Chesnay Schepler wrote:

The vote has passed unanimously in favor of migrating to a separate
Travis account.

I will now set things up such that no PullRequest is no longer run on
the ASF servers.
This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5
workers, which is the same the ASF gives us). Over the course of the
next week we'll setup the Ververica subscription to increase this limit.

 From now now, a bot will mirror all new and updated PullRequests to a
mirror repository (https://github.com/flink-ci/flink-ci) and write an
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing
Travis and it was working without major issues.

The biggest change that contributors will see is that there's no
longer a icon next to each commit. We may revisit this in the future.

I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:

I've raised a JIRA
with INFRA to
inquire whether it would be possible to switch to a different Travis
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full
control of the flink repository (for example, we cannot access the
settings page).

If this is indeed possible, Ververica is willing sponsor a Travis
account for the Flink project.
This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by
external companies I would like to vote on this.

Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis
account, provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored
Travis account

The vote will be open for at least 24h, and until we have
confirmation from INFRA. The voting period may be shorter than the
usual 3 days since our current is effectively not working.

On 04/07/2019 06:51, Bowen Li wrote:

Re: > Are they using their own Travis CI pool, or did the switch to
an entirely different CI service?

I reached out to Wes and Krisztián from Apache Arrow PMC. They are
currently moving away from ASF's Travis to their own in-house metal
machines at [1] with custom CI application at [2]. They've seen
significant improvement w.r.t both much higher performance and
basically no resource waiting time, "night-and-day" difference
quoting Wes.

Re: > If we can just switch to our own Travis pool, just for our
project, then this might be something we can do fairly quickly?

I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3]


https://docs.travis-ci.com/user/migrate/open-source-repository-migration

[4]
https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com



On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler
mailto:ches...@apache.org>> wrote:

 Are they using their own Travis CI pool, or did the switch to an
 entirely different CI service?

 If we can just switch to our own Travis pool, just for our
 project, then
 this might be something we can do fairly quickly?

 On 03/07/2019 05:55, Bowen Li wrote:
 > I responded in the INFRA ticket [1] that I believe they are
 using a wrong
 > metric against Flink and the total build time is a completely
 different
 > thing than guaranteed build capacity.
 >
 > My response:
 >
 > "As mentione

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Kurt Young]

Thanks for all your efforts Chesnay, it indeed improves a lot for our
develop experience. BTW, do you know how to find the master branch
information which the CI runs with?

For example, like this one:
https://travis-ci.com/flink-ci/flink/jobs/214542568
It shows pass with the commits, which rebased on the master when the CI
is triggered. But it's both possible that the master branch CI runs on is
the
same or different with current master. If it's the same, I can simply rely
on the
passed information to push commits, but if it's not, I think i should find
another
way to re-trigger tests based on the newest master.

Do you know where can I get such information?

Best,
Kurt


On Tue, Jul 9, 2019 at 3:27 AM Chesnay Schepler  wrote:

> The kinks have been worked out; the bot is running again and pr builds
> are yet again no longer running on ASF resources.
>
> PRs are mirrored to: https://github.com/flink-ci/flink
> Bot source: https://github.com/flink-ci/ci-bot
>
> On 08/07/2019 17:14, Chesnay Schepler wrote:
> > I have temporarily re-enabled running PR builds on the ASF account;
> > migrating to the Travis subscription caused some issues in the bot
> > that I have to fix first.
> >
> > On 07/07/2019 23:01, Chesnay Schepler wrote:
> >> The vote has passed unanimously in favor of migrating to a separate
> >> Travis account.
> >>
> >> I will now set things up such that no PullRequest is no longer run on
> >> the ASF servers.
> >> This is a major setup in reducing our usage of ASF resources.
> >> For the time being we'll use free Travis plan for flink-ci (i.e. 5
> >> workers, which is the same the ASF gives us). Over the course of the
> >> next week we'll setup the Ververica subscription to increase this limit.
> >>
> >> From now now, a bot will mirror all new and updated PullRequests to a
> >> mirror repository (https://github.com/flink-ci/flink-ci) and write an
> >> update into the PR once the build is complete.
> >> I have ran the bots for the past 3 days in parallel to our existing
> >> Travis and it was working without major issues.
> >>
> >> The biggest change that contributors will see is that there's no
> >> longer a icon next to each commit. We may revisit this in the future.
> >>
> >> I'll setup a repo with the source of the bot later.
> >>
> >> On 04/07/2019 10:46, Chesnay Schepler wrote:
> >>> I've raised a JIRA
> >>> with INFRA to
> >>> inquire whether it would be possible to switch to a different Travis
> >>> account, and if so what steps would need to be taken.
> >>> We need a proper confirmation from INFRA since we are not in full
> >>> control of the flink repository (for example, we cannot access the
> >>> settings page).
> >>>
> >>> If this is indeed possible, Ververica is willing sponsor a Travis
> >>> account for the Flink project.
> >>> This would provide us with more than enough resources than we need.
> >>>
> >>> Since this makes the project more reliant on resources provided by
> >>> external companies I would like to vote on this.
> >>>
> >>> Please vote on this proposal, as follows:
> >>> [ ] +1, Approve the migration to a Ververica-sponsored Travis
> >>> account, provided that INFRA approves
> >>> [ ] -1, Do not approach the migration to a Ververica-sponsored
> >>> Travis account
> >>>
> >>> The vote will be open for at least 24h, and until we have
> >>> confirmation from INFRA. The voting period may be shorter than the
> >>> usual 3 days since our current is effectively not working.
> >>>
> >>> On 04/07/2019 06:51, Bowen Li wrote:
>  Re: > Are they using their own Travis CI pool, or did the switch to
>  an entirely different CI service?
> 
>  I reached out to Wes and Krisztián from Apache Arrow PMC. They are
>  currently moving away from ASF's Travis to their own in-house metal
>  machines at [1] with custom CI application at [2]. They've seen
>  significant improvement w.r.t both much higher performance and
>  basically no resource waiting time, "night-and-day" difference
>  quoting Wes.
> 
>  Re: > If we can just switch to our own Travis pool, just for our
>  project, then this might be something we can do fairly quickly?
> 
>  I believe so, according to [3] and [4]
> 
> 
>  [1] https://ci.ursalabs.org/ 
>  [2] https://github.com/ursa-labs/ursabot
>  [3]
> 
> https://docs.travis-ci.com/user/migrate/open-source-repository-migration
> 
>  [4]
>  https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com
> 
> 
> 
>  On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler
>  mailto:ches...@apache.org>> wrote:
> 
>  Are they using their own Travis CI pool, or did the switch to an
>  entirely different CI service?
> 
>  If we can just switch to our own Travis pool, just for our
>  project, then
>  this might be something we can do fairly quickly?
> >>

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

The kinks have been worked out; the bot is running again and pr builds 
are yet again no longer running on ASF resources.


PRs are mirrored to: https://github.com/flink-ci/flink
Bot source: https://github.com/flink-ci/ci-bot

On 08/07/2019 17:14, Chesnay Schepler wrote:
I have temporarily re-enabled running PR builds on the ASF account; 
migrating to the Travis subscription caused some issues in the bot 
that I have to fix first.


On 07/07/2019 23:01, Chesnay Schepler wrote:
The vote has passed unanimously in favor of migrating to a separate 
Travis account.


I will now set things up such that no PullRequest is no longer run on 
the ASF servers.

This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5 
workers, which is the same the ASF gives us). Over the course of the 
next week we'll setup the Ververica subscription to increase this limit.


From now now, a bot will mirror all new and updated PullRequests to a 
mirror repository (https://github.com/flink-ci/flink-ci) and write an 
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing 
Travis and it was working without major issues.


The biggest change that contributors will see is that there's no 
longer a icon next to each commit. We may revisit this in the future.


I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:
I've raised a JIRA 
with INFRA to 
inquire whether it would be possible to switch to a different Travis 
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full 
control of the flink repository (for example, we cannot access the 
settings page).


If this is indeed possible, Ververica is willing sponsor a Travis 
account for the Flink project.

This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by 
external companies I would like to vote on this.


Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis 
account, provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored 
Travis account


The vote will be open for at least 24h, and until we have 
confirmation from INFRA. The voting period may be shorter than the 
usual 3 days since our current is effectively not working.


On 04/07/2019 06:51, Bowen Li wrote:
Re: > Are they using their own Travis CI pool, or did the switch to 
an entirely different CI service?


I reached out to Wes and Krisztián from Apache Arrow PMC. They are 
currently moving away from ASF's Travis to their own in-house metal 
machines at [1] with custom CI application at [2]. They've seen 
significant improvement w.r.t both much higher performance and 
basically no resource waiting time, "night-and-day" difference 
quoting Wes.


Re: > If we can just switch to our own Travis pool, just for our 
project, then this might be something we can do fairly quickly?


I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3] 
https://docs.travis-ci.com/user/migrate/open-source-repository-migration 

[4] 
https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com




On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler 
mailto:ches...@apache.org>> wrote:


    Are they using their own Travis CI pool, or did the switch to an
    entirely different CI service?

    If we can just switch to our own Travis pool, just for our
    project, then
    this might be something we can do fairly quickly?

    On 03/07/2019 05:55, Bowen Li wrote:
    > I responded in the INFRA ticket [1] that I believe they are
    using a wrong
    > metric against Flink and the total build time is a completely
    different
    > thing than guaranteed build capacity.
    >
    > My response:
    >
    > "As mentioned above, since I started to pay attention to Flink's
    build
    > queue a few tens of days ago, I'm in Seattle and I saw no build
    was kicking
    > off in PST daytime in weekdays for Flink. Our teammates in China
    and Europe
    > have also reported similar observations. So we need to evaluate
    how the
    > large total build time came from - if 1) your number and 2) our
    > observations from three locations that cover pretty much a full
    day, are
    > all true, I **guess** one reason can be that - highly likely the
    extra
    > build time came from weekends when other Apache projects may be
    idle and
    > Flink just drains hard its congested queue.
    >
    > Please be aware of that we're not complaining about the lack of
    resources
    > in general, I'm complaining about the lack of **stable, 
dedicated**

    > resources. An example for the latter one is, currently even

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

I have temporarily re-enabled running PR builds on the ASF account; 
migrating to the Travis subscription caused some issues in the bot that 
I have to fix first.


On 07/07/2019 23:01, Chesnay Schepler wrote:
The vote has passed unanimously in favor of migrating to a separate 
Travis account.


I will now set things up such that no PullRequest is no longer run on 
the ASF servers.

This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5 
workers, which is the same the ASF gives us). Over the course of the 
next week we'll setup the Ververica subscription to increase this limit.


From now now, a bot will mirror all new and updated PullRequests to a 
mirror repository (https://github.com/flink-ci/flink-ci) and write an 
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing 
Travis and it was working without major issues.


The biggest change that contributors will see is that there's no 
longer a icon next to each commit. We may revisit this in the future.


I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:
I've raised a JIRA 
with INFRA to 
inquire whether it would be possible to switch to a different Travis 
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full 
control of the flink repository (for example, we cannot access the 
settings page).


If this is indeed possible, Ververica is willing sponsor a Travis 
account for the Flink project.

This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by 
external companies I would like to vote on this.


Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis 
account, provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored Travis 
account


The vote will be open for at least 24h, and until we have 
confirmation from INFRA. The voting period may be shorter than the 
usual 3 days since our current is effectively not working.


On 04/07/2019 06:51, Bowen Li wrote:
Re: > Are they using their own Travis CI pool, or did the switch to 
an entirely different CI service?


I reached out to Wes and Krisztián from Apache Arrow PMC. They are 
currently moving away from ASF's Travis to their own in-house metal 
machines at [1] with custom CI application at [2]. They've seen 
significant improvement w.r.t both much higher performance and 
basically no resource waiting time, "night-and-day" difference 
quoting Wes.


Re: > If we can just switch to our own Travis pool, just for our 
project, then this might be something we can do fairly quickly?


I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3] 
https://docs.travis-ci.com/user/migrate/open-source-repository-migration 

[4] 
https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com




On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler > wrote:


    Are they using their own Travis CI pool, or did the switch to an
    entirely different CI service?

    If we can just switch to our own Travis pool, just for our
    project, then
    this might be something we can do fairly quickly?

    On 03/07/2019 05:55, Bowen Li wrote:
    > I responded in the INFRA ticket [1] that I believe they are
    using a wrong
    > metric against Flink and the total build time is a completely
    different
    > thing than guaranteed build capacity.
    >
    > My response:
    >
    > "As mentioned above, since I started to pay attention to Flink's
    build
    > queue a few tens of days ago, I'm in Seattle and I saw no build
    was kicking
    > off in PST daytime in weekdays for Flink. Our teammates in China
    and Europe
    > have also reported similar observations. So we need to evaluate
    how the
    > large total build time came from - if 1) your number and 2) our
    > observations from three locations that cover pretty much a full
    day, are
    > all true, I **guess** one reason can be that - highly likely the
    extra
    > build time came from weekends when other Apache projects may be
    idle and
    > Flink just drains hard its congested queue.
    >
    > Please be aware of that we're not complaining about the lack of
    resources
    > in general, I'm complaining about the lack of **stable, 
dedicated**

    > resources. An example for the latter one is, currently even if
    no build is
    > in Flink's queue and I submit a request to be the queue head 
in PST

    > morning, my build won't even start in 6-8+h. That is an absurd
    amount of
    > waiting time.
    >
    > That's saying, if ASF INFRA decides to adopt a quota system a

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

Yes we can do that; for the time being you can add an empty commit to 
re-trigger the CI.



On 08/07/2019 03:49, Congxian Qiu wrote:

As we used flink bot to trigger the CI test, could we add a command for
flink bot to retrigger the CI(sometimes we may encounter some flaky tests)

Best,
Congxian


Chesnay Schepler  于2019年7月8日周一 上午5:01写道：


The vote has passed unanimously in favor of migrating to a separate
Travis account.

I will now set things up such that no PullRequest is no longer run on
the ASF servers.
This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5
workers, which is the same the ASF gives us). Over the course of the
next week we'll setup the Ververica subscription to increase this limit.

  From now now, a bot will mirror all new and updated PullRequests to a
mirror repository (https://github.com/flink-ci/flink-ci) and write an
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing
Travis and it was working without major issues.

The biggest change that contributors will see is that there's no longer
a icon next to each commit. We may revisit this in the future.

I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:

I've raised a JIRA
with INFRA to
inquire whether it would be possible to switch to a different Travis
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full
control of the flink repository (for example, we cannot access the
settings page).

If this is indeed possible, Ververica is willing sponsor a Travis
account for the Flink project.
This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by
external companies I would like to vote on this.

Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis account,
provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored Travis
account

The vote will be open for at least 24h, and until we have confirmation
from INFRA. The voting period may be shorter than the usual 3 days
since our current is effectively not working.

On 04/07/2019 06:51, Bowen Li wrote:

Re: > Are they using their own Travis CI pool, or did the switch to
an entirely different CI service?

I reached out to Wes and Krisztián from Apache Arrow PMC. They are
currently moving away from ASF's Travis to their own in-house metal
machines at [1] with custom CI application at [2]. They've seen
significant improvement w.r.t both much higher performance and
basically no resource waiting time, "night-and-day" difference
quoting Wes.

Re: > If we can just switch to our own Travis pool, just for our
project, then this might be something we can do fairly quickly?

I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3]


https://docs.travis-ci.com/user/migrate/open-source-repository-migration

[4]

https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com



On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler mailto:ches...@apache.org>> wrote:

 Are they using their own Travis CI pool, or did the switch to an
 entirely different CI service?

 If we can just switch to our own Travis pool, just for our
 project, then
 this might be something we can do fairly quickly?

 On 03/07/2019 05:55, Bowen Li wrote:
 > I responded in the INFRA ticket [1] that I believe they are
 using a wrong
 > metric against Flink and the total build time is a completely
 different
 > thing than guaranteed build capacity.
 >
 > My response:
 >
 > "As mentioned above, since I started to pay attention to Flink's
 build
 > queue a few tens of days ago, I'm in Seattle and I saw no build
 was kicking
 > off in PST daytime in weekdays for Flink. Our teammates in China
 and Europe
 > have also reported similar observations. So we need to evaluate
 how the
 > large total build time came from - if 1) your number and 2) our
 > observations from three locations that cover pretty much a full
 day, are
 > all true, I **guess** one reason can be that - highly likely the
 extra
 > build time came from weekends when other Apache projects may be
 idle and
 > Flink just drains hard its congested queue.
 >
 > Please be aware of that we're not complaining about the lack of
 resources
 > in general, I'm complaining about the lack of **stable,
dedicated**
 > resources. An example for the latter one is, currently even if
 no build is
 > in Flink's queue and I submit a request to be the queue head in
PST
 > morning, my build won't even start in 6-8

Re: [RESULT][VOTE] Migrate to sponsored Travis account

[Congxian Qiu]

As we used flink bot to trigger the CI test, could we add a command for
flink bot to retrigger the CI(sometimes we may encounter some flaky tests)

Best,
Congxian


Chesnay Schepler  于2019年7月8日周一 上午5:01写道：

> The vote has passed unanimously in favor of migrating to a separate
> Travis account.
>
> I will now set things up such that no PullRequest is no longer run on
> the ASF servers.
> This is a major setup in reducing our usage of ASF resources.
> For the time being we'll use free Travis plan for flink-ci (i.e. 5
> workers, which is the same the ASF gives us). Over the course of the
> next week we'll setup the Ververica subscription to increase this limit.
>
>  From now now, a bot will mirror all new and updated PullRequests to a
> mirror repository (https://github.com/flink-ci/flink-ci) and write an
> update into the PR once the build is complete.
> I have ran the bots for the past 3 days in parallel to our existing
> Travis and it was working without major issues.
>
> The biggest change that contributors will see is that there's no longer
> a icon next to each commit. We may revisit this in the future.
>
> I'll setup a repo with the source of the bot later.
>
> On 04/07/2019 10:46, Chesnay Schepler wrote:
> > I've raised a JIRA
> > with INFRA to
> > inquire whether it would be possible to switch to a different Travis
> > account, and if so what steps would need to be taken.
> > We need a proper confirmation from INFRA since we are not in full
> > control of the flink repository (for example, we cannot access the
> > settings page).
> >
> > If this is indeed possible, Ververica is willing sponsor a Travis
> > account for the Flink project.
> > This would provide us with more than enough resources than we need.
> >
> > Since this makes the project more reliant on resources provided by
> > external companies I would like to vote on this.
> >
> > Please vote on this proposal, as follows:
> > [ ] +1, Approve the migration to a Ververica-sponsored Travis account,
> > provided that INFRA approves
> > [ ] -1, Do not approach the migration to a Ververica-sponsored Travis
> > account
> >
> > The vote will be open for at least 24h, and until we have confirmation
> > from INFRA. The voting period may be shorter than the usual 3 days
> > since our current is effectively not working.
> >
> > On 04/07/2019 06:51, Bowen Li wrote:
> >> Re: > Are they using their own Travis CI pool, or did the switch to
> >> an entirely different CI service?
> >>
> >> I reached out to Wes and Krisztián from Apache Arrow PMC. They are
> >> currently moving away from ASF's Travis to their own in-house metal
> >> machines at [1] with custom CI application at [2]. They've seen
> >> significant improvement w.r.t both much higher performance and
> >> basically no resource waiting time, "night-and-day" difference
> >> quoting Wes.
> >>
> >> Re: > If we can just switch to our own Travis pool, just for our
> >> project, then this might be something we can do fairly quickly?
> >>
> >> I believe so, according to [3] and [4]
> >>
> >>
> >> [1] https://ci.ursalabs.org/ 
> >> [2] https://github.com/ursa-labs/ursabot
> >> [3]
> >>
> https://docs.travis-ci.com/user/migrate/open-source-repository-migration
> >> [4]
> https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com
> >>
> >>
> >>
> >> On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler  >> > wrote:
> >>
> >> Are they using their own Travis CI pool, or did the switch to an
> >> entirely different CI service?
> >>
> >> If we can just switch to our own Travis pool, just for our
> >> project, then
> >> this might be something we can do fairly quickly?
> >>
> >> On 03/07/2019 05:55, Bowen Li wrote:
> >> > I responded in the INFRA ticket [1] that I believe they are
> >> using a wrong
> >> > metric against Flink and the total build time is a completely
> >> different
> >> > thing than guaranteed build capacity.
> >> >
> >> > My response:
> >> >
> >> > "As mentioned above, since I started to pay attention to Flink's
> >> build
> >> > queue a few tens of days ago, I'm in Seattle and I saw no build
> >> was kicking
> >> > off in PST daytime in weekdays for Flink. Our teammates in China
> >> and Europe
> >> > have also reported similar observations. So we need to evaluate
> >> how the
> >> > large total build time came from - if 1) your number and 2) our
> >> > observations from three locations that cover pretty much a full
> >> day, are
> >> > all true, I **guess** one reason can be that - highly likely the
> >> extra
> >> > build time came from weekends when other Apache projects may be
> >> idle and
> >> > Flink just drains hard its congested queue.
> >> >
> >> > Please be aware of that we're not complaining about the lack of
> >> resources
> >> > in genera

[RESULT][VOTE] Migrate to sponsored Travis account

[Chesnay Schepler]

The vote has passed unanimously in favor of migrating to a separate 
Travis account.


I will now set things up such that no PullRequest is no longer run on 
the ASF servers.

This is a major setup in reducing our usage of ASF resources.
For the time being we'll use free Travis plan for flink-ci (i.e. 5 
workers, which is the same the ASF gives us). Over the course of the 
next week we'll setup the Ververica subscription to increase this limit.


From now now, a bot will mirror all new and updated PullRequests to a 
mirror repository (https://github.com/flink-ci/flink-ci) and write an 
update into the PR once the build is complete.
I have ran the bots for the past 3 days in parallel to our existing 
Travis and it was working without major issues.


The biggest change that contributors will see is that there's no longer 
a icon next to each commit. We may revisit this in the future.


I'll setup a repo with the source of the bot later.

On 04/07/2019 10:46, Chesnay Schepler wrote:
I've raised a JIRA 
with INFRA to 
inquire whether it would be possible to switch to a different Travis 
account, and if so what steps would need to be taken.
We need a proper confirmation from INFRA since we are not in full 
control of the flink repository (for example, we cannot access the 
settings page).


If this is indeed possible, Ververica is willing sponsor a Travis 
account for the Flink project.

This would provide us with more than enough resources than we need.

Since this makes the project more reliant on resources provided by 
external companies I would like to vote on this.


Please vote on this proposal, as follows:
[ ] +1, Approve the migration to a Ververica-sponsored Travis account, 
provided that INFRA approves
[ ] -1, Do not approach the migration to a Ververica-sponsored Travis 
account


The vote will be open for at least 24h, and until we have confirmation 
from INFRA. The voting period may be shorter than the usual 3 days 
since our current is effectively not working.


On 04/07/2019 06:51, Bowen Li wrote:
Re: > Are they using their own Travis CI pool, or did the switch to 
an entirely different CI service?


I reached out to Wes and Krisztián from Apache Arrow PMC. They are 
currently moving away from ASF's Travis to their own in-house metal 
machines at [1] with custom CI application at [2]. They've seen 
significant improvement w.r.t both much higher performance and 
basically no resource waiting time, "night-and-day" difference 
quoting Wes.


Re: > If we can just switch to our own Travis pool, just for our 
project, then this might be something we can do fairly quickly?


I believe so, according to [3] and [4]


[1] https://ci.ursalabs.org/ 
[2] https://github.com/ursa-labs/ursabot
[3] 
https://docs.travis-ci.com/user/migrate/open-source-repository-migration

[4] https://docs.travis-ci.com/user/migrate/open-source-on-travis-ci-com



On Wed, Jul 3, 2019 at 12:01 AM Chesnay Schepler > wrote:


    Are they using their own Travis CI pool, or did the switch to an
    entirely different CI service?

    If we can just switch to our own Travis pool, just for our
    project, then
    this might be something we can do fairly quickly?

    On 03/07/2019 05:55, Bowen Li wrote:
    > I responded in the INFRA ticket [1] that I believe they are
    using a wrong
    > metric against Flink and the total build time is a completely
    different
    > thing than guaranteed build capacity.
    >
    > My response:
    >
    > "As mentioned above, since I started to pay attention to Flink's
    build
    > queue a few tens of days ago, I'm in Seattle and I saw no build
    was kicking
    > off in PST daytime in weekdays for Flink. Our teammates in China
    and Europe
    > have also reported similar observations. So we need to evaluate
    how the
    > large total build time came from - if 1) your number and 2) our
    > observations from three locations that cover pretty much a full
    day, are
    > all true, I **guess** one reason can be that - highly likely the
    extra
    > build time came from weekends when other Apache projects may be
    idle and
    > Flink just drains hard its congested queue.
    >
    > Please be aware of that we're not complaining about the lack of
    resources
    > in general, I'm complaining about the lack of **stable, 
dedicated**

    > resources. An example for the latter one is, currently even if
    no build is
    > in Flink's queue and I submit a request to be the queue head in 
PST

    > morning, my build won't even start in 6-8+h. That is an absurd
    amount of
    > waiting time.
    >
    > That's saying, if ASF INFRA decides to adopt a quota system and
    grants
    > Flink five DEDICATED servers that runs all the time only for
    Flink, that'll
    > be PERFECT and can totally solve our problem now.
    >
    > Please be aware of that we're not complaining