Re: Failures in Arrow tests run outside of continuous integration

2019-06-23 Thread Wes McKinney
The Docker nightlies are almost back to passing

https://github.com/ursa-labs/crossbow/branches/all?page=1=nightly-484=%E2%9C%93

I put up a WIP patch for the c_glib build but will need help with
that, and a patch for R. On the others

* I will look into turbodbc to see if it can be fixed easily, it
requires an upstream patch that hasn't been merged yet
* If someone could look into the Dask Docker build that would be great

- Wes

On Thu, Jun 20, 2019 at 7:42 AM Francois Saint-Jacques
 wrote:
>
> Ubuntu 16.04 ships with either 3.4 or 3.5
> Centos 6 with epel ships with 3.4, with scl it's 3.5 or 3.6
>
> I'd say keep just for ub16.
>
> François


Re: Failures in Arrow tests run outside of continuous integration

2019-06-20 Thread Francois Saint-Jacques
Ubuntu 16.04 ships with either 3.4 or 3.5
Centos 6 with epel ships with 3.4, with scl it's 3.5 or 3.6

I'd say keep just for ub16.

François


Re: Failures in Arrow tests run outside of continuous integration

2019-06-20 Thread Krisztián Szűcs
These are the packaging tasks, the red ones are blocking the release:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-479

The conda tasks should be ported to pipelines, because the travis builds
simply stopped working (without saying anything to either stderr or stdout)
and conda smithy now uses pipelines with more maintainable configurations:
https://github.com/conda-forge/arrow-cpp-feedstock/tree/master/.azure-pipelines

On Thu, Jun 20, 2019 at 11:16 AM Antoine Pitrou  wrote:

>
> Le 20/06/2019 à 11:14, Krisztián Szűcs a écrit :
> > Travis provides 50 minutes timeout for free builds [1], so We need to run
> > those
> > tests either on ursabot or another hosted CI service. With crossbow we're
> > not
> > limited to certain CI services, so we can use CircleCI and Azure
> Pipelines
> > too,
> > and simultaneously.
> > Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm
> handling
> > this. Other tasks still need help though.
>
> What are "other tasks"?
>
>
> >
> > [1]:
> https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
> > [2]: https://circleci.com/docs/2.0/configuration-reference/#jobs
> >
> > On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler  wrote:
> >
> >> The last Spark Integration test timed out. Is there any way to increase
> the
> >> timeout for Crossbow tests?
> >>
> >> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney 
> wrote:
> >>
> >>> hi folks,
> >>>
> >>> We have a number of integration tests and extra-CI build
> >>> configurations that are run via Crossbow, and many of them are
> >>> failing:
> >>>
> >>>
> >>>
> >>
> https://github.com/ursa-labs/crossbow/branches/all?page=1=nightly-480=%E2%9C%93
> >>>
> >>> At some point we would like to notify the dev@ list about these
> >>> failures when they happen (or in a nightly e-mail), but in the
> >>> meantime some of these probably should be fixed (e.g. the Spark
> >>> integration build is failing) before the project can release again
> >>>
> >>> Thanks
> >>> Wes
> >>>
> >>
> >
>


Re: Failures in Arrow tests run outside of continuous integration

2019-06-20 Thread Antoine Pitrou


Le 20/06/2019 à 11:14, Krisztián Szűcs a écrit :
> Travis provides 50 minutes timeout for free builds [1], so We need to run
> those
> tests either on ursabot or another hosted CI service. With crossbow we're
> not
> limited to certain CI services, so we can use CircleCI and Azure Pipelines
> too,
> and simultaneously.
> Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm handling
> this. Other tasks still need help though.

What are "other tasks"?


> 
> [1]: https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
> [2]: https://circleci.com/docs/2.0/configuration-reference/#jobs
> 
> On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler  wrote:
> 
>> The last Spark Integration test timed out. Is there any way to increase the
>> timeout for Crossbow tests?
>>
>> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney  wrote:
>>
>>> hi folks,
>>>
>>> We have a number of integration tests and extra-CI build
>>> configurations that are run via Crossbow, and many of them are
>>> failing:
>>>
>>>
>>>
>> https://github.com/ursa-labs/crossbow/branches/all?page=1=nightly-480=%E2%9C%93
>>>
>>> At some point we would like to notify the dev@ list about these
>>> failures when they happen (or in a nightly e-mail), but in the
>>> meantime some of these probably should be fixed (e.g. the Spark
>>> integration build is failing) before the project can release again
>>>
>>> Thanks
>>> Wes
>>>
>>
> 


Re: Failures in Arrow tests run outside of continuous integration

2019-06-20 Thread Krisztián Szűcs
Travis provides 50 minutes timeout for free builds [1], so We need to run
those
tests either on ursabot or another hosted CI service. With crossbow we're
not
limited to certain CI services, so we can use CircleCI and Azure Pipelines
too,
and simultaneously.
Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm handling
this. Other tasks still need help though.

[1]: https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
[2]: https://circleci.com/docs/2.0/configuration-reference/#jobs

On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler  wrote:

> The last Spark Integration test timed out. Is there any way to increase the
> timeout for Crossbow tests?
>
> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney  wrote:
>
> > hi folks,
> >
> > We have a number of integration tests and extra-CI build
> > configurations that are run via Crossbow, and many of them are
> > failing:
> >
> >
> >
> https://github.com/ursa-labs/crossbow/branches/all?page=1=nightly-480=%E2%9C%93
> >
> > At some point we would like to notify the dev@ list about these
> > failures when they happen (or in a nightly e-mail), but in the
> > meantime some of these probably should be fixed (e.g. the Spark
> > integration build is failing) before the project can release again
> >
> > Thanks
> > Wes
> >
>


Re: Failures in Arrow tests run outside of continuous integration

2019-06-19 Thread Bryan Cutler
The last Spark Integration test timed out. Is there any way to increase the
timeout for Crossbow tests?

On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney  wrote:

> hi folks,
>
> We have a number of integration tests and extra-CI build
> configurations that are run via Crossbow, and many of them are
> failing:
>
>
> https://github.com/ursa-labs/crossbow/branches/all?page=1=nightly-480=%E2%9C%93
>
> At some point we would like to notify the dev@ list about these
> failures when they happen (or in a nightly e-mail), but in the
> meantime some of these probably should be fixed (e.g. the Spark
> integration build is failing) before the project can release again
>
> Thanks
> Wes
>