date:20201230

Re: [C++] Weird Rust Linter error in CICD & Float/Double equality

2020-12-30 Thread Andrew Lamb

https://github.com/apache/arrow/pull/9041 has a fix -- I'll merge it as
soon as it passes CI

On Wed, Dec 30, 2020 at 7:01 AM Andrew Lamb  wrote:

> I think this may be related to something merged in
> https://github.com/apache/arrow/pull/9041#issuecomment-752266447
>
> We'll get a fix in ASAP -- sorry about that
>
> On Wed, Dec 30, 2020 at 4:29 AM Ying Zhou  wrote:
>
>> Hi,
>>
>> When finalizing my Arrow2ORC C++ pull request I found a weird
>> Rust-related and IPC-related error in the Linter that didn’t happen just 2
>> days ago despite my code having nothing to do with either Rust or IPC. Here
>> is the check:
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680
>> 
>> Here is the part of the output I think is relevant:
>>
>> source_dir /arrow/cpp/src --quiet
>> 1413
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1413>[1/1]
>> cd /tmp/arrow-lint-_9i678pu/cpp-build && /usr/local/bin/python
>> /arrow/cpp/build-support/lint_cpp_cli.py /arrow/cpp/src
>> 1414
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1414>INFO:archery:Running
>> Python formatter (autopep8)
>> 1415
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1415>INFO:archery:Running
>> Python linter (flake8)
>> 1416
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1416>INFO:archery:Running
>> cmake-format linters
>> 1417
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1417>WARNING:archery:run-cmake-format
>> modifies files, regardless of --fix
>> 1418
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1418>INFO:archery:Running
>> apache-rat linter
>> 1419
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1419>INFO:archery:Running
>> R linter
>> 1420
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1420>INFO:archery:Running
>> Rust linter
>> 1421
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1421>Diff
>> in /arrow/rust/arrow/src/ipc/reader.rs at line 160:
>> 1422
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1422>
>>let null_count = struct_node.null_count() as usize;
>> 1423
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1423>
>>let struct_array = if null_count > 0 {
>> 1424
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1424>
>>// create struct array from fields, arrays and null data
>> 1425
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1425>-
>>   StructArray::from((
>> 1426
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1426>-
>>   struct_arrays,
>> 1427
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1427>-
>>   null_buffer,
>> 1428
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1428>-
>>   ))
>> 1429
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1429>+
>>   StructArray::from((struct_arrays, null_buffer))
>> 1430
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1430>
>>} else {
>> 1431
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1431>
>>StructArray::from(struct_arrays)
>> 1432
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1432>
>>};
>> 1433
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1433>INFO:archery:Running
>> Docker linter
>> 1434
>>  <
>> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1434>Error:
>> `docker-compose --file /home/runner/work/arrow/arrow/docker-compose.yml run
>> --rm ubuntu-lint` exited with a non-zero exit code 1, see the process log
>> above.
>>
>> There is no C++ or reasonbly C++-related error. Does anyone know why the
>> error happen?
>>
>> I also would like to ask about table equality when some columns are
>> float/double. In this case do we have some built-in epsilon so that 0.6 ==
>> 0.6 work? Right now I have separate tests for these types that looks like
>> the following and they are pretty clumsy:
>>
>> EXPECT_TRUE(outputTable->schema()->Equals(*(table->schema(;
>>   EXPECT_TRUE(outputTable->column(0)
>>   ->chunk(0)
>>   ->Slice(0, numRows / 2)
>>   ->ApproxEquals(table->column(0)->chunk(1)));
>>   EXPECT_TRUE(outputTable->column(0)
>

Re: [C++] Weird Rust Linter error in CICD & Float/Double equality

2020-12-30 Thread Andrew Lamb

I think this may be related to something merged in
https://github.com/apache/arrow/pull/9041#issuecomment-752266447

We'll get a fix in ASAP -- sorry about that

On Wed, Dec 30, 2020 at 4:29 AM Ying Zhou  wrote:

> Hi,
>
> When finalizing my Arrow2ORC C++ pull request I found a weird Rust-related
> and IPC-related error in the Linter that didn’t happen just 2 days ago
> despite my code having nothing to do with either Rust or IPC. Here is the
> check:
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680 <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680>
> Here is the part of the output I think is relevant:
>
> source_dir /arrow/cpp/src --quiet
> 1413
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1413>[1/1]
> cd /tmp/arrow-lint-_9i678pu/cpp-build && /usr/local/bin/python
> /arrow/cpp/build-support/lint_cpp_cli.py /arrow/cpp/src
> 1414
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1414>INFO:archery:Running
> Python formatter (autopep8)
> 1415
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1415>INFO:archery:Running
> Python linter (flake8)
> 1416
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1416>INFO:archery:Running
> cmake-format linters
> 1417
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1417>WARNING:archery:run-cmake-format
> modifies files, regardless of --fix
> 1418
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1418>INFO:archery:Running
> apache-rat linter
> 1419
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1419>INFO:archery:Running
> R linter
> 1420
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1420>INFO:archery:Running
> Rust linter
> 1421
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1421>Diff
> in /arrow/rust/arrow/src/ipc/reader.rs at line 160:
> 1422
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1422>
>let null_count = struct_node.null_count() as usize;
> 1423
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1423>
>let struct_array = if null_count > 0 {
> 1424
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1424>
>// create struct array from fields, arrays and null data
> 1425
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1425>-
>   StructArray::from((
> 1426
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1426>-
>   struct_arrays,
> 1427
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1427>-
>   null_buffer,
> 1428
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1428>-
>   ))
> 1429
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1429>+
>   StructArray::from((struct_arrays, null_buffer))
> 1430
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1430>
>} else {
> 1431
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1431>
>StructArray::from(struct_arrays)
> 1432
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1432>
>};
> 1433
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1433>INFO:archery:Running
> Docker linter
> 1434
>  <
> https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680#step:7:1434>Error:
> `docker-compose --file /home/runner/work/arrow/arrow/docker-compose.yml run
> --rm ubuntu-lint` exited with a non-zero exit code 1, see the process log
> above.
>
> There is no C++ or reasonbly C++-related error. Does anyone know why the
> error happen?
>
> I also would like to ask about table equality when some columns are
> float/double. In this case do we have some built-in epsilon so that 0.6 ==
> 0.6 work? Right now I have separate tests for these types that looks like
> the following and they are pretty clumsy:
>
> EXPECT_TRUE(outputTable->schema()->Equals(*(table->schema(;
>   EXPECT_TRUE(outputTable->column(0)
>   ->chunk(0)
>   ->Slice(0, numRows / 2)
>   ->ApproxEquals(table->column(0)->chunk(1)));
>   EXPECT_TRUE(outputTable->column(0)
>   ->chunk(0)
>   ->Slice(numRows / 2, numRows / 2)
>   ->ApproxEquals(table->column(0)->chunk(3)));
>   EXPECT_TRUE(outputTable->column(1)
>   ->chunk(0)
>   ->Slice(0, numRows / 2)
>

[NIGHTLY] Arrow Build Report for Job nightly-2020-12-30-0

2020-12-30 Thread Crossbow



Arrow Build Report for Job nightly-2020-12-30-0

All tasks: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0

Failed Tasks:
- test-conda-python-3.7-hdfs-2.9.2:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-test-conda-python-3.7-hdfs-2.9.2
- test-conda-python-3.7-spark-branch-3.0:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-test-conda-python-3.7-spark-branch-3.0
- test-conda-python-3.8-dask-master:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-test-conda-python-3.8-dask-master
- test-conda-python-3.8-jpype:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-test-conda-python-3.8-jpype
- test-ubuntu-18.04-docs:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-test-ubuntu-18.04-docs
- wheel-win-cp36m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-wheel-win-cp36m
- wheel-win-cp37m:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-wheel-win-cp37m
- wheel-win-cp38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-wheel-win-cp38

Succeeded Tasks:
- centos-7-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-travis-centos-7-aarch64
- centos-7-amd64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-centos-7-amd64
- centos-8-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-travis-centos-8-aarch64
- centos-8-amd64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-centos-8-amd64
- conda-clean:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-clean
- conda-linux-gcc-py36-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-drone-conda-linux-gcc-py36-aarch64
- conda-linux-gcc-py36-cpu-r36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py36-cpu-r36
- conda-linux-gcc-py36-cuda:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py36-cuda
- conda-linux-gcc-py37-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-drone-conda-linux-gcc-py37-aarch64
- conda-linux-gcc-py37-cpu-r40:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py37-cpu-r40
- conda-linux-gcc-py37-cuda:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py37-cuda
- conda-linux-gcc-py38-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-drone-conda-linux-gcc-py38-aarch64
- conda-linux-gcc-py38-cpu:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py38-cpu
- conda-linux-gcc-py38-cuda:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py38-cuda
- conda-linux-gcc-py39-aarch64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-drone-conda-linux-gcc-py39-aarch64
- conda-linux-gcc-py39-cpu:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py39-cpu
- conda-linux-gcc-py39-cuda:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-linux-gcc-py39-cuda
- conda-osx-clang-py36-r36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-osx-clang-py36-r36
- conda-osx-clang-py37-r40:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-osx-clang-py37-r40
- conda-osx-clang-py38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-osx-clang-py38
- conda-win-vs2017-py36-r36:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-win-vs2017-py36-r36
- conda-win-vs2017-py37-r40:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-win-vs2017-py37-r40
- conda-win-vs2017-py38:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-azure-conda-win-vs2017-py38
- debian-buster-amd64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-github-debian-buster-amd64
- debian-buster-arm64:
  URL: 
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-30-0-travis-debian-buster-arm64
- example-cpp-minimal-build-static-system-dependency:
  URL: 
https://github.com/ursa-labs/

[C++] Weird Rust Linter error in CICD & Float/Double equality

2020-12-30 Thread Ying Zhou

Hi,

When finalizing my Arrow2ORC C++ pull request I found a weird Rust-related and 
IPC-related error in the Linter that didn’t happen just 2 days ago despite my 
code having nothing to do with either Rust or IPC. Here is the check: 
https://github.com/apache/arrow/pull/8648/checks?check_run_id=1625423680 
 Here 
is the part of the output I think is relevant:

source_dir /arrow/cpp/src --quiet
1413
 
[1/1]
 cd /tmp/arrow-lint-_9i678pu/cpp-build && /usr/local/bin/python 
/arrow/cpp/build-support/lint_cpp_cli.py /arrow/cpp/src
1414
 
INFO:archery:Running
 Python formatter (autopep8)
1415
 
INFO:archery:Running
 Python linter (flake8)
1416
 
INFO:archery:Running
 cmake-format linters
1417
 
WARNING:archery:run-cmake-format
 modifies files, regardless of --fix
1418
 
INFO:archery:Running
 apache-rat linter
1419
 
INFO:archery:Running
 R linter
1420
 
INFO:archery:Running
 Rust linter
1421
 
Diff
 in /arrow/rust/arrow/src/ipc/reader.rs at line 160:
1422
 

 let null_count = struct_node.null_count() as usize;
1423
 

 let struct_array = if null_count > 0 {
1424
 

 // create struct array from fields, arrays and null data
1425
 
-
StructArray::from((
1426
 
-
struct_arrays,
1427
 
-
null_buffer,
1428
 
-
))
1429
 
+
StructArray::from((struct_arrays, null_buffer))
1430
 

 } else {
1431
 

 StructArray::from(struct_arrays)
1432
 

 };
1433
 
INFO:archery:Running
 Docker linter
1434
 
Error:
 `docker-compose --file /home/runner/work/arrow/arrow/docker-compose.yml run 
--rm ubuntu-lint` exited with a non-zero exit code 1, see the process log above.

There is no C++ or reasonbly C++-related error. Does anyone know why the error 
happen?

I also would like to ask about table equality when some columns are 
float/double. In this case do we have some built-in epsilon so that 0.6 == 0.6 
work? Right now I have separate tests for these types that looks like the 
following and they are pretty clumsy:

EXPECT_TRUE(outputTable->schema()->Equals(*(table->schema(;
  EXPECT_TRUE(outputTable->column(0)
  ->chunk(0)
  ->Slice(0, numRows / 2)
  ->ApproxEquals(table->column(0)->chunk(1)));
  EXPECT_TRUE(outputTable->column(0)
  ->chunk(0)
  ->Slice(numRows / 2, numRows / 2)
  ->ApproxEquals(table->column(0)->chunk(3)));
  EXPECT_TRUE(outputTable->column(1)
  ->chunk(0)
  ->Slice(0, numRows / 2)
  ->ApproxEquals(table->column(1)->chunk(1)));
  EXPECT_TRUE(outputTable->column(1)
  ->chunk(0)
  ->Slice(numRows / 2, numRows / 2)
  ->ApproxEquals(table->column(1)->chunk(3)));

Thanks,
Ying

Re: [C++] Weird Rust Linter error in CICD & Float/Double equality

Re: [C++] Weird Rust Linter error in CICD & Float/Double equality

[NIGHTLY] Arrow Build Report for Job nightly-2020-12-30-0

[C++] Weird Rust Linter error in CICD & Float/Double equality

4 matches

Site Navigation

Mail list logo

Footer information