[jira] [Updated] (ARROW-17369) [CI][Java] Automatically extract dependent library versions from pom.xml for s390x
[
https://issues.apache.org/jira/browse/ARROW-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-17369:
-
Description: Current script {{java_build.sh}} requires a manual update for
dependent library versions. (was: Current script {{java_build.sh }}requires a
manual update for dependent library versions.)
> [CI][Java] Automatically extract dependent library versions from pom.xml for
> s390x
> --
>
> Key: ARROW-17369
> URL: https://issues.apache.org/jira/browse/ARROW-17369
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
>Affects Versions: 10.0.0
>Reporter: Kazuaki Ishizaki
>Priority: Minor
>
> Current script {{java_build.sh}} requires a manual update for dependent
> library versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17369) [CI][Java] Automatically extract dependent library versions from pom.xml for s390x
Kazuaki Ishizaki created ARROW-17369:
Summary: [CI][Java] Automatically extract dependent library
versions from pom.xml for s390x
Key: ARROW-17369
URL: https://issues.apache.org/jira/browse/ARROW-17369
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Affects Versions: 10.0.0
Reporter: Kazuaki Ishizaki
Current script {{java_build.sh }}requires a manual update for dependent library
versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-17112) [Java] TestArrowReaderWriter.testFileFooterSizeOverflow causes a failure on s390x
[
https://issues.apache.org/jira/browse/ARROW-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-17112:
-
Summary: [Java] TestArrowReaderWriter.testFileFooterSizeOverflow causes a
failure on s390x (was: [Jafva]
TestArrowReaderWriter.testFileFooterSizeOverflow causes a failure on s390x)
> [Java] TestArrowReaderWriter.testFileFooterSizeOverflow causes a failure on
> s390x
> -
>
> Key: ARROW-17112
> URL: https://issues.apache.org/jira/browse/ARROW-17112
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 9.0.0
>Reporter: Kazuaki Ishizaki
>Priority: Minor
>
> On big-endian plantform such as s390x,
> {{TestArrowReaderWriter.testFileFooterSizeOverflow}} causes a failure.
> {code}
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestArrowReaderWriter.testFileFooterSizeOverflow:913
> expected:<...alid footer length: [2147483647]> but was:<...alid footer
> length: [-129]>
> [INFO]
> [ERROR] Tests run: 610, Failures: 1, Errors: 0, Skipped: 4
> [INFO]
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Arrow Java Root POM 9.0.0-SNAPSHOT .. SUCCESS [ 2.182
> s]
> [INFO] Arrow Format ... SUCCESS [ 0.995
> s]
> [INFO] Arrow Memory ... SUCCESS [ 0.761
> s]
> [INFO] Arrow Memory - Core SUCCESS [ 1.582
> s]
> [INFO] Arrow Memory - Unsafe .. SUCCESS [ 1.600
> s]
> [INFO] Arrow Memory - Netty ... SUCCESS [ 1.966
> s]
> [INFO] Arrow Vectors .. FAILURE [ 16.779
> s]
> [INFO] Arrow Compression .. SKIPPED
> [INFO] Arrow Tools SKIPPED
> [INFO] Arrow JDBC Adapter . SKIPPED
> [INFO] Arrow Plasma Client SUCCESS [ 1.171
> s]
> [INFO] Arrow Flight ... SUCCESS [ 0.741
> s]
> [INFO] Arrow Flight Core .. SKIPPED
> [INFO] Arrow Flight GRPC .. SKIPPED
> [INFO] Arrow Flight SQL ... SKIPPED
> [INFO] Arrow Flight Integration Tests . SKIPPED
> [INFO] Arrow AVRO Adapter . SKIPPED
> [INFO] Arrow Algorithms ... SKIPPED
> [INFO] Arrow Performance Benchmarks 9.0.0-SNAPSHOT SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 23.584 s (Wall Clock)
> [INFO] Finished at: 2022-07-18T12:32:21Z
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test)
> on project arrow-vector: There are test failures.
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17112) [Jafva] TestArrowReaderWriter.testFileFooterSizeOverflow causes a failure on s390x
Kazuaki Ishizaki created ARROW-17112:
Summary: [Jafva] TestArrowReaderWriter.testFileFooterSizeOverflow
causes a failure on s390x
Key: ARROW-17112
URL: https://issues.apache.org/jira/browse/ARROW-17112
Project: Apache Arrow
Issue Type: Bug
Components: Java
Affects Versions: 9.0.0
Reporter: Kazuaki Ishizaki
On big-endian plantform such as s390x,
{{TestArrowReaderWriter.testFileFooterSizeOverflow}} causes a failure.
{code}
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestArrowReaderWriter.testFileFooterSizeOverflow:913
expected:<...alid footer length: [2147483647]> but was:<...alid footer length:
[-129]>
[INFO]
[ERROR] Tests run: 610, Failures: 1, Errors: 0, Skipped: 4
[INFO]
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Arrow Java Root POM 9.0.0-SNAPSHOT .. SUCCESS [ 2.182 s]
[INFO] Arrow Format ... SUCCESS [ 0.995 s]
[INFO] Arrow Memory ... SUCCESS [ 0.761 s]
[INFO] Arrow Memory - Core SUCCESS [ 1.582 s]
[INFO] Arrow Memory - Unsafe .. SUCCESS [ 1.600 s]
[INFO] Arrow Memory - Netty ... SUCCESS [ 1.966 s]
[INFO] Arrow Vectors .. FAILURE [ 16.779 s]
[INFO] Arrow Compression .. SKIPPED
[INFO] Arrow Tools SKIPPED
[INFO] Arrow JDBC Adapter . SKIPPED
[INFO] Arrow Plasma Client SUCCESS [ 1.171 s]
[INFO] Arrow Flight ... SUCCESS [ 0.741 s]
[INFO] Arrow Flight Core .. SKIPPED
[INFO] Arrow Flight GRPC .. SKIPPED
[INFO] Arrow Flight SQL ... SKIPPED
[INFO] Arrow Flight Integration Tests . SKIPPED
[INFO] Arrow AVRO Adapter . SKIPPED
[INFO] Arrow Algorithms ... SKIPPED
[INFO] Arrow Performance Benchmarks 9.0.0-SNAPSHOT SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 23.584 s (Wall Clock)
[INFO] Finished at: 2022-07-18T12:32:21Z
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on
project arrow-vector: There are test failures.
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17101) [Java] Prepare new protoc-gen-grpc-java for s390x
Kazuaki Ishizaki created ARROW-17101: Summary: [Java] Prepare new protoc-gen-grpc-java for s390x Key: ARROW-17101 URL: https://issues.apache.org/jira/browse/ARROW-17101 Project: Apache Arrow Issue Type: Improvement Components: Java Affects Versions: 9.0.0 Reporter: Kazuaki Ishizaki Recent s390x build for Java causes [failures|https://app.travis-ci.com/github/apache/arrow/jobs/576822591#L2933] due to missing relevant version of protoc-gen-grpc-java. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ARROW-12885) [C++] Error: template with C linkage template
[ https://issues.apache.org/jira/browse/ARROW-12885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528304#comment-17528304 ] Kazuaki Ishizaki commented on ARROW-12885: -- I got an AIX instance for a few weeks. I will work to fix errors on AIX. > [C++] Error: template with C linkage template > --- > > Key: ARROW-12885 > URL: https://issues.apache.org/jira/browse/ARROW-12885 > Project: Apache Arrow > Issue Type: Bug > Components: C++ > Environment: IBM i | AS400 | AIX >Reporter: Menno >Priority: Major > Attachments: 2021-05-26 16_31_09-Window.png, thrift_ep-build-err.log > > > When installing arrow on IBM i it fails the install at the thrift dependency > install with the following output: > !2021-05-26 16_31_09-Window.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (ARROW-12885) [C++] Error: template with C linkage template
[
https://issues.apache.org/jira/browse/ARROW-12885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17526393#comment-17526393
]
Kazuaki Ishizaki commented on ARROW-12885:
--
According to [this
issue|[https://github.com/protocolbuffers/protobuf/issues/8373],] AIX does not
have {{{}endian.h{}}}. Do we apply this change?
> [C++] Error: template with C linkage template
> ---
>
> Key: ARROW-12885
> URL: https://issues.apache.org/jira/browse/ARROW-12885
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: IBM i | AS400 | AIX
>Reporter: Menno
>Priority: Major
> Attachments: 2021-05-26 16_31_09-Window.png, thrift_ep-build-err.log
>
>
> When installing arrow on IBM i it fails the install at the thrift dependency
> install with the following output:
> !2021-05-26 16_31_09-Window.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (ARROW-16218) [CI][Java][Flight] Test failure on s390x
[ https://issues.apache.org/jira/browse/ARROW-16218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523826#comment-17523826 ] Kazuaki Ishizaki commented on ARROW-16218: -- I know JNI libraries should be updated for the latest netty. However, the latest netty includes more JNIs. Thus, at first, we need to identify which JNI libraries are required. > [CI][Java][Flight] Test failure on s390x > > > Key: ARROW-16218 > URL: https://issues.apache.org/jira/browse/ARROW-16218 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration, FlightRPC, Java >Reporter: Antoine Pitrou >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Seems to happen quite reliably on Travis-CI: > https://app.travis-ci.com/github/apache/arrow/builds/249511328 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (ARROW-15778) [Java] Endianness field not emitted in IPC stream
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513132#comment-17513132 ] Kazuaki Ishizaki commented on ARROW-15778: -- [~bryanc] Thank you for your suggestion. Sure, I will create a simple PR as is, and will open an follow-up issue to add a proper test. > [Java] Endianness field not emitted in IPC stream > - > > Key: ARROW-15778 > URL: https://issues.apache.org/jira/browse/ARROW-15778 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Reporter: Antoine Pitrou >Priority: Major > Fix For: 8.0.0 > > > It seems the Java IPC writer implementation does not emit the Endianness > information at all (making it Little by default). This complicates > interoperability with the C++ IPC reader, which does read this information > and acts on it to decide whether it needs to byteswap the incoming data. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (ARROW-15778) [Java] Endianness field not emitted in IPC stream
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502054#comment-17502054 ] Kazuaki Ishizaki commented on ARROW-15778: -- This fix is already created [here|https://github.com/kiszk/arrow/commit/db06d043b07c63069169b5a88a4393aa229c095c]. We confirmed this issue was solved by this fix. The challenge is to create a test case. I realized that the latest arrow requires more native libraries for the build. It takes some time. So, I will create a test case with an older version arrow. > [Java] Endianness field not emitted in IPC stream > - > > Key: ARROW-15778 > URL: https://issues.apache.org/jira/browse/ARROW-15778 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Reporter: Antoine Pitrou >Priority: Major > Fix For: 8.0.0 > > > It seems the Java IPC writer implementation does not emit the Endianness > information at all (making it Little by default). This complicates > interoperability with the C++ IPC reader, which does read this information > and acts on it to decide whether it needs to byteswap the incoming data. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (ARROW-3476) [Java] mvn test in memory fails on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki resolved ARROW-3476.
-
Fix Version/s: 3.0.0
Resolution: Fixed
This is already solved since Arrow 3.0 or later support big-endian in C++
binding.
> [Java] mvn test in memory fails on a big-endian platform
>
>
> Key: ARROW-3476
> URL: https://issues.apache.org/jira/browse/ARROW-3476
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Reporter: Kazuaki Ishizaki
>Priority: Major
> Fix For: 3.0.0
>
>
> Apache Arrow is becoming commonplace to exchange data among important
> emerging analytics frameworks such as Pandas, Numpy, and Spark.
> [IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process
> critical transactions such as bank or credit card. Users of IBM Z want to
> extract insights from these transactions using the emerging analytics systems
> on IBM Z Linux. These analytics pipelines can be also fast and effective on
> IBM Z Linux by using Apache Arrow on memory.
> From the technical perspective, since IBM Z Linux uses big-endian data
> format, it is not possible to use Apache Arrow in this pipeline. If Apache
> Arrow could support big-endian, the use case would be expanded.
> When I ran test case of Apache arrow on a big-endian platform (ppc64be),
> {{mvn test}} in memory causes a failure due to an assertion.
> In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during
> an allocation of a {{RootAllocator}} class.
> {code}
> $ uname -a
> Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC
> 2016 ppc64 ppc64 ppc64 GNU/Linux
> $ arch
> ppc64
> $ cd java/memory
> $ mvn test
> [INFO] Scanning for projects...
> [INFO]
>
> [INFO]
>
> [INFO] Building Arrow Memory 0.12.0-SNAPSHOT
> [INFO]
>
> [INFO]
> ...
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082
> s - in org.apache.arrow.memory.TestAccountant
> [INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012
> s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Running org.apache.arrow.memory.TestBaseAllocator
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746
> s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
> [ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time
> elapsed: 0.313 s <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian
> systems.
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> [ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed:
> 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (ARROW-3476) [Java] mvn test in memory fails on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497953#comment-17497953
]
Kazuaki Ishizaki commented on ARROW-3476:
-
Sure, let me close this issue since this was already solved.
> [Java] mvn test in memory fails on a big-endian platform
>
>
> Key: ARROW-3476
> URL: https://issues.apache.org/jira/browse/ARROW-3476
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Reporter: Kazuaki Ishizaki
>Priority: Major
>
> Apache Arrow is becoming commonplace to exchange data among important
> emerging analytics frameworks such as Pandas, Numpy, and Spark.
> [IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process
> critical transactions such as bank or credit card. Users of IBM Z want to
> extract insights from these transactions using the emerging analytics systems
> on IBM Z Linux. These analytics pipelines can be also fast and effective on
> IBM Z Linux by using Apache Arrow on memory.
> From the technical perspective, since IBM Z Linux uses big-endian data
> format, it is not possible to use Apache Arrow in this pipeline. If Apache
> Arrow could support big-endian, the use case would be expanded.
> When I ran test case of Apache arrow on a big-endian platform (ppc64be),
> {{mvn test}} in memory causes a failure due to an assertion.
> In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during
> an allocation of a {{RootAllocator}} class.
> {code}
> $ uname -a
> Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC
> 2016 ppc64 ppc64 ppc64 GNU/Linux
> $ arch
> ppc64
> $ cd java/memory
> $ mvn test
> [INFO] Scanning for projects...
> [INFO]
>
> [INFO]
>
> [INFO] Building Arrow Memory 0.12.0-SNAPSHOT
> [INFO]
>
> [INFO]
> ...
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082
> s - in org.apache.arrow.memory.TestAccountant
> [INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012
> s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Running org.apache.arrow.memory.TestBaseAllocator
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746
> s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
> [ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time
> elapsed: 0.313 s <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian
> systems.
> at
> org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> [ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed:
> 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (ARROW-15778) [Java] Endianness field not emitted in IPC stream
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497949#comment-17497949 ] Kazuaki Ishizaki commented on ARROW-15778: -- [~apitrou] Thank you. In another issue, I am suspecting the endianness in schema. I will look at this. > [Java] Endianness field not emitted in IPC stream > - > > Key: ARROW-15778 > URL: https://issues.apache.org/jira/browse/ARROW-15778 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Reporter: Antoine Pitrou >Priority: Major > Fix For: 8.0.0 > > > It seems the Java IPC writer implementation does not emit the Endianness > information at all (making it Little by default). This complicates > interoperability with the C++ IPC reader, which does read this information > and acts on it to decide whether it needs to byteswap the incoming data. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (ARROW-15563) [C++] Compilation failure on s390x platform
[
https://issues.apache.org/jira/browse/ARROW-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493397#comment-17493397
]
Kazuaki Ishizaki commented on ARROW-15563:
--
This problem does not occur in the master branch. This is because [the latest
parse_number.h|https://github.com/apache/arrow/blob/master/cpp/src/arrow/vendored/fast_float/parse_number.h]
does not have this statement.
So, it would be good to pick up the recent version of Arrow.
> [C++] Compilation failure on s390x platform
> ---
>
> Key: ARROW-15563
> URL: https://issues.apache.org/jira/browse/ARROW-15563
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 3.0.0
> Environment: s390x (IBM LinuxONE)
>Reporter: Chandra Shekhar Reddy
>Priority: Major
>
>
> {code:java}
> (pyarrow-dev) [root@s390x]# cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME
> -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=debug
> -DARROW_WITH_BZ2=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON
> -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON
> -DARROW_WITH_BROTLI=ON -DARROW_PARQUET=ON -DARROW_PYTHON=ON
> -DARROW_BUILD_TESTS=ON ..
> -- Building using CMake version: 3.22.2
> -- The C compiler identification is GNU 9.2.1
> -- The CXX compiler identification is GNU 8.5.0
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /usr/bin/cc - skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Arrow version: 3.0.0 (full: '3.0.0')
> -- Arrow SO version: 300 (full: 300.0.0)
> -- clang-tidy not found
> -- clang-format not found
> -- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
> -- infer not found
> -- Found Python3: /usr/bin/python3.9 (found version "3.9.6") found
> components: Interpreter
> -- Found cpplint executable at
> /root/git/repos/arrow/cpp/build-support/cpplint.py
> -- System processor: s390x
> -- Arrow build warning level: CHECKIN
> Using ld linker
> Configured for DEBUG build (set with cmake
> -DCMAKE_BUILD_TYPE={release,debug,...})
> -- Build Type: DEBUG
> -- Using AUTO approach to find dependencies
> -- ARROW_ABSL_BUILD_VERSION: 0f3bb466b868b523cf1dc9b2aaaed65c77b28862
> -- ARROW_AWSSDK_BUILD_VERSION: 1.8.90
> -- ARROW_AWS_CHECKSUMS_BUILD_VERSION: v0.1.5
> -- ARROW_AWS_C_COMMON_BUILD_VERSION: v0.4.59
> -- ARROW_AWS_C_EVENT_STREAM_BUILD_VERSION: v0.1.5
> -- ARROW_BOOST_BUILD_VERSION: 1.71.0
> -- ARROW_BROTLI_BUILD_VERSION: v1.0.7
> -- ARROW_BZIP2_BUILD_VERSION: 1.0.8
> -- ARROW_CARES_BUILD_VERSION: 1.16.1
> -- ARROW_GBENCHMARK_BUILD_VERSION: v1.5.2
> -- ARROW_GFLAGS_BUILD_VERSION: v2.2.2
> -- ARROW_GLOG_BUILD_VERSION: v0.4.0
> -- ARROW_GRPC_BUILD_VERSION: v1.33.2
> -- ARROW_GTEST_BUILD_VERSION: 1.10.0
> -- ARROW_JEMALLOC_BUILD_VERSION: 5.2.1
> -- ARROW_LZ4_BUILD_VERSION: v1.9.2
> -- ARROW_MIMALLOC_BUILD_VERSION: v1.6.4
> -- ARROW_ORC_BUILD_VERSION: 1.6.2
> -- ARROW_PROTOBUF_BUILD_VERSION: v3.13.0
> -- ARROW_RAPIDJSON_BUILD_VERSION: 1a803826f1197b5e30703afe4b9c0e7dd48074f5
> -- ARROW_RE2_BUILD_VERSION: 2019-08-01
> -- ARROW_SNAPPY_BUILD_VERSION: 1.1.8
> -- ARROW_THRIFT_BUILD_VERSION: 0.12.0
> -- ARROW_THRIFT_BUILD_MD5_CHECKSUM: 3deebbb4d1ca77dd9c9e009a1ea02183
> -- ARROW_UTF8PROC_BUILD_VERSION: v2.5.0
> -- ARROW_ZLIB_BUILD_VERSION: 1.2.11
> -- ARROW_ZSTD_BUILD_VERSION: v1.4.5
> -- Looking for pthread.h
> -- Looking for pthread.h - found
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
> -- Check if compiler accepts -pthread
> -- Check if compiler accepts -pthread - yes
> -- Found Threads: TRUE
> -- Checking for module 'thrift'
> -- Package 'thrift', required by 'virtual:world', not found
> -- Could NOT find Thrift: Found unsuitable version "", but required is at
> least "0.11.0" (found THRIFT_LIB-NOTFOUND)
> -- Looking for __SIZEOF_INT128__
> -- Looking for __SIZEOF_INT128__ - found
> -- Found Boost: /usr/include (found suitable version "1.66.0", minimum
> required is "1.58") found components: regex system filesystem
> -- Boost include dir: /usr/include
> -- Boost libraries: Boost::system;Boost::filesystem
> -- Could NOT find Snappy (missing: Snappy_LIB Snappy_INCLUDE_DIR)
> -- Building snappy from source
> -- Checking for modules 'libbrotlicommon;libbrotlienc;libbrotlidec'
> -- Package 'libbrotlicommon', required by 'virtual:world', not found
> -- Package 'libbrotlienc', required by 'virtual:world', not found
> -- Package 'libbrotlidec', required by 'virtual
[jira] [Commented] (ARROW-15563) [C++] Compilation failure on s390x platform
[
https://issues.apache.org/jira/browse/ARROW-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488531#comment-17488531
]
Kazuaki Ishizaki commented on ARROW-15563:
--
Sure, I will look at this
> [C++] Compilation failure on s390x platform
> ---
>
> Key: ARROW-15563
> URL: https://issues.apache.org/jira/browse/ARROW-15563
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 3.0.0
> Environment: s390x (IBM LinuxONE)
>Reporter: Chandra Shekhar Reddy
>Priority: Major
>
>
> {code:java}
> (pyarrow-dev) [root@s390x]# cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME
> -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=debug
> -DARROW_WITH_BZ2=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON
> -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON
> -DARROW_WITH_BROTLI=ON -DARROW_PARQUET=ON -DARROW_PYTHON=ON
> -DARROW_BUILD_TESTS=ON ..
> -- Building using CMake version: 3.22.2
> -- The C compiler identification is GNU 9.2.1
> -- The CXX compiler identification is GNU 8.5.0
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /usr/bin/cc - skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Arrow version: 3.0.0 (full: '3.0.0')
> -- Arrow SO version: 300 (full: 300.0.0)
> -- clang-tidy not found
> -- clang-format not found
> -- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
> -- infer not found
> -- Found Python3: /usr/bin/python3.9 (found version "3.9.6") found
> components: Interpreter
> -- Found cpplint executable at
> /root/git/repos/arrow/cpp/build-support/cpplint.py
> -- System processor: s390x
> -- Arrow build warning level: CHECKIN
> Using ld linker
> Configured for DEBUG build (set with cmake
> -DCMAKE_BUILD_TYPE={release,debug,...})
> -- Build Type: DEBUG
> -- Using AUTO approach to find dependencies
> -- ARROW_ABSL_BUILD_VERSION: 0f3bb466b868b523cf1dc9b2aaaed65c77b28862
> -- ARROW_AWSSDK_BUILD_VERSION: 1.8.90
> -- ARROW_AWS_CHECKSUMS_BUILD_VERSION: v0.1.5
> -- ARROW_AWS_C_COMMON_BUILD_VERSION: v0.4.59
> -- ARROW_AWS_C_EVENT_STREAM_BUILD_VERSION: v0.1.5
> -- ARROW_BOOST_BUILD_VERSION: 1.71.0
> -- ARROW_BROTLI_BUILD_VERSION: v1.0.7
> -- ARROW_BZIP2_BUILD_VERSION: 1.0.8
> -- ARROW_CARES_BUILD_VERSION: 1.16.1
> -- ARROW_GBENCHMARK_BUILD_VERSION: v1.5.2
> -- ARROW_GFLAGS_BUILD_VERSION: v2.2.2
> -- ARROW_GLOG_BUILD_VERSION: v0.4.0
> -- ARROW_GRPC_BUILD_VERSION: v1.33.2
> -- ARROW_GTEST_BUILD_VERSION: 1.10.0
> -- ARROW_JEMALLOC_BUILD_VERSION: 5.2.1
> -- ARROW_LZ4_BUILD_VERSION: v1.9.2
> -- ARROW_MIMALLOC_BUILD_VERSION: v1.6.4
> -- ARROW_ORC_BUILD_VERSION: 1.6.2
> -- ARROW_PROTOBUF_BUILD_VERSION: v3.13.0
> -- ARROW_RAPIDJSON_BUILD_VERSION: 1a803826f1197b5e30703afe4b9c0e7dd48074f5
> -- ARROW_RE2_BUILD_VERSION: 2019-08-01
> -- ARROW_SNAPPY_BUILD_VERSION: 1.1.8
> -- ARROW_THRIFT_BUILD_VERSION: 0.12.0
> -- ARROW_THRIFT_BUILD_MD5_CHECKSUM: 3deebbb4d1ca77dd9c9e009a1ea02183
> -- ARROW_UTF8PROC_BUILD_VERSION: v2.5.0
> -- ARROW_ZLIB_BUILD_VERSION: 1.2.11
> -- ARROW_ZSTD_BUILD_VERSION: v1.4.5
> -- Looking for pthread.h
> -- Looking for pthread.h - found
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
> -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
> -- Check if compiler accepts -pthread
> -- Check if compiler accepts -pthread - yes
> -- Found Threads: TRUE
> -- Checking for module 'thrift'
> -- Package 'thrift', required by 'virtual:world', not found
> -- Could NOT find Thrift: Found unsuitable version "", but required is at
> least "0.11.0" (found THRIFT_LIB-NOTFOUND)
> -- Looking for __SIZEOF_INT128__
> -- Looking for __SIZEOF_INT128__ - found
> -- Found Boost: /usr/include (found suitable version "1.66.0", minimum
> required is "1.58") found components: regex system filesystem
> -- Boost include dir: /usr/include
> -- Boost libraries: Boost::system;Boost::filesystem
> -- Could NOT find Snappy (missing: Snappy_LIB Snappy_INCLUDE_DIR)
> -- Building snappy from source
> -- Checking for modules 'libbrotlicommon;libbrotlienc;libbrotlidec'
> -- Package 'libbrotlicommon', required by 'virtual:world', not found
> -- Package 'libbrotlienc', required by 'virtual:world', not found
> -- Package 'libbrotlidec', required by 'virtual:world', not found
> -- Could NOT find Brotli (missing: BROTLI_COMMON_LIBRARY BROTLI_ENC_LIBRARY
> BROTLI_DEC_LIBRARY BROTLI_INCLUDE_DIR)
> -- Building brotli from source
> -- Building without OpenSSL support. Minimum OpenSSL version 1.0.2 required.
> CM
[jira] [Comment Edited] (ARROW-8484) [C++] TestArrayImport tests cause failures on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448353#comment-17448353
]
Kazuaki Ishizaki edited comment on ARROW-8484 at 11/24/21, 4:08 AM:
[~apitrou] Good catch. It should be closed as resolved.
was (Author: kiszk):
[~apitrou] Good catch. It should be closed.
> [C++] TestArrayImport tests cause failures on a big-endian platform
> ---
>
> Key: ARROW-8484
> URL: https://issues.apache.org/jira/browse/ARROW-8484
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> The current code causes two types of failures of TestArrayImport on a a
> big-endian platform as follows:
> {code}
> 12: [ RUN ] TestSchemaImport.Struct
> 12: /home/ishizaki/Arrow/arrow/cpp/build-support/run-test.sh: line 92: 19528
> Segmentation fault (core dumped) $TEST_EXECUTABLE "$@" 2>&1
> 12: 19529 Done| $ROOT/build-support/asan_symbolize.py
> 12: 19530 Done| ${CXXFILT:-c++filt}
> 12: 19531 Done|
> $ROOT/build-support/stacktrace_addr2line.pl $TEST_EXECUTABLE
> 12: 19532 Done| $pipe_cmd 2>&1
> 12: 19533 Done| tee $LOGFILE
> {code}
> {code}
> 12: [ RUN ] TestArrayImport.PrimitiveWithOffset
> 12: /home/ishizaki/Arrow/arrow/cpp/src/arrow/testing/gtest_util.cc:77: Failure
> 12: Failed
> 12:
> 12: @@ -0, +0 @@
> 12: -1027
> 12: -1541
> 12: -2055
> 12: +772
> 12: +1286
> 12: +1800
> 12: Expected:
> 12: [
> 12: 1027,
> 12: 1541,
> 12: 2055
> 12: ]
> 12: Actual:
> 12: [
> 12: 772,
> 12: 1286,
> 12: 1800
> 12: ]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (ARROW-8484) [C++] TestArrayImport tests cause failures on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki resolved ARROW-8484.
-
Resolution: Fixed
> [C++] TestArrayImport tests cause failures on a big-endian platform
> ---
>
> Key: ARROW-8484
> URL: https://issues.apache.org/jira/browse/ARROW-8484
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> The current code causes two types of failures of TestArrayImport on a a
> big-endian platform as follows:
> {code}
> 12: [ RUN ] TestSchemaImport.Struct
> 12: /home/ishizaki/Arrow/arrow/cpp/build-support/run-test.sh: line 92: 19528
> Segmentation fault (core dumped) $TEST_EXECUTABLE "$@" 2>&1
> 12: 19529 Done| $ROOT/build-support/asan_symbolize.py
> 12: 19530 Done| ${CXXFILT:-c++filt}
> 12: 19531 Done|
> $ROOT/build-support/stacktrace_addr2line.pl $TEST_EXECUTABLE
> 12: 19532 Done| $pipe_cmd 2>&1
> 12: 19533 Done| tee $LOGFILE
> {code}
> {code}
> 12: [ RUN ] TestArrayImport.PrimitiveWithOffset
> 12: /home/ishizaki/Arrow/arrow/cpp/src/arrow/testing/gtest_util.cc:77: Failure
> 12: Failed
> 12:
> 12: @@ -0, +0 @@
> 12: -1027
> 12: -1541
> 12: -2055
> 12: +772
> 12: +1286
> 12: +1800
> 12: Expected:
> 12: [
> 12: 1027,
> 12: 1541,
> 12: 2055
> 12: ]
> 12: Actual:
> 12: [
> 12: 772,
> 12: 1286,
> 12: 1800
> 12: ]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (ARROW-8484) [C++] TestArrayImport tests cause failures on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448353#comment-17448353
]
Kazuaki Ishizaki commented on ARROW-8484:
-
[~apitrou] Good catch. It should be closed.
> [C++] TestArrayImport tests cause failures on a big-endian platform
> ---
>
> Key: ARROW-8484
> URL: https://issues.apache.org/jira/browse/ARROW-8484
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> The current code causes two types of failures of TestArrayImport on a a
> big-endian platform as follows:
> {code}
> 12: [ RUN ] TestSchemaImport.Struct
> 12: /home/ishizaki/Arrow/arrow/cpp/build-support/run-test.sh: line 92: 19528
> Segmentation fault (core dumped) $TEST_EXECUTABLE "$@" 2>&1
> 12: 19529 Done| $ROOT/build-support/asan_symbolize.py
> 12: 19530 Done| ${CXXFILT:-c++filt}
> 12: 19531 Done|
> $ROOT/build-support/stacktrace_addr2line.pl $TEST_EXECUTABLE
> 12: 19532 Done| $pipe_cmd 2>&1
> 12: 19533 Done| tee $LOGFILE
> {code}
> {code}
> 12: [ RUN ] TestArrayImport.PrimitiveWithOffset
> 12: /home/ishizaki/Arrow/arrow/cpp/src/arrow/testing/gtest_util.cc:77: Failure
> 12: Failed
> 12:
> 12: @@ -0, +0 @@
> 12: -1027
> 12: -1541
> 12: -2055
> 12: +772
> 12: +1286
> 12: +1800
> 12: Expected:
> 12: [
> 12: 1027,
> 12: 1541,
> 12: 2055
> 12: ]
> 12: Actual:
> 12: [
> 12: 772,
> 12: 1286,
> 12: 1800
> 12: ]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (ARROW-14686) [Python] Current endian decision for numpy does not work for built-in data type.
[
https://issues.apache.org/jira/browse/ARROW-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-14686:
-
Description:
Although the current NumPy converter code tries to detect endian against a
given object, it does not work for a built-in object. This is because a
build-in object may have {{native |}} byte order.
This issue causes test failure as follows:
{code:java}
FAILED pyarrow/tests/test_array.py::test_array_from_numpy_unicode -
UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3: code
point not in range(0x11)
FAILED pyarrow/tests/test_array.py::test_array_from_strided -
UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3: code
point not in range(0x11)
FAILED pyarrow/tests/test_array.py::test_array_from_numpy_str_utf8 -
UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3: code
point not in range(0x11) {code}
was:Although the current NumPy converter code tries to detect endian against
a given object, it does not work for a built-in object. This is because a
build-in object may have `native |` byte order.
> [Python] Current endian decision for numpy does not work for built-in data
> type.
>
>
> Key: ARROW-14686
> URL: https://issues.apache.org/jira/browse/ARROW-14686
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Affects Versions: 7.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Major
> Fix For: 7.0.0
>
>
> Although the current NumPy converter code tries to detect endian against a
> given object, it does not work for a built-in object. This is because a
> build-in object may have {{native |}} byte order.
> This issue causes test failure as follows:
> {code:java}
> FAILED pyarrow/tests/test_array.py::test_array_from_numpy_unicode -
> UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3:
> code point not in range(0x11)
> FAILED pyarrow/tests/test_array.py::test_array_from_strided -
> UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3:
> code point not in range(0x11)
> FAILED pyarrow/tests/test_array.py::test_array_from_numpy_str_utf8 -
> UnicodeDecodeError: 'utf-32-le' codec can't decode bytes in position 0-3:
> code point not in range(0x11) {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (ARROW-14686) [Python] Current endian decision for numpy does not work for built-in data type.
Kazuaki Ishizaki created ARROW-14686: Summary: [Python] Current endian decision for numpy does not work for built-in data type. Key: ARROW-14686 URL: https://issues.apache.org/jira/browse/ARROW-14686 Project: Apache Arrow Issue Type: Improvement Components: Python Affects Versions: 7.0.0 Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki Fix For: 7.0.0 Although the current NumPy converter code tries to detect endian against a given object, it does not work for a built-in object. This is because a build-in object may have `native |` byte order. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-14685) [Python] Test case hard-codes endian information
Kazuaki Ishizaki created ARROW-14685:
Summary: [Python] Test case hard-codes endian information
Key: ARROW-14685
URL: https://issues.apache.org/jira/browse/ARROW-14685
Project: Apache Arrow
Issue Type: Test
Components: Python
Affects Versions: 7.0.0
Reporter: Kazuaki Ishizaki
Assignee: Kazuaki Ishizaki
Fix For: 7.0.0
The current test case causes a failure on big-endian platform since the endian
information `>` is hard-coded.
{{FAILED pyarrow/tests/test_pandas.py::test_timestamp_as_object_out_of_range -
AssertionError: assert (dtype('O'), dtype('>M8[ns]')) == (dtype('O'),
dtype('
[jira] [Created] (ARROW-14625) [Python][CI] Enable CI for Python on s390x
Kazuaki Ishizaki created ARROW-14625: Summary: [Python][CI] Enable CI for Python on s390x Key: ARROW-14625 URL: https://issues.apache.org/jira/browse/ARROW-14625 Project: Apache Arrow Issue Type: Test Components: Python Affects Versions: 7.0.0 Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki We will support big-endian for Python. As the first step, we will enable Python CI on s390x using TravisCI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (ARROW-13630) [CI][C++] Travis s390x CI job is failing and blocks endianness related code verification
[
https://issues.apache.org/jira/browse/ARROW-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17400569#comment-17400569
]
Kazuaki Ishizaki commented on ARROW-13630:
--
Created a PR to reduce parallelism at
https://github.com/apache/arrow/pull/10952. Let us see what will happen
> [CI][C++] Travis s390x CI job is failing and blocks endianness related code
> verification
>
>
> Key: ARROW-13630
> URL: https://issues.apache.org/jira/browse/ARROW-13630
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Yibo Cai
>Priority: Major
>
> Possibly due to travis memory limit, though only {{-j4}}
> https://app.travis-ci.com/github/apache/arrow/jobs/531277678#L1132
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-13630) [CI][C++] Travis s390x CI job is failing and blocks endianness related code verification
[
https://issues.apache.org/jira/browse/ARROW-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17399865#comment-17399865
]
Kazuaki Ishizaki commented on ARROW-13630:
--
Thank you for creating an issue. Let us try to reduce -j at first.
> [CI][C++] Travis s390x CI job is failing and blocks endianness related code
> verification
>
>
> Key: ARROW-13630
> URL: https://issues.apache.org/jira/browse/ARROW-13630
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Yibo Cai
>Priority: Major
>
> Possibly due to travis memory limit, though only {{-j4}}
> https://app.travis-ci.com/github/apache/arrow/jobs/531277678#L1132
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-13032) [Java] Update gauva version
[ https://issues.apache.org/jira/browse/ARROW-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved ARROW-13032. -- Fix Version/s: 5.0.0 Resolution: Fixed Issue resolved by pull request 10501 [https://github.com/apache/arrow/pull/10501] > [Java] Update gauva version > --- > > Key: ARROW-13032 > URL: https://issues.apache.org/jira/browse/ARROW-13032 > Project: Apache Arrow > Issue Type: Task > Components: Java >Reporter: Projjal Chanda >Assignee: Projjal Chanda >Priority: Major > Labels: pull-request-available > Fix For: 5.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Update guava version due to reported vulnerabilities in the existing version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-13047) [Website] Add kiszk to committer list
[ https://issues.apache.org/jira/browse/ARROW-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved ARROW-13047. -- Resolution: Fixed Issue resolved by pull request 118 [https://github.com/apache/arrow-site/pull/118] > [Website] Add kiszk to committer list > - > > Key: ARROW-13047 > URL: https://issues.apache.org/jira/browse/ARROW-13047 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Kazuaki Ishizaki >Assignee: Kazuaki Ishizaki >Priority: Major > Fix For: 5.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-13026) [C++][CI] s390x job setup fails
[
https://issues.apache.org/jira/browse/ARROW-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-13026:
Assignee: Kazuaki Ishizaki
> [C++][CI] s390x job setup fails
> ---
>
> Key: ARROW-13026
> URL: https://issues.apache.org/jira/browse/ARROW-13026
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Kazuaki Ishizaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 5.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> It looks like the upstream LLVM packages for s390x are broken or unavailable:
> {code}
> Err:8 https://apt.llvm.org/focal llvm-toolchain-focal-12/main s390x libllvm12
> s390x 1:12.0.1~++20210604112550+6279fd114acb-1~exp1~20210604213327.98
> 404 Not Found [IP: 199.232.38.49 443]
> {code}
> https://travis-ci.com/github/apache/arrow/jobs/512489573#L586
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-13026) [C++][CI] s390x job setup fails
[
https://issues.apache.org/jira/browse/ARROW-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki resolved ARROW-13026.
--
Fix Version/s: 5.0.0
Resolution: Fixed
Issue resolved by pull request 10525
[https://github.com/apache/arrow/pull/10525]
> [C++][CI] s390x job setup fails
> ---
>
> Key: ARROW-13026
> URL: https://issues.apache.org/jira/browse/ARROW-13026
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Priority: Major
> Labels: pull-request-available
> Fix For: 5.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It looks like the upstream LLVM packages for s390x are broken or unavailable:
> {code}
> Err:8 https://apt.llvm.org/focal llvm-toolchain-focal-12/main s390x libllvm12
> s390x 1:12.0.1~++20210604112550+6279fd114acb-1~exp1~20210604213327.98
> 404 Not Found [IP: 199.232.38.49 443]
> {code}
> https://travis-ci.com/github/apache/arrow/jobs/512489573#L586
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-13026) [C++][CI] s390x job setup fails
[
https://issues.apache.org/jira/browse/ARROW-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362554#comment-17362554
]
Kazuaki Ishizaki commented on ARROW-13026:
--
[The repository|https://apt.llvm.org/focal/pool/main/l/llvm-toolchain-12/]
provides binary for s390x. However, the version is different (20210604 ... (try
to download) v.s. 20210605... (file)).
> [C++][CI] s390x job setup fails
> ---
>
> Key: ARROW-13026
> URL: https://issues.apache.org/jira/browse/ARROW-13026
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Priority: Major
>
> It looks like the upstream LLVM packages for s390x are broken or unavailable:
> {code}
> Err:8 https://apt.llvm.org/focal llvm-toolchain-focal-12/main s390x libllvm12
> s390x 1:12.0.1~++20210604112550+6279fd114acb-1~exp1~20210604213327.98
> 404 Not Found [IP: 199.232.38.49 443]
> {code}
> https://travis-ci.com/github/apache/arrow/jobs/512489573#L586
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-13026) [C++][CI] s390x job setup fails
[
https://issues.apache.org/jira/browse/ARROW-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362548#comment-17362548
]
Kazuaki Ishizaki commented on ARROW-13026:
--
@kou Thank you for your suggestion. Looks good. [This
commit|https://github.com/apache/arrow/commit/c408048d1152efb973245402510dcd57d2ffee5a]
removed {{LLVM: "10"}}.
> [C++][CI] s390x job setup fails
> ---
>
> Key: ARROW-13026
> URL: https://issues.apache.org/jira/browse/ARROW-13026
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Priority: Major
>
> It looks like the upstream LLVM packages for s390x are broken or unavailable:
> {code}
> Err:8 https://apt.llvm.org/focal llvm-toolchain-focal-12/main s390x libllvm12
> s390x 1:12.0.1~++20210604112550+6279fd114acb-1~exp1~20210604213327.98
> 404 Not Found [IP: 199.232.38.49 443]
> {code}
> https://travis-ci.com/github/apache/arrow/jobs/512489573#L586
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-13047) [Website] Add kiszk to committer list
Kazuaki Ishizaki created ARROW-13047: Summary: [Website] Add kiszk to committer list Key: ARROW-13047 URL: https://issues.apache.org/jira/browse/ARROW-13047 Project: Apache Arrow Issue Type: Improvement Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki Fix For: 5.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-13026) [C++][CI] s390x job setup fails
[
https://issues.apache.org/jira/browse/ARROW-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360261#comment-17360261
]
Kazuaki Ishizaki commented on ARROW-13026:
--
Thanks, I will watch this.
> [C++][CI] s390x job setup fails
> ---
>
> Key: ARROW-13026
> URL: https://issues.apache.org/jira/browse/ARROW-13026
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Priority: Major
>
> It looks like the upstream LLVM packages for s390x are broken or unavailable:
> {code}
> Err:8 https://apt.llvm.org/focal llvm-toolchain-focal-12/main s390x libllvm12
> s390x 1:12.0.1~++20210604112550+6279fd114acb-1~exp1~20210604213327.98
> 404 Not Found [IP: 199.232.38.49 443]
> {code}
> https://travis-ci.com/github/apache/arrow/jobs/512489573#L586
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-12419) [Java] flatc is not used in mvn
Kazuaki Ishizaki created ARROW-12419: Summary: [Java] flatc is not used in mvn Key: ARROW-12419 URL: https://issues.apache.org/jira/browse/ARROW-12419 Project: Apache Arrow Issue Type: Improvement Components: Java Affects Versions: 4.0.0 Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki ARROW-12111 removed the usage of flatc during the build process in mvn. Thus, it is not necessary to explicitly download flatc for s390x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ARROW-11666) [Integration] Add endianness "gold" integration file for decimal256
[ https://issues.apache.org/jira/browse/ARROW-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321902#comment-17321902 ] Kazuaki Ishizaki edited comment on ARROW-11666 at 4/15/21, 4:23 AM: This was already merged at [https://github.com/apache/arrow-testing/pull/58] was (Author: kiszk): I was already merged at [https://github.com/apache/arrow-testing/pull/58] > [Integration] Add endianness "gold" integration file for decimal256 > --- > > Key: ARROW-11666 > URL: https://issues.apache.org/jira/browse/ARROW-11666 > Project: Apache Arrow > Issue Type: Task > Components: Integration >Reporter: Antoine Pitrou >Assignee: Kazuaki Ishizaki >Priority: Major > Labels: pull-request-available > Fix For: 5.0.0 > > > Decimal256 data is missing from the current little-/big-endian "gold" > integration files in the arrow-testing repository: > [https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-stream/integration/1.0.0-littleendian] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11666) [Integration] Add endianness "gold" integration file for decimal256
[ https://issues.apache.org/jira/browse/ARROW-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321902#comment-17321902 ] Kazuaki Ishizaki commented on ARROW-11666: -- I was already merged at [https://github.com/apache/arrow-testing/pull/58] > [Integration] Add endianness "gold" integration file for decimal256 > --- > > Key: ARROW-11666 > URL: https://issues.apache.org/jira/browse/ARROW-11666 > Project: Apache Arrow > Issue Type: Task > Components: Integration >Reporter: Antoine Pitrou >Assignee: Kazuaki Ishizaki >Priority: Major > Labels: pull-request-available > Fix For: 5.0.0 > > > Decimal256 data is missing from the current little-/big-endian "gold" > integration files in the arrow-testing repository: > [https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-stream/integration/1.0.0-littleendian] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-12330) [Developer] Restore values in counters column of Archery benchmark
[
https://issues.apache.org/jira/browse/ARROW-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-12330:
-
Description:
The issue is that ARROW-11189 always suppressed values in {{counters}} column
of Archery benchmark
{code:java}
% archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json
HEAD HEAD~1
...
---
Benchmark Time CPU Iterations UserCounters...
---
SetBitsTo/28.15 ns 8.15 ns 81991087
bytes_per_second=234.044M/s
SetBitsTo/16 7.78 ns 7.78 ns 89928878
bytes_per_second=1.91429G/s
SetBitsTo/1024 13.9 ns 13.9 ns 50372172
bytes_per_second=68.6182G/s
SetBitsTo/131072 3508 ns 3508 ns 199335
bytes_per_second=34.7944G/s
--
Non-regressions: (4)
--
benchmark baselinecontender change % counters
SetBitsTo/161.877 GiB/sec1.914 GiB/sec 1.975 {}
SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {}
SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {}
SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {}
{code}
was:
The issue is that ARROW-11189 always suppressed values in {{counters}} column
of Archery benchmark
{code}
% archery benchmark run --benchmark-filter="SetBitsTo" --output=head2.json HEAD
HEAD~1
...
---
Benchmark Time CPU Iterations UserCounters...
---
SetBitsTo/28.15 ns 8.15 ns 81991087
bytes_per_second=234.044M/s
SetBitsTo/16 7.78 ns 7.78 ns 89928878
bytes_per_second=1.91429G/s
SetBitsTo/1024 13.9 ns 13.9 ns 50372172
bytes_per_second=68.6182G/s
SetBitsTo/131072 3508 ns 3508 ns 199335
bytes_per_second=34.7944G/s
--
Non-regressions: (4)
--
benchmark baselinecontender change % counters
SetBitsTo/161.877 GiB/sec1.914 GiB/sec 1.975 {}
SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {}
SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {}
SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {}
{code}
> [Developer] Restore values in counters column of Archery benchmark
> --
>
> Key: ARROW-12330
> URL: https://issues.apache.org/jira/browse/ARROW-12330
> Project: Apache Arrow
> Issue Type: Bug
> Components: Developer Tools
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
> Fix For: 4.0.0
>
>
> The issue is that ARROW-11189 always suppressed values in {{counters}} column
> of Archery benchmark
> {code:java}
> % archery benchmark diff --benchmark-filter="SetBitsTo" --output=head2.json
> HEAD HEAD~1
> ...
> ---
> Benchmark Time CPU Iterations UserCounters...
> ---
> SetBitsTo/28.15 ns 8.15 ns 81991087
> bytes_per_second=234.044M/s
> SetBitsTo/16 7.78 ns 7.78 ns 89928878
> bytes_per_second=1.91429G/s
> SetBitsTo/1024 13.9 ns 13.9 ns 50372172
> bytes_per_second=68.6182G/s
> SetBitsTo/131072 3508 ns 3508 ns 199335
> bytes_per_second=34.7944G/s
> --
> Non-regressions: (4)
> --
> benchmark baselinecontender change % counters
> SetBitsTo/161.877 GiB/sec1.914 GiB/sec 1.975 {}
> SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {}
> SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {}
>SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-12330) [Developer] Restore values in counters column of Archery benchmark
[
https://issues.apache.org/jira/browse/ARROW-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-12330:
Assignee: Kazuaki Ishizaki
> [Developer] Restore values in counters column of Archery benchmark
> --
>
> Key: ARROW-12330
> URL: https://issues.apache.org/jira/browse/ARROW-12330
> Project: Apache Arrow
> Issue Type: Bug
> Components: Developer Tools
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
> Fix For: 4.0.0
>
>
> The issue is that ARROW-11189 always suppressed values in {{counters}} column
> of Archery benchmark
> {code}
> % archery benchmark run --benchmark-filter="SetBitsTo" --output=head2.json
> HEAD HEAD~1
> ...
> ---
> Benchmark Time CPU Iterations UserCounters...
> ---
> SetBitsTo/28.15 ns 8.15 ns 81991087
> bytes_per_second=234.044M/s
> SetBitsTo/16 7.78 ns 7.78 ns 89928878
> bytes_per_second=1.91429G/s
> SetBitsTo/1024 13.9 ns 13.9 ns 50372172
> bytes_per_second=68.6182G/s
> SetBitsTo/131072 3508 ns 3508 ns 199335
> bytes_per_second=34.7944G/s
> --
> Non-regressions: (4)
> --
> benchmark baselinecontender change % counters
> SetBitsTo/161.877 GiB/sec1.914 GiB/sec 1.975 {}
> SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {}
> SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {}
>SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-12330) [Developer] Restore values in counters column of Archery benchmark
Kazuaki Ishizaki created ARROW-12330:
Summary: [Developer] Restore values in counters column of Archery
benchmark
Key: ARROW-12330
URL: https://issues.apache.org/jira/browse/ARROW-12330
Project: Apache Arrow
Issue Type: Bug
Components: Developer Tools
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
Fix For: 4.0.0
The issue is that ARROW-11189 always suppressed values in {{counters}} column
of Archery benchmark
{code}
% archery benchmark run --benchmark-filter="SetBitsTo" --output=head2.json HEAD
HEAD~1
...
---
Benchmark Time CPU Iterations UserCounters...
---
SetBitsTo/28.15 ns 8.15 ns 81991087
bytes_per_second=234.044M/s
SetBitsTo/16 7.78 ns 7.78 ns 89928878
bytes_per_second=1.91429G/s
SetBitsTo/1024 13.9 ns 13.9 ns 50372172
bytes_per_second=68.6182G/s
SetBitsTo/131072 3508 ns 3508 ns 199335
bytes_per_second=34.7944G/s
--
Non-regressions: (4)
--
benchmark baselinecontender change % counters
SetBitsTo/161.877 GiB/sec1.914 GiB/sec 1.975 {}
SetBitsTo/2 230.566 MiB/sec 234.044 MiB/sec 1.509 {}
SetBitsTo/131072 34.722 GiB/sec 34.794 GiB/sec 0.207 {}
SetBitsTo/1024 68.593 GiB/sec 68.618 GiB/sec 0.037 {}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-11666) [Integration] Add endianness "gold" integration file for decimal256
[ https://issues.apache.org/jira/browse/ARROW-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reassigned ARROW-11666: Assignee: Kazuaki Ishizaki > [Integration] Add endianness "gold" integration file for decimal256 > --- > > Key: ARROW-11666 > URL: https://issues.apache.org/jira/browse/ARROW-11666 > Project: Apache Arrow > Issue Type: Task > Components: Integration >Reporter: Antoine Pitrou >Assignee: Kazuaki Ishizaki >Priority: Major > Fix For: 4.0.0 > > > Decimal256 data is missing from the current little-/big-endian "gold" > integration files in the arrow-testing repository: > [https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-stream/integration/1.0.0-littleendian] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11698) [Website] Write blog post about C++ endianness compatibility
[ https://issues.apache.org/jira/browse/ARROW-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287755#comment-17287755 ] Kazuaki Ishizaki commented on ARROW-11698: -- Sounds good. It would be nice to add big-endian support on Java, too. This is because our community do not announce it while Arrow 3.0 includes the feature. What do you think? > [Website] Write blog post about C++ endianness compatibility > > > Key: ARROW-11698 > URL: https://issues.apache.org/jira/browse/ARROW-11698 > Project: Apache Arrow > Issue Type: Task > Components: Website >Reporter: Antoine Pitrou >Priority: Minor > > It might be nice to announce the cross-endian compatibility effort on the > website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11666) [Integration] Add endianness "gold" integration file for decimal256
[ https://issues.apache.org/jira/browse/ARROW-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286331#comment-17286331 ] Kazuaki Ishizaki commented on ARROW-11666: -- Sure, I will do this. > [Integration] Add endianness "gold" integration file for decimal256 > --- > > Key: ARROW-11666 > URL: https://issues.apache.org/jira/browse/ARROW-11666 > Project: Apache Arrow > Issue Type: Task > Components: Integration >Reporter: Antoine Pitrou >Priority: Major > Fix For: 4.0.0 > > > Decimal256 data is missing from the current little-/big-endian "gold" > integration files in the arrow-testing repository: > [https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-stream/integration/1.0.0-littleendian] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11501) [C++] endianness check does not work on Solaris
[
https://issues.apache.org/jira/browse/ARROW-11501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17285873#comment-17285873
]
Kazuaki Ishizaki commented on ARROW-11501:
--
[~npr] Is there any update on this?
> [C++] endianness check does not work on Solaris
> ---
>
> Key: ARROW-11501
> URL: https://issues.apache.org/jira/browse/ARROW-11501
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: C++
>Reporter: Neal Richardson
>Priority: Major
>
> {code}
> In file included from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/type_traits.h:26:0,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/scalar.h:36,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/datum.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/expression.h:32,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.cc:18:
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/util/bit_util.h:26:42:
>
> fatal error: endian.h: No such file or directory
> {code}
> Googling the error message shows some known issues and workarounds for this
> on Solaris, e.g.:
> * https://github.com/Sereal/Sereal/issues/139
> * https://gitlab.torproject.org/legacy/trac/-/issues/11426
> cc [~kiszk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11501) [C++] endianness check does not work on Solaris
[
https://issues.apache.org/jira/browse/ARROW-11501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280077#comment-17280077
]
Kazuaki Ishizaki commented on ARROW-11501:
--
I see. Could you please try this code by applying a patch by hand? Or, should I
create a PR as WIP?
> [C++] endianness check does not work on Solaris
> ---
>
> Key: ARROW-11501
> URL: https://issues.apache.org/jira/browse/ARROW-11501
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: C++
>Reporter: Neal Richardson
>Priority: Major
>
> {code}
> In file included from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/type_traits.h:26:0,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/scalar.h:36,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/datum.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/expression.h:32,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.cc:18:
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/util/bit_util.h:26:42:
>
> fatal error: endian.h: No such file or directory
> {code}
> Googling the error message shows some known issues and workarounds for this
> on Solaris, e.g.:
> * https://github.com/Sereal/Sereal/issues/139
> * https://gitlab.torproject.org/legacy/trac/-/issues/11426
> cc [~kiszk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (ARROW-11501) [C++] endianness check does not work on Solaris
[
https://issues.apache.org/jira/browse/ARROW-11501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280070#comment-17280070
]
Kazuaki Ishizaki edited comment on ARROW-11501 at 2/6/21, 4:04 AM:
---
Thank you for raising the issue. I can address this, too.
was (Author: kiszk):
Thank you for raising the issue. I will address this.
> [C++] endianness check does not work on Solaris
> ---
>
> Key: ARROW-11501
> URL: https://issues.apache.org/jira/browse/ARROW-11501
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: C++
>Reporter: Neal Richardson
>Priority: Major
>
> {code}
> In file included from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/type_traits.h:26:0,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/scalar.h:36,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/datum.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/expression.h:32,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.cc:18:
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/util/bit_util.h:26:42:
>
> fatal error: endian.h: No such file or directory
> {code}
> Googling the error message shows some known issues and workarounds for this
> on Solaris, e.g.:
> * https://github.com/Sereal/Sereal/issues/139
> * https://gitlab.torproject.org/legacy/trac/-/issues/11426
> cc [~kiszk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11501) [C++] endianness check does not work on Solaris
[
https://issues.apache.org/jira/browse/ARROW-11501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280072#comment-17280072
]
Kazuaki Ishizaki commented on ARROW-11501:
--
Is it the simplest solution to add the following code around
[here|https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/bit_util.h#L20-L27]?
{code}
...
#elif defined(sun) || defined(__sun)
#include // IWYU pragma: keep
#else
#include // IWYU pragma: keep
#endif
{code}
> [C++] endianness check does not work on Solaris
> ---
>
> Key: ARROW-11501
> URL: https://issues.apache.org/jira/browse/ARROW-11501
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: C++
>Reporter: Neal Richardson
>Priority: Major
>
> {code}
> In file included from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/type_traits.h:26:0,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/scalar.h:36,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/datum.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/expression.h:32,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.cc:18:
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/util/bit_util.h:26:42:
>
> fatal error: endian.h: No such file or directory
> {code}
> Googling the error message shows some known issues and workarounds for this
> on Solaris, e.g.:
> * https://github.com/Sereal/Sereal/issues/139
> * https://gitlab.torproject.org/legacy/trac/-/issues/11426
> cc [~kiszk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11501) [C++] endianness check does not work on Solaris
[
https://issues.apache.org/jira/browse/ARROW-11501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280070#comment-17280070
]
Kazuaki Ishizaki commented on ARROW-11501:
--
Thank you for raising the issue. I will address this.
> [C++] endianness check does not work on Solaris
> ---
>
> Key: ARROW-11501
> URL: https://issues.apache.org/jira/browse/ARROW-11501
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: C++
>Reporter: Neal Richardson
>Priority: Major
>
> {code}
> In file included from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/type_traits.h:26:0,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/scalar.h:36,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/datum.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/expression.h:32,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.h:28,
> from
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/dataset/dataset.cc:18:
> /export/home/XXVfZhv/Rtemp/RtmpoK9Cps/file3f4a341e5d8f/cpp/src/arrow/util/bit_util.h:26:42:
>
> fatal error: endian.h: No such file or directory
> {code}
> Googling the error message shows some known issues and workarounds for this
> on Solaris, e.g.:
> * https://github.com/Sereal/Sereal/issues/139
> * https://gitlab.torproject.org/legacy/trac/-/issues/11426
> cc [~kiszk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10860) [Java] Avoid integer overflow for Json file reader
Kazuaki Ishizaki created ARROW-10860:
Summary: [Java] Avoid integer overflow for Json file reader
Key: ARROW-10860
URL: https://issues.apache.org/jira/browse/ARROW-10860
Project: Apache Arrow
Issue Type: Bug
Components: Java
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
This issue is similar to https://issues.apache.org/jira/browse/ARROW-10662.
For the current implementation in the templates, {{int * int}} multiplication
is used to calculate buffer offset. The result may be larger than
Integer.MAX_VALUE, which will lead to integer overflow and unexpected behaviors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-10699) [C++] BitmapUInt64Reader doesn't work on big-endian
[ https://issues.apache.org/jira/browse/ARROW-10699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237770#comment-17237770 ] Kazuaki Ishizaki commented on ARROW-10699: -- [~apitrou] Thank you for fixing this. I missed this. > [C++] BitmapUInt64Reader doesn't work on big-endian > --- > > Key: ARROW-10699 > URL: https://issues.apache.org/jira/browse/ARROW-10699 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Continuous Integration >Reporter: Antoine Pitrou >Assignee: Antoine Pitrou >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > I didn't notice this when merging ARROW-10655 (the s390x CI is allowed to > fail). > https://travis-ci.com/github/apache/arrow/jobs/445803711#L3534 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-10621) [Java] flight-cpre test causes a failure on s390x
[
https://issues.apache.org/jira/browse/ARROW-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-10621:
Assignee: Kazuaki Ishizaki
> [Java] flight-cpre test causes a failure on s390x
> -
>
> Key: ARROW-10621
> URL: https://issues.apache.org/jira/browse/ARROW-10621
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> Java flight-core test on s390x causes a failure as follows:
>
> {code:java}
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO]
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (src) @ flight-core ---
> [INFO] Compiling 1 proto file(s) to
> /arrow/java/flight/flight-core/target/generated-sources/protobuf
> [ERROR] PROTOC FAILED:
> /arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
> error while loading shared libraries: libprotoc.so.18: cannot open shared
> object file: No such file or directory
> [ERROR] /arrow/java/flight/flight-core/../../../format/Flight.proto [0:0]:
> /arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
> error while loading shared libraries: libprotoc.so.18: cannot open shared
> object file: No such file or directory
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10621) [Java] flight-cpre test causes a failure on s390x
Kazuaki Ishizaki created ARROW-10621:
Summary: [Java] flight-cpre test causes a failure on s390x
Key: ARROW-10621
URL: https://issues.apache.org/jira/browse/ARROW-10621
Project: Apache Arrow
Issue Type: Bug
Components: Java
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
Java flight-core test on s390x causes a failure as follows:
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO]
[INFO] --- protobuf-maven-plugin:0.5.0:compile (src) @ flight-core ---
[INFO] Compiling 1 proto file(s) to
/arrow/java/flight/flight-core/target/generated-sources/protobuf
[ERROR] PROTOC FAILED:
/arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
error while loading shared libraries: libprotoc.so.18: cannot open shared
object file: No such file or directory
[ERROR] /arrow/java/flight/flight-core/../../../format/Flight.proto [0:0]:
/arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
error while loading shared libraries: libprotoc.so.18: cannot open shared
object file: No such file or directory
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-10610) [C++] arrow-utility-test and arrow-csv-test causes failures on a big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-10610:
Assignee: Kazuaki Ishizaki
> [C++] arrow-utility-test and arrow-csv-test causes failures on a big-endian
> platform
>
>
> Key: ARROW-10610
> URL: https://issues.apache.org/jira/browse/ARROW-10610
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> After [https://github.com/apache/arrow/pull/8494] has been merged, the
> following tests, which use {{1eN}} format, cause failures.
> {code}
> [ RUN ] FloatingPointConversion.Basics
> /arrow/cpp/src/arrow/testing/gtest_util.cc:128: Failure
> Failed
> @@ -1, +1 @@
> --1e+30
> +0
> [ FAILED ] FloatingPointConversion.Basics (3 ms)
> ...
> [ RUN ] StringConversion.ToFloat
> /arrow/cpp/src/arrow/util/value_parsing_test.cc:35: Failure
> Expected equality of these values:
> out
> Which is: 0
> expected
> Which is: -1e+20
> Conversion failed for '-1e20'
> [ FAILED ] StringConversion.ToFloat (0 ms)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10610) [C++] arrow-utility-test and arrow-csv-test causes failures on a big-endian platform
Kazuaki Ishizaki created ARROW-10610:
Summary: [C++] arrow-utility-test and arrow-csv-test causes
failures on a big-endian platform
Key: ARROW-10610
URL: https://issues.apache.org/jira/browse/ARROW-10610
Project: Apache Arrow
Issue Type: Bug
Components: C++
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
After [https://github.com/apache/arrow/pull/8494] has been merged, the
following tests, which use {{1eN}} format, cause failures.
{code}
[ RUN ] FloatingPointConversion.Basics
/arrow/cpp/src/arrow/testing/gtest_util.cc:128: Failure
Failed
@@ -1, +1 @@
--1e+30
+0
[ FAILED ] FloatingPointConversion.Basics (3 ms)
...
[ RUN ] StringConversion.ToFloat
/arrow/cpp/src/arrow/util/value_parsing_test.cc:35: Failure
Expected equality of these values:
out
Which is: 0
expected
Which is: -1e+20
Conversion failed for '-1e20'
[ FAILED ] StringConversion.ToFloat (0 ms)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-10384) [c++] Fix typos and spelling
[
https://issues.apache.org/jira/browse/ARROW-10384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-10384:
Assignee: Kazuaki Ishizaki
> [c++] Fix typos and spelling
>
>
> Key: ARROW-10384
> URL: https://issues.apache.org/jira/browse/ARROW-10384
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> Fix typo under {{cpp}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10384) [c++] Fix typos and spelling
Kazuaki Ishizaki created ARROW-10384:
Summary: [c++] Fix typos and spelling
Key: ARROW-10384
URL: https://issues.apache.org/jira/browse/ARROW-10384
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
Fix typo under {{cpp}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-10382) [rust] Fix typos and spelling
[
https://issues.apache.org/jira/browse/ARROW-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-10382:
Assignee: Kazuaki Ishizaki
> [rust] Fix typos and spelling
> -
>
> Key: ARROW-10382
> URL: https://issues.apache.org/jira/browse/ARROW-10382
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> Fix typo under {{rust}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-10383) [docs] Fix typos and spelling
[
https://issues.apache.org/jira/browse/ARROW-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-10383:
Assignee: Kazuaki Ishizaki
> [docs] Fix typos and spelling
> -
>
> Key: ARROW-10383
> URL: https://issues.apache.org/jira/browse/ARROW-10383
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Documentation
>Affects Versions: 3.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Minor
>
> Fix typo under {{docs}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10383) [docs] Fix typos and spelling
Kazuaki Ishizaki created ARROW-10383:
Summary: [docs] Fix typos and spelling
Key: ARROW-10383
URL: https://issues.apache.org/jira/browse/ARROW-10383
Project: Apache Arrow
Issue Type: Improvement
Components: Documentation
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
Fix typo under {{docs}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10382) [rust] Fix typos and spelling
Kazuaki Ishizaki created ARROW-10382:
Summary: [rust] Fix typos and spelling
Key: ARROW-10382
URL: https://issues.apache.org/jira/browse/ARROW-10382
Project: Apache Arrow
Issue Type: Improvement
Components: Rust
Affects Versions: 3.0.0
Reporter: Kazuaki Ishizaki
Fix typo under {{rust}} directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10200) [Java][CI] Fix failure of Java CI on s390x
Kazuaki Ishizaki created ARROW-10200:
Summary: [Java][CI] Fix failure of Java CI on s390x
Key: ARROW-10200
URL: https://issues.apache.org/jira/browse/ARROW-10200
Project: Apache Arrow
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
ARROW-9701 causes the following failure due to missing a library
{{libprotobuf.so.18}}.
{code:java}
...
[ERROR] PROTOC FAILED:
/arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
error while loading shared libraries: libprotobuf.so.18: cannot open shared
object file: No such file or directory
[ERROR] /arrow/java/flight/flight-core/../../../format/Flight.proto [0:0]:
/arrow/java/flight/flight-core/target/protoc-plugins/protoc-3.7.1-linux-s390_64.exe:
error while loading shared libraries: libprotobuf.so.18: cannot open shared
object file: No such file or directory
...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-10031) Support Java benchmark in Ursabot
Kazuaki Ishizaki created ARROW-10031: Summary: Support Java benchmark in Ursabot Key: ARROW-10031 URL: https://issues.apache.org/jira/browse/ARROW-10031 Project: Apache Arrow Issue Type: New Feature Components: CI, Java Affects Versions: 2.0.0 Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki Based on [the suggestion|https://mail-archives.apache.org/mod_mbox/arrow-dev/202008.mbox/%3ccabnn7+q35j7qwshjbx8omdewkt+f1p_m7r1_f6szs4dqc+l...@mail.gmail.com%3e], Ursabot will support Java benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-9862) Throw an exception in UnsafeDirectLittleEndian on Big-Endian platform
[
https://issues.apache.org/jira/browse/ARROW-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-9862:
---
Assignee: Kazuaki Ishizaki
> Throw an exception in UnsafeDirectLittleEndian on Big-Endian platform
> -
>
> Key: ARROW-9862
> URL: https://issues.apache.org/jira/browse/ARROW-9862
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The current code throws an intended exception on a big-endian platform while
> this class supports native endianness for the primitive data types (up to
> 64-bit).
> {code:java}
> throw new IllegalStateException("Arrow only runs on LittleEndian systems.");
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-9861) [Java] Failed Arrow Vector on big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki reassigned ARROW-9861:
---
Assignee: Kazuaki Ishizaki
> [Java] Failed Arrow Vector on big-endian platform
> -
>
> Key: ARROW-9861
> URL: https://issues.apache.org/jira/browse/ARROW-9861
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Kazuaki Ishizaki
>Assignee: Kazuaki Ishizaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The following test failure occurs on a big-endian platform
> {code:java}
> mvn -B -Drat.skip=true
> -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
> -Dflatc.download.skip=true -rf :arrow-vector test
> ...
> [INFO] Running org.apache.arrow.vector.TestDecimalVector
> [ERROR] Tests run: 9, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.008
> s <<< FAILURE! - in org.apache.arrow.vector.TestDecimalVector
> [ERROR] setUsingArrowBufOfLEInts Time elapsed: 0.001 s <<< FAILURE!
> java.lang.AssertionError: expected:<705.32> but was:<-20791293.44>
> at
> org.apache.arrow.vector.TestDecimalVector.setUsingArrowBufOfLEInts(TestDecimalVector.java:295)
> [ERROR] setUsingArrowLongLEBytes Time elapsed: 0.001 s <<< FAILURE!
> java.lang.AssertionError: expected:<9223372036854775807> but was:<-129>
> at
> org.apache.arrow.vector.TestDecimalVector.setUsingArrowLongLEBytes(TestDecimalVector.java:322)
> [ERROR] testLongReadWrite Time elapsed: 0.001 s <<< FAILURE!
> java.lang.AssertionError: expected:<-2> but was:<-72057594037927937>
> at
> org.apache.arrow.vector.TestDecimalVector.testLongReadWrite(TestDecimalVector.java:176)
> ...
> [ERROR] Failures:
> [ERROR] TestDecimalVector.setUsingArrowBufOfLEInts:295 expected:<705.32>
> but was:<-20791293.44>
> [ERROR] TestDecimalVector.setUsingArrowLongLEBytes:322
> expected:<9223372036854775807> but was:<-129>
> [ERROR] TestDecimalVector.testLongReadWrite:176 expected:<-2> but
> was:<-72057594037927937>
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9862) Throw an exception in UnsafeDirectLittleEndian on Big-Endian platform
Kazuaki Ishizaki created ARROW-9862:
---
Summary: Throw an exception in UnsafeDirectLittleEndian on
Big-Endian platform
Key: ARROW-9862
URL: https://issues.apache.org/jira/browse/ARROW-9862
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
The current code throws an intended exception on a big-endian platform while
this class supports native endianness for the primitive data types (up to
64-bit).
{code:java}
throw new IllegalStateException("Arrow only runs on LittleEndian systems.");
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9861) [Java] Failed Arrow Vector on big-endian platform
Kazuaki Ishizaki created ARROW-9861:
---
Summary: [Java] Failed Arrow Vector on big-endian platform
Key: ARROW-9861
URL: https://issues.apache.org/jira/browse/ARROW-9861
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
The following test failure occurs on a big-endian platform
{code:java}
mvn -B -Drat.skip=true
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
-Dflatc.download.skip=true -rf :arrow-vector test
...
[INFO] Running org.apache.arrow.vector.TestDecimalVector
[ERROR] Tests run: 9, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.008 s
<<< FAILURE! - in org.apache.arrow.vector.TestDecimalVector
[ERROR] setUsingArrowBufOfLEInts Time elapsed: 0.001 s <<< FAILURE!
java.lang.AssertionError: expected:<705.32> but was:<-20791293.44>
at
org.apache.arrow.vector.TestDecimalVector.setUsingArrowBufOfLEInts(TestDecimalVector.java:295)
[ERROR] setUsingArrowLongLEBytes Time elapsed: 0.001 s <<< FAILURE!
java.lang.AssertionError: expected:<9223372036854775807> but was:<-129>
at
org.apache.arrow.vector.TestDecimalVector.setUsingArrowLongLEBytes(TestDecimalVector.java:322)
[ERROR] testLongReadWrite Time elapsed: 0.001 s <<< FAILURE!
java.lang.AssertionError: expected:<-2> but was:<-72057594037927937>
at
org.apache.arrow.vector.TestDecimalVector.testLongReadWrite(TestDecimalVector.java:176)
...
[ERROR] Failures:
[ERROR] TestDecimalVector.setUsingArrowBufOfLEInts:295 expected:<705.32> but
was:<-20791293.44>
[ERROR] TestDecimalVector.setUsingArrowLongLEBytes:322
expected:<9223372036854775807> but was:<-129>
[ERROR] TestDecimalVector.testLongReadWrite:176 expected:<-2> but
was:<-72057594037927937>
...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9709) [Java] Test cases in arrow-vector assume little-endian platform
Kazuaki Ishizaki created ARROW-9709:
---
Summary: [Java] Test cases in arrow-vector assume little-endian
platform
Key: ARROW-9709
URL: https://issues.apache.org/jira/browse/ARROW-9709
Project: Apache Arrow
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
Assignee: Kazuaki Ishizaki
{{MessageSerializerTest.testWriteMessageBufferAligned}},
{{TestArrowReaderWriter.testChannelReadFully}} and
{{TestArrowReaderWriter.testChannelReadFullyEos}} assume only a little-endian
platform.
Two tests in {{TestArrowReaderWriter}} fails on a big-endian platform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9706) [Java] Tests in TestLargeListVector fails on big endian platform
Kazuaki Ishizaki created ARROW-9706:
---
Summary: [Java] Tests in TestLargeListVector fails on big endian
platform
Key: ARROW-9706
URL: https://issues.apache.org/jira/browse/ARROW-9706
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
Assignee: Kazuaki Ishizaki
Multiple test cases (e.g. {{testSetLastSetUsage}}) in {{TestLargeListVector}}
fail on a big-endian platform.
This is because these test cases read offset as integer while the width of the
offset is 8-byte. This means that only the first 4-byte are read. It works only
for a little-endian platform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9704) [Java] TestEndianness.testLittleEndian fails on big endian platform
Kazuaki Ishizaki created ARROW-9704:
---
Summary: [Java] TestEndianness.testLittleEndian fails on big
endian platform
Key: ARROW-9704
URL: https://issues.apache.org/jira/browse/ARROW-9704
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Reporter: Kazuaki Ishizaki
Assignee: Kazuaki Ishizaki
Fix For: 2.0.0
{{TestEndianness.testLittleEndian}} assumes that the data layout of int is
little-endian. Thus, this test fails on a big-endian platform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9701) [Java][CI] Add a test job on s390x
Kazuaki Ishizaki created ARROW-9701: --- Summary: [Java][CI] Add a test job on s390x Key: ARROW-9701 URL: https://issues.apache.org/jira/browse/ARROW-9701 Project: Apache Arrow Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Kazuaki Ishizaki Assignee: Kazuaki Ishizaki -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9681) [Java] Failed Arrow Memory - Core on big-endian platform
[
https://issues.apache.org/jira/browse/ARROW-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kazuaki Ishizaki updated ARROW-9681:
Description:
The following two errors occur on a big-endian platform.
{code:java}
% mvn -B -Drat.skip=true
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
-Dflatc.download.skip=true -rf :arrow-memory-core test
...
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.arrow.memory.TestBoundaryChecking
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s
- in org.apache.arrow.memory.TestBoundaryChecking
[INFO] Running org.apache.arrow.memory.TestAllocationManager
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s -
in org.apache.arrow.memory.TestAllocationManager
[INFO] Running org.apache.arrow.memory.TestArrowBuf
13:50:53.066 [main] INFO org.apache.arrow.memory.BaseAllocator - Debug mode
enabled.
13:50:53.068 [main] INFO org.apache.arrow.memory.DefaultAllocationManagerOption
- allocation manager type not specified, using netty as the default type
13:50:53.069 [main] INFO org.apache.arrow.memory.CheckAllocator - Using
DefaultAllocationManager at
memory/memory-core/target/test-classes/org/apache/arrow/memory/DefaultAllocationManagerFactory.class
13:50:53.071 [main] DEBUG org.apache.arrow.memory.util.MemoryUtil - Constructor
for direct buffer found and made accessible
13:50:53.071 [main] DEBUG org.apache.arrow.memory.util.MemoryUtil - direct
buffer constructor: available
13:50:53.073 [main] DEBUG
org.apache.arrow.memory.rounding.DefaultRoundingPolicy -
-Dorg.apache.memory.allocator.pageSize: 8192
13:50:53.073 [main] DEBUG
org.apache.arrow.memory.rounding.DefaultRoundingPolicy -
-Dorg.apache.memory.allocator.maxOrder: 11
13:50:53.075 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.076 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.077 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.082 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.083 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.017 s
<<< FAILURE! - in org.apache.arrow.memory.TestArrowBuf
[ERROR] testSetBytesBigEndian Time elapsed: 0.004 s <<< FAILURE!
org.junit.internal.ArrayComparisonFailure: arrays first differed at element
[0]; expected:<0> but was:<7>
at
org.apache.arrow.memory.TestArrowBuf.testSetBytesBigEndian(TestArrowBuf.java:145)
Caused by: java.lang.AssertionError: expected:<0> but was:<7>
at
org.apache.arrow.memory.TestArrowBuf.testSetBytesBigEndian(TestArrowBuf.java:145)
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in
org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Running org.apache.arrow.memory.TestAccountant
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 s
- in org.apache.arrow.memory.TestAccountant
[INFO] Running org.apache.arrow.memory.util.TestArrowBufPointer
13:50:53.133 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.134 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.135 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.136 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.136 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.138 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s
- in org.apache.arrow.memory.util.TestArrowBufPointer
[INFO] Running org.apache.arrow.memory.util.hash.TestArrowBufHasher
13:50:53.139 [main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].
13:50:53.141 [main] DEBUG org.apache.arrow.memory.ArrowBuf - event log for:
ArrowBuf[21]
5531898924530725 create()
at org.apache.arrow.memory.util.StackTrace.(StackTrace.java:34)
at
org.apache.arrow.memory.util.HistoricalLog$Event.(HistoricalLog.java:175)
at
org.apache.arrow.memory.util.HistoricalLog.recordEvent(HistoricalLog.java:83)
at org.apache.arrow.memory.ArrowBuf.(ArrowBuf.java:96)
at org.apache.arrow.memory.BufferLedger.newArrowBuf(BufferLedger.java:271)
at
org.apache.arrow.memory.BaseAllocator.bufferWithoutReservation(BaseAllocator.java:299)
at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:275)
at org.apache.arrow.memory.RootAllocator.buffer(RootAllocat
[jira] [Created] (ARROW-9681) [Java] Failed Arrow Memory - Core on big-endian platform
Kazuaki Ishizaki created ARROW-9681:
---
Summary: [Java] Failed Arrow Memory - Core on big-endian platform
Key: ARROW-9681
URL: https://issues.apache.org/jira/browse/ARROW-9681
Project: Apache Arrow
Issue Type: Improvement
Components: Java
Affects Versions: 2.0.0
Reporter: Kazuaki Ishizaki
The following error occurs on a big-endian platform.
{code:java}
% mvn -B -Drat.skip=true
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
-Dflatc.download.skip=true -rf :arrow-memory-core test% mvn -B -Drat.skip=true
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
-Dflatc.download.skip=true -rf :arrow-memory-core test...[INFO]
---[INFO] T E S T S[INFO]
---[INFO] Running
org.apache.arrow.memory.TestBoundaryChecking[INFO] Tests run: 4, Failures: 0,
Errors: 0, Skipped: 0, Time elapsed: 0.003 s - in
org.apache.arrow.memory.TestBoundaryChecking[INFO] Running
org.apache.arrow.memory.TestAllocationManager[INFO] Tests run: 1, Failures: 0,
Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in
org.apache.arrow.memory.TestAllocationManager[INFO] Running
org.apache.arrow.memory.TestArrowBuf13:50:53.066 [main] INFO
org.apache.arrow.memory.BaseAllocator - Debug mode enabled.13:50:53.068 [main]
INFO org.apache.arrow.memory.DefaultAllocationManagerOption - allocation
manager type not specified, using netty as the default type13:50:53.069 [main]
INFO org.apache.arrow.memory.CheckAllocator - Using DefaultAllocationManager at
memory/memory-core/target/test-classes/org/apache/arrow/memory/DefaultAllocationManagerFactory.class13:50:53.071
[main] DEBUG org.apache.arrow.memory.util.MemoryUtil - Constructor for direct
buffer found and made accessible13:50:53.071 [main] DEBUG
org.apache.arrow.memory.util.MemoryUtil - direct buffer constructor:
available13:50:53.073 [main] DEBUG
org.apache.arrow.memory.rounding.DefaultRoundingPolicy -
-Dorg.apache.memory.allocator.pageSize: 819213:50:53.073 [main] DEBUG
org.apache.arrow.memory.rounding.DefaultRoundingPolicy -
-Dorg.apache.memory.allocator.maxOrder: 1113:50:53.075 [main] DEBUG
org.apache.arrow.memory.BaseAllocator - closed allocator[ROOT].13:50:53.076
[main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].13:50:53.077 [main] DEBUG org.apache.arrow.memory.BaseAllocator
- closed allocator[ROOT].13:50:53.082 [main] DEBUG
org.apache.arrow.memory.BaseAllocator - closed allocator[ROOT].13:50:53.083
[main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 0.017 s <<< FAILURE! - in org.apache.arrow.memory.TestArrowBuf[ERROR]
testSetBytesBigEndian Time elapsed: 0.004 s <<<
FAILURE!org.junit.internal.ArrayComparisonFailure: arrays first differed at
element [0]; expected:<0> but was:<7> at
org.apache.arrow.memory.TestArrowBuf.testSetBytesBigEndian(TestArrowBuf.java:145)Caused
by: java.lang.AssertionError: expected:<0> but was:<7> at
org.apache.arrow.memory.TestArrowBuf.testSetBytesBigEndian(TestArrowBuf.java:145)
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap[INFO] Tests
run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in
org.apache.arrow.memory.TestLowCostIdentityHashMap[INFO] Running
org.apache.arrow.memory.TestAccountant[INFO] Tests run: 3, Failures: 0, Errors:
0, Skipped: 0, Time elapsed: 0.045 s - in
org.apache.arrow.memory.TestAccountant[INFO] Running
org.apache.arrow.memory.util.TestArrowBufPointer13:50:53.133 [main] DEBUG
org.apache.arrow.memory.BaseAllocator - closed allocator[ROOT].13:50:53.134
[main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].13:50:53.135 [main] DEBUG org.apache.arrow.memory.BaseAllocator
- closed allocator[ROOT].13:50:53.136 [main] DEBUG
org.apache.arrow.memory.BaseAllocator - closed allocator[ROOT].13:50:53.136
[main] DEBUG org.apache.arrow.memory.BaseAllocator - closed
allocator[ROOT].13:50:53.138 [main] DEBUG org.apache.arrow.memory.BaseAllocator
- closed allocator[ROOT].[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped:
0, Time elapsed: 0.005 s - in
org.apache.arrow.memory.util.TestArrowBufPointer[INFO] Running
org.apache.arrow.memory.util.hash.TestArrowBufHasher13:50:53.139 [main] DEBUG
org.apache.arrow.memory.BaseAllocator - closed allocator[ROOT].13:50:53.141
[main] DEBUG org.apache.arrow.memory.ArrowBuf - event log for: ArrowBuf[21]
5531898924530725 create() at
org.apache.arrow.memory.util.StackTrace.(StackTrace.java:34) at
org.apache.arrow.memory.util.HistoricalLog$Event.(HistoricalLog.java:175)
at
org.apache.arrow.memory.util.HistoricalLog.recordEvent(HistoricalLog.java:83)
at org.apache.arrow.memory.
[jira] [Commented] (ARROW-9512) [CI][Gandiva] Fix Gandiva nightly build failures
[ https://issues.apache.org/jira/browse/ARROW-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160592#comment-17160592 ] Kazuaki Ishizaki commented on ARROW-9512: - Thank you > [CI][Gandiva] Fix Gandiva nightly build failures > > > Key: ARROW-9512 > URL: https://issues.apache.org/jira/browse/ARROW-9512 > Project: Apache Arrow > Issue Type: Bug >Reporter: Projjal Chanda >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Gandiva nightlies have been failing for a past couple of days -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9417) [C++][IPC] size in message written in native endian
Kazuaki Ishizaki created ARROW-9417:
---
Summary: [C++][IPC] size in message written in native endian
Key: ARROW-9417
URL: https://issues.apache.org/jira/browse/ARROW-9417
Project: Apache Arrow
Issue Type: Test
Components: C++
Reporter: Kazuaki Ishizaki
According to [this
specification|https://github.com/apache/arrow/blob/master/docs/source/format/Columnar.rst],
{{metadata length}} and {{footer size}} should be written in little endian
However, the current implementation writes them in native endian.
On little-endian
{code}
: 41 52 52 4f 57 31 00 00 ff ff ff ff 88 07 00 00 10 00 00 00 00 00 0a
00 0e 00 06 00 05 00 08 00 ARROW1..
...
5700: 62 6f 6f 6c 5f 6e 75 6c 6c 61 62 6c 65 00 00 00 d0 07 00 00 41 52 52
4f 57 31bool_nullable...ARROW1
{code}
On big-endian
{code}
: 41 52 52 4f 57 31 00 00 ff ff ff ff 00 00 07 90 10 00 00 00 00 00 0a
00 0e 00 06 00 05 00 08 00 ARROW1..
~~~
...
5700: 04 00 00 00 0d 00 00 00 62 6f 6f 6c 5f 6e 75 6c 6c 61 62 6c 65 00 00
00 00 00 07 d0 41 52 52 4f bool_nullable...ARRO
~~~
5720: 57 31
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-9238) [C++][CI] A few test coverages of round-robin in ipc and flight
[ https://issues.apache.org/jira/browse/ARROW-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reassigned ARROW-9238: --- Assignee: Kazuaki Ishizaki > [C++][CI] A few test coverages of round-robin in ipc and flight > --- > > Key: ARROW-9238 > URL: https://issues.apache.org/jira/browse/ARROW-9238 > Project: Apache Arrow > Issue Type: Test > Components: C++, CI, FlightRPC >Reporter: Kazuaki Ishizaki >Assignee: Kazuaki Ishizaki >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The current round-robin tests under ipc and flight cover only a few types > (e.g. int32 and dictionary). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9238) [C++][CI] A few test coverages of round-robin in ipc and flight
Kazuaki Ishizaki created ARROW-9238: --- Summary: [C++][CI] A few test coverages of round-robin in ipc and flight Key: ARROW-9238 URL: https://issues.apache.org/jira/browse/ARROW-9238 Project: Apache Arrow Issue Type: Test Components: C++, CI, FlightRPC Reporter: Kazuaki Ishizaki The current round-robin tests under ipc and flight cover only a few types (e.g. int32 and dictionary). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8797) [C++] Support Flight RPC among diffent endian platforms
[ https://issues.apache.org/jira/browse/ARROW-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141703#comment-17141703 ] Kazuaki Ishizaki commented on ARROW-8797: - https://github.com/apache/arrow/pull/7507 is trying to create test cases. > [C++] Support Flight RPC among diffent endian platforms > --- > > Key: ARROW-8797 > URL: https://issues.apache.org/jira/browse/ARROW-8797 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, FlightRPC >Reporter: Kazuaki Ishizaki >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > This PR is derived from discussion in ARROW-8790. The current implementation > of the Flight RPC exchanges messages using a native endlian format. It does > not allow us to exchange data though Flight RPC among different endian > machines (e.g. x86_64 <-> s390x). > There is a field to show an endianness at[ > https://github.com/apache/arrow/blob/master/format/Schema.fbs#L343-L346|https://github.com/apache/arrow/blob/master/format/Schema.fbs#L343-L346], > which is added by ARROW-245. -- This message was sent by Atlassian Jira (v8.3.4#803005)
