[jira] [Resolved] (ARROW-2582) [GLib] Add negate functions for Decimal128
[ https://issues.apache.org/jira/browse/ARROW-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-2582. - Resolution: Fixed Fix Version/s: 0.10.0 Issue resolved by pull request 2047 [https://github.com/apache/arrow/pull/2047] > [GLib] Add negate functions for Decimal128 > -- > > Key: ARROW-2582 > URL: https://issues.apache.org/jira/browse/ARROW-2582 > Project: Apache Arrow > Issue Type: Improvement >Reporter: yosuke shiro >Priority: Major > Labels: pull-request-available > Fix For: 0.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ARROW-2582) [GLib] Add negate functions for Decimal128
[ https://issues.apache.org/jira/browse/ARROW-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou reassigned ARROW-2582: --- Assignee: yosuke shiro > [GLib] Add negate functions for Decimal128 > -- > > Key: ARROW-2582 > URL: https://issues.apache.org/jira/browse/ARROW-2582 > Project: Apache Arrow > Issue Type: Improvement >Reporter: yosuke shiro >Assignee: yosuke shiro >Priority: Major > Labels: pull-request-available > Fix For: 0.10.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ARROW-2558) [Plasma] avoid walk through all the objects when a client disconnects
[ https://issues.apache.org/jira/browse/ARROW-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Moritz resolved ARROW-2558. --- Resolution: Fixed Fix Version/s: 0.10.0 Issue resolved by pull request 2015 [https://github.com/apache/arrow/pull/2015] > [Plasma] avoid walk through all the objects when a client disconnects > - > > Key: ARROW-2558 > URL: https://issues.apache.org/jira/browse/ARROW-2558 > Project: Apache Arrow > Issue Type: Improvement > Components: Plasma (C++) >Reporter: Zhijun Fu >Priority: Minor > Labels: pull-request-available > Fix For: 0.10.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Currently plasma stores list-of-clients in ObjectTableEntry, which is used to > track which clients are using a given object, this serves for two purposes: > - If an object is in use. > - If the client trying to abort an object is the one who created it. > A problem with list-of-clients approach is that when a client disconnects, we > need to walk through all the objects and remove the client pointer from the > list for each object. > Instead, we could add a reference count in ObjectTableEntry, and store > list-of-object-ids in client structure. This could both goals that the > original approach is targeting, while when a client disconnects, it just walk > through its object-ids and dereference each ObjectTableEntry, there's no need > to walk through all objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2586) Make child builders of ListBuilder and StructBuilder shared_ptr's
[ https://issues.apache.org/jira/browse/ARROW-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2586: -- Labels: pull-request-available (was: ) > Make child builders of ListBuilder and StructBuilder shared_ptr's > - > > Key: ARROW-2586 > URL: https://issues.apache.org/jira/browse/ARROW-2586 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Joshua Storck >Priority: Major > Labels: pull-request-available > > This is needed for changes in this PR that make it possible to deserialize > arbitrary nested structures in parquet (ARROW-1644): > https://github.com/apache/parquet-cpp/pull/462 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ARROW-2584) [JS] Node v10 issues
[ https://issues.apache.org/jira/browse/ARROW-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Hulette resolved ARROW-2584. -- Resolution: Fixed Fix Version/s: JS-0.4.0 Issue resolved by pull request 2049 [https://github.com/apache/arrow/pull/2049] > [JS] Node v10 issues > > > Key: ARROW-2584 > URL: https://issues.apache.org/jira/browse/ARROW-2584 > Project: Apache Arrow > Issue Type: Bug > Components: JavaScript >Reporter: Brian Hulette >Assignee: Paul Taylor >Priority: Major > Labels: pull-request-available > Fix For: JS-0.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Build and tests fail with node v10. Fix these issues and bump CI to use node > v10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-2586) Make child builders of ListBuilder and StructBuilder shared_ptr's
Joshua Storck created ARROW-2586: Summary: Make child builders of ListBuilder and StructBuilder shared_ptr's Key: ARROW-2586 URL: https://issues.apache.org/jira/browse/ARROW-2586 Project: Apache Arrow Issue Type: Improvement Reporter: Joshua Storck This is needed for changes in this PR that make it possible to deserialize arbitrary nested structures in parquet (ARROW-1644): https://github.com/apache/parquet-cpp/pull/462 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2574) [CI] Collect and publish Python coverage
[ https://issues.apache.org/jira/browse/ARROW-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2574: -- Labels: pull-request-available (was: ) > [CI] Collect and publish Python coverage > > > Key: ARROW-2574 > URL: https://issues.apache.org/jira/browse/ARROW-2574 > Project: Apache Arrow > Issue Type: Improvement > Components: Continuous Integration, Python >Affects Versions: 0.9.0 >Reporter: Antoine Pitrou >Assignee: Antoine Pitrou >Priority: Major > Labels: pull-request-available > > Now that our Travis-CI setup is able to collect and publish C++ and Rust > coverage, we should do the same for Python and Cython modules in pyarrow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2584) [JS] Node v10 issues
[ https://issues.apache.org/jira/browse/ARROW-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2584: -- Labels: pull-request-available (was: ) > [JS] Node v10 issues > > > Key: ARROW-2584 > URL: https://issues.apache.org/jira/browse/ARROW-2584 > Project: Apache Arrow > Issue Type: Bug > Components: JavaScript >Reporter: Brian Hulette >Assignee: Paul Taylor >Priority: Major > Labels: pull-request-available > > Build and tests fail with node v10. Fix these issues and bump CI to use node > v10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2585) Add Decimal128::FromBigEndian
[ https://issues.apache.org/jira/browse/ARROW-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2585: -- Labels: pull-request-available (was: ) > Add Decimal128::FromBigEndian > - > > Key: ARROW-2585 > URL: https://issues.apache.org/jira/browse/ARROW-2585 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Joshua Storck >Priority: Major > Labels: pull-request-available > > This code is being moved from > https://github.com/apache/parquet-cpp/blob/8046481235e558344c3aa059c83ee86b9f67/src/parquet/arrow/reader.cc#L1049 > for us in this PR: https://github.com/apache/parquet-cpp/pull/462 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-2585) Add Decimal128::FromBigEndian
Joshua Storck created ARROW-2585: Summary: Add Decimal128::FromBigEndian Key: ARROW-2585 URL: https://issues.apache.org/jira/browse/ARROW-2585 Project: Apache Arrow Issue Type: Improvement Reporter: Joshua Storck This code is being moved from https://github.com/apache/parquet-cpp/blob/8046481235e558344c3aa059c83ee86b9f67/src/parquet/arrow/reader.cc#L1049 for us in this PR: https://github.com/apache/parquet-cpp/pull/462 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2579) Appending to streamable table file format doesn't seem to work
[ https://issues.apache.org/jira/browse/ARROW-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475947#comment-16475947 ] Antoine Pitrou commented on ARROW-2579: --- It would be better with a standalone reproducer. Otherwise we don't know whether it depends on using specific datatypes or features. > Appending to streamable table file format doesn't seem to work > -- > > Key: ARROW-2579 > URL: https://issues.apache.org/jira/browse/ARROW-2579 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Affects Versions: 0.9.0 >Reporter: Rob Ambalu >Priority: Major > > As far as I can tell it looks like appending to a streaming file format isn’t > currently supported, is that right? > RecordBatchStreamWriter always writes the schema up front, and it doesn’t > look like a schema is expected mid file ( assuming im doing this append test > correctly, this is the error I hit when I try to read back this file into > python: > Traceback (most recent call last): > File "/home/ra7293/rba_arrow_mmap.py", line 9, in > table = reader.read_all() > File "ipc.pxi", line 302, in pyarrow.lib._RecordBatchReader.read_all > File "error.pxi", line 79, in pyarrow.lib.check_status > pyarrow.lib.ArrowIOError: Message not expected type: record batch, was: 1 > > This reader script works fine if I write once / don’t append. > Seeing as IO interfaces support Append, streaming should support it as well ( > if for whatever reason this cant be supported, RecordBatchStreamWriter should > throw if configured with an OutputStreamer that is attempting to append ) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ARROW-2561) [C++] Crash in cuda-test shutdown with coverage enabled
[
https://issues.apache.org/jira/browse/ARROW-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antoine Pitrou reassigned ARROW-2561:
-
Assignee: Antoine Pitrou
> [C++] Crash in cuda-test shutdown with coverage enabled
> ---
>
> Key: ARROW-2561
> URL: https://issues.apache.org/jira/browse/ARROW-2561
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, GPU
>Affects Versions: 0.9.0
>Reporter: Antoine Pitrou
>Assignee: Antoine Pitrou
>Priority: Major
> Labels: pull-request-available
>
> If I enable both CUDA and code coverage (using
> {{-DARROW_GENERATE_COVERAGE=on}}), {{cuda-test}} sometimes crashes at
> shutdown with the following message:
> {code}
> *** Error in `./build-test/debug/cuda-test': corrupted size vs. prev_size:
> 0x01612bb0 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc3d61e47e5]
> /lib/x86_64-linux-gnu/libc.so.6(+0x7e9dc)[0x7fc3d61eb9dc]
> /lib/x86_64-linux-gnu/libc.so.6(+0x81cde)[0x7fc3d61eecde]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fc3d61f1184]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x9350f3)[0x7fc3d5a510f3]
> /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0x9a)[0x7fc3d61a736a]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x3415e3)[0x7fc3d545d5e3]
> {code}
> (the CUDA tests themselves pass)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2561) [C++] Crash in cuda-test shutdown with coverage enabled
[
https://issues.apache.org/jira/browse/ARROW-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-2561:
--
Labels: pull-request-available (was: )
> [C++] Crash in cuda-test shutdown with coverage enabled
> ---
>
> Key: ARROW-2561
> URL: https://issues.apache.org/jira/browse/ARROW-2561
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, GPU
>Affects Versions: 0.9.0
>Reporter: Antoine Pitrou
>Assignee: Antoine Pitrou
>Priority: Major
> Labels: pull-request-available
>
> If I enable both CUDA and code coverage (using
> {{-DARROW_GENERATE_COVERAGE=on}}), {{cuda-test}} sometimes crashes at
> shutdown with the following message:
> {code}
> *** Error in `./build-test/debug/cuda-test': corrupted size vs. prev_size:
> 0x01612bb0 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc3d61e47e5]
> /lib/x86_64-linux-gnu/libc.so.6(+0x7e9dc)[0x7fc3d61eb9dc]
> /lib/x86_64-linux-gnu/libc.so.6(+0x81cde)[0x7fc3d61eecde]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fc3d61f1184]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x9350f3)[0x7fc3d5a510f3]
> /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0x9a)[0x7fc3d61a736a]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x3415e3)[0x7fc3d545d5e3]
> {code}
> (the CUDA tests themselves pass)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-2584) [JS] Node v10 issues
Brian Hulette created ARROW-2584: Summary: [JS] Node v10 issues Key: ARROW-2584 URL: https://issues.apache.org/jira/browse/ARROW-2584 Project: Apache Arrow Issue Type: Bug Components: JavaScript Reporter: Brian Hulette Assignee: Paul Taylor Build and tests fail with node v10. Fix these issues and bump CI to use node v10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2561) [C++] Crash in cuda-test shutdown with coverage enabled
[
https://issues.apache.org/jira/browse/ARROW-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475826#comment-16475826
]
Antoine Pitrou commented on ARROW-2561:
---
Here is the {{valgrind}} output for {{cuda-test}}:
https://gist.github.com/pitrou/bb4fee35c55f58818225bdfdace916f2
In short, it looks like some double-free in a static {{std::string}} in
{{libarrow.so}}. Most probably some C++ shared / static library issue. The
following patch seems to fix things here, but I'm not sure how correct is. What
do you think?
{code}
diff --git a/cpp/src/arrow/gpu/CMakeLists.txt b/cpp/src/arrow/gpu/CMakeLists.txt
index be47413..7a019db 100644
--- a/cpp/src/arrow/gpu/CMakeLists.txt
+++ b/cpp/src/arrow/gpu/CMakeLists.txt
@@ -43,7 +43,7 @@ ADD_ARROW_LIB(arrow_gpu
DEPENDENCIES metadata_fbs
SHARED_LINK_FLAGS ""
SHARED_LINK_LIBS ${ARROW_GPU_SHARED_LINK_LIBS}
- STATIC_LINK_LIBS ""
+ STATIC_LINK_LIBS ${ARROW_GPU_SHARED_LINK_LIBS}
)
# CUDA build version
@@ -72,7 +72,7 @@ install(
DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig/")
set(ARROW_GPU_TEST_LINK_LIBS
- arrow_gpu_shared
+ arrow_gpu_static
${ARROW_TEST_LINK_LIBS})
if (ARROW_BUILD_TESTS)
{code}
> [C++] Crash in cuda-test shutdown with coverage enabled
> ---
>
> Key: ARROW-2561
> URL: https://issues.apache.org/jira/browse/ARROW-2561
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, GPU
>Affects Versions: 0.9.0
>Reporter: Antoine Pitrou
>Priority: Major
>
> If I enable both CUDA and code coverage (using
> {{-DARROW_GENERATE_COVERAGE=on}}), {{cuda-test}} sometimes crashes at
> shutdown with the following message:
> {code}
> *** Error in `./build-test/debug/cuda-test': corrupted size vs. prev_size:
> 0x01612bb0 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fc3d61e47e5]
> /lib/x86_64-linux-gnu/libc.so.6(+0x7e9dc)[0x7fc3d61eb9dc]
> /lib/x86_64-linux-gnu/libc.so.6(+0x81cde)[0x7fc3d61eecde]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fc3d61f1184]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x9350f3)[0x7fc3d5a510f3]
> /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0x9a)[0x7fc3d61a736a]
> /home/antoine/arrow/cpp/build-test/debug/libarrow.so.10(+0x3415e3)[0x7fc3d545d5e3]
> {code}
> (the CUDA tests themselves pass)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-2583) [Rust] Buffer should be typeless
Andy Grove created ARROW-2583: - Summary: [Rust] Buffer should be typeless Key: ARROW-2583 URL: https://issues.apache.org/jira/browse/ARROW-2583 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: Andy Grove Fix For: 0.10.0 See comments in [https://github.com/apache/arrow/pull/1971] for background on this but the summary is that Buffer should just deal with untyped memory e.g. `* const u8` and all type-handling should be moved to the Array layer e.g. `BufferArray`. This would be more consistent with the other implementations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2582) [GLib] Add negate functions for Decimal128
[ https://issues.apache.org/jira/browse/ARROW-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-2582: -- Labels: pull-request-available (was: ) > [GLib] Add negate functions for Decimal128 > -- > > Key: ARROW-2582 > URL: https://issues.apache.org/jira/browse/ARROW-2582 > Project: Apache Arrow > Issue Type: Improvement >Reporter: yosuke shiro >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-2582) [GLib] Add negate functions for Decimal128
yosuke shiro created ARROW-2582: --- Summary: [GLib] Add negate functions for Decimal128 Key: ARROW-2582 URL: https://issues.apache.org/jira/browse/ARROW-2582 Project: Apache Arrow Issue Type: Improvement Reporter: yosuke shiro -- This message was sent by Atlassian JIRA (v7.6.3#76005)
