[jira] [Commented] (ARROW-1253) [C++] Use pre-built toolchain libraries where prudent to speed up CI builds
[ https://issues.apache.org/jira/browse/ARROW-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099496#comment-16099496 ] Wes McKinney commented on ARROW-1253: - PR: https://github.com/apache/arrow/pull/884 > [C++] Use pre-built toolchain libraries where prudent to speed up CI builds > --- > > Key: ARROW-1253 > URL: https://issues.apache.org/jira/browse/ARROW-1253 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: 0.6.0 > > > Our Travis CI builds, particularly the ones including Python, are spending a > lot of time rebuilding the thirdparty toolchain. We should limit how many > times we build the toolchain to just enough to be confident that it yields a > working system -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ARROW-1252) [Website] Update for 0.5.0 release, add blog post summarizing changes from 0.4.x
[ https://issues.apache.org/jira/browse/ARROW-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099495#comment-16099495 ] Wes McKinney commented on ARROW-1252: - PR: https://github.com/apache/arrow/pull/885 > [Website] Update for 0.5.0 release, add blog post summarizing changes from > 0.4.x > > > Key: ARROW-1252 > URL: https://issues.apache.org/jira/browse/ARROW-1252 > Project: Apache Arrow > Issue Type: Task > Components: Website >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: 0.5.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1263) [C++] CpuInfo should be able to get CPU features on Windows
Max Risuhin created ARROW-1263: -- Summary: [C++] CpuInfo should be able to get CPU features on Windows Key: ARROW-1263 URL: https://issues.apache.org/jira/browse/ARROW-1263 Project: Apache Arrow Issue Type: Bug Components: C++ Environment: Windows Reporter: Max Risuhin Assignee: Max Risuhin CpuInfo able to determine CPU features on unix using method `CpuInfo::IsSupported` Currently it detects such CPU features as SSSE3, SSE4_1, SSE4_2 and POPCNT support. Windows support of above requires specific implementation. Useful link https://msdn.microsoft.com/en-us/library/hskdteyh.aspx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1262) Packaging automation in arrow-dist
Wes McKinney created ARROW-1262: --- Summary: Packaging automation in arrow-dist Key: ARROW-1262 URL: https://issues.apache.org/jira/browse/ARROW-1262 Project: Apache Arrow Issue Type: Task Components: Packaging Reporter: Wes McKinney Fix For: 0.6.0 This JIRA is an umbrella JIRA for tasks to streamline our binary builds at release time as much as possible. We may also be able to set up nightly builds for testing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1261) [Java] Add container type for Map logical type
Wes McKinney created ARROW-1261: --- Summary: [Java] Add container type for Map logical type Key: ARROW-1261 URL: https://issues.apache.org/jira/browse/ARROW-1261 Project: Apache Arrow Issue Type: New Feature Components: Java - Vectors Reporter: Wes McKinney As follow up to ARROW-1246 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1219) [C++] Use more vanilla Google C++ formatting
[ https://issues.apache.org/jira/browse/ARROW-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1219. - Resolution: Fixed Issue resolved by pull request 848 [https://github.com/apache/arrow/pull/848] > [C++] Use more vanilla Google C++ formatting > > > Key: ARROW-1219 > URL: https://issues.apache.org/jira/browse/ARROW-1219 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: 0.6.0 > > > I find the Google coding style used in TensorFlow, Apache Kudu, and other > projects to be a little more readable and having more stylistic consistency > with Python code, too. I will put up a patch and we can discuss. Mostly > function signatures are affected -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-1260) [Plasma] Use factory method to create Python PlasmaClient
[ https://issues.apache.org/jira/browse/ARROW-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-1260: --- Assignee: Philipp Moritz > [Plasma] Use factory method to create Python PlasmaClient > - > > Key: ARROW-1260 > URL: https://issues.apache.org/jira/browse/ARROW-1260 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Assignee: Philipp Moritz >Priority: Minor > Fix For: 0.6.0 > > > see the last comment from Wes in https://github.com/apache/arrow/pull/797 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1260) [Plasma] Use factory method to create Python PlasmaClient
[ https://issues.apache.org/jira/browse/ARROW-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1260. - Resolution: Fixed Issue resolved by pull request 883 [https://github.com/apache/arrow/pull/883] > [Plasma] Use factory method to create Python PlasmaClient > - > > Key: ARROW-1260 > URL: https://issues.apache.org/jira/browse/ARROW-1260 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Priority: Minor > Fix For: 0.6.0 > > > see the last comment from Wes in https://github.com/apache/arrow/pull/797 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1246) [Format] Add Map logical type to metadata
[ https://issues.apache.org/jira/browse/ARROW-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1246. - Resolution: Fixed Issue resolved by pull request 876 [https://github.com/apache/arrow/pull/876] > [Format] Add Map logical type to metadata > - > > Key: ARROW-1246 > URL: https://issues.apache.org/jira/browse/ARROW-1246 > Project: Apache Arrow > Issue Type: New Feature > Components: Format >Reporter: Wes McKinney >Assignee: Wes McKinney > Fix For: 0.6.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1245) [Integration] Java Integration Tests Disabled
[ https://issues.apache.org/jira/browse/ARROW-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1245. - Resolution: Fixed Issue resolved by pull request 875 [https://github.com/apache/arrow/pull/875] > [Integration] Java Integration Tests Disabled > - > > Key: ARROW-1245 > URL: https://issues.apache.org/jira/browse/ARROW-1245 > Project: Apache Arrow > Issue Type: Bug >Reporter: Bryan Cutler >Assignee: Bryan Cutler > > JavaTester in Integration tests is commented out. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1259) [Plasma] Speed up Plasma tests
[ https://issues.apache.org/jira/browse/ARROW-1259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1259. - Resolution: Fixed Issue resolved by pull request 882 [https://github.com/apache/arrow/pull/882] > [Plasma] Speed up Plasma tests > -- > > Key: ARROW-1259 > URL: https://issues.apache.org/jira/browse/ARROW-1259 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Assignee: Philipp Moritz > Fix For: 0.6.0 > > > At the moment Plasma Linux entry in the build matrix seems to hit the Travis > limits sometimes. One way to speed the build significantly is by making the > subscribe a little less demanding; this gets the time taken down from > 40-45min to 25min. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1249) Expose the fillEmpties function from NullableVector.mutator
[ https://issues.apache.org/jira/browse/ARROW-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1249. - Resolution: Fixed Issue resolved by pull request 880 [https://github.com/apache/arrow/pull/880] > Expose the fillEmpties function from NullableVector.mutator > -- > > Key: ARROW-1249 > URL: https://issues.apache.org/jira/browse/ARROW-1249 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Vectors >Reporter: SIDDHARTH TEOTIA >Assignee: SIDDHARTH TEOTIA >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1260) [Plasma] Use factory method to create Python PlasmaClient
Philipp Moritz created ARROW-1260: - Summary: [Plasma] Use factory method to create Python PlasmaClient Key: ARROW-1260 URL: https://issues.apache.org/jira/browse/ARROW-1260 Project: Apache Arrow Issue Type: Improvement Reporter: Philipp Moritz Priority: Minor Fix For: 0.6.0 see the last comment from Wes in https://github.com/apache/arrow/pull/797 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1259) [Plasma] Speed up Plasma tests
Philipp Moritz created ARROW-1259: - Summary: [Plasma] Speed up Plasma tests Key: ARROW-1259 URL: https://issues.apache.org/jira/browse/ARROW-1259 Project: Apache Arrow Issue Type: Improvement Reporter: Philipp Moritz Assignee: Philipp Moritz Fix For: 0.6.0 At the moment Plasma Linux entry in the build matrix seems to hit the Travis limits sometimes. One way to speed the build significantly is by making the subscribe a little less demanding; this gets the time taken down from 40-45min to 25min. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1258) [C++] Suppress dlmalloc warnings on Clang
Wes McKinney created ARROW-1258:
---
Summary: [C++] Suppress dlmalloc warnings on Clang
Key: ARROW-1258
URL: https://issues.apache.org/jira/browse/ARROW-1258
Project: Apache Arrow
Issue Type: Improvement
Components: Plasma (C++)
Reporter: Wes McKinney
Fix For: 0.6.0
{code}
In file included from ../src/plasma/malloc.cc:45:
../src/plasma/thirdparty/dlmalloc.c:592:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS /* Spin locks for gcc >= 4.1, older gcc on x86, MSC >= 1310 */
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:592:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:648:9: warning: '_GNU_SOURCE' macro
redefined [-Wmacro-redefined]
#define _GNU_SOURCE /* Turns on mremap() definition */
^
:341:9: note: previous definition is here
#define _GNU_SOURCE 1
^
In file included from ../src/plasma/malloc.cc:45:
../src/plasma/thirdparty/dlmalloc.c:1492:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:1492:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:1802:6: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if !USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:1802:6: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:2594:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:2594:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:2644:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:2644:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:2740:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note: expanded from macro
'USE_LOCKS'
#define USE_LOCKS ((defined(USE_SPIN_LOCKS) && USE_SPIN_LOCKS != 0) || \
^
../src/plasma/thirdparty/dlmalloc.c:2740:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
../src/plasma/thirdparty/dlmalloc.c:589:22: note: expanded from macro
'USE_LOCKS'
(defined(USE_RECURSIVE_LOCKS) && USE_RECURSIVE_LOCKS != 0))
^
../src/plasma/thirdparty/dlmalloc.c:4563:5: warning: macro expansion producing
'defined' has undefined behavior [-Wexpansion-to-defined]
#if USE_LOCKS
^
../src/plasma/thirdparty/dlmalloc.c:588:22: note:
[jira] [Comment Edited] (ARROW-1254) failing installation on osx / linux
[ https://issues.apache.org/jira/browse/ARROW-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099259#comment-16099259 ] Jeff Reback edited comment on ARROW-1254 at 7/24/17 11:21 PM: -- fixed via: https://github.com/pandas-dev/pandas/pull/17065 was (Author: jreback): fixed via: https://github.com/pandas-dev/pandas/pull/16727 > failing installation on osx / linux > --- > > Key: ARROW-1254 > URL: https://issues.apache.org/jira/browse/ARROW-1254 > Project: Apache Arrow > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Jeff Reback >Assignee: Uwe L. Korn > > This might be pre-mature as it looks like arrow-cpp is the released version, > but pyarrow is still a pre-build (maybe still building)? > xref https://github.com/pandas-dev/pandas/issues/17064 > This install is on osx, but seeing issue on travis (linux) > ``` > The following packages will be downloaded: > package|build > ---|- > arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge > python-dateutil-2.6.0 | py36_0 236 KB defaults > pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge > >Total: 2.6 MB > The following NEW packages will be INSTALLED: > python-dateutil: 2.6.0-py36_0 defaults > The following packages will be UPDATED: > arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 > conda-forge > pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 > conda-forge > The following packages will be DOWNGRADED due to dependency conflicts: > jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 > conda-forge > Proceed ([y]/n)? y > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ARROW-1254) failing installation on osx / linux
[ https://issues.apache.org/jira/browse/ARROW-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099259#comment-16099259 ] Jeff Reback commented on ARROW-1254: fixed via: https://github.com/pandas-dev/pandas/pull/16727 > failing installation on osx / linux > --- > > Key: ARROW-1254 > URL: https://issues.apache.org/jira/browse/ARROW-1254 > Project: Apache Arrow > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Jeff Reback >Assignee: Uwe L. Korn > > This might be pre-mature as it looks like arrow-cpp is the released version, > but pyarrow is still a pre-build (maybe still building)? > xref https://github.com/pandas-dev/pandas/issues/17064 > This install is on osx, but seeing issue on travis (linux) > ``` > The following packages will be downloaded: > package|build > ---|- > arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge > python-dateutil-2.6.0 | py36_0 236 KB defaults > pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge > >Total: 2.6 MB > The following NEW packages will be INSTALLED: > python-dateutil: 2.6.0-py36_0 defaults > The following packages will be UPDATED: > arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 > conda-forge > pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 > conda-forge > The following packages will be DOWNGRADED due to dependency conflicts: > jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 > conda-forge > Proceed ([y]/n)? y > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ARROW-1195) [C++] CpuInfo doesn't get cache size on Windows
[ https://issues.apache.org/jira/browse/ARROW-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-1195: Fix Version/s: 0.6.0 > [C++] CpuInfo doesn't get cache size on Windows > --- > > Key: ARROW-1195 > URL: https://issues.apache.org/jira/browse/ARROW-1195 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Max Risuhin > Fix For: 0.6.0 > > > CpuInfo::cache_sizes_ values are not initialized with actual values on > Windows in method CpuInfo::Init(). Default values are used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1195) [C++] CpuInfo doesn't get cache size on Windows
[ https://issues.apache.org/jira/browse/ARROW-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1195. - Resolution: Fixed Issue resolved by pull request 877 [https://github.com/apache/arrow/pull/877] > [C++] CpuInfo doesn't get cache size on Windows > --- > > Key: ARROW-1195 > URL: https://issues.apache.org/jira/browse/ARROW-1195 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Max Risuhin > > CpuInfo::cache_sizes_ values are not initialized with actual values on > Windows in method CpuInfo::Init(). Default values are used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-1195) [C++] CpuInfo doesn't get cache size on Windows
[ https://issues.apache.org/jira/browse/ARROW-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-1195: --- Assignee: Max Risuhin > [C++] CpuInfo doesn't get cache size on Windows > --- > > Key: ARROW-1195 > URL: https://issues.apache.org/jira/browse/ARROW-1195 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Max Risuhin >Assignee: Max Risuhin > Fix For: 0.6.0 > > > CpuInfo::cache_sizes_ values are not initialized with actual values on > Windows in method CpuInfo::Init(). Default values are used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1257) [Plasma] Plasma documentation
Philipp Moritz created ARROW-1257: - Summary: [Plasma] Plasma documentation Key: ARROW-1257 URL: https://issues.apache.org/jira/browse/ARROW-1257 Project: Apache Arrow Issue Type: Improvement Components: Plasma (C++) Reporter: Philipp Moritz Fix For: 0.6.0 The documentation for Plasma should contain the following section: - Installation and Getting Started - The Plasma Python API (a high level overview and sphinx generated documentation) - The Plasma C++ API (a high level overview and doxygen documentation) - Using Arrow and Pandas with Plasma -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1256) [Plasma] Fix compile warnings on macOS
Philipp Moritz created ARROW-1256: - Summary: [Plasma] Fix compile warnings on macOS Key: ARROW-1256 URL: https://issues.apache.org/jira/browse/ARROW-1256 Project: Apache Arrow Issue Type: Improvement Components: Plasma (C++) Reporter: Philipp Moritz Priority: Minor Fix For: 0.6.0 see https://github.com/ray-project/ray/pull/769 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ARROW-1255) [Plasma] Check plasma flatbuffer messages with the flatbuffer verifier
[ https://issues.apache.org/jira/browse/ARROW-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Moritz updated ARROW-1255: -- Affects Version/s: (was: 0.5.0) > [Plasma] Check plasma flatbuffer messages with the flatbuffer verifier > -- > > Key: ARROW-1255 > URL: https://issues.apache.org/jira/browse/ARROW-1255 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Priority: Minor > Fix For: 0.6.0 > > > The proposal here is to check requests that are send to plasma via the > flatbuffers protocol for validity using the flatbuffers verifier. For > efficiency reasons this should only be done in debug builds. > This will prevent typos like https://github.com/apache/arrow/pull/878 and > will also prevent errors from possible future plasma clients written in > different languages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ARROW-1255) [Plasma] Check plasma flatbuffer messages with the flatbuffer verifier
[ https://issues.apache.org/jira/browse/ARROW-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Moritz updated ARROW-1255: -- Summary: [Plasma] Check plasma flatbuffer messages with the flatbuffer verifier (was: [Plasma] Check plasma flatbuffer messages with the flatuffer verifier) > [Plasma] Check plasma flatbuffer messages with the flatbuffer verifier > -- > > Key: ARROW-1255 > URL: https://issues.apache.org/jira/browse/ARROW-1255 > Project: Apache Arrow > Issue Type: Improvement >Affects Versions: 0.5.0 >Reporter: Philipp Moritz >Priority: Minor > Fix For: 0.6.0 > > > The proposal here is to check requests that are send to plasma via the > flatbuffers protocol for validity using the flatbuffers verifier. For > efficiency reasons this should only be done in debug builds. > This will prevent typos like https://github.com/apache/arrow/pull/878 and > will also prevent errors from possible future plasma clients written in > different languages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1255) [Plasma] Check plasma flatbuffer messages with the flatuffer verifier
Philipp Moritz created ARROW-1255: - Summary: [Plasma] Check plasma flatbuffer messages with the flatuffer verifier Key: ARROW-1255 URL: https://issues.apache.org/jira/browse/ARROW-1255 Project: Apache Arrow Issue Type: Improvement Affects Versions: 0.5.0 Reporter: Philipp Moritz Priority: Minor Fix For: 0.6.0 The proposal here is to check requests that are send to plasma via the flatbuffers protocol for validity using the flatbuffers verifier. For efficiency reasons this should only be done in debug builds. This will prevent typos like https://github.com/apache/arrow/pull/878 and will also prevent errors from possible future plasma clients written in different languages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1149) [Plasma] Create Cython client library for Plasma
[ https://issues.apache.org/jira/browse/ARROW-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1149. - Resolution: Fixed Issue resolved by pull request 797 [https://github.com/apache/arrow/pull/797] > [Plasma] Create Cython client library for Plasma > > > Key: ARROW-1149 > URL: https://issues.apache.org/jira/browse/ARROW-1149 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Philipp Moritz >Assignee: Philipp Moritz > Fix For: 0.6.0 > > > This would address the point raised in > https://github.com/apache/arrow/pull/742 by [~wesmckinn] and replace the > current C Python API with Cython. > Advantages: > - More uniform with the rest of Arrow (especially pyarrow) > - can share code with pyarrow > - makes error handling more uniform > - easier to program than the C Python API > As part of this, we would also fix the visibility of symbols in the plasma > library and port the python tests we have for plasma (which are currently in > the Ray repo) to this repo. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ARROW-1237) Expose the ability to set lastSet
[ https://issues.apache.org/jira/browse/ARROW-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau updated ARROW-1237: -- Fix Version/s: 0.6.0 > Expose the ability to set lastSet > -- > > Key: ARROW-1237 > URL: https://issues.apache.org/jira/browse/ARROW-1237 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Vectors >Reporter: SIDDHARTH TEOTIA >Assignee: SIDDHARTH TEOTIA >Priority: Minor > Fix For: 0.6.0 > > > Expose the ability to set lastSet on vectors such that > Mutator.setValueCount() doesn't blow away the vector if we have previously > loaded it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1239) issue with current version of git-commit-id-plugin
[
https://issues.apache.org/jira/browse/ARROW-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau resolved ARROW-1239.
---
Resolution: Fixed
Issue resolved by pull request 869
[https://github.com/apache/arrow/pull/869]
> issue with current version of git-commit-id-plugin
> ---
>
> Key: ARROW-1239
> URL: https://issues.apache.org/jira/browse/ARROW-1239
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java - Memory, Java - Vectors
> Environment: maven
>Reporter: Antony Mayi
>Assignee: Antony Mayi
> Fix For: 0.6.0
>
>
> Currently used version of the git-commit-id-plugin maven plugin (2.1.9)
> doesn't work with recent git structures. This is for majority of the users
> not manifested since Arrow has the java maven root in the project
> subdirectory ({{/java}}) instead of top level so this plugin normally
> doesn't kick in if maven is executed from the subdirectory (usual case - ie
> {{cd java; mvn install}} - works fine) as the plugin doesn't see the {{.git}}
> directory but it does kick in and fail if executed from the main arrow top
> level dir as {{mvn -f java/pom.xml install}} (where the {{.git}} sits):
> {quote}
> $ mvn -f java/pom.xml package
> ...
> [ERROR] Failed to execute goal
> pl.project13.maven:git-commit-id-plugin:2.1.9:revision (for-jars) on project
> arrow-java-root: Execution for-jars of goal
> pl.project13.maven:git-commit-id-plugin:2.1.9:revision failed: Bare
> Repository has neither a working tree, nor an index -> [Help 1]
> {quote}
> Simple fix is upgrading the plugin to recent version (the minimal working
> version appears to be 2.1.13).
> This is required for seamless integration with Jenkins (ARROW-1234).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (ARROW-1237) Expose the ability to set lastSet
[ https://issues.apache.org/jira/browse/ARROW-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacques Nadeau resolved ARROW-1237. --- Resolution: Fixed Issue resolved by pull request 868 [https://github.com/apache/arrow/pull/868] > Expose the ability to set lastSet > -- > > Key: ARROW-1237 > URL: https://issues.apache.org/jira/browse/ARROW-1237 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Vectors >Reporter: SIDDHARTH TEOTIA >Assignee: SIDDHARTH TEOTIA >Priority: Minor > > Expose the ability to set lastSet on vectors such that > Mutator.setValueCount() doesn't blow away the vector if we have previously > loaded it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-1240) security: upgrade logback to address CVE-2017-5929
[ https://issues.apache.org/jira/browse/ARROW-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-1240: --- Assignee: Matt Darwin > security: upgrade logback to address CVE-2017-5929 > -- > > Key: ARROW-1240 > URL: https://issues.apache.org/jira/browse/ARROW-1240 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Memory, Java - Vectors >Affects Versions: 0.4.1 >Reporter: Matt Darwin >Assignee: Matt Darwin > Fix For: 0.6.0 > > > logback versions before 1.2.0 are affected by "a rather severe serialization > vulnerability in SocketServer and ServerSocketReceiver". > We should upgrade logback from 1.0.13 to the latest version (currently 1.2.3) > in order to address this. > See https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5929 > and > https://logback.qos.ch/news.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ARROW-1240) security: upgrade logback to address CVE-2017-5929
[ https://issues.apache.org/jira/browse/ARROW-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-1240: Fix Version/s: (was: 0.5.0) 0.6.0 > security: upgrade logback to address CVE-2017-5929 > -- > > Key: ARROW-1240 > URL: https://issues.apache.org/jira/browse/ARROW-1240 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Memory, Java - Vectors >Affects Versions: 0.4.1 >Reporter: Matt Darwin >Assignee: Matt Darwin > Fix For: 0.6.0 > > > logback versions before 1.2.0 are affected by "a rather severe serialization > vulnerability in SocketServer and ServerSocketReceiver". > We should upgrade logback from 1.0.13 to the latest version (currently 1.2.3) > in order to address this. > See https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5929 > and > https://logback.qos.ch/news.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1240) security: upgrade logback to address CVE-2017-5929
[ https://issues.apache.org/jira/browse/ARROW-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1240. - Resolution: Fixed Issue resolved by pull request 879 [https://github.com/apache/arrow/pull/879] > security: upgrade logback to address CVE-2017-5929 > -- > > Key: ARROW-1240 > URL: https://issues.apache.org/jira/browse/ARROW-1240 > Project: Apache Arrow > Issue Type: Bug > Components: Java - Memory, Java - Vectors >Affects Versions: 0.4.1 >Reporter: Matt Darwin > Fix For: 0.5.0 > > > logback versions before 1.2.0 are affected by "a rather severe serialization > vulnerability in SocketServer and ServerSocketReceiver". > We should upgrade logback from 1.0.13 to the latest version (currently 1.2.3) > in order to address this. > See https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5929 > and > https://logback.qos.ch/news.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ARROW-1254) failing installation on osx / linux
[ https://issues.apache.org/jira/browse/ARROW-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-1254. - Resolution: Not A Problem Because of the way conda-forge works (takes a long time to update multiple interdependent packages), this is usually going to happen around release time. The way to avoid it is to pin the package version rather than installing "head". I think when we do "pre" builds on conda-forge we should pin the pre build so that when we update to final it doesn't break things if pyarrow (last package in the dependency chain) hasn't been updated > failing installation on osx / linux > --- > > Key: ARROW-1254 > URL: https://issues.apache.org/jira/browse/ARROW-1254 > Project: Apache Arrow > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Jeff Reback >Assignee: Uwe L. Korn > > This might be pre-mature as it looks like arrow-cpp is the released version, > but pyarrow is still a pre-build (maybe still building)? > xref https://github.com/pandas-dev/pandas/issues/17064 > This install is on osx, but seeing issue on travis (linux) > ``` > The following packages will be downloaded: > package|build > ---|- > arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge > python-dateutil-2.6.0 | py36_0 236 KB defaults > pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge > >Total: 2.6 MB > The following NEW packages will be INSTALLED: > python-dateutil: 2.6.0-py36_0 defaults > The following packages will be UPDATED: > arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 > conda-forge > pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 > conda-forge > The following packages will be DOWNGRADED due to dependency conflicts: > jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 > conda-forge > Proceed ([y]/n)? y > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-905) [Docs] Add Dockerfile for reproducible documentation generation
[ https://issues.apache.org/jira/browse/ARROW-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe L. Korn reassigned ARROW-905: - Assignee: Uwe L. Korn > [Docs] Add Dockerfile for reproducible documentation generation > --- > > Key: ARROW-905 > URL: https://issues.apache.org/jira/browse/ARROW-905 > Project: Apache Arrow > Issue Type: Improvement > Components: Documentation >Reporter: Wes McKinney >Assignee: Uwe L. Korn > Fix For: 0.6.0 > > > We want to make sure that any committer can update the documentation on > http://arrow.apache.org/. Since development environments may vary, it would > be useful to have a point-of-truth bash script or Dockerfile for building all > of the project API documentation for upload to the website. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ARROW-1254) failing installation on osx / linux
[ https://issues.apache.org/jira/browse/ARROW-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16098490#comment-16098490 ] Uwe L. Korn commented on ARROW-1254: PR: https://github.com/conda-forge/pyarrow-feedstock/pull/34 > failing installation on osx / linux > --- > > Key: ARROW-1254 > URL: https://issues.apache.org/jira/browse/ARROW-1254 > Project: Apache Arrow > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Jeff Reback >Assignee: Uwe L. Korn > > This might be pre-mature as it looks like arrow-cpp is the released version, > but pyarrow is still a pre-build (maybe still building)? > xref https://github.com/pandas-dev/pandas/issues/17064 > This install is on osx, but seeing issue on travis (linux) > ``` > The following packages will be downloaded: > package|build > ---|- > arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge > python-dateutil-2.6.0 | py36_0 236 KB defaults > pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge > >Total: 2.6 MB > The following NEW packages will be INSTALLED: > python-dateutil: 2.6.0-py36_0 defaults > The following packages will be UPDATED: > arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 > conda-forge > pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 > conda-forge > The following packages will be DOWNGRADED due to dependency conflicts: > jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 > conda-forge > Proceed ([y]/n)? y > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-439) [Python] Add option in "to_pandas" conversions to yield Categorical from String/Binary arrays
[ https://issues.apache.org/jira/browse/ARROW-439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florian Jetter reassigned ARROW-439: Assignee: Florian Jetter > [Python] Add option in "to_pandas" conversions to yield Categorical from > String/Binary arrays > - > > Key: ARROW-439 > URL: https://issues.apache.org/jira/browse/ARROW-439 > Project: Apache Arrow > Issue Type: New Feature > Components: Python >Reporter: Wes McKinney >Assignee: Florian Jetter > Fix For: 0.6.0 > > > This will depend on bringing in general dictionary encoding / hashing > machinery -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ARROW-1254) failing installation on osx / linux
[ https://issues.apache.org/jira/browse/ARROW-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe L. Korn reassigned ARROW-1254: -- Assignee: Uwe L. Korn > failing installation on osx / linux > --- > > Key: ARROW-1254 > URL: https://issues.apache.org/jira/browse/ARROW-1254 > Project: Apache Arrow > Issue Type: Bug >Affects Versions: 0.5.0 >Reporter: Jeff Reback >Assignee: Uwe L. Korn > > This might be pre-mature as it looks like arrow-cpp is the released version, > but pyarrow is still a pre-build (maybe still building)? > xref https://github.com/pandas-dev/pandas/issues/17064 > This install is on osx, but seeing issue on travis (linux) > ``` > The following packages will be downloaded: > package|build > ---|- > arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge > python-dateutil-2.6.0 | py36_0 236 KB defaults > pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge > >Total: 2.6 MB > The following NEW packages will be INSTALLED: > python-dateutil: 2.6.0-py36_0 defaults > The following packages will be UPDATED: > arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 > conda-forge > pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 > conda-forge > The following packages will be DOWNGRADED due to dependency conflicts: > jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 > conda-forge > Proceed ([y]/n)? y > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ARROW-1254) failing installation on osx / linux
Jeff Reback created ARROW-1254: -- Summary: failing installation on osx / linux Key: ARROW-1254 URL: https://issues.apache.org/jira/browse/ARROW-1254 Project: Apache Arrow Issue Type: Bug Affects Versions: 0.5.0 Reporter: Jeff Reback This might be pre-mature as it looks like arrow-cpp is the released version, but pyarrow is still a pre-build (maybe still building)? xref https://github.com/pandas-dev/pandas/issues/17064 This install is on osx, but seeing issue on travis (linux) ``` The following packages will be downloaded: package|build ---|- arrow-cpp-0.5.0| np113py36_0 1.8 MB conda-forge python-dateutil-2.6.0 | py36_0 236 KB defaults pyarrow-0.5.0.pre | np113py36_2 511 KB conda-forge Total: 2.6 MB The following NEW packages will be INSTALLED: python-dateutil: 2.6.0-py36_0 defaults The following packages will be UPDATED: arrow-cpp: 0.4.1-np113py36_2 conda-forge --> 0.5.0-np113py36_0 conda-forge pyarrow: 0.4.1-np113py36_0 conda-forge --> 0.5.0.pre-np113py36_2 conda-forge The following packages will be DOWNGRADED due to dependency conflicts: jemalloc:5.0.0-0 conda-forge --> 4.4.0-0 conda-forge Proceed ([y]/n)? y ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
