[jira] [Created] (ARROW-5080) [Release] Add a script to release Rust packages

2019-03-31 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-5080:
---

 Summary: [Release] Add a script to release Rust packages
 Key: ARROW-5080
 URL: https://issues.apache.org/jira/browse/ARROW-5080
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-5080) [Release] Add a script to release Rust packages

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-5080:
--
Labels: pull-request-available  (was: )

> [Release] Add a script to release Rust packages
> ---
>
> Key: ARROW-5080
> URL: https://issues.apache.org/jira/browse/ARROW-5080
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-5079) [Release] Add a script to release C# package

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-5079:
--
Labels: pull-request-available  (was: )

> [Release] Add a script to release C# package
> 
>
> Key: ARROW-5079
> URL: https://issues.apache.org/jira/browse/ARROW-5079
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5079) [Release] Add a script to release C# package

2019-03-31 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-5079:
---

 Summary: [Release] Add a script to release C# package
 Key: ARROW-5079
 URL: https://issues.apache.org/jira/browse/ARROW-5079
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-4301) [Java][Gandiva] Maven snapshot version update does not seem to update Gandiva submodule

2019-03-31 Thread Kouhei Sutou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806361#comment-16806361
 ] 

Kouhei Sutou commented on ARROW-4301:
-

Thanks for the information.
I've applied a fix by https://github.com/apache/arrow/pull/4087 manually.
We should fix this until 0.14.0.

> [Java][Gandiva] Maven snapshot version update does not seem to update Gandiva 
> submodule
> ---
>
> Key: ARROW-4301
> URL: https://issues.apache.org/jira/browse/ARROW-4301
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: C++ - Gandiva, Java
>Reporter: Wes McKinney
>Assignee: Praveen Kumar Desabandu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> See 
> https://github.com/apache/arrow/commit/a486db8c1476be1165981c4fe22996639da8e550.
>  This is breaking the build so I'm going to patch manually



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ARROW-5053) [Rust] [DataFusion] Use env var for location of arrow test data

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou resolved ARROW-5053.
-
Resolution: Fixed

Issue resolved by pull request 4068
[https://github.com/apache/arrow/pull/4068]

> [Rust] [DataFusion] Use env var for location of arrow test data
> ---
>
> Key: ARROW-5053
> URL: https://issues.apache.org/jira/browse/ARROW-5053
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust - DataFusion
>Affects Versions: 0.13.0
>Reporter: Andy Grove
>Assignee: Andy Grove
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> A small number of tests have hard coded relative path for arrow test data 
> files, meaning that the tests fail in the release tarball.
> We should use an ARROW_TEST_DATA env var similar to how we deal with 
> PARQUET_TEST_DATA



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5078) [Documentation] Sphinx is failed by RemovedInSphinx30Warning

2019-03-31 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-5078:
---

 Summary: [Documentation] Sphinx is failed by 
RemovedInSphinx30Warning
 Key: ARROW-5078
 URL: https://issues.apache.org/jira/browse/ARROW-5078
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Documentation
Reporter: Kouhei Sutou


https://travis-ci.org/apache/arrow/jobs/513850506

{noformat}
/home/travis/build/apache/arrow/pyarrow-test-3.6/lib/python3.6/site-packages/sphinx/util/docutils.py:311:
 RemovedInSphinx30Warning: function based directive support is now deprecated. 
Use class based directive instead.
  RemovedInSphinx30Warning)
Exception occurred:
  File 
"/home/travis/build/apache/arrow/pyarrow-test-3.6/lib/python3.6/site-packages/breathe/renderer/sphinxrenderer.py",
 line 38, in parse_definition
ast = parser.parse_declaration("class")
TypeError: parse_declaration() missing 1 required positional argument: 
'directiveType'
The full traceback has been saved in /tmp/sphinx-err-i64grj3x.log, if you want 
to report the issue to the developers.
Please also report this if it was a user error, so that a better error message 
can be provided next time.
A bug report can be filed in the tracker at 
. Thanks!
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-5076) [Packaging] Improve post binary upload performance

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-5076:
--
Labels: pull-request-available  (was: )

> [Packaging] Improve post binary upload performance
> --
>
> Key: ARROW-5076
> URL: https://issues.apache.org/jira/browse/ARROW-5076
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5077) [Rust] Release process should change Cargo.toml to use release versions

2019-03-31 Thread Andy Grove (JIRA)
Andy Grove created ARROW-5077:
-

 Summary: [Rust] Release process should change Cargo.toml to use 
release versions
 Key: ARROW-5077
 URL: https://issues.apache.org/jira/browse/ARROW-5077
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Rust
Affects Versions: 0.13.0
Reporter: Andy Grove
 Fix For: 0.14.0


In the dev tree we use relative path dependencies between arrow, parquet, and 
datafusion, which means we can't just run cargo publish for each crate from the 
release source tarball.

It would be good to have the relaese packaging change the Cargo.toml for 
parquet and datafusion to have dependencies on a versioned release instead of a 
relative path to remove this manual step when publishing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5076) [Packaging] Improve post binary upload performance

2019-03-31 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-5076:
---

 Summary: [Packaging] Improve post binary upload performance
 Key: ARROW-5076
 URL: https://issues.apache.org/jira/browse/ARROW-5076
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-5075) [Release] Add 0.13.0 release note

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-5075:
--
Labels: pull-request-available  (was: )

> [Release] Add 0.13.0 release note
> -
>
> Key: ARROW-5075
> URL: https://issues.apache.org/jira/browse/ARROW-5075
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Website
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5075) [Release] Add 0.13.0 release note

2019-03-31 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-5075:
---

 Summary: [Release] Add 0.13.0 release note
 Key: ARROW-5075
 URL: https://issues.apache.org/jira/browse/ARROW-5075
 Project: Apache Arrow
  Issue Type: Improvement
  Components: Website
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ARROW-5039) [Rust] [DataFusion] Fix bugs in CAST support

2019-03-31 Thread Andy Grove (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Grove resolved ARROW-5039.
---
Resolution: Fixed

Issue resolved by pull request 4054
[https://github.com/apache/arrow/pull/4054]

> [Rust] [DataFusion] Fix bugs in CAST support
> 
>
> Key: ARROW-5039
> URL: https://issues.apache.org/jira/browse/ARROW-5039
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust, Rust - DataFusion
>Affects Versions: 0.13.0
>Reporter: Andy Grove
>Assignee: Andy Grove
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-1918) [JS] Integration portion of verify-release-candidate.sh fails

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-1918:

Fix Version/s: (was: JS-0.5.0)

> [JS] Integration portion of verify-release-candidate.sh fails
> -
>
> Key: ARROW-1918
> URL: https://issues.apache.org/jira/browse/ARROW-1918
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: JavaScript
>Affects Versions: 0.8.0
>Reporter: Wes McKinney
>Assignee: Brian Hulette
>Priority: Major
>
> I'm going to temporarily disable this in my fixes in ARROW-1917



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (ARROW-1918) [JS] Integration portion of verify-release-candidate.sh fails

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou closed ARROW-1918.
---
Resolution: Not A Problem

> [JS] Integration portion of verify-release-candidate.sh fails
> -
>
> Key: ARROW-1918
> URL: https://issues.apache.org/jira/browse/ARROW-1918
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: JavaScript
>Affects Versions: 0.8.0
>Reporter: Wes McKinney
>Assignee: Brian Hulette
>Priority: Major
>
> I'm going to temporarily disable this in my fixes in ARROW-1917



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-1700) [JS] Implement Node.js client for Plasma store

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-1700:

Fix Version/s: (was: JS-0.5.0)
   0.14.0

> [JS] Implement Node.js client for Plasma store
> --
>
> Key: ARROW-1700
> URL: https://issues.apache.org/jira/browse/ARROW-1700
> Project: Apache Arrow
>  Issue Type: New Feature
>  Components: C++ - Plasma, JavaScript
>Reporter: Robert Nishihara
>Priority: Major
> Fix For: 0.14.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-2410) [JS] Add DataFrame.scanAsync

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-2410:

Fix Version/s: (was: JS-0.5.0)
   0.14.0

> [JS] Add DataFrame.scanAsync
> 
>
> Key: ARROW-2410
> URL: https://issues.apache.org/jira/browse/ARROW-2410
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: JavaScript
>Reporter: Brian Hulette
>Priority: Major
> Fix For: 0.14.0
>
>
> Add a version of `DataFrame.scan`, `scanAsync` that yields periodically. The 
> yield frequency could be specified either as a number of record batches, or a 
> number of records.
> This scan should also be cancellable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-2797) [JS] comparison predicates don't work on 64-bit integers

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-2797:

Fix Version/s: (was: JS-0.5.0)
   0.14.0

> [JS] comparison predicates don't work on 64-bit integers
> 
>
> Key: ARROW-2797
> URL: https://issues.apache.org/jira/browse/ARROW-2797
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: JavaScript
>Affects Versions: JS-0.3.1
>Reporter: Brian Hulette
>Assignee: Brian Hulette
>Priority: Major
> Fix For: 0.14.0
>
>
> The 64-bit integer vector {{get}} function returns a 2-element array, which 
> doesn't compare propery in the comparison predicates. We should special case 
> the comparisons for 64-bit integers and timestamps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-3667) [JS] Incorrectly reads record batches with an all null column

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-3667:

Fix Version/s: (was: JS-0.5.0)

> [JS] Incorrectly reads record batches with an all null column
> -
>
> Key: ARROW-3667
> URL: https://issues.apache.org/jira/browse/ARROW-3667
> Project: Apache Arrow
>  Issue Type: Bug
>Affects Versions: JS-0.3.1
>Reporter: Brian Hulette
>Assignee: Paul Taylor
>Priority: Major
> Fix For: JS-0.4.1
>
>
> The JS library seems to incorrectly read any columns that come after an 
> all-null column in IPC buffers produced by pyarrow.
> Here's a python script that generates two arrow buffers, one with an all-null 
> column followed by a utf-8 column, and a second with those two reversed
> {code:python}
> import pyarrow as pa
> import pandas as pd
> def serialize_to_arrow(df, fd, compress=True):
>   batch = pa.RecordBatch.from_pandas(df)
>   writer = pa.RecordBatchFileWriter(fd, batch.schema)
>   writer.write_batch(batch)
>   writer.close()
> if __name__ == "__main__":
> df = pd.DataFrame(data={'nulls': [None, None, None], 'not nulls': ['abc', 
> 'def', 'ghi']}, columns=['nulls', 'not nulls'])
> with open('bad.arrow', 'wb') as fd:
> serialize_to_arrow(df, fd)
> df = pd.DataFrame(df, columns=['not nulls', 'nulls'])
> with open('good.arrow', 'wb') as fd:
> serialize_to_arrow(df, fd)
> {code}
> JS incorrectly interprets the [null, not null] case:
> {code:javascript}
> > var arrow = require('apache-arrow')
> undefined
> > var fs = require('fs')
> undefined
> > arrow.Table.from(fs.readFileSync('good.arrow')).getColumn('not 
> > nulls').get(0)
> 'abc'
> > arrow.Table.from(fs.readFileSync('bad.arrow')).getColumn('not nulls').get(0)
> '\u\u\u\u\u0003\u\u\u\u0006\u\u\u\t\u\u\u'
> {code}
> Presumably this is because pyarrow is omitting some (or all) of the buffers 
> associated with the all-null column, but the JS IPC reader is still looking 
> for them, causing the buffer count to get out of sync.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-3523) [JS] Assign dictionary IDs in IPC writer rather than on creation

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-3523:

Fix Version/s: (was: JS-0.5.0)
   0.14.0

> [JS] Assign dictionary IDs in IPC writer rather than on creation
> 
>
> Key: ARROW-3523
> URL: https://issues.apache.org/jira/browse/ARROW-3523
> Project: Apache Arrow
>  Issue Type: Improvement
>Reporter: Brian Hulette
>Priority: Major
> Fix For: 0.14.0
>
>
>  Currently the JS implementation relies on on the user assigning IDs for 
> dictionaries that they create, we should do something like the C++ 
> implementation, which uses a dictionary id memo to assign and retrieve 
> dictionary ids in the IPC writer 
> (https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/metadata-internal.cc#L495).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Reopened] (ARROW-1918) [JS] Integration portion of verify-release-candidate.sh fails

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou reopened ARROW-1918:
-

> [JS] Integration portion of verify-release-candidate.sh fails
> -
>
> Key: ARROW-1918
> URL: https://issues.apache.org/jira/browse/ARROW-1918
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: JavaScript
>Affects Versions: 0.8.0
>Reporter: Wes McKinney
>Assignee: Brian Hulette
>Priority: Major
> Fix For: JS-0.5.0
>
>
> I'm going to temporarily disable this in my fixes in ARROW-1917



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-951) [JS] Fix generated API documentation

2019-03-31 Thread Kouhei Sutou (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kouhei Sutou updated ARROW-951:
---
Fix Version/s: (was: JS-0.5.0)
   0.14.0

> [JS] Fix generated API documentation
> 
>
> Key: ARROW-951
> URL: https://issues.apache.org/jira/browse/ARROW-951
> Project: Apache Arrow
>  Issue Type: Task
>  Components: JavaScript
>Reporter: Brian Hulette
>Priority: Minor
>  Labels: documentation
> Fix For: 0.14.0
>
>
> The current generated API documentation doesn't respect the project's 
> namespaces, it simply lists all exported objects. We should see if we can 
> make typedoc display the project's structure (even if it means re-structuring 
> the code a bit), or find another approach for doc generation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ARROW-5074) [C++/Python] When installing into a SYSTEM prefix, RPATHs are not correctly set

2019-03-31 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-5074:
--

 Summary: [C++/Python] When installing into a SYSTEM prefix, RPATHs 
are not correctly set
 Key: ARROW-5074
 URL: https://issues.apache.org/jira/browse/ARROW-5074
 Project: Apache Arrow
  Issue Type: Bug
  Components: C++, Packaging, Python
Reporter: Uwe L. Korn


When installing the Arrow libraries into a system with a prefix (mostly a conda 
env), the RPATHs are not correctly set by CMake (there is no RPATH). Thus we 
need to use {{LD_LIBRARY_PATH}} in consumers. When packages are built using 
{{conda-build}}, this takes cares of that in its post-processing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-5056) [Packaging] Adjust conda recipes to use ORC conda-forge package on unix systems

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-5056:
--
Labels: pull-request-available  (was: )

> [Packaging] Adjust conda recipes to use ORC conda-forge package on unix 
> systems 
> 
>
> Key: ARROW-5056
> URL: https://issues.apache.org/jira/browse/ARROW-5056
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Packaging
>Reporter: Krisztian Szucs
>Assignee: Krisztian Szucs
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
>
> Instead of building orc_ep.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-465) [C++] Investigate usage of madvise

2019-03-31 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806206#comment-16806206
 ] 

Antoine Pitrou commented on ARROW-465:
--

Does the OS make the allocation in the background using a separate thread? 
Otherwise, it seems you're just moving the latency around without hiding it.

> [C++] Investigate usage of madvise 
> ---
>
> Key: ARROW-465
> URL: https://issues.apache.org/jira/browse/ARROW-465
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: C++
>Reporter: Uwe L. Korn
>Priority: Major
> Fix For: 0.14.0
>
>
> In some usecases (e.g. Pandas->Arrow conversion) our main constraint is page 
> faulting not yet accessed pages. 
> With {{madvise}} we can indicate our planned actions to the OS and may 
> improve the performance a bit in these cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-465) [C++] Investigate usage of madvise

2019-03-31 Thread Uwe L. Korn (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806204#comment-16806204
 ] 

Uwe L. Korn commented on ARROW-465:
---

The context of this ticket was that I was browsing the {{jemalloc}} source code 
and performance traces. We spent a lot time during the builders with repeately 
allocating new pages when writing to a newly allocated memory segment. With 
specifying {{MADV_WILLNEED}} we might reduce the time we wait for new pages. In 
the end, I want to the OS to allocate all pages I have requested with 
{{(je_)malloc}} immediately and not every page on first access (when a TLB miss 
occurs).

> [C++] Investigate usage of madvise 
> ---
>
> Key: ARROW-465
> URL: https://issues.apache.org/jira/browse/ARROW-465
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: C++
>Reporter: Uwe L. Korn
>Priority: Major
> Fix For: 0.14.0
>
>
> In some usecases (e.g. Pandas->Arrow conversion) our main constraint is page 
> faulting not yet accessed pages. 
> With {{madvise}} we can indicate our planned actions to the OS and may 
> improve the performance a bit in these cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ARROW-3680) [Go] implement Float16 array

2019-03-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ARROW-3680:
--
Labels: pull-request-available  (was: )

> [Go] implement Float16 array
> 
>
> Key: ARROW-3680
> URL: https://issues.apache.org/jira/browse/ARROW-3680
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Go
>Reporter: Sebastien Binet
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-2938) [Packaging] Make the source release via crossbow

2019-03-31 Thread Krisztian Szucs (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806075#comment-16806075
 ] 

Krisztian Szucs commented on ARROW-2938:


Agree, closing.

> [Packaging] Make the source release via crossbow
> 
>
> Key: ARROW-2938
> URL: https://issues.apache.org/jira/browse/ARROW-2938
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Packaging
>Reporter: Krisztian Szucs
>Assignee: Krisztian Szucs
>Priority: Major
>
> And make it possible to upload source distribution (signature and checksums 
> as well) to github releases. This will make ARROW-2910 testable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (ARROW-2938) [Packaging] Make the source release via crossbow

2019-03-31 Thread Krisztian Szucs (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARROW-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Szucs closed ARROW-2938.
--
Resolution: Won't Fix

> [Packaging] Make the source release via crossbow
> 
>
> Key: ARROW-2938
> URL: https://issues.apache.org/jira/browse/ARROW-2938
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Packaging
>Reporter: Krisztian Szucs
>Assignee: Krisztian Szucs
>Priority: Major
>
> And make it possible to upload source distribution (signature and checksums 
> as well) to github releases. This will make ARROW-2910 testable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)