[jira] [Comment Edited] (ARROW-4144) [Java] Arrow-to-JDBC
[ https://issues.apache.org/jira/browse/ARROW-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123402#comment-17123402 ] Micah Kornfield edited comment on ARROW-4144 at 6/2/20, 6:56 AM: - [~uwe] have you come across a use-case for writing to JDBC sources? was (Author: emkornfi...@gmail.com): @uwe have you come across a use-case for writing to JDBC sources? > [Java] Arrow-to-JDBC > > > Key: ARROW-4144 > URL: https://issues.apache.org/jira/browse/ARROW-4144 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Michael Pigott >Assignee: Chen >Priority: Major > > ARROW-1780 reads a query from a JDBC data source and converts the ResultSet > to an Arrow VectorSchemaRoot. However, there is no built-in adapter for > writing an Arrow VectorSchemaRoot back to the database. > ARROW-3966 adds JDBC field metadata: > * The Catalog Name > * The Table Name > * The Field Name > * The Field Type > We can use this information to ask for the field information from the > database via the > [DatabaseMetaData|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html] > object. We can then create INSERT or UPDATE statements based on the [list > of primary > keys|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)] > in the table: > * If the value in the VectorSchemaRoot corresponding to the primary key is > NULL, insert that record into the database. > * If the value in the VectorSchemaRoot corresponding to the primary key is > not NULL, update the existing record in the database. > We can also perform the same data conversion in reverse based on the field > types queried from the database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-4144) [Java] Arrow-to-JDBC
[ https://issues.apache.org/jira/browse/ARROW-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123402#comment-17123402 ] Micah Kornfield commented on ARROW-4144: @uwe have you come across a use-case for writing to JDBC sources? > [Java] Arrow-to-JDBC > > > Key: ARROW-4144 > URL: https://issues.apache.org/jira/browse/ARROW-4144 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Michael Pigott >Assignee: Chen >Priority: Major > > ARROW-1780 reads a query from a JDBC data source and converts the ResultSet > to an Arrow VectorSchemaRoot. However, there is no built-in adapter for > writing an Arrow VectorSchemaRoot back to the database. > ARROW-3966 adds JDBC field metadata: > * The Catalog Name > * The Table Name > * The Field Name > * The Field Type > We can use this information to ask for the field information from the > database via the > [DatabaseMetaData|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html] > object. We can then create INSERT or UPDATE statements based on the [list > of primary > keys|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)] > in the table: > * If the value in the VectorSchemaRoot corresponding to the primary key is > NULL, insert that record into the database. > * If the value in the VectorSchemaRoot corresponding to the primary key is > not NULL, update the existing record in the database. > We can also perform the same data conversion in reverse based on the field > types queried from the database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8972) [Java] Support range value comparison for large varchar/varbinary vectors
[ https://issues.apache.org/jira/browse/ARROW-8972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micah Kornfield resolved ARROW-8972. Resolution: Fixed > [Java] Support range value comparison for large varchar/varbinary vectors > - > > Key: ARROW-8972 > URL: https://issues.apache.org/jira/browse/ARROW-8972 > Project: Apache Arrow > Issue Type: New Feature > Components: Java >Reporter: Liya Fan >Assignee: Liya Fan >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Support comparing a range of values for LargeVarCharVector and > LargeVarBinaryVector. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9000) Java build crashes with JDK14
[
https://issues.apache.org/jira/browse/ARROW-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Micah Kornfield updated ARROW-9000:
---
Component/s: Java
> Java build crashes with JDK14
> -
>
> Key: ARROW-9000
> URL: https://issues.apache.org/jira/browse/ARROW-9000
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Current master tree does not build with JDK14. The issue seems to be caused
> by error prone plugin:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project arrow-memory: Compilation failure
> [ERROR]
> /Users/laurent/devel/arrow/java/memory/src/main/java/org/apache/arrow/memory/BufferLedger.java:[545,15]
> error: An unhandled exception was thrown by the Error Prone static analysis
> plugin.
> [ERROR] Please report this at
> https://github.com/google/error-prone/issues/new and include the following:
> [ERROR]
> [ERROR] error-prone version: 2.3.3
> [ERROR] BugPattern: TypeParameterUnusedInFormals
> [ERROR] Stack Trace:
> [ERROR] java.lang.NoSuchFieldError: bound
> [ERROR] at
> com.google.errorprone.bugpatterns.TypeParameterUnusedInFormals.matchMethod(TypeParameterUnusedInFormals.java:71)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:433)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:725)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:916)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:535)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:823)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:546)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:603)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:55)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
> [ERROR] at
> com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:151)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1423)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1370)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:316)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
> [ERROR] at jdk.compiler/com.sun.tools
[jira] [Updated] (ARROW-9000) [Java] build crashes with JDK14
[
https://issues.apache.org/jira/browse/ARROW-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Micah Kornfield updated ARROW-9000:
---
Summary: [Java] build crashes with JDK14 (was: Java build crashes with
JDK14)
> [Java] build crashes with JDK14
> ---
>
> Key: ARROW-9000
> URL: https://issues.apache.org/jira/browse/ARROW-9000
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Current master tree does not build with JDK14. The issue seems to be caused
> by error prone plugin:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project arrow-memory: Compilation failure
> [ERROR]
> /Users/laurent/devel/arrow/java/memory/src/main/java/org/apache/arrow/memory/BufferLedger.java:[545,15]
> error: An unhandled exception was thrown by the Error Prone static analysis
> plugin.
> [ERROR] Please report this at
> https://github.com/google/error-prone/issues/new and include the following:
> [ERROR]
> [ERROR] error-prone version: 2.3.3
> [ERROR] BugPattern: TypeParameterUnusedInFormals
> [ERROR] Stack Trace:
> [ERROR] java.lang.NoSuchFieldError: bound
> [ERROR] at
> com.google.errorprone.bugpatterns.TypeParameterUnusedInFormals.matchMethod(TypeParameterUnusedInFormals.java:71)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:433)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:725)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:916)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:535)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:823)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:546)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:603)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:55)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
> [ERROR] at
> com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:151)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1423)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1370)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:316)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.co
[jira] [Resolved] (ARROW-9000) [Java] build crashes with JDK14
[
https://issues.apache.org/jira/browse/ARROW-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Micah Kornfield resolved ARROW-9000.
Resolution: Fixed
> [Java] build crashes with JDK14
> ---
>
> Key: ARROW-9000
> URL: https://issues.apache.org/jira/browse/ARROW-9000
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Current master tree does not build with JDK14. The issue seems to be caused
> by error prone plugin:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project arrow-memory: Compilation failure
> [ERROR]
> /Users/laurent/devel/arrow/java/memory/src/main/java/org/apache/arrow/memory/BufferLedger.java:[545,15]
> error: An unhandled exception was thrown by the Error Prone static analysis
> plugin.
> [ERROR] Please report this at
> https://github.com/google/error-prone/issues/new and include the following:
> [ERROR]
> [ERROR] error-prone version: 2.3.3
> [ERROR] BugPattern: TypeParameterUnusedInFormals
> [ERROR] Stack Trace:
> [ERROR] java.lang.NoSuchFieldError: bound
> [ERROR] at
> com.google.errorprone.bugpatterns.TypeParameterUnusedInFormals.matchMethod(TypeParameterUnusedInFormals.java:71)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:433)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:725)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:916)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:535)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:823)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:546)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:603)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:55)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
> [ERROR] at
> com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:151)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1423)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1370)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:316)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
> [ERROR] at jdk.compiler/com.sun
[jira] [Commented] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121531#comment-17121531 ] Yibo Cai commented on ARROW-8999: - Looks like some hidden bugs like https://github.com/apache/arrow/pull/7123. I met with same error once last week. My patch https://github.com/apache/arrow/pull/7135 handles buffer directly, would double check if all boundary conditions are safe. > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Priority: Major > Fix For: 1.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-4144) [Java] Arrow-to-JDBC
[ https://issues.apache.org/jira/browse/ARROW-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121516#comment-17121516 ] Michael Pigott commented on ARROW-4144: --- Hi, and thanks for taking a look at this! I saw Arrow as a tool that could be used to transfer data between systems, and not just for in-memory analysis. However, I don't need this today, and if nobody else has asked for it, then there might be more valuable JIRA tickets than mine for you to dedicate your time to? Thanks again! Mike > [Java] Arrow-to-JDBC > > > Key: ARROW-4144 > URL: https://issues.apache.org/jira/browse/ARROW-4144 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Michael Pigott >Assignee: Chen >Priority: Major > > ARROW-1780 reads a query from a JDBC data source and converts the ResultSet > to an Arrow VectorSchemaRoot. However, there is no built-in adapter for > writing an Arrow VectorSchemaRoot back to the database. > ARROW-3966 adds JDBC field metadata: > * The Catalog Name > * The Table Name > * The Field Name > * The Field Type > We can use this information to ask for the field information from the > database via the > [DatabaseMetaData|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html] > object. We can then create INSERT or UPDATE statements based on the [list > of primary > keys|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)] > in the table: > * If the value in the VectorSchemaRoot corresponding to the primary key is > NULL, insert that record into the database. > * If the value in the VectorSchemaRoot corresponding to the primary key is > not NULL, update the existing record in the database. > We can also perform the same data conversion in reverse based on the field > types queried from the database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-4144) [Java] Arrow-to-JDBC
[ https://issues.apache.org/jira/browse/ARROW-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen reassigned ARROW-4144: --- Assignee: Chen > [Java] Arrow-to-JDBC > > > Key: ARROW-4144 > URL: https://issues.apache.org/jira/browse/ARROW-4144 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Michael Pigott >Assignee: Chen >Priority: Major > > ARROW-1780 reads a query from a JDBC data source and converts the ResultSet > to an Arrow VectorSchemaRoot. However, there is no built-in adapter for > writing an Arrow VectorSchemaRoot back to the database. > ARROW-3966 adds JDBC field metadata: > * The Catalog Name > * The Table Name > * The Field Name > * The Field Type > We can use this information to ask for the field information from the > database via the > [DatabaseMetaData|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html] > object. We can then create INSERT or UPDATE statements based on the [list > of primary > keys|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)] > in the table: > * If the value in the VectorSchemaRoot corresponding to the primary key is > NULL, insert that record into the database. > * If the value in the VectorSchemaRoot corresponding to the primary key is > not NULL, update the existing record in the database. > We can also perform the same data conversion in reverse based on the field > types queried from the database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8992) [CI][C++] march not passing correctly for docker-compose run
[ https://issues.apache.org/jira/browse/ARROW-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121506#comment-17121506 ] Elliott Kipp commented on ARROW-8992: - I only modified the volumes section of the docker-compose, (per [https://github.com/apache/arrow/pull/6907#issuecomment-612940401]) I didn't apply the rest of the patch. I believe I did a pull of the master at one point too, with the same result, but I'll double check tomorrow. > [CI][C++] march not passing correctly for docker-compose run > > > Key: ARROW-8992 > URL: https://issues.apache.org/jira/browse/ARROW-8992 > Project: Apache Arrow > Issue Type: Bug > Components: Packaging, Python >Affects Versions: 0.17.0, 0.17.1 > Environment: Mendel Linux 4.0 >Reporter: Elliott Kipp >Priority: Critical > Attachments: CMakeError.log, CMakeOutput.log > > > [https://github.com/apache/arrow/issues/7307] > Building on the new ASUS Tinker Edge T, running Mendel Linux 4.0 (Day). > docker-compose build commands work fine with no errors: > DEBIAN=10 ARCH=arm64v8 docker-compose build debian-cpp && DEBIAN=10 > ARCH=arm64v8 docker-compose build debian-python > DEBIAN=10 ARCH=arm64v8 docker-compose run debian-python - fails with the > following: > – Running cmake for pyarrow > cmake -DPYTHON_EXECUTABLE=/usr/local/bin/python -G Ninja > -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=on -DPYARROW_BUILD_GANDIVA=on > -DPYARROW_BUILD_DATASET=on -DPYARROW_BUILD_ORC=on -DPYARROW_BUILD_PARQUET=on > -DPYARROW_BUILD_PLASMA=on -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off > -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off > -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off > -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on > -DCMAKE_BUILD_TYPE=debug /arrow/python > – The C compiler identification is GNU 8.3.0 > – The CXX compiler identification is GNU 8.3.0 > – Check for working C compiler: /usr/lib/ccache/gcc > – Check for working C compiler: /usr/lib/ccache/gcc – works > – Detecting C compiler ABI info > – Detecting C compiler ABI info - done > – Detecting C compile features > – Detecting C compile features - done > – Check for working CXX compiler: /usr/lib/ccache/g++ > – Check for working CXX compiler: /usr/lib/ccache/g++ – works > – Detecting CXX compiler ABI info > – Detecting CXX compiler ABI info - done > – Detecting CXX compile features > – Detecting CXX compile features - done > – System processor: aarch64 > – Performing Test CXX_SUPPORTS_ARMV8_ARCH > – Performing Test CXX_SUPPORTS_ARMV8_ARCH - Failed > – Arrow build warning level: PRODUCTION > CMake Error at cmake_modules/SetupCxxFlags.cmake:338 (message): > Unsupported arch flag: -march=. > Call Stack (most recent call first): > CMakeLists.txt:100 (include) > – Configuring incomplete, errors occurred! > See also "/build/python/temp.linux-aarch64-3.7/CMakeFiles/CMakeOutput.log". > See also "/build/python/temp.linux-aarch64-3.7/CMakeFiles/CMakeError.log". > error: command 'cmake' failed with exit status 1 > Tried the tarball release for both 0.17.0 and 0.17.1, same result. Also tried > compiling manually (following these instructions: > [https://dzone.com/articles/building-pyarrow-with-cuda-support]) with the > same result. > Only modifications I made to source are editing the docker-compose volumes, > as described here: [https://github.com/apache/arrow/pull/6907] > Jira opened, per request at: [https://github.com/apache/arrow/issues/7307] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-4144) [Java] Arrow-to-JDBC
[ https://issues.apache.org/jira/browse/ARROW-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121502#comment-17121502 ] Chen commented on ARROW-4144: - is it necessary to do it,?Arrow is designed for in-memory data analysis, the data won't be modified. recently i pay much attention on Arrow, i want to make a contribution to the project. if this issue is necessary, i am happy to resolve it. > [Java] Arrow-to-JDBC > > > Key: ARROW-4144 > URL: https://issues.apache.org/jira/browse/ARROW-4144 > Project: Apache Arrow > Issue Type: Improvement > Components: Java >Reporter: Michael Pigott >Priority: Major > > ARROW-1780 reads a query from a JDBC data source and converts the ResultSet > to an Arrow VectorSchemaRoot. However, there is no built-in adapter for > writing an Arrow VectorSchemaRoot back to the database. > ARROW-3966 adds JDBC field metadata: > * The Catalog Name > * The Table Name > * The Field Name > * The Field Type > We can use this information to ask for the field information from the > database via the > [DatabaseMetaData|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html] > object. We can then create INSERT or UPDATE statements based on the [list > of primary > keys|https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)] > in the table: > * If the value in the VectorSchemaRoot corresponding to the primary key is > NULL, insert that record into the database. > * If the value in the VectorSchemaRoot corresponding to the primary key is > not NULL, update the existing record in the database. > We can also perform the same data conversion in reverse based on the field > types queried from the database. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (ARROW-4530) [C++] Review Aggregate kernel state allocation/ownership semantics
[ https://issues.apache.org/jira/browse/ARROW-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney closed ARROW-4530. --- Resolution: Later > [C++] Review Aggregate kernel state allocation/ownership semantics > -- > > Key: ARROW-4530 > URL: https://issues.apache.org/jira/browse/ARROW-4530 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (ARROW-8955) [C++] Use kernels for casting Scalar values instead of bespoke implementation
[ https://issues.apache.org/jira/browse/ARROW-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney closed ARROW-8955. --- Fix Version/s: (was: 1.0.0) Resolution: Duplicate duplicate of ARROW-9006 > [C++] Use kernels for casting Scalar values instead of bespoke implementation > - > > Key: ARROW-8955 > URL: https://issues.apache.org/jira/browse/ARROW-8955 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Priority: Major > > See details of casting in arrow/scalar.cc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8844) [C++] Optimize TransferBitmap unaligned case
[ https://issues.apache.org/jira/browse/ARROW-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-8844. - Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 7300 [https://github.com/apache/arrow/pull/7300] > [C++] Optimize TransferBitmap unaligned case > > > Key: ARROW-8844 > URL: https://issues.apache.org/jira/browse/ARROW-8844 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Yibo Cai >Assignee: Yibo Cai >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > TransferBitmap(CopyBitmap, InvertBitmap) unaligned case is processed > bit-by-bit[1]. Similar trick in this PR[2] may also be helpful here to > improve performance by processing in words. > [1] > https://github.com/apache/arrow/blob/e5a33f1220705aec6a224b55d2a6f47fbd957603/cpp/src/arrow/util/bit_util.cc#L121-L134 > [2] https://github.com/apache/arrow/pull/7135 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9006) [C++] Use Cast kernels to implement Scalar::Parse and Scalar::CastTo
Wes McKinney created ARROW-9006: --- Summary: [C++] Use Cast kernels to implement Scalar::Parse and Scalar::CastTo Key: ARROW-9006 URL: https://issues.apache.org/jira/browse/ARROW-9006 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Wes McKinney Fix For: 1.0.0 We should not maintain distinct (and possibly differently behaving) implementations of elementwise array casting and scalar casting. The new kernels framework provides for relatively easily generating kernels that can process arrays or scalars. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8929) [C++] Change compute::Arity:VarArgs min_args default to 0
[
https://issues.apache.org/jira/browse/ARROW-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney resolved ARROW-8929.
-
Resolution: Fixed
Issue resolved by pull request 7322
[https://github.com/apache/arrow/pull/7322]
> [C++] Change compute::Arity:VarArgs min_args default to 0
> -
>
> Key: ARROW-8929
> URL: https://issues.apache.org/jira/browse/ARROW-8929
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The issue of minimum number of arguments is separate from providing an
> {{InputType}} for input type checking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-9005) Support sort expression
[ https://issues.apache.org/jira/browse/ARROW-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9005: -- Labels: pull-request-available (was: ) > Support sort expression > --- > > Key: ARROW-9005 > URL: https://issues.apache.org/jira/browse/ARROW-9005 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust - DataFusion >Reporter: QP Hou >Assignee: QP Hou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9005) Support sort expression
QP Hou created ARROW-9005: - Summary: Support sort expression Key: ARROW-9005 URL: https://issues.apache.org/jira/browse/ARROW-9005 Project: Apache Arrow Issue Type: Improvement Components: Rust - DataFusion Reporter: QP Hou Assignee: QP Hou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9004) [C++][Gandiva] Upgrade to LLVM 10
[ https://issues.apache.org/jira/browse/ARROW-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9004: -- Labels: pull-request-available (was: ) > [C++][Gandiva] Upgrade to LLVM 10 > - > > Key: ARROW-9004 > URL: https://issues.apache.org/jira/browse/ARROW-9004 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ - Gandiva >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9004) [C++][Gandiva] Upgrade to LLVM 10
Kouhei Sutou created ARROW-9004: --- Summary: [C++][Gandiva] Upgrade to LLVM 10 Key: ARROW-9004 URL: https://issues.apache.org/jira/browse/ARROW-9004 Project: Apache Arrow Issue Type: Improvement Components: C++ - Gandiva Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-3520) [C++] Implement List Flatten kernel
[ https://issues.apache.org/jira/browse/ARROW-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-3520: --- Assignee: Wes McKinney > [C++] Implement List Flatten kernel > --- > > Key: ARROW-3520 > URL: https://issues.apache.org/jira/browse/ARROW-3520 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Fix For: 2.0.0 > > > see also ARROW-45 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8929) [C++] Change compute::Arity:VarArgs min_args default to 0
[
https://issues.apache.org/jira/browse/ARROW-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-8929:
--
Labels: pull-request-available (was: )
> [C++] Change compute::Arity:VarArgs min_args default to 0
> -
>
> Key: ARROW-8929
> URL: https://issues.apache.org/jira/browse/ARROW-8929
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The issue of minimum number of arguments is separate from providing an
> {{InputType}} for input type checking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9003) [C++] Add VectorFunction wrapping arrow::Concatenate
Wes McKinney created ARROW-9003: --- Summary: [C++] Add VectorFunction wrapping arrow::Concatenate Key: ARROW-9003 URL: https://issues.apache.org/jira/browse/ARROW-9003 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Wes McKinney Fix For: 1.0.0 This would be a varargs function -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8929) [C++] Change compute::Arity:VarArgs min_args default to 0
[
https://issues.apache.org/jira/browse/ARROW-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney reassigned ARROW-8929:
---
Assignee: Wes McKinney
> [C++] Change compute::Arity:VarArgs min_args default to 0
> -
>
> Key: ARROW-8929
> URL: https://issues.apache.org/jira/browse/ARROW-8929
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Fix For: 1.0.0
>
>
> The issue of minimum number of arguments is separate from providing an
> {{InputType}} for input type checking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-8985) [Format] Add "byte width" field with default of 16 to Decimal Flatbuffers type for forward compatibility
[ https://issues.apache.org/jira/browse/ARROW-8985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8985: -- Labels: pull-request-available (was: ) > [Format] Add "byte width" field with default of 16 to Decimal Flatbuffers > type for forward compatibility > > > Key: ARROW-8985 > URL: https://issues.apache.org/jira/browse/ARROW-8985 > Project: Apache Arrow > Issue Type: Improvement > Components: Format >Reporter: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > This will permit larger or smaller decimals to be added to the format later > without having to add a new Type union value -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8896) [C++] Reimplement dictionary unpacking in Cast kernels using Take
[ https://issues.apache.org/jira/browse/ARROW-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8896: -- Labels: pull-request-available (was: ) > [C++] Reimplement dictionary unpacking in Cast kernels using Take > - > > Key: ARROW-8896 > URL: https://issues.apache.org/jira/browse/ARROW-8896 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > As suggested by [~apitrou] this should yield less code to maintain -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8896) [C++] Reimplement dictionary unpacking in Cast kernels using Take
[ https://issues.apache.org/jira/browse/ARROW-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-8896: --- Assignee: Wes McKinney > [C++] Reimplement dictionary unpacking in Cast kernels using Take > - > > Key: ARROW-8896 > URL: https://issues.apache.org/jira/browse/ARROW-8896 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Fix For: 1.0.0 > > > As suggested by [~apitrou] this should yield less code to maintain -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9002) [C++] Unable to load libjvm on ppc64le architecture for hdfs.connect()
Yuan Cheng created ARROW-9002: - Summary: [C++] Unable to load libjvm on ppc64le architecture for hdfs.connect() Key: ARROW-9002 URL: https://issues.apache.org/jira/browse/ARROW-9002 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.17.1 Environment: Ubuntu 16.04.4 LTS Reporter: Yuan Cheng Attachments: patch.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8931) [Rust] Support lexical sort in arrow compute kernel
[ https://issues.apache.org/jira/browse/ARROW-8931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neville Dipale resolved ARROW-8931. --- Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 7265 [https://github.com/apache/arrow/pull/7265] > [Rust] Support lexical sort in arrow compute kernel > --- > > Key: ARROW-8931 > URL: https://issues.apache.org/jira/browse/ARROW-8931 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: QP Hou >Assignee: QP Hou >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8917) [C++][Compute] Formalize "metafunction" concept
[
https://issues.apache.org/jira/browse/ARROW-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-8917:
--
Labels: pull-request-available (was: )
> [C++][Compute] Formalize "metafunction" concept
> ---
>
> Key: ARROW-8917
> URL: https://issues.apache.org/jira/browse/ARROW-8917
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> A metafunction is a function that provides the {{Execute}} API but does not
> contain any kernels. Such functions can also handle non-Array/Scalar inputs
> like RecordBatch or Table.
> This will enable bindings to invoke such functions (like take, filter) like
> {code}
> call_function('take', [table, indices])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9001) [R] Box outputs as correct type in call_function
Wes McKinney created ARROW-9001: --- Summary: [R] Box outputs as correct type in call_function Key: ARROW-9001 URL: https://issues.apache.org/jira/browse/ARROW-9001 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Wes McKinney Fix For: 1.0.0 This would prevent segfaults by putting the SEXP in the wrong kind of R6 container -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8843) [C++] Optimize BitmapEquals unaligned case
[ https://issues.apache.org/jira/browse/ARROW-8843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques resolved ARROW-8843. --- Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 7285 [https://github.com/apache/arrow/pull/7285] > [C++] Optimize BitmapEquals unaligned case > -- > > Key: ARROW-8843 > URL: https://issues.apache.org/jira/browse/ARROW-8843 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Yibo Cai >Assignee: Yibo Cai >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > BitmapEquals unaligned case compares two bitmap bit-by-bit[1]. Similar tricks > in this PR[2] may also be helpful here to improve performance by processing > in words. > [1] > https://github.com/apache/arrow/blob/e5a33f1220705aec6a224b55d2a6f47fbd957603/cpp/src/arrow/util/bit_util.cc#L248-L254 > [2] https://github.com/apache/arrow/pull/7135 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8485) [Integration][Java] Implement extension types integration
[ https://issues.apache.org/jira/browse/ARROW-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Li resolved ARROW-8485. - Resolution: Fixed Issue resolved by pull request 7270 [https://github.com/apache/arrow/pull/7270] > [Integration][Java] Implement extension types integration > - > > Key: ARROW-8485 > URL: https://issues.apache.org/jira/browse/ARROW-8485 > Project: Apache Arrow > Issue Type: Improvement > Components: Integration, Java >Reporter: Antoine Pitrou >Assignee: Ryan Murray >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Java should support the extension type integration tests added in ARROW-5649. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques resolved ARROW-8997. --- Fix Version/s: 1.0.0 Resolution: Fixed Issue resolved by pull request 7316 [https://github.com/apache/arrow/pull/7316] > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Improvement > Components: Archery >Reporter: Francois Saint-Jacques >Assignee: Francois Saint-Jacques >Priority: Minor > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9000) Java build crashes with JDK14
[
https://issues.apache.org/jira/browse/ARROW-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-9000:
--
Labels: pull-request-available (was: )
> Java build crashes with JDK14
> -
>
> Key: ARROW-9000
> URL: https://issues.apache.org/jira/browse/ARROW-9000
> Project: Apache Arrow
> Issue Type: Bug
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Current master tree does not build with JDK14. The issue seems to be caused
> by error prone plugin:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project arrow-memory: Compilation failure
> [ERROR]
> /Users/laurent/devel/arrow/java/memory/src/main/java/org/apache/arrow/memory/BufferLedger.java:[545,15]
> error: An unhandled exception was thrown by the Error Prone static analysis
> plugin.
> [ERROR] Please report this at
> https://github.com/google/error-prone/issues/new and include the following:
> [ERROR]
> [ERROR] error-prone version: 2.3.3
> [ERROR] BugPattern: TypeParameterUnusedInFormals
> [ERROR] Stack Trace:
> [ERROR] java.lang.NoSuchFieldError: bound
> [ERROR] at
> com.google.errorprone.bugpatterns.TypeParameterUnusedInFormals.matchMethod(TypeParameterUnusedInFormals.java:71)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:433)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:725)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:916)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:535)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:823)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:546)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:150)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:603)
> [ERROR] at
> jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
> [ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:55)
> [ERROR] at
> com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
> [ERROR] at
> com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:151)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1423)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1370)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:316)
> [ERROR] at
> jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
> [ERROR] at jdk.compiler/com.sun.tools.javac.Ma
[jira] [Created] (ARROW-9000) Java build crashes with JDK14
Laurent Goujon created ARROW-9000:
-
Summary: Java build crashes with JDK14
Key: ARROW-9000
URL: https://issues.apache.org/jira/browse/ARROW-9000
Project: Apache Arrow
Issue Type: Bug
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Current master tree does not build with JDK14. The issue seems to be caused by
error prone plugin:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile (default-compile)
on project arrow-memory: Compilation failure
[ERROR]
/Users/laurent/devel/arrow/java/memory/src/main/java/org/apache/arrow/memory/BufferLedger.java:[545,15]
error: An unhandled exception was thrown by the Error Prone static analysis
plugin.
[ERROR] Please report this at
https://github.com/google/error-prone/issues/new and include the following:
[ERROR]
[ERROR] error-prone version: 2.3.3
[ERROR] BugPattern: TypeParameterUnusedInFormals
[ERROR] Stack Trace:
[ERROR] java.lang.NoSuchFieldError: bound
[ERROR] at
com.google.errorprone.bugpatterns.TypeParameterUnusedInFormals.matchMethod(TypeParameterUnusedInFormals.java:71)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:433)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:725)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:150)
[ERROR] at
jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:916)
[ERROR] at
jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:535)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:150)
[ERROR] at
jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:823)
[ERROR] at
jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:71)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:45)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
[ERROR] at
jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:546)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:150)
[ERROR] at
jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:603)
[ERROR] at
jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:55)
[ERROR] at
com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
[ERROR] at
com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:151)
[ERROR] at
jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
[ERROR] at
jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1423)
[ERROR] at
jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1370)
[ERROR] at
jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
[ERROR] at
jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:316)
[ERROR] at
jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
[ERROR] at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:57)
[ERROR] at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:43)
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[jira] [Resolved] (ARROW-7784) [C++] diff.cc is extremely slow to compile
[
https://issues.apache.org/jira/browse/ARROW-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Kietzman resolved ARROW-7784.
-
Resolution: Fixed
Issue resolved by pull request 7311
[https://github.com/apache/arrow/pull/7311]
> [C++] diff.cc is extremely slow to compile
> --
>
> Key: ARROW-7784
> URL: https://issues.apache.org/jira/browse/ARROW-7784
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Antoine Pitrou
>Assignee: Wes McKinney
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> This comes up especially when doing an optimized build. {{diff.cc}} is always
> enabled even if all components are disabled, and it takes multiple seconds to
> compile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-8990) [C++] Benchmark hash table against thirdparty options, possibly vendor a thirdparty hash table library
[ https://issues.apache.org/jira/browse/ARROW-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121205#comment-17121205 ] Maarten Breddels commented on ARROW-8990: - FYI, I've been using that library and [https://github.com/skarupke/flat_hash_map] for Vaex. After some benchmarking settled for the tsl one, but my research/benchmark wasn't very thorough, because the idea was I could easily switch if needed. But because the performance was great, I never looked back actually, so I'd be interested in the benchmark result. By the same author, the [https://github.com/Tessil/hat-trie] library can also be very interesting to take a look at. > [C++] Benchmark hash table against thirdparty options, possibly vendor a > thirdparty hash table library > -- > > Key: ARROW-8990 > URL: https://issues.apache.org/jira/browse/ARROW-8990 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Priority: Major > > While we have our own hash table implementation, it would be worthwhile to > set up some benchmarks so that we can compare against std::unordered_map and > some other thirdparty libraries for hash tables to know whether we should > possibly use a thirdparty library. See e.g. > https://tessil.github.io/2016/08/29/benchmark-hopscotch-map.html > Libraries to consider: > * https://github.com/sparsehash/sparsehash -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8997: -- Labels: pull-request-available (was: ) > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Improvement > Components: Archery >Reporter: Francois Saint-Jacques >Assignee: Francois Saint-Jacques >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques reassigned ARROW-8997: - Assignee: Francois Saint-Jacques > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Improvement > Components: Archery >Reporter: Francois Saint-Jacques >Assignee: Francois Saint-Jacques >Priority: Minor > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-7009) [C++] Refactor filter/take kernels to use Datum instead of overloads
[ https://issues.apache.org/jira/browse/ARROW-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney reassigned ARROW-7009: --- Assignee: Wes McKinney (was: Ben Kietzman) > [C++] Refactor filter/take kernels to use Datum instead of overloads > > > Key: ARROW-7009 > URL: https://issues.apache.org/jira/browse/ARROW-7009 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ >Reporter: Neal Richardson >Assignee: Wes McKinney >Priority: Minor > Fix For: 1.0.0 > > > Followup to ARROW-6784. See discussion on > [https://github.com/apache/arrow/pull/5686,|https://github.com/apache/arrow/pull/5686] > as well as ARROW-6959. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8917) [C++][Compute] Formalize "metafunction" concept
[
https://issues.apache.org/jira/browse/ARROW-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney reassigned ARROW-8917:
---
Assignee: Wes McKinney
> [C++][Compute] Formalize "metafunction" concept
> ---
>
> Key: ARROW-8917
> URL: https://issues.apache.org/jira/browse/ARROW-8917
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Wes McKinney
>Priority: Major
> Fix For: 1.0.0
>
>
> A metafunction is a function that provides the {{Execute}} API but does not
> contain any kernels. Such functions can also handle non-Array/Scalar inputs
> like RecordBatch or Table.
> This will enable bindings to invoke such functions (like take, filter) like
> {code}
> call_function('take', [table, indices])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-8992) [CI][C++] march not passing correctly for docker-compose run
[ https://issues.apache.org/jira/browse/ARROW-8992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-8992: Summary: [CI][C++] march not passing correctly for docker-compose run (was: march not passing correctly for docker-compose run) > [CI][C++] march not passing correctly for docker-compose run > > > Key: ARROW-8992 > URL: https://issues.apache.org/jira/browse/ARROW-8992 > Project: Apache Arrow > Issue Type: Bug > Components: Packaging, Python >Affects Versions: 0.17.0, 0.17.1 > Environment: Mendel Linux 4.0 >Reporter: Elliott Kipp >Priority: Critical > Attachments: CMakeError.log, CMakeOutput.log > > > [https://github.com/apache/arrow/issues/7307] > Building on the new ASUS Tinker Edge T, running Mendel Linux 4.0 (Day). > docker-compose build commands work fine with no errors: > DEBIAN=10 ARCH=arm64v8 docker-compose build debian-cpp && DEBIAN=10 > ARCH=arm64v8 docker-compose build debian-python > DEBIAN=10 ARCH=arm64v8 docker-compose run debian-python - fails with the > following: > – Running cmake for pyarrow > cmake -DPYTHON_EXECUTABLE=/usr/local/bin/python -G Ninja > -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=on -DPYARROW_BUILD_GANDIVA=on > -DPYARROW_BUILD_DATASET=on -DPYARROW_BUILD_ORC=on -DPYARROW_BUILD_PARQUET=on > -DPYARROW_BUILD_PLASMA=on -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off > -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off > -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off > -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on > -DCMAKE_BUILD_TYPE=debug /arrow/python > – The C compiler identification is GNU 8.3.0 > – The CXX compiler identification is GNU 8.3.0 > – Check for working C compiler: /usr/lib/ccache/gcc > – Check for working C compiler: /usr/lib/ccache/gcc – works > – Detecting C compiler ABI info > – Detecting C compiler ABI info - done > – Detecting C compile features > – Detecting C compile features - done > – Check for working CXX compiler: /usr/lib/ccache/g++ > – Check for working CXX compiler: /usr/lib/ccache/g++ – works > – Detecting CXX compiler ABI info > – Detecting CXX compiler ABI info - done > – Detecting CXX compile features > – Detecting CXX compile features - done > – System processor: aarch64 > – Performing Test CXX_SUPPORTS_ARMV8_ARCH > – Performing Test CXX_SUPPORTS_ARMV8_ARCH - Failed > – Arrow build warning level: PRODUCTION > CMake Error at cmake_modules/SetupCxxFlags.cmake:338 (message): > Unsupported arch flag: -march=. > Call Stack (most recent call first): > CMakeLists.txt:100 (include) > – Configuring incomplete, errors occurred! > See also "/build/python/temp.linux-aarch64-3.7/CMakeFiles/CMakeOutput.log". > See also "/build/python/temp.linux-aarch64-3.7/CMakeFiles/CMakeError.log". > error: command 'cmake' failed with exit status 1 > Tried the tarball release for both 0.17.0 and 0.17.1, same result. Also tried > compiling manually (following these instructions: > [https://dzone.com/articles/building-pyarrow-with-cuda-support]) with the > same result. > Only modifications I made to source are editing the docker-compose volumes, > as described here: [https://github.com/apache/arrow/pull/6907] > Jira opened, per request at: [https://github.com/apache/arrow/issues/7307] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8995) [C++] Scalar formatting code used in array/diff.cc should be reusable
[
https://issues.apache.org/jira/browse/ARROW-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121145#comment-17121145
]
Wes McKinney commented on ARROW-8995:
-
Since there is already {{Scalar::ToString}}, let's use that. Does not seem
justified to have more than one way to format scalar values.
> [C++] Scalar formatting code used in array/diff.cc should be reusable
> -
>
> Key: ARROW-8995
> URL: https://issues.apache.org/jira/browse/ARROW-8995
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Priority: Major
>
> Formatting Array values as strings is not specific to the diff.cc code, so it
> may make sense to move this code elsewhere where it can be used generally
> (perhaps a method like {{Array::FormatValue}}?).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-5854) [Python] Expose compare kernels on Array class
[
https://issues.apache.org/jira/browse/ARROW-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney resolved ARROW-5854.
-
Resolution: Fixed
Issue resolved by pull request 7273
[https://github.com/apache/arrow/pull/7273]
> [Python] Expose compare kernels on Array class
> --
>
> Key: ARROW-5854
> URL: https://issues.apache.org/jira/browse/ARROW-5854
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Reporter: Joris Van den Bossche
>Assignee: Joris Van den Bossche
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Expose the compare kernel for comparing with scalar or array (ARROW-3087,
> ARROW-4990) on the python Array class.
> This can implement the {{\_\_eq\_\_}} et al dunder methods on the Array class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
Wes McKinney created ARROW-8999: --- Summary: [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build Key: ARROW-8999 URL: https://issues.apache.org/jira/browse/ARROW-8999 Project: Apache Arrow Issue Type: Bug Components: Python Reporter: Wes McKinney Fix For: 1.0.0 I've been seeing this segfault periodically the last week, does anyone have an idea what might be wrong? https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-6482) [Rust] Investigate enabling features in regex crate to reduce compile times
[ https://issues.apache.org/jira/browse/ARROW-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121073#comment-17121073 ] Paddy Horan commented on ARROW-6482: Yep, it's the unicode features I had in mind. > [Rust] Investigate enabling features in regex crate to reduce compile times > --- > > Key: ARROW-6482 > URL: https://issues.apache.org/jira/browse/ARROW-6482 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.14.1 >Reporter: Paddy Horan >Priority: Minor > Labels: beginner > Fix For: 1.0.0 > > > The regex crate recently added a feature flag to reduce compile times and > binary size if certain unicode related features are not needed. We should > investigate using this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8998) [Python] Make NumPy an optional runtime dependency
Wes McKinney created ARROW-8998: --- Summary: [Python] Make NumPy an optional runtime dependency Key: ARROW-8998 URL: https://issues.apache.org/jira/browse/ARROW-8998 Project: Apache Arrow Issue Type: New Feature Components: Python Reporter: Wes McKinney Since in the relatively near future, one will be able to do non-trivial analytical operations and query processing natively on Arrow data structures through pyarrow, it does not make sense to require users to always install NumPy when that install pyarrow. I propose to split the NumPy-depending parts of libarrow_python into a libarrow_numpy (which also must be bundled) and moving this part of the codebase into a separate Cython module. This refactoring should be relatively painless though there may be a number of packaging details to chase up since this would introduce a new shared library to be installed in various packaging targets. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8998) [Python] Make NumPy an optional runtime dependency
[ https://issues.apache.org/jira/browse/ARROW-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-8998: Description: Since in the relatively near future, one will be able to do non-trivial analytical operations and query processing natively on Arrow data structures through pyarrow, it does not make sense to require users to always install NumPy when they install pyarrow. I propose to split the NumPy-depending parts of libarrow_python into a libarrow_numpy (which also must be bundled) and moving this part of the codebase into a separate Cython module. This refactoring should be relatively painless though there may be a number of packaging details to chase up since this would introduce a new shared library to be installed in various packaging targets. was: Since in the relatively near future, one will be able to do non-trivial analytical operations and query processing natively on Arrow data structures through pyarrow, it does not make sense to require users to always install NumPy when that install pyarrow. I propose to split the NumPy-depending parts of libarrow_python into a libarrow_numpy (which also must be bundled) and moving this part of the codebase into a separate Cython module. This refactoring should be relatively painless though there may be a number of packaging details to chase up since this would introduce a new shared library to be installed in various packaging targets. > [Python] Make NumPy an optional runtime dependency > -- > > Key: ARROW-8998 > URL: https://issues.apache.org/jira/browse/ARROW-8998 > Project: Apache Arrow > Issue Type: New Feature > Components: Python >Reporter: Wes McKinney >Priority: Major > > Since in the relatively near future, one will be able to do non-trivial > analytical operations and query processing natively on Arrow data structures > through pyarrow, it does not make sense to require users to always install > NumPy when they install pyarrow. I propose to split the NumPy-depending parts > of libarrow_python into a libarrow_numpy (which also must be bundled) and > moving this part of the codebase into a separate Cython module. > This refactoring should be relatively painless though there may be a number > of packaging details to chase up since this would introduce a new shared > library to be installed in various packaging targets. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques updated ARROW-8997: -- Issue Type: Improvement (was: Bug) > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Improvement > Components: Archery >Reporter: Francois Saint-Jacques >Priority: Minor > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques updated ARROW-8997: -- Priority: Minor (was: Major) > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Bug > Components: Archery >Reporter: Francois Saint-Jacques >Priority: Minor > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
[ https://issues.apache.org/jira/browse/ARROW-8997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques updated ARROW-8997: -- Component/s: Archery > [Archery] Benchmark formatter should have friendly units > > > Key: ARROW-8997 > URL: https://issues.apache.org/jira/browse/ARROW-8997 > Project: Apache Arrow > Issue Type: Bug > Components: Archery >Reporter: Francois Saint-Jacques >Priority: Major > > The current output is not friendly to glance at. Usage of humanfriendly can > help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8997) [Archery] Benchmark formatter should have friendly units
Francois Saint-Jacques created ARROW-8997: - Summary: [Archery] Benchmark formatter should have friendly units Key: ARROW-8997 URL: https://issues.apache.org/jira/browse/ARROW-8997 Project: Apache Arrow Issue Type: Bug Reporter: Francois Saint-Jacques The current output is not friendly to glance at. Usage of humanfriendly can help here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8986) [Archery][ursabot] Fix benchmark diff checkout of origin/master
[ https://issues.apache.org/jira/browse/ARROW-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francois Saint-Jacques reassigned ARROW-8986: - Assignee: Francois Saint-Jacques > [Archery][ursabot] Fix benchmark diff checkout of origin/master > --- > > Key: ARROW-8986 > URL: https://issues.apache.org/jira/browse/ARROW-8986 > Project: Apache Arrow > Issue Type: Bug >Reporter: Francois Saint-Jacques >Assignee: Francois Saint-Jacques >Priority: Minor > > https://github.com/apache/arrow/pull/7300#issuecomment-635967095 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-6482) [Rust] Investigate enabling features in regex crate to reduce compile times
[ https://issues.apache.org/jira/browse/ARROW-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120956#comment-17120956 ] Neville Dipale commented on ARROW-6482: --- [~paddyhoran] I've briefly looked at this, and it seems like the only feature we'd want to remove would be Unicode support ([https://docs.rs/regex/1.3.9/regex/#crate-features]). > [Rust] Investigate enabling features in regex crate to reduce compile times > --- > > Key: ARROW-6482 > URL: https://issues.apache.org/jira/browse/ARROW-6482 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Affects Versions: 0.14.1 >Reporter: Paddy Horan >Priority: Minor > Labels: beginner > Fix For: 1.0.0 > > > The regex crate recently added a feature flag to reduce compile times and > binary size if certain unicode related features are not needed. We should > investigate using this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-5213) [Format] Script for updating various checked-in Flatbuffers files
[ https://issues.apache.org/jira/browse/ARROW-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120952#comment-17120952 ] Neville Dipale commented on ARROW-5213: --- [~andygrove] I'd like to remove this from the 1.0.0 milestone. We can't build files until the next flatbuffers release due to a bug in the codegen of required fields, and the other already known bugs. There's an alternative flatbuf implementation being worked on at [https://github.com/butte-rs/butte], but I don't know what its timelines are. Perhaps until one of the 2 options is fine, we could continue manually generating the files? > [Format] Script for updating various checked-in Flatbuffers files > - > > Key: ARROW-5213 > URL: https://issues.apache.org/jira/browse/ARROW-5213 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools, Format, Go, Rust >Reporter: Wes McKinney >Assignee: Andy Grove >Priority: Minor > Fix For: 1.0.0 > > > Some subprojects have begun checking in generated Flatbuffers files to source > control. This presents a maintainability issue when there are additions or > changes made to the .fbs sources. It would be useful to be able to automate > the update of these files so it doesn't have to happen on a manual / > case-by-case basis -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8996) [C++] Runtime SSE path for Aggregate Sum kernel of dense
[ https://issues.apache.org/jira/browse/ARROW-8996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8996: -- Labels: pull-request-available (was: ) > [C++] Runtime SSE path for Aggregate Sum kernel of dense > > > Key: ARROW-8996 > URL: https://issues.apache.org/jira/browse/ARROW-8996 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Frank Du >Assignee: Frank Du >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Thanks to the new kernel framework, now it can override a SIMD kernel version > at runtime. > > Below is the things we need implemented for a SSE version of sum kernel to > dense data: > 1. Add SSE intrinsic path for aggregate sum dense. > 2. Add build support to append the compiler flag for SIMD code file. > 3. Register the SSE version at runtime as the CPU feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8996) [C++] Runtime SSE path for Aggregate Sum kernel of dense
Frank Du created ARROW-8996: --- Summary: [C++] Runtime SSE path for Aggregate Sum kernel of dense Key: ARROW-8996 URL: https://issues.apache.org/jira/browse/ARROW-8996 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Frank Du Assignee: Frank Du Thanks to the new kernel framework, now it can override a SIMD kernel version at runtime. Below is the things we need implemented for a SSE version of sum kernel to dense data: 1. Add SSE intrinsic path for aggregate sum dense. 2. Add build support to append the compiler flag for SIMD code file. 3. Register the SSE version at runtime as the CPU feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
