[DISCUSS] Possibility of 12.0.2 release

2023-06-23 Thread Bryan Cutler
Hi All, I recently became aware of CVE issue https://github.com/advisories/GHSA-6mjq-h674-j845 with the Java netty libraries and using the fixed Netty library in version 4.1.94.Final required a patch for Arrow, already merged in https://github.com/apache/arrow/issues/36209. I know the freeze for

[jira] [Updated] (ARROW-18360) [Python] Incorrectly passing schema=None to do_put crashes

2022-11-17 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-18360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-18360: - Priority: Minor (was: Major) > [Python] Incorrectly passing schema=None to do_put cras

[jira] [Created] (ARROW-18360) [Python] Incorrectly passing schema=None to do_put crashes

2022-11-17 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-18360: Summary: [Python] Incorrectly passing schema=None to do_put crashes Key: ARROW-18360 URL: https://issues.apache.org/jira/browse/ARROW-18360 Project: Apache Arrow

[jira] [Created] (ARROW-18360) [Python] Incorrectly passing schema=None to do_put crashes

2022-11-17 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-18360: Summary: [Python] Incorrectly passing schema=None to do_put crashes Key: ARROW-18360 URL: https://issues.apache.org/jira/browse/ARROW-18360 Project: Apache Arrow

[jira] [Assigned] (SPARK-38098) Add support for ArrayType of nested StructType to arrow-based conversion

2022-09-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-38098: Assignee: Luca Canali > Add support for ArrayType of nested StructType to arrow-ba

[jira] [Resolved] (SPARK-38098) Add support for ArrayType of nested StructType to arrow-based conversion

2022-09-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-38098. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35391 [https

Re: Introducing "Pandas API on Spark" component in JIRA, and use "PS" PR title component

2022-05-19 Thread Bryan Cutler
+1, sounds good On Wed, May 18, 2022 at 9:16 PM Dongjoon Hyun wrote: > +1 > > Thank you for the suggestion, Hyukjin. > > Dongjoon. > > On Wed, May 18, 2022 at 11:08 AM Bjørn Jørgensen > wrote: > >> +1 >> But can will have PR Title and PR label the same, PS >> >> ons. 18. mai 2022 kl. 18:57 skr

[jira] [Resolved] (SPARK-39160) Remove workaround for ARROW-1948

2022-05-12 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-39160. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36518 [https

[jira] [Assigned] (SPARK-39160) Remove workaround for ARROW-1948

2022-05-12 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-39160: Assignee: Cheng Pan > Remove workaround for ARROW-1

Re: [VOTE] Release Apache Arrow 8.0.0 - RC3

2022-05-04 Thread Bryan Cutler
+1 (non-binding) I ran: TEST_DEFAULT=0 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 ARROW_GANDIVA=OFF ARROW_PLASMA=OFF dev/release/verify-release-candidate.sh 8.0.0 3 On Wed, May 4, 2022 at 3:23 PM Sutou Kouhei wrote: > +1 > > I ran the followings on Debian GNU/Linux sid: > > * TEST_DEFAULT

Re: [ANNOUNCE] New Arrow committer: Liang-Chi Hsieh

2022-04-27 Thread Bryan Cutler
Congratulations!! That's great news and really glad to have you on the project! On Wed, Apr 27, 2022, 11:44 AM Andrew Lamb wrote: > On behalf of the Arrow PMC, I'm happy to announce that Liang-Chi Hsieh > has accepted an invitation to become a committer on Apache > Arrow. Welcome, and thank you

Re: [JAVA] JDK Support Policy?

2022-04-05 Thread Bryan Cutler
Thanks for bringing this up Micah. Given that we have finite resources for CI, I think the oldest active LTS version sounds pretty reasonable. Ultimately it should be community driven and balance between the available resources we have and peoples time to patch any issues that come up. On Tue, Mar

[jira] [Assigned] (ARROW-15778) [Java] Endianness field not emitted in IPC stream

2022-04-05 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-15778: Assignee: Kazuaki Ishizaki > [Java] Endianness field not emitted in IPC str

[jira] [Resolved] (ARROW-15778) [Java] Endianness field not emitted in IPC stream

2022-04-05 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-15778. -- Resolution: Fixed Issue resolved by pull request 12777 [https://github.com/apache/arrow/pull

Re: [VOTE] Extend Arrow Flight SQL with GetXdbcTypeInfo, SQL type info in schemas

2022-03-28 Thread Bryan Cutler
+1 (non-binding) On Mon, Mar 28, 2022, 7:07 AM Andrew Lamb wrote: > Thank you David for pushing this through -- I think the overall FlightSQL > story is very compelling for the Arrow ecosystem > > I am also +1 on the idea, but I haven't had enough time to study the > implementation in detail yet

[jira] [Commented] (ARROW-15778) [Java] Endianness field not emitted in IPC stream

2022-03-24 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512169#comment-17512169 ] Bryan Cutler commented on ARROW-15778: -- [~kiszk] since your patch looks t

[jira] [Commented] (ARROW-15954) [Java][Packaging] Nigthly build is broken due dependencies problems for flight-core module

2022-03-16 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507812#comment-17507812 ] Bryan Cutler commented on ARROW-15954: -- I think after the grpc and netty upg

[jira] [Assigned] (ARROW-15954) [Java][Packaging] Nigthly build is broken due dependencies problems for flight-core module

2022-03-16 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-15954: Assignee: Bryan Cutler > [Java][Packaging] Nigthly build is broken due dependenc

Re: [ANNOUNCE] New Arrow committers: Raphael Taylor-Davies, Wang Xudong, Yijie Shen, and Kun Liu

2022-03-14 Thread Bryan Cutler
Congrats to all! On Thu, Mar 10, 2022 at 12:11 AM Alenka Frim wrote: > Congratulations all! > > On Thu, Mar 10, 2022 at 1:55 AM Yang hao <1371656737...@gmail.com> wrote: > > > Congratulations to all! > > > > From: Benson Muite > > Date: Thursday, March 10, 2022 at 03:45 > > To: dev@arrow.apache

Re: [VOTE] Release Apache Arrow 7.0.0 - Java artifacts

2022-03-14 Thread Bryan Cutler
+1 (non-binding) On Mon, Mar 14, 2022 at 10:26 AM David Li wrote: > My vote: +1 (binding) > > Are any other PMC members able to take a quick look? > > Thanks, > David > > On Sat, Mar 12, 2022, at 07:31, Kun Liu wrote: > > +1 non-binding > > just uploaded the missing pom and don't need to releas

[jira] [Updated] (ARROW-15795) Add a getter for the timeZone in TimeStampMicroTZVector

2022-03-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-15795: - Fix Version/s: 7.0.1 > Add a getter for the timeZone in TimeStampMicroTZVec

[jira] [Assigned] (ARROW-15795) Add a getter for the timeZone in TimeStampMicroTZVector

2022-03-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-15795: Assignee: Fabien > Add a getter for the timeZone in TimeStampMicroTZVec

[jira] [Resolved] (ARROW-15795) Add a getter for the timeZone in TimeStampMicroTZVector

2022-03-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-15795. -- Fix Version/s: 8.0.0 Resolution: Fixed Issue resolved by pull request 12521 [https

[jira] [Created] (ARROW-15831) [Java] Upgrade Flight dependencies

2022-03-02 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15831: Summary: [Java] Upgrade Flight dependencies Key: ARROW-15831 URL: https://issues.apache.org/jira/browse/ARROW-15831 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-15831) [Java] Upgrade Flight dependencies

2022-03-02 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15831: Summary: [Java] Upgrade Flight dependencies Key: ARROW-15831 URL: https://issues.apache.org/jira/browse/ARROW-15831 Project: Apache Arrow Issue Type

Re: Flight/FlightSQL Optimization for Small Results?

2022-03-01 Thread Bryan Cutler
I think this would be a useful feature and be nice to have in Flight core. For cases like previewing data, you usually just want to get a small amount of data quickly. Would it make sense to make this part of DoGet since it still would be returning a record batch? Perhaps a Ticket could be made to

[jira] [Resolved] (ARROW-11396) [JAVA] Update the access permission from private to protected

2022-02-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-11396. -- Resolution: Won't Fix > [JAVA] Update the access permission from private to p

[jira] [Assigned] (ARROW-14665) [Java] JdbcToArrowUtils ResultSet iteration bug

2022-02-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-14665: Assignee: Zac > [Java] JdbcToArrowUtils ResultSet iteration

[jira] [Assigned] (ARROW-15272) [Java] ArrowVectorIterator eats initialization exceptions when close fails

2022-02-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-15272: Assignee: Andrew Higgins > [Java] ArrowVectorIterator eats initialization excepti

[jira] [Resolved] (ARROW-15272) [Java] ArrowVectorIterator eats initialization exceptions when close fails

2022-02-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-15272. -- Fix Version/s: 8.0.0 Resolution: Fixed Issue resolved by pull request 12094 [https

[jira] [Comment Edited] (ARROW-14549) VectorSchemaRoot is not refreshed when value is null

2022-02-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498249#comment-17498249 ] Bryan Cutler edited comment on ARROW-14549 at 2/25/22, 6:2

[jira] [Resolved] (ARROW-14549) VectorSchemaRoot is not refreshed when value is null

2022-02-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-14549. -- Resolution: Not A Problem > VectorSchemaRoot is not refreshed when value is n

[jira] [Commented] (ARROW-14549) VectorSchemaRoot is not refreshed when value is null

2022-02-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498249#comment-17498249 ] Bryan Cutler commented on ARROW-14549: -- [~hu6360567] Calling `allocateNew()`

[jira] [Resolved] (ARROW-14665) [Java] JdbcToArrowUtils ResultSet iteration bug

2022-02-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-14665. -- Fix Version/s: 8.0.0 Resolution: Fixed Issue resolved by pull request 11667 [https

[jira] [Resolved] (ARROW-15722) [Java] Improve error message for ListVector with wrong number of children

2022-02-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-15722. -- Fix Version/s: 8.0.0 Resolution: Fixed Issue resolved by pull request 12456 [https

Re: Is 7.0.0 release missing the Java arrow-flight POM?

2022-02-21 Thread Bryan Cutler
les at > > https://repository.apache.org/#stagingRepositories with your > > ASF account. > > Note that you MUST not press the "Close" button! I'll remove > > them by pressing "Drop" button when we fix this. > > > > > > Th

[jira] [Created] (ARROW-15746) [Java] Add arrow-flight pom to list of artifacts to deploy

2022-02-21 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15746: Summary: [Java] Add arrow-flight pom to list of artifacts to deploy Key: ARROW-15746 URL: https://issues.apache.org/jira/browse/ARROW-15746 Project: Apache Arrow

[jira] [Created] (ARROW-15746) [Java] Add arrow-flight pom to list of artifacts to deploy

2022-02-21 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15746: Summary: [Java] Add arrow-flight pom to list of artifacts to deploy Key: ARROW-15746 URL: https://issues.apache.org/jira/browse/ARROW-15746 Project: Apache Arrow

Re: Is 7.0.0 release missing the Java arrow-flight POM?

2022-02-18 Thread Bryan Cutler
, Feb 17, 2022 at 10:06 PM Bryan Cutler wrote: > Sure, I'll take a look at the script. > > On Thu, Feb 17, 2022 at 4:39 PM Sutou Kouhei wrote: > >> Hi, >> >> Ah, arrow-flight-*.pom exists on our CI artifacts: >> >> https://github.com/ursacomputing/cr

Re: Is 7.0.0 release missing the Java arrow-flight POM?

2022-02-17 Thread Bryan Cutler
w why our upload script > https://github.com/apache/arrow/blob/master/dev/release/06-java-upload.sh > doesn't upload it... > > Could you take a look at it? > > > Thanks, > -- > kou > > In > "Re: Is 7.0.0 release missing the Java arrow-flight POM?"

[jira] [Updated] (ARROW-15486) [Release][Java] Verify staged maven artifacts

2022-02-17 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-15486: - Summary: [Release][Java] Verify staged maven artifacts (was: [Relase][Java] Verify staged

Re: Is 7.0.0 release missing the Java arrow-flight POM?

2022-02-17 Thread Bryan Cutler
ven.org/maven2/org/apache/arrow/flight-grpc/ > > Can we use flight-core (and flight-grpc) instead of > arrow-flight? > > Thanks, > -- > kou > > In > "Is 7.0.0 release missing the Java arrow-flight POM?" on Thu, 17 Feb > 2022 09:48:57 -0800, > Bry

[jira] [Created] (ARROW-15722) [Java] Improve error message for ListVector with wrong number of children

2022-02-17 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15722: Summary: [Java] Improve error message for ListVector with wrong number of children Key: ARROW-15722 URL: https://issues.apache.org/jira/browse/ARROW-15722 Project

[jira] [Created] (ARROW-15722) [Java] Improve error message for ListVector with wrong number of children

2022-02-17 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-15722: Summary: [Java] Improve error message for ListVector with wrong number of children Key: ARROW-15722 URL: https://issues.apache.org/jira/browse/ARROW-15722 Project

Is 7.0.0 release missing the Java arrow-flight POM?

2022-02-17 Thread Bryan Cutler
Hi All, Congrats on the 7.0.0 release! I was trying it out and got an error not being able to find arrow-flight-7.0.0.pom. This looks like a new parent POM for Flight, so I checked maven central and don't see it deployed there. Not sure what could have happened, but maybe it's only me. Anyone else

Re: [ANNOUNCE] New Arrow PMC chair: Kouhei Sutou

2022-01-27 Thread Bryan Cutler
Congratulations Kou, thanks for all your work! On Thu, Jan 27, 2022, 4:36 PM Sutou Kouhei wrote: > Thanks everyone!!! > > In > "[ANNOUNCE] New Arrow PMC chair: Kouhei Sutou" on Tue, 25 Jan 2022 > 11:32:56 -0500, > Wes McKinney wrote: > > > I am pleased to announce that we have a new PMC ch

[jira] [Assigned] (SPARK-34521) spark.createDataFrame does not support Pandas StringDtype extension type

2021-12-16 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-34521: Assignee: Nicolas Azrak > spark.createDataFrame does not support Pandas StringDt

[jira] [Resolved] (SPARK-34521) spark.createDataFrame does not support Pandas StringDtype extension type

2021-12-15 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-34521. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34509 [https

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

2021-10-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Attachment: (was: 0--1172099527-254246775-1412485878) > Complete support for remain

[jira] [Resolved] (ARROW-13814) [CI] Nightly integration build with spark master failing to compile spark

2021-10-05 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-13814. -- Resolution: Fixed Issue resolved by pull request 11282 [https://github.com/apache/arrow/pull

[jira] [Resolved] (ARROW-14014) FlightClient.ClientStreamListener not notified on error when parsing invalid trailers

2021-10-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-14014. -- Fix Version/s: 6.0.0 Resolution: Fixed Issue resolved by pull request 11311 [https

[jira] [Assigned] (ARROW-14014) FlightClient.ClientStreamListener not notified on error when parsing invalid trailers

2021-10-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-14014: Assignee: Bryan Cutler > FlightClient.ClientStreamListener not notified on error w

[jira] [Resolved] (ARROW-14198) [Java] Upgrade Netty and gRPC dependencies

2021-10-04 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-14198. -- Fix Version/s: 6.0.0 Resolution: Fixed Issue resolved by pull request 11293 [https

[jira] [Commented] (ARROW-14014) FlightClient.ClientStreamListener not notified on error when parsing invalid trailers

2021-10-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423442#comment-17423442 ] Bryan Cutler commented on ARROW-14014: -- I've run into this a cou

[jira] [Created] (ARROW-14198) [Java] Upgrade Netty and gRPC dependencies

2021-10-01 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-14198: Summary: [Java] Upgrade Netty and gRPC dependencies Key: ARROW-14198 URL: https://issues.apache.org/jira/browse/ARROW-14198 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-14198) [Java] Upgrade Netty and gRPC dependencies

2021-10-01 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-14198: Summary: [Java] Upgrade Netty and gRPC dependencies Key: ARROW-14198 URL: https://issues.apache.org/jira/browse/ARROW-14198 Project: Apache Arrow Issue Type

[jira] [Resolved] (ARROW-9647) [Java] Cannot install arrow-memory 1.0.0 from maven central

2021-10-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-9647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-9647. - Resolution: Not A Problem > [Java] Cannot install arrow-memory 1.0.0 from maven cent

[jira] [Assigned] (ARROW-13814) [CI] Nightly integration build with spark master failing to compile spark

2021-09-30 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-13814: Assignee: Bryan Cutler > [CI] Nightly integration build with spark master failing

Re: [Java] When to close ArrowStreamWriter?

2021-09-15 Thread Bryan Cutler
Hi Daniel, You need to call either ArrowStreamWriter.end() or ArrowStreamWriter.close() when done writing batches to ensure the end-of-stream identifier is written. end() will just write the end-of-stream, while close() will call end() and then close the writer resources like the write channel. O

[jira] [Created] (ARROW-13872) [Java] ExtensionTypeVector does not work with RangeEqualsVisitor

2021-09-02 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13872: Summary: [Java] ExtensionTypeVector does not work with RangeEqualsVisitor Key: ARROW-13872 URL: https://issues.apache.org/jira/browse/ARROW-13872 Project: Apache

[jira] [Created] (ARROW-13872) [Java] ExtensionTypeVector does not work with RangeEqualsVisitor

2021-09-02 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13872: Summary: [Java] ExtensionTypeVector does not work with RangeEqualsVisitor Key: ARROW-13872 URL: https://issues.apache.org/jira/browse/ARROW-13872 Project: Apache

[jira] [Commented] (ARROW-13814) [CI] Nightly integration build with spark master failing to compile spark

2021-08-31 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407801#comment-17407801 ] Bryan Cutler commented on ARROW-13814: -- Not sure why compilation would fail

[jira] [Assigned] (ARROW-12679) [Java] JDBC adapter does not preserve SQL-nullability

2021-07-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-12679: Assignee: Joris Peeters (was: Bryan Cutler) > [Java] JDBC adapter does not preserve

[jira] [Assigned] (ARROW-12679) [Java] JDBC adapter does not preserve SQL-nullability

2021-07-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned ARROW-12679: Assignee: Bryan Cutler (was: Joris Peeters) > [Java] JDBC adapter does not preserve

Re: [VOTE] Arrow should state a convention for encoding instants as Timestamp with "UTC" as the time zone

2021-06-30 Thread Bryan Cutler
+1 non-binding On Wed, Jun 30, 2021, 2:53 AM Weston Pace wrote: > This vote is a result of previous discussion[1][2]. This vote is also > a prerequisite for the PR in [5]. > > --- > Some date & time libraries have three temporal concepts. For the sake > of this document we will call them Local

Re: [STRAW POLL] (How) should Arrow define storage for "Instant"s

2021-06-28 Thread Bryan Cutler
C first choice, E second On Mon, Jun 28, 2021, 8:40 AM Julian Hyde wrote: > D > > (2nd choice E if we’re doing ranked-choice voting) > > Julian > > > On Jun 24, 2021, at 12:24 PM, Weston Pace wrote: > > > > The discussion in [1] led to the following question. Before we > > proceed on a vote i

Re: [ANNOUNCE] New Arrow PMC member: David M Li

2021-06-23 Thread Bryan Cutler
Congrats David! On Tue, Jun 22, 2021, 7:24 PM Micah Kornfield wrote: > Congrats David! > > On Tue, Jun 22, 2021 at 7:13 PM Fan Liya wrote: > > > Congratulations David! > > > > Best, > > Liya Fan > > > > > > On Wed, Jun 23, 2021 at 9:44 AM Yibo Cai wrote: > > > > > Congrats David! > > > > > > O

[jira] [Resolved] (ARROW-13044) [Java] Union vectors should extend ValueVector

2021-06-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-13044. -- Fix Version/s: 5.0.0 Resolution: Fixed Issue resolved by pull request 10513 [https

[jira] [Created] (ARROW-13076) [Java] Enable ExtensionType to use StructVector and UnionVector for underlying storage

2021-06-14 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13076: Summary: [Java] Enable ExtensionType to use StructVector and UnionVector for underlying storage Key: ARROW-13076 URL: https://issues.apache.org/jira/browse/ARROW-13076

[jira] [Created] (ARROW-13076) [Java] Enable ExtensionType to use StructVector and UnionVector for underlying storage

2021-06-14 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13076: Summary: [Java] Enable ExtensionType to use StructVector and UnionVector for underlying storage Key: ARROW-13076 URL: https://issues.apache.org/jira/browse/ARROW-13076

[jira] [Updated] (ARROW-13044) [Java] Union vectors should extend ValueVector

2021-06-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-13044: - Description: I was going to try using a DenseUnionVector as the underlying vector of an

[jira] [Updated] (ARROW-13044) [Java] Union vectors should extend ValueVector

2021-06-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-13044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-13044: - Summary: [Java] Union vectors should extend ValueVector (was: [Java] Union vectors should

[jira] [Created] (ARROW-13044) [Java] Union vectors should extend BaseValueVector

2021-06-10 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13044: Summary: [Java] Union vectors should extend BaseValueVector Key: ARROW-13044 URL: https://issues.apache.org/jira/browse/ARROW-13044 Project: Apache Arrow

[jira] [Created] (ARROW-13044) [Java] Union vectors should extend BaseValueVector

2021-06-10 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-13044: Summary: [Java] Union vectors should extend BaseValueVector Key: ARROW-13044 URL: https://issues.apache.org/jira/browse/ARROW-13044 Project: Apache Arrow

[jira] [Commented] (ARROW-8666) [Java] DenseUnionVector has no way to set offset/validity directly

2021-06-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361229#comment-17361229 ] Bryan Cutler commented on ARROW-8666: - [~lidavidm] can this be closed then, or

Re: [ANNOUNCE] New Arrow committer: Kazuaki Ishizaki

2021-06-07 Thread Bryan Cutler
Congratulations!! On Sun, Jun 6, 2021, 7:28 PM Sutou Kouhei wrote: > Hi, > > On behalf of the Arrow PMC, I'm happy to announce that > Kazuaki Ishizaki has accepted an invitation to become a > committer on Apache Arrow. Welcome, and thank you for your > contributions! > > > Thanks, > -- > kou >

Re: [ANNOUNCE] New Arrow PMC member: Benjamin Kietzman

2021-05-06 Thread Bryan Cutler
Congrats Ben! On Thu, May 6, 2021 at 12:05 PM Antoine Pitrou wrote: > > Congratulations Ben :-) > > > Le 06/05/2021 à 21:02, Rok Mihevc a écrit : > > Congrats! > > > > On Thu, May 6, 2021 at 10:49 AM Krisztián Szűcs < > szucs.kriszt...@gmail.com> > > wrote: > > > >> Congrats Ben! > >> > >> On Th

Re: Welcoming six new Apache Spark committers

2021-03-29 Thread Bryan Cutler
Congratulations everyone! On Sun, Mar 28, 2021 at 11:00 PM ML Books wrote: > Congrats all > > On Sat, Mar 27, 2021, 1:58 AM Matei Zaharia > wrote: > >> Hi all, >> >> The Spark PMC recently voted to add several new committers. Please join >> me in welcoming them to their new role! Our new commit

Re: [VOTE] SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Bryan Cutler
+1 (non-binding) On Fri, Mar 26, 2021 at 9:49 AM Maciej wrote: > +1 (nonbinding) > > On 3/26/21 3:52 PM, Hyukjin Kwon wrote: > > Hi all, > > I’d like to start a vote for SPIP: Support pandas API layer on PySpark. > > The proposal is to embrace Koalas in PySpark to have the pandas API layer > on

Re: [DISCUSS] Support pandas API layer on PySpark

2021-03-16 Thread Bryan Cutler
+1 the proposal sounds good to me. Having a familiar API built-in will really help new users get into using Spark that might only have Pandas experience. It sounds like maintenance costs should be manageable, once the hurdle with setting up tests is done. Just out of curiosity, does Koalas pretty m

[jira] [Comment Edited] (SPARK-34463) toPandas failed with error: buffer source array is read-only when Arrow with self-destruct is enabled

2021-03-02 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293893#comment-17293893 ] Bryan Cutler edited comment on SPARK-34463 at 3/2/21, 6:1

[jira] [Commented] (SPARK-34463) toPandas failed with error: buffer source array is read-only when Arrow with self-destruct is enabled

2021-03-02 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293893#comment-17293893 ] Bryan Cutler commented on SPARK-34463: -- As David said, it depends on what is

Re: [C++][CI] Make "C++ on s390x" build mandatory?

2021-02-23 Thread Bryan Cutler
+1 sgtm On Tue, Feb 23, 2021, 9:47 AM Micah Kornfield wrote: > +1, but let's keep an eye on it to make sure it remains stable. > > On Tue, Feb 23, 2021 at 5:34 AM Kazuaki Ishizaki > wrote: > > > Thank you. +1 for this proposal, > > > > Kazuaki Ishizaki > > > > > > > > From: Benjamin Kietzman

[jira] [Resolved] (ARROW-11223) [Java] BaseVariableWidthVector/BaseLargeVariableWidthVector setNull and getBufferSizeFor is buggy

2021-02-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved ARROW-11223. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 9187 [https

[jira] [Updated] (ARROW-11739) [Java] Add API for getBufferSizeFor() with density to BaseVariableWidthVector

2021-02-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-11739: - Description: Following the discussion on https://github.com/apache/arrow/pull/9187. Proposed

[jira] [Updated] (ARROW-11739) [Java] Add API for getBufferSize() with density to BaseVariableWidthVector

2021-02-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-11739: - Description: Following the discussion on https://github.com/apache/arrow/pull/9187. The

[jira] [Updated] (ARROW-11739) [Java] Add API for getBufferSizeFor() with density to BaseVariableWidthVector

2021-02-22 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/ARROW-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated ARROW-11739: - Summary: [Java] Add API for getBufferSizeFor() with density to BaseVariableWidthVector (was

[jira] [Created] (ARROW-11739) [Java] Add API for getBufferSize() with density to BaseVariableWidthVector

2021-02-22 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-11739: Summary: [Java] Add API for getBufferSize() with density to BaseVariableWidthVector Key: ARROW-11739 URL: https://issues.apache.org/jira/browse/ARROW-11739 Project

[jira] [Created] (ARROW-11739) [Java] Add API for getBufferSize() with density to BaseVariableWidthVector

2021-02-22 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-11739: Summary: [Java] Add API for getBufferSize() with density to BaseVariableWidthVector Key: ARROW-11739 URL: https://issues.apache.org/jira/browse/ARROW-11739 Project

[jira] [Assigned] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2021-02-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-32953: Assignee: David Li > Lower memory usage in toPandas with Arrow self_destr

[jira] [Resolved] (SPARK-32953) Lower memory usage in toPandas with Arrow self_destruct

2021-02-10 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-32953. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 29818 [https

[jira] [Created] (ARROW-11382) [Java] NullVector field name can't be set

2021-01-25 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-11382: Summary: [Java] NullVector field name can't be set Key: ARROW-11382 URL: https://issues.apache.org/jira/browse/ARROW-11382 Project: Apache Arrow Issue

[jira] [Created] (ARROW-11382) [Java] NullVector field name can't be set

2021-01-25 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-11382: Summary: [Java] NullVector field name can't be set Key: ARROW-11382 URL: https://issues.apache.org/jira/browse/ARROW-11382 Project: Apache Arrow Issue

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-20 Thread Bryan Cutler
+1 (non-binding) I verified binaries and source with the following: ARROW_TMPDIR=/tmp/arrow-test ARROW_GANDIVA=0 ARROW_PLASMA=0 TEST_DEFAULT=0 TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON=1 TEST_JAVA=1 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 dev/release/verify-release-candidate.sh source 3.0.0 2 I

Re: [Java] PR review for ARROW-11173

2021-01-19 Thread Bryan Cutler
Hi Nick, I left a note in the PR that I will try to review soon, thanks! On Sun, Jan 17, 2021 at 8:22 PM Nick Bruno wrote: > Hi All, > > I'd like to get feedback on the pull request I created a little over a > week ago - https://github.com/apache/arrow/pull/9151 > > It adds support for Map type

Re: [Java] Flight fails to handle list vector

2021-01-14 Thread Bryan Cutler
Hi John, could you include the error with stacktrace? On Sat, Jan 9, 2021 at 9:34 PM John Peterson wrote: > I believe I'm running into a bug with Flight but I'd like to confirm and > get some advice on a potential fix. I'm not sure where to look or what > could be causing it. > > The code in que

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2020-12-28 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255712#comment-17255712 ] Bryan Cutler commented on SPARK-24632: -- Ping [~huaxingao] in case you have

[jira] [Resolved] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

2020-12-11 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-33576. -- Resolution: Duplicate Going to resolve as a duplicate, but please reopen if you find it is

[jira] [Commented] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

2020-12-11 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248064#comment-17248064 ] Bryan Cutler commented on SPARK-33576: -- [~darshats] I believe the only cur

[jira] [Commented] (SPARK-33576) PythonException: An exception was thrown from a UDF: 'OSError: Invalid IPC message: negative bodyLength'.

2020-12-01 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241769#comment-17241769 ] Bryan Cutler commented on SPARK-33576: -- Is this due to the 2GB limit? As in h

  1   2   3   4   5   6   7   8   9   10   >