Re: [PR] GH-39737: [Release][Docs] Update post release documentation task [arrow]

2024-02-05 Thread via GitHub
kou commented on PR #39762: URL: https://github.com/apache/arrow/pull/39762#issuecomment-1926413528 Ah, you need to enter your passphrase for `/Users/alenkafrim/.ssh/id_ed25519`. Did you enter it? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] GH-39737: [Release][Docs] Update post release documentation task [arrow]

2024-02-05 Thread via GitHub
AlenkaF commented on PR #39762: URL: https://github.com/apache/arrow/pull/39762#issuecomment-1926430536 Yes, I did. First time it worked, now it gets stuck every time ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] GH-39737: [Release][Docs] Update post release documentation task [arrow]

2024-02-05 Thread via GitHub
AlenkaF commented on PR #39762: URL: https://github.com/apache/arrow/pull/39762#issuecomment-1926434643 Ha, I tried hitting enter again (twice that is) and got: ``` [email protected]: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure y

Re: [PR] GH-39737: [Release][Docs] Update post release documentation task [arrow]

2024-02-05 Thread via GitHub
kou commented on PR #39762: URL: https://github.com/apache/arrow/pull/39762#issuecomment-1926439007 No problem. :-) You may want to use ssh-agent or something to avoid entering passphrase multiple times. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] Projections require unique expression names error [arrow-datafusion]

2024-02-05 Thread via GitHub
Jefffrey commented on issue #9130: URL: https://github.com/apache/arrow-datafusion/issues/9130#issuecomment-1926442064 Related: https://github.com/apache/arrow-datafusion/issues/6543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Allow the contributor to self-assign the issue [arrow-datafusion]

2024-02-05 Thread via GitHub
Jefffrey commented on issue #9124: URL: https://github.com/apache/arrow-datafusion/issues/9124#issuecomment-1926443371 This seems like a good idea :+1: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Registering a record batch with the name json_table causes ParserError [arrow-datafusion]

2024-02-05 Thread via GitHub
Jefffrey commented on issue #9122: URL: https://github.com/apache/arrow-datafusion/issues/9122#issuecomment-1926449887 That makes sense. Looks like @viirya has submitted the PR to fix this in sqlparser-rs, so will need that to land, then wait for next sqlparser-rs release to fix this bug

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
Tom-Newton commented on PR #39622: URL: https://github.com/apache/arrow/pull/39622#issuecomment-1926477850 @kou please could you re-trigger archery builds. It looks like the java-jars ones succeeded apart from one which failed with a timeout reading from the cache https://github.com/ursaco

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
kou commented on PR #39622: URL: https://github.com/apache/arrow/pull/39622#issuecomment-1926491102 Done! (Re-ran) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] GH-39759: [Docs] Update pydata-sphinx-theme to 0.15.x [arrow]

2024-02-05 Thread via GitHub
Divyansh200102 commented on PR #39879: URL: https://github.com/apache/arrow/pull/39879#issuecomment-1926499797 > I have opened an issue for the failing docs preview build: https://github.com/apache/arrow/issues/39942 Okay thank you @AlenkaF -- This is an automated message from the

Re: [I] [FlightSQL] Support `DoExchange` (in addition to `DoPut`) to bind parameters and execute prepared statements [arrow]

2024-02-05 Thread via GitHub
kou commented on issue #37741: URL: https://github.com/apache/arrow/issues/37741#issuecomment-1926517038 How about just returning `null` or an empty `FlightInfo` (no endpoint)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] [FlightSQL] Support `DoExchange` (in addition to `DoPut`) to bind parameters and execute prepared statements [arrow]

2024-02-05 Thread via GitHub
kou commented on issue #37741: URL: https://github.com/apache/arrow/issues/37741#issuecomment-1926525710 Ah, we may want to return `*flight.Reader` like [`Client.DoGet()`](https://pkg.go.dev/github.com/apache/arrow/go/[email protected]/arrow/flight/flightsql#Client.DoGe

Re: [PR] GH-39759: [Docs] Update pydata-sphinx-theme to 0.15.x [arrow]

2024-02-05 Thread via GitHub
Divyansh200102 commented on PR #39879: URL: https://github.com/apache/arrow/pull/39879#issuecomment-1926549310 BTW @AlenkaF does this pr still needs working or the issue is in @github-actions crossbow submit preview-docs? -- This is an automated message from the Apache Git Service. To res

Re: [PR] GH-33745: [Docs] Updated "struct_field" kernel documentation [arrow]

2024-02-05 Thread via GitHub
Divyansh200102 commented on PR #39851: URL: https://github.com/apache/arrow/pull/39851#issuecomment-1926551977 > Could you use `GH-33745: ` instead of `MINOR: `? Is this pr ready to merge @kou or do I need to make any changes? -- This is an automated message from the Apache Git Serv

Re: [I] [CI][R][Docs] Docs preview jobs failing due to R Markdown error [arrow]

2024-02-05 Thread via GitHub
jorisvandenbossche commented on issue #39942: URL: https://github.com/apache/arrow/issues/39942#issuecomment-1926558728 FWIW I assume it's not related to the _preview_, it's just the doc build in general that is failing? (also on the main branch, the last commits have red CI for docs) --

Re: [I] [CI][R][Docs] Docs jobs failing due to R Markdown error [arrow]

2024-02-05 Thread via GitHub
AlenkaF commented on issue #39942: URL: https://github.com/apache/arrow/issues/39942#issuecomment-1926568486 Yes, true! Changed the description and title. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] GH-39759: [Docs] Update pydata-sphinx-theme to 0.15.x [arrow]

2024-02-05 Thread via GitHub
AlenkaF commented on PR #39879: URL: https://github.com/apache/arrow/pull/39879#issuecomment-1926593007 Preview build can't currently test the changes of this PR. The issue causing the failure of the docs build has to be fixed first (not related to the work here), so we will need to wait fo

Re: [PR] GH-39759: [Docs] Update pydata-sphinx-theme to 0.15.x [arrow]

2024-02-05 Thread via GitHub
jorisvandenbossche commented on PR #39879: URL: https://github.com/apache/arrow/pull/39879#issuecomment-1926618166 I updated a bunch of Python versions in our CI last year (https://github.com/apache/arrow/pull/35548), but it's probably time to do that again, and more thoroughly this time (i

Re: [I] [FlightSQL] Support `DoExchange` (in addition to `DoPut`) to bind parameters and execute prepared statements [arrow]

2024-02-05 Thread via GitHub
alamb commented on issue #37741: URL: https://github.com/apache/arrow/issues/37741#issuecomment-1926620916 > @alamb I think we are on the same page that DoExchange does not require passing a FlightInfo — the problem is that the existing prepared statement interfaces do require returning a F

Re: [I] A More General Approach for Optimizing Projections in Physical Plans [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #9111: URL: https://github.com/apache/arrow-datafusion/issues/9111#issuecomment-1926625356 > I haven't deeply examined the rule in the logical plan, but I know the rule there is less complicated. However, after all the rules have been worked in a physical plan, it

Re: [PR] Minor: Fix Self referential links in readme [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on code in PR #9119: URL: https://github.com/apache/arrow-datafusion/pull/9119#discussion_r1477941492 ## datafusion/core/README.md: ## @@ -19,8 +19,15 @@ # DataFusion Core -[DataFusion][df] is an extensible query execution framework, written in Rust, that u

Re: [PR] GH-39759: [Docs] Update pydata-sphinx-theme to 0.15.x [arrow]

2024-02-05 Thread via GitHub
AlenkaF commented on PR #39879: URL: https://github.com/apache/arrow/pull/39879#issuecomment-1926636979 Worth a separate PR then? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] MINOR: [Docs][Java] Fix incorrect example [arrow]

2024-02-05 Thread via GitHub
raulcd commented on PR #39941: URL: https://github.com/apache/arrow/pull/39941#issuecomment-1926690132 @davisusanibar @danepitkin can you review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Add example of converting RecordBatches to JSON objects [arrow-rs]

2024-02-05 Thread via GitHub
alamb commented on code in PR #5364: URL: https://github.com/apache/arrow-rs/pull/5364#discussion_r1477986890 ## arrow-json/src/writer.rs: ## @@ -74,7 +74,39 @@ //! [`LineDelimitedWriter`] and [`ArrayWriter`] will omit writing keys with null values. //! In order to explicitly

Re: [PR] MINOR: [Docs][Java] Fix installation documentation for BOM file [arrow]

2024-02-05 Thread via GitHub
raulcd commented on PR #39939: URL: https://github.com/apache/arrow/pull/39939#issuecomment-1926692775 The failure of the `preview-docs` job is related to: https://github.com/apache/arrow/issues/39942 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Add example of converting RecordBatches to JSON objects [arrow-rs]

2024-02-05 Thread via GitHub
alamb commented on code in PR #5364: URL: https://github.com/apache/arrow-rs/pull/5364#discussion_r1477988745 ## arrow-json/src/writer.rs: ## @@ -74,7 +74,35 @@ //! [`LineDelimitedWriter`] and [`ArrayWriter`] will omit writing keys with null values. //! In order to explicitly

Re: [I] join statement causes panic [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb closed issue #7931: join statement causes panic URL: https://github.com/apache/arrow-datafusion/issues/7931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [I] join statement causes panic [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #7931: URL: https://github.com/apache/arrow-datafusion/issues/7931#issuecomment-1926712968 Nice work @korowa (again) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
raulcd commented on PR #39622: URL: https://github.com/apache/arrow/pull/39622#issuecomment-1926713593 @github-actions crossbow submit -g wheel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] `ctx.read_parquet` and `ctx.register_parquet` don't load schema metadata [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #9081: URL: https://github.com/apache/arrow-datafusion/issues/9081#issuecomment-1926719680 That is a great find @brayanjuls -- nice 🕵️ . I wonder if the code that does schema inference is ignorning metadata somehow This code seems to imply it is handling me

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
github-actions[bot] commented on PR #39622: URL: https://github.com/apache/arrow/pull/39622#issuecomment-1926718536 Revision: 87640675eb35b9e7ee53b4f75e906e39bf12 Submitted crossbow builds: [ursacomputing/crossbow @ actions-5e4cec918f](https://github.com/ursacomputing/crossbow/bra

Re: [PR] Add example of converting RecordBatches to JSON objects [arrow-rs]

2024-02-05 Thread via GitHub
alamb merged PR #5364: URL: https://github.com/apache/arrow-rs/pull/5364 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
raulcd commented on code in PR #39622: URL: https://github.com/apache/arrow/pull/39622#discussion_r1478006878 ## .env: ## @@ -92,10 +92,10 @@ DEVTOOLSET_VERSION= # Used through docker-compose.yml and serves as the default version for the # ci/scripts/install_vcpkg.sh script. P

Re: [PR] Add example of converting RecordBatches to JSON objects [arrow-rs]

2024-02-05 Thread via GitHub
alamb commented on code in PR #5364: URL: https://github.com/apache/arrow-rs/pull/5364#discussion_r1478008839 ## arrow-json/src/writer.rs: ## @@ -74,7 +74,39 @@ //! [`LineDelimitedWriter`] and [`ArrayWriter`] will omit writing keys with null values. //! In order to explicitly

Re: [I] DataFusion weekly project plan (Andrew Lamb) - Feb 5, 2024 [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #9121: URL: https://github.com/apache/arrow-datafusion/issues/9121#issuecomment-1926726817 DataFusion - [ ] https://github.com/apache/arrow-datafusion/pull/9080 - [ ] https://github.com/apache/arrow-datafusion/pull/9064 - [ ] https://github.com/apache/arrow-

[PR] GH-39943: [CI][Python] Update manylinux images to avoid GPG problems downloading packages [arrow]

2024-02-05 Thread via GitHub
raulcd opened a new pull request, #39944: URL: https://github.com/apache/arrow/pull/39944 ### Rationale for this change Old manylinux images seem to have issues with a GPG key in order to download packages. ### What changes are included in this PR? Update the manylinux i

Re: [PR] GH-39943: [CI][Python] Update manylinux images to avoid GPG problems downloading packages [arrow]

2024-02-05 Thread via GitHub
raulcd commented on PR #39944: URL: https://github.com/apache/arrow/pull/39944#issuecomment-1926733970 @github-actions crossbow submit -g wheel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
raulcd commented on PR #39622: URL: https://github.com/apache/arrow/pull/39622#issuecomment-1926735048 The `manylinux_2_28` failures are unrelated and I've created and issue and a PR to fix them: https://github.com/apache/arrow/issues/39943 -- This is an automated message from the Apache

Re: [PR] GH-39943: [CI][Python] Update manylinux images to avoid GPG problems downloading packages [arrow]

2024-02-05 Thread via GitHub
github-actions[bot] commented on PR #39944: URL: https://github.com/apache/arrow/pull/39944#issuecomment-1926738333 Revision: 87e885c0fc22a392e7d44f00ea868899560933b7 Submitted crossbow builds: [ursacomputing/crossbow @ actions-9acd7b44a1](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-39621: [CI][Packaging] Update vcpkg to 2023.11.20 release [arrow]

2024-02-05 Thread via GitHub
Tom-Newton commented on code in PR #39622: URL: https://github.com/apache/arrow/pull/39622#discussion_r1478027887 ## .env: ## @@ -92,10 +92,10 @@ DEVTOOLSET_VERSION= # Used through docker-compose.yml and serves as the default version for the # ci/scripts/install_vcpkg.sh scrip

Re: [I] A More General Approach for Optimizing Projections in Physical Plans [arrow-datafusion]

2024-02-05 Thread via GitHub
berkaysynnada commented on issue #9111: URL: https://github.com/apache/arrow-datafusion/issues/9111#issuecomment-1926766394 If I remember correctly `LogicalPlan` rule works with qualified column names. Let's think there is a plan such as: _A <- B <- C_ and B

Re: [PR] Support join filter for `SortMergeJoin` [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9080: URL: https://github.com/apache/arrow-datafusion/pull/9080#issuecomment-1926772802 Thanks @viirya -- I plan to review this later today -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] GH-33745: [Docs] Updated "struct_field" kernel documentation [arrow]

2024-02-05 Thread via GitHub
kou commented on code in PR #39851: URL: https://github.com/apache/arrow/pull/39851#discussion_r1478042122 ## docs/source/cpp/compute.rst: ## @@ -1816,7 +1816,7 @@ Structural transforms The output type is an Array of items for the ``FIRST``/``LAST`` options and an Array of

Re: [PR] GH-39737: [Release][Docs] Update post release documentation task [arrow]

2024-02-05 Thread via GitHub
raulcd commented on code in PR #39762: URL: https://github.com/apache/arrow/pull/39762#discussion_r1478042873 ## dev/release/post-08-docs.sh: ## @@ -84,6 +84,18 @@ if [ "$is_major_release" = "yes" ] ; then previous_series=${previous_version%.*} mv docs_temp docs/${previous

Re: [I] A More General Approach for Optimizing Projections in Physical Plans [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #9111: URL: https://github.com/apache/arrow-datafusion/issues/9111#issuecomment-1926784866 > All expressions can be rewritten if a change occurs in the plan because of this kind of index changes. Right -- I think the difference is that in the Physical realm,

Re: [I] [CI][R][Docs] Docs jobs failing due to R Markdown error [arrow]

2024-02-05 Thread via GitHub
kou commented on issue #39942: URL: https://github.com/apache/arrow/issues/39942#issuecomment-1926813724 > Quitting from lines 192-195 [unnamed-chunk-14] (arrow.Rmd) https://github.com/apache/arrow/blob/ed78986aa6971484f40a5780922128636a47d175/r/vignettes/arrow.Rmd#L192-L195 Th

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478076768 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1004 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478086838 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1004 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478088185 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1004 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478098378 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1005 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478099334 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1005 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478102841 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1005 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478106057 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1005 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [I] A More General Approach for Optimizing Projections in Physical Plans [arrow-datafusion]

2024-02-05 Thread via GitHub
berkaysynnada commented on issue #9111: URL: https://github.com/apache/arrow-datafusion/issues/9111#issuecomment-1926855711 Yes, as you said, in short, the rule consists of tracking the required columns across the plan and rewriting the plans by new expressions with updated indices. --

Re: [PR] Partial Sort Plan Implementation [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo commented on code in PR #9125: URL: https://github.com/apache/arrow-datafusion/pull/9125#discussion_r1478106835 ## datafusion/physical-plan/src/sorts/partial_sort.rs: ## @@ -0,0 +1,1005 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

Re: [I] `ctx.read_parquet` and `ctx.register_parquet` don't load schema metadata [arrow-datafusion]

2024-02-05 Thread via GitHub
brayanjuls commented on issue #9081: URL: https://github.com/apache/arrow-datafusion/issues/9081#issuecomment-1926858967 @alamb I debugged this section of the code and it is not ignoring the metadata. The issue happens in the following code when listing the table. The metadata is being ig

Re: [I] Add documentation to datafusion-substrait crate about what it does [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #8820: URL: https://github.com/apache/arrow-datafusion/issues/8820#issuecomment-1926864511 Thanks for trying and being so communicative @caicancai - I'll put it on my own list -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] `ctx.read_parquet` and `ctx.register_parquet` don't load schema metadata [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on issue #9081: URL: https://github.com/apache/arrow-datafusion/issues/9081#issuecomment-1926872866 > The metadata is being ignored when building the table schema. Creating the schema from the schema instead of fields in line 551 solved the issue in my local environment

Re: [PR] Remove External Table Backwards Compatibility Options [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9105: URL: https://github.com/apache/arrow-datafusion/pull/9105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [I] Remove External Table Backwards Compatibility Options [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb closed issue #8547: Remove External Table Backwards Compatibility Options URL: https://github.com/apache/arrow-datafusion/issues/8547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Remove External Table Backwards Compatibility Options [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9105: URL: https://github.com/apache/arrow-datafusion/pull/9105#issuecomment-1926879975 Thanks again @yyy1000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat: support `LargeList` in `flatten` [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9110: URL: https://github.com/apache/arrow-datafusion/pull/9110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] feat: support `LargeList` in `flatten` [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9110: URL: https://github.com/apache/arrow-datafusion/pull/9110#issuecomment-1926880619 Thanks @Weijun-H and @comphead for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] feat: improve `make_date` performance [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9112: URL: https://github.com/apache/arrow-datafusion/pull/9112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] feat: improve `make_date` performance [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9112: URL: https://github.com/apache/arrow-datafusion/pull/9112#issuecomment-1926881533 Thanks again @r3stl355 and @Omega359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Refactor min/max value update in Parquet statistics [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on code in PR #9120: URL: https://github.com/apache/arrow-datafusion/pull/9120#discussion_r1478127426 ## datafusion/core/src/datasource/file_format/parquet.rs: ## @@ -303,155 +303,71 @@ fn summarize_min_max( i: usize, stat: &ParquetStatistics, ) { +

Re: [I] `ctx.read_parquet` and `ctx.register_parquet` don't load schema metadata [arrow-datafusion]

2024-02-05 Thread via GitHub
brayanjuls commented on issue #9081: URL: https://github.com/apache/arrow-datafusion/issues/9081#issuecomment-1926883626 Yes, I would like to open a PR to fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Refactor min/max value update in Parquet statistics [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9120: URL: https://github.com/apache/arrow-datafusion/pull/9120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Refactor min/max value update in Parquet statistics [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9120: URL: https://github.com/apache/arrow-datafusion/pull/9120#issuecomment-1926887175 That is a really nice cleanup -- thank you @Weijun-H and @Jefffrey -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] chore: Fix incorrect comment in substrait consumer [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9123: URL: https://github.com/apache/arrow-datafusion/pull/9123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] chore: Fix incorrect comment in substrait consumer [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9123: URL: https://github.com/apache/arrow-datafusion/pull/9123#issuecomment-1926888556 Thanks @caicancai -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Update parse_protobuf_file_scan_config to remove any partition columns from the file_schema in FileScanConfig [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on code in PR #9126: URL: https://github.com/apache/arrow-datafusion/pull/9126#discussion_r1478136051 ## datafusion/proto/src/physical_plan/from_proto.rs: ## @@ -560,9 +560,21 @@ pub fn parse_protobuf_file_scan_config( output_ordering.push(sort_expr);

Re: [PR] Minor: Fix Self referential links in readme [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9119: URL: https://github.com/apache/arrow-datafusion/pull/9119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] MINOR: [Docs][Java] Fix incorrect example [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on PR #39941: URL: https://github.com/apache/arrow/pull/39941#issuecomment-1926939223 @danepitkin @davisusanibar this change looks correct to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] ARROW-15641: [C++][Python] UDF Aggregate Function Implementation [arrow]

2024-02-05 Thread via GitHub
vibhatha closed pull request #14527: ARROW-15641: [C++][Python] UDF Aggregate Function Implementation URL: https://github.com/apache/arrow/pull/14527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] GH-14965: [C++] Enable Substrait ReadRel Projection in Acero [arrow]

2024-02-05 Thread via GitHub
vibhatha closed pull request #14966: GH-14965: [C++] Enable Substrait ReadRel Projection in Acero URL: https://github.com/apache/arrow/pull/14966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-37413: [Java] ZSTD compression issue [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on PR #38396: URL: https://github.com/apache/arrow/pull/38396#issuecomment-1927004105 @pitrou @lidavidm shall we close this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-39900: [Java][CI] To upload Maven and Memory Netty Buffer Patch into Apache Nightly repository [arrow]

2024-02-05 Thread via GitHub
davisusanibar commented on PR #39901: URL: https://github.com/apache/arrow/pull/39901#issuecomment-1927009730 @kou 1.) Upload Task: https://github.com/ursacomputing/crossbow/actions/runs/032405/job/21205283534 ``` $ archery crossbow \ --queue-path $(pwd)/crossb

Re: [I] [Python] Feature to append row groups to existing parquet file [arrow]

2024-02-05 Thread via GitHub
vkhodygo commented on issue #33362: URL: https://github.com/apache/arrow/issues/33362#issuecomment-1927039910 Any progress so far? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Change ScalarValue::Struct to ArrayRef [arrow-datafusion]

2024-02-05 Thread via GitHub
jayzhan211 commented on PR #7893: URL: https://github.com/apache/arrow-datafusion/pull/7893#issuecomment-1927041848 Rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] GH-38703: [C++][FS][Azure] Implement DeleteFile() [arrow]

2024-02-05 Thread via GitHub
av8or1 commented on code in PR #39840: URL: https://github.com/apache/arrow/pull/39840#discussion_r1478287543 ## cpp/src/arrow/filesystem/azurefs_test.cc: ## @@ -1382,6 +1382,32 @@ TEST_F(TestAzuriteFileSystem, DeleteDirContentsFailureNonexistent) { this->TestDeleteDirConten

Re: [PR] GH-38703: [C++][FS][Azure] Implement DeleteFile() [arrow]

2024-02-05 Thread via GitHub
av8or1 commented on code in PR #39840: URL: https://github.com/apache/arrow/pull/39840#discussion_r1478287543 ## cpp/src/arrow/filesystem/azurefs_test.cc: ## @@ -1382,6 +1382,32 @@ TEST_F(TestAzuriteFileSystem, DeleteDirContentsFailureNonexistent) { this->TestDeleteDirConten

Re: [I] [C++][Parquet] Thread crash at exit in Arrow-15.0.0 dataset scan [arrow]

2024-02-05 Thread via GitHub
mattaubury commented on issue #39862: URL: https://github.com/apache/arrow/issues/39862#issuecomment-1927068623 Thanks for trying it out @mapleFU. I suspect this is very timing / number of threads related so might not show up on some machines (I'm running on some reasonably high end Intel s

Re: [PR] GH-39900: [Java][CI] To upload Maven and Memory Netty Buffer Patch into Apache Nightly repository [arrow]

2024-02-05 Thread via GitHub
davisusanibar commented on PR #39901: URL: https://github.com/apache/arrow/pull/39901#issuecomment-1927069370 > @kou > > 1.) Upload Task: https://github.com/ursacomputing/crossbow/actions/runs/032405/job/21205283534 > > ``` > $ archery crossbow \ > --queue-path

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on code in PR #39694: URL: https://github.com/apache/arrow/pull/39694#discussion_r1478303027 ## java/dev/checkstyle/checkstyle.xml: ## @@ -223,14 +224,8 @@ - - -

Re: [PR] improved support for struct columns with missing values [arrow-julia]

2024-02-05 Thread via GitHub
baumgold commented on PR #498: URL: https://github.com/apache/arrow-julia/pull/498#issuecomment-1927084596 @quinnj / @ericphanson - Any questions/comments/concerns here? If not I'd like to merge and release. Thanks. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on code in PR #39694: URL: https://github.com/apache/arrow/pull/39694#discussion_r1478307482 ## java/dev/checkstyle/checkstyle.xml: ## @@ -223,14 +224,8 @@ - - -

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on PR #39694: URL: https://github.com/apache/arrow/pull/39694#issuecomment-1927085609 @github-actions crossbow submit *java* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on code in PR #39694: URL: https://github.com/apache/arrow/pull/39694#discussion_r1478307482 ## java/dev/checkstyle/checkstyle.xml: ## @@ -223,14 +224,8 @@ - - -

Re: [PR] GH-37938: [Swift] initial impl of C Data interface [arrow]

2024-02-05 Thread via GitHub
abandy commented on PR #39091: URL: https://github.com/apache/arrow/pull/39091#issuecomment-1927086372 Added local integration/sanity testing for the c data interface between go and swift. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
github-actions[bot] commented on PR #39694: URL: https://github.com/apache/arrow/pull/39694#issuecomment-1927090402 Revision: ca838fb9d9b548e2a15b01867e875ffdd410cb56 Submitted crossbow builds: [ursacomputing/crossbow @ actions-d6ad59a58d](https://github.com/ursacomputing/crossbow/bra

Re: [PR] Add `ColumnarValue::values_to_arrays`, deprecate `columnar_values_to_array` [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb merged PR #9114: URL: https://github.com/apache/arrow-datafusion/pull/9114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] Update minimum rust version to 1.72 [arrow-datafusion]

2024-02-05 Thread via GitHub
Omega359 commented on PR #8997: URL: https://github.com/apache/arrow-datafusion/pull/8997#issuecomment-1927094492 @alamb You missed the cli Dockerfile unfortunately - it's still 1.70 and won't build without bumping up the version. -- This is an automated message from the Apache Git Servi

Re: [PR] Add `ColumnarValue::values_to_arrays`, deprecate `columnar_values_to_array` [arrow-datafusion]

2024-02-05 Thread via GitHub
alamb commented on PR #9114: URL: https://github.com/apache/arrow-datafusion/pull/9114#issuecomment-1927093356 Thanks again for the review @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [Java] Bump com.puppycrawl.tools:checkstyle from 8.19 to 8.29 [arrow]

2024-02-05 Thread via GitHub
vibhatha commented on code in PR #39694: URL: https://github.com/apache/arrow/pull/39694#discussion_r1478303027 ## java/dev/checkstyle/checkstyle.xml: ## @@ -223,14 +224,8 @@ - - -

Re: [I] join statement causes panic [arrow-datafusion]

2024-02-05 Thread via GitHub
DDtKey commented on issue #7931: URL: https://github.com/apache/arrow-datafusion/issues/7931#issuecomment-1927108803 I can conform it seems to work fine, I had another scenario with the same issue and it works against main branch. Thank you @korowa! -- This is an automated message fro

[PR] Consider table scan filter during analysis of optimize projections [arrow-datafusion]

2024-02-05 Thread via GitHub
mustafasrepo opened a new pull request, #9131: URL: https://github.com/apache/arrow-datafusion/pull/9131 ## Which issue does this PR close? Closes [#9109](https://github.com/apache/arrow-datafusion/issues/9109). ## Rationale for this change ## What changes

Re: [PR] Update parse_protobuf_file_scan_config to remove any partition columns from the file_schema in FileScanConfig [arrow-datafusion]

2024-02-05 Thread via GitHub
bcmcmill commented on code in PR #9126: URL: https://github.com/apache/arrow-datafusion/pull/9126#discussion_r1478331095 ## datafusion/proto/src/physical_plan/from_proto.rs: ## @@ -560,9 +560,21 @@ pub fn parse_protobuf_file_scan_config( output_ordering.push(sort_expr);

Re: [PR] PyBallista - Python SQL client for Ballista [arrow-ballista]

2024-02-05 Thread via GitHub
andygrove merged PR #970: URL: https://github.com/apache/arrow-ballista/pull/970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] GH-34865: [C++][Java][Flight RPC] Add Session management messages [arrow]

2024-02-05 Thread via GitHub
lidavidm commented on code in PR #34817: URL: https://github.com/apache/arrow/pull/34817#discussion_r1478339887 ## format/Flight.proto: ## Review Comment: @pitrou @zeroshade @jduo @emkornfield are we satisfied with the API now? I'd like to avoid bikeshedding during the act

  1   2   3   4   >