[GitHub] [arrow-datafusion] viirya closed issue #5314: Use compute_op_dyn_scalar for datatime

2023-03-02 Thread via GitHub
viirya closed issue #5314: Use compute_op_dyn_scalar for datatime URL: https://github.com/apache/arrow-datafusion/issues/5314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [arrow-datafusion] viirya merged pull request #5315: Use compute_op_dyn_scalar for datatime

2023-03-02 Thread via GitHub
viirya merged PR #5315: URL: https://github.com/apache/arrow-datafusion/pull/5315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow-datafusion] viirya commented on pull request #5315: Use compute_op_dyn_scalar for datatime

2023-03-02 Thread via GitHub
viirya commented on PR #5315: URL: https://github.com/apache/arrow-datafusion/pull/5315#issuecomment-1451449523 Thanks for review. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [arrow-datafusion] ursabot commented on pull request #5315: Use compute_op_dyn_scalar for datatime

2023-03-02 Thread via GitHub
ursabot commented on PR #5315: URL: https://github.com/apache/arrow-datafusion/pull/5315#issuecomment-1451461112 Benchmark runs are scheduled for baseline = a4b47d8c88cb859b2ffa7341d65763cf221ff2be and contender = f68214dc6553ede5f23ba54cac6150f4c1e18e8c. f68214dc6553ede5f23ba54cac6150f4c

[GitHub] [arrow-datafusion] waynexia opened a new issue, #5449: Add `name()` method to `UserDefinedLogicalNode`

2023-03-02 Thread via GitHub
waynexia opened a new issue, #5449: URL: https://github.com/apache/arrow-datafusion/issues/5449 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated when

[GitHub] [arrow-datafusion] waynexia commented on issue #5449: Add `name()` method to `UserDefinedLogicalNode`

2023-03-02 Thread via GitHub
waynexia commented on issue #5449: URL: https://github.com/apache/arrow-datafusion/issues/5449#issuecomment-1451472680 Another example from iox: https://github.com/influxdata/influxdb_iox/blob/f3267f992ae338973ea71df169b91015c6dc1473/iox_query/src/exec/context.rs#L94 -- This is an automa

[GitHub] [arrow] XinyuZeng commented on issue #21238: [C++] Improve C++ Orc Adapter performance and memory footprint

2023-03-02 Thread via GitHub
XinyuZeng commented on issue #21238: URL: https://github.com/apache/arrow/issues/21238#issuecomment-1451478855 @iajoiner Do you still plan working on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow-datafusion] waynexia opened a new pull request, #5450: feat: add name() method to UserDefinedLogicalNode

2023-03-02 Thread via GitHub
waynexia opened a new pull request, #5450: URL: https://github.com/apache/arrow-datafusion/pull/5450 # Which issue does this PR close? Closes #5449. # Rationale for this change Add `name()` method so that there is no need to test a `UserDefinedLogicalNode

[GitHub] [arrow] jorisvandenbossche commented on issue #34374: [C++] Investigate regressions caused by changing row group size from 64Mi to 1Mi.

2023-03-02 Thread via GitHub
jorisvandenbossche commented on issue #34374: URL: https://github.com/apache/arrow/issues/34374#issuecomment-1451488304 Opened https://github.com/apache/arrow/issues/34410 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [arrow-datafusion] dependabot[bot] opened a new pull request, #5451: build(deps): update sqlparser requirement from 0.30 to 0.31

2023-03-02 Thread via GitHub
dependabot[bot] opened a new pull request, #5451: URL: https://github.com/apache/arrow-datafusion/pull/5451 Updates the requirements on [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) to permit the latest version. Changelog Sourced from https://github.com/sqlparser-rs/sq

[GitHub] [arrow] Fokko commented on a diff in pull request #34184: GH-34154: [Python] Add `is_nan` expression

2023-03-02 Thread via GitHub
Fokko commented on code in PR #34184: URL: https://github.com/apache/arrow/pull/34184#discussion_r1122788284 ## python/pyarrow/array.pxi: ## @@ -1253,6 +1253,16 @@ cdef class Array(_PandasConvertible): options = _pc().NullOptions(nan_is_null=nan_is_null) return

[GitHub] [arrow-datafusion] yukkit opened a new pull request, #5452: fix: build union schema with child has same column name but qualifier…

2023-03-02 Thread via GitHub
yukkit opened a new pull request, #5452: URL: https://github.com/apache/arrow-datafusion/pull/5452 … is different # Which issue does this PR close? Closes #5410 . # Rationale for this change # What changes are included in this PR? # A

[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #34275: GH-21761: [Python] Change array constructor to accept pyarrow array

2023-03-02 Thread via GitHub
jorisvandenbossche commented on code in PR #34275: URL: https://github.com/apache/arrow/pull/34275#discussion_r1122826799 ## python/pyarrow/tests/test_array.py: ## @@ -3381,3 +3381,15 @@ def test_struct_array_sort(): {"a": 5, "b": "foo"}, None ] + + +def t

[GitHub] [arrow] raulcd commented on issue #34381: [Dev] PR Workflow incorrectly tagging committer reviews that have their Apache membership set to private

2023-03-02 Thread via GitHub
raulcd commented on issue #34381: URL: https://github.com/apache/arrow/issues/34381#issuecomment-1451623346 Thanks @kou I forgot this existed. Yes that should work. I'll take the committer GitHub usernames from there. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] ursabot commented on pull request #34373: GH-34333: [Python] Test run_query with a registered scalar UDF

2023-03-02 Thread via GitHub
ursabot commented on PR #34373: URL: https://github.com/apache/arrow/pull/34373#issuecomment-1451635578 Benchmark runs are scheduled for baseline = 4c1448e85011c24f2dde087dc75035c91be7afcd and contender = e8107bfa58ef5ad50c5c40d3f54bb7a96bdf2d0e. e8107bfa58ef5ad50c5c40d3f54bb7a96bdf2d0e is

[GitHub] [arrow] extabgrad commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
extabgrad commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451636140 MATLAB’s Deep Learning Toolbox uses n-dimensional arrays which fit quite well with the proposal. It also has a special datatype called “dlarray” which is responsible for automatic diffe

[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #34408: GH-34361: [C++] work-in-progress: Add skeleton of the new APIs for handling null checks correctly for all types

2023-03-02 Thread via GitHub
jorisvandenbossche commented on code in PR #34408: URL: https://github.com/apache/arrow/pull/34408#discussion_r1122928924 ## cpp/src/arrow/array/data.h: ## @@ -229,15 +256,88 @@ struct ARROW_EXPORT ArrayData { void SetNullCount(int64_t v) { null_count.store(v); } - /// \

[GitHub] [arrow] rok commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
rok commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451695480 Thanks you for the input and description of MATLAB's Deep Learning Toolbox @extabgrad ! > Therefore our main input would be to request that permutation and dim_names are not mutually e

[GitHub] [arrow-nanoarrow] jorisvandenbossche commented on pull request #142: fix: include compilers in conda instructions

2023-03-02 Thread via GitHub
jorisvandenbossche commented on PR #142: URL: https://github.com/apache/arrow-nanoarrow/pull/142#issuecomment-1451715720 I needed this to get the verification working locally. If this is not included, it is detecting and using my system gcc, which resulted in some compilation errors.

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122952139 ## cpp/src/arrow/extension/fixed_shape_tensor.h: ## @@ -0,0 +1,87 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122952877 ## cpp/src/arrow/extension/fixed_shape_tensor.cc: ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122953445 ## cpp/src/arrow/extension/fixed_shape_tensor_test.cc: ## @@ -0,0 +1,188 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122956726 ## cpp/src/arrow/extension/fixed_shape_tensor_test.cc: ## @@ -0,0 +1,188 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122960281 ## cpp/src/arrow/extension/fixed_shape_tensor.h: ## @@ -0,0 +1,87 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122960585 ## cpp/src/arrow/extension/fixed_shape_tensor.cc: ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

[GitHub] [arrow] jorisvandenbossche commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
jorisvandenbossche commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451727185 > > We would also like to ensure that the format will support complex data. > > Just to be clear: by complex you mean diverse not complex as in complex numbers? An

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122962212 ## cpp/src/arrow/extension/fixed_shape_tensor.cc: ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

[GitHub] [arrow] rok commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
rok commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1122962984 ## cpp/src/arrow/extension/fixed_shape_tensor.h: ## @@ -0,0 +1,87 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

[GitHub] [arrow-rs] MachaelLee opened a new pull request, #3787: feat: add a function to parse a string without timezone to local timestamp

2023-03-02 Thread via GitHub
MachaelLee opened a new pull request, #3787: URL: https://github.com/apache/arrow-rs/pull/3787 # Which issue does this PR close? None Closes #. # Rationale for this change Before modification with #2814, function `string_to_timestamp_nanos` return a local timest

[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #34184: GH-34154: [Python] Add `is_nan` expression

2023-03-02 Thread via GitHub
jorisvandenbossche commented on code in PR #34184: URL: https://github.com/apache/arrow/pull/34184#discussion_r1122969200 ## python/pyarrow/array.pxi: ## @@ -1253,6 +1253,16 @@ cdef class Array(_PandasConvertible): options = _pc().NullOptions(nan_is_null=nan_is_null)

[GitHub] [arrow] jorisvandenbossche commented on issue #34403: [Python] Is there a way to construct the metadata_collector for an existing partitioned dataset?

2023-03-02 Thread via GitHub
jorisvandenbossche commented on issue #34403: URL: https://github.com/apache/arrow/issues/34403#issuecomment-1451740151 Yes, that's certainly possible. If you check the example in the docs about this (https://arrow.apache.org/docs/dev/python/parquet.html#writing-metadata-and-common-m

[GitHub] [arrow-datafusion] jiangzhx commented on issue #5444: Expr.alias function not work with count aggregation

2023-03-02 Thread via GitHub
jiangzhx commented on issue #5444: URL: https://github.com/apache/arrow-datafusion/issues/5444#issuecomment-1451745719 found some thing not right. https://github.com/apache/arrow-datafusion/blob/f68214dc6553ede5f23ba54cac6150f4c1e18e8c/datafusion/core/src/execution/context.rs#L1566

[GitHub] [arrow] AlenkaF commented on a diff in pull request #34275: GH-21761: [Python] Change array constructor to accept pyarrow array

2023-03-02 Thread via GitHub
AlenkaF commented on code in PR #34275: URL: https://github.com/apache/arrow/pull/34275#discussion_r1122989311 ## python/pyarrow/array.pxi: ## @@ -232,6 +232,11 @@ def array(object obj, type=None, mask=None, size=None, from_pandas=None, else: c_from_pandas = from_

[GitHub] [arrow] thisisnic commented on pull request #34399: GH-34398: [R] Update NEWS.md for 11.0.0.3

2023-03-02 Thread via GitHub
thisisnic commented on PR #34399: URL: https://github.com/apache/arrow/pull/34399#issuecomment-1451759350 > Looks good! > > You can include the schema commit if you want to, but I would personally just include the dplyr fix in this particular patch release unless the schema PR was in

[GitHub] [arrow] wgtmac commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
wgtmac commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1451760791 I am looking for a similar feature and came across this issue. I know we have supported JDBC to Arrow but not in the inverse direction. Do we have any plan to support it? @lidavidm

[GitHub] [arrow-ballista] Dandandan opened a new issue, #694: Make `with_task_launcher` pub

2023-03-02 Thread via GitHub
Dandandan opened a new issue, #694: URL: https://github.com/apache/arrow-ballista/issues/694 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** I would like to modify the task launcher. **Describe the solution you'd like** M

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #3778: Use NullBuffer in ArrayData (#3775)

2023-03-02 Thread via GitHub
tustvold commented on code in PR #3778: URL: https://github.com/apache/arrow-rs/pull/3778#discussion_r1123012909 ## arrow-data/src/data/mod.rs: ## @@ -1704,6 +1710,7 @@ pub struct ArrayDataBuilder { len: usize, null_count: Option, Review Comment: Yes, so that we ca

[GitHub] [arrow-ballista] Dandandan opened a new pull request, #695: Make task launcher pub

2023-03-02 Thread via GitHub
Dandandan opened a new pull request, #695: URL: https://github.com/apache/arrow-ballista/pull/695 # Which issue does this PR close? Closes #. # Rationale for this change # What changes are included in this PR? # Are there any user-facing changes?

[GitHub] [arrow-datafusion] Dandandan commented on pull request #5408: refactor count_distinct to not to have update and merge

2023-03-02 Thread via GitHub
Dandandan commented on PR #5408: URL: https://github.com/apache/arrow-datafusion/pull/5408#issuecomment-1451792586 Merging this in 24 hours if no other comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] lidavidm commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
lidavidm commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1451797340 https://github.com/apache/arrow-adbc/tree/main/java/driver/jdbc can do it but needs assistance generating the SQL https://github.com/apache/arrow/blob/main/java/adapter/jdbc/

[GitHub] [arrow-datafusion] jiangzhx commented on issue #5444: Expr.alias function not work with count aggregation

2023-03-02 Thread via GitHub
jiangzhx commented on issue #5444: URL: https://github.com/apache/arrow-datafusion/issues/5444#issuecomment-1451815153 more and more research. https://github.com/apache/arrow-datafusion/blob/f68214dc6553ede5f23ba54cac6150f4c1e18e8c/datafusion/core/src/physical_optimizer/aggregate_sta

[GitHub] [arrow-rs] tustvold closed issue #3775: Use NullBuffer in ArrayData

2023-03-02 Thread via GitHub
tustvold closed issue #3775: Use NullBuffer in ArrayData URL: https://github.com/apache/arrow-rs/issues/3775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[GitHub] [arrow-rs] tustvold merged pull request #3778: Use NullBuffer in ArrayData (#3775)

2023-03-02 Thread via GitHub
tustvold merged PR #3778: URL: https://github.com/apache/arrow-rs/pull/3778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]

[GitHub] [arrow-rs] tustvold commented on pull request #3786: minor: make Parquet CLI input args consistent

2023-03-02 Thread via GitHub
tustvold commented on PR #3786: URL: https://github.com/apache/arrow-rs/pull/3786#issuecomment-1451829532 What do you think of unifying in the opposite direction, i.e. making the other tools take positional arguments? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] extabgrad commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
extabgrad commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451831380 > Just to be clear: by complex you mean diverse not complex as in complex numbers? No, I mean complex numbers, which are increasingly used in AI workflows. > `shape`, `dim_

[GitHub] [arrow-rs] ursabot commented on pull request #3778: Use NullBuffer in ArrayData (#3775)

2023-03-02 Thread via GitHub
ursabot commented on PR #3778: URL: https://github.com/apache/arrow-rs/pull/3778#issuecomment-1451832486 Benchmark runs are scheduled for baseline = 7852e763fea66b33a2766b6d6421cafcf6a58c29 and contender = eff058fc7a156d1b22569bd60a747d98960d97e7. eff058fc7a156d1b22569bd60a747d98960d97e7 i

[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #8510: GH-15483: [C++] Add a Fixed Shape Tensor canonical ExtensionType

2023-03-02 Thread via GitHub
jorisvandenbossche commented on code in PR #8510: URL: https://github.com/apache/arrow/pull/8510#discussion_r1123064708 ## cpp/src/arrow/extension/fixed_shape_tensor_test.cc: ## @@ -0,0 +1,188 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribu

[GitHub] [arrow-rs] crepererum opened a new pull request, #3788: refactor: assorted `FlightSqlServiceClient` improvements

2023-03-02 Thread via GitHub
crepererum opened a new pull request, #3788: URL: https://github.com/apache/arrow-rs/pull/3788 # Which issue does this PR close? \- # Rationale for this change Assorted `FlightSqlServiceClient` improvements: - **TLS config:** Do NOT alter existing method signatures if the

[GitHub] [arrow-datafusion] jiangzhx commented on issue #5444: Expr.alias function not work with count aggregation

2023-03-02 Thread via GitHub
jiangzhx commented on issue #5444: URL: https://github.com/apache/arrow-datafusion/issues/5444#issuecomment-1451841791 The following code can resolve my problem. but anyone can give some good suggestion to make code look nice. ``` #[tokio::test] async fn count_df() -> Result<()>

[GitHub] [arrow-ballista] Dandandan merged pull request #695: Mark `SchedulerServer::with_task_launcher` as pub

2023-03-02 Thread via GitHub
Dandandan merged PR #695: URL: https://github.com/apache/arrow-ballista/pull/695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-ballista] Dandandan closed issue #694: Make `with_task_launcher` pub

2023-03-02 Thread via GitHub
Dandandan closed issue #694: Make `with_task_launcher` pub URL: https://github.com/apache/arrow-ballista/issues/694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[GitHub] [arrow-rs] XinyuZeng commented on pull request #3786: minor: make Parquet CLI input args consistent

2023-03-02 Thread via GitHub
XinyuZeng commented on PR #3786: URL: https://github.com/apache/arrow-rs/pull/3786#issuecomment-1451852976 Positional arguments may not be well suited for tools like parquet-fromcsv and parquet-rewrite, which have too many arguments (and many file names). But for the tools that only require

[GitHub] [arrow-rs] crepererum opened a new pull request, #3789: feat: simple flight sql CLI client

2023-03-02 Thread via GitHub
crepererum opened a new pull request, #3789: URL: https://github.com/apache/arrow-rs/pull/3789 **:information_source: Marked as WIP because this is based on #3788.** # Which issue does this PR close? \- # Rationale for this change It's a bit of a pity that the Rust ecosyst

[GitHub] [arrow-nanoarrow] paleolimbot commented on pull request #142: fix: include compilers in conda instructions

2023-03-02 Thread via GitHub
paleolimbot commented on PR #142: URL: https://github.com/apache/arrow-nanoarrow/pull/142#issuecomment-1451855643 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #142: fix: include compilers in conda instructions

2023-03-02 Thread via GitHub
paleolimbot merged PR #142: URL: https://github.com/apache/arrow-nanoarrow/pull/142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #141: [R] Test failure on 0.1-RC1 verification

2023-03-02 Thread via GitHub
paleolimbot commented on issue #141: URL: https://github.com/apache/arrow-nanoarrow/issues/141#issuecomment-1451864660 Yes, it's expected that verification passes without the 'arrow' R package (although some extra tests will run if it is installed and those should pass as well). If you did

[GitHub] [arrow] github-actions[bot] commented on pull request #34275: GH-34411: [Python] Change array constructor to accept pyarrow array

2023-03-02 Thread via GitHub
github-actions[bot] commented on PR #34275: URL: https://github.com/apache/arrow/pull/34275#issuecomment-1451867487 * Closes: #34411 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [arrow] jorisvandenbossche merged pull request #34275: GH-34411: [Python] Change array constructor to accept pyarrow array

2023-03-02 Thread via GitHub
jorisvandenbossche merged PR #34275: URL: https://github.com/apache/arrow/pull/34275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

[GitHub] [arrow] jorisvandenbossche commented on issue #21761: [Python] accept pyarrow values / scalars in constructor functions ?

2023-03-02 Thread via GitHub
jorisvandenbossche commented on issue #21761: URL: https://github.com/apache/arrow/issues/21761#issuecomment-1451870190 The part about accepting an Array object is solved now with https://github.com/apache/arrow/pull/34275. Leaving this issue open for the Scalar case. -- This is an autom

[GitHub] [arrow] lidavidm commented on issue #34393: [Java] Arrow deserialization performance is so poor in Java

2023-03-02 Thread via GitHub
lidavidm commented on issue #34393: URL: https://github.com/apache/arrow/issues/34393#issuecomment-1451874520 The Java library doesn't really have compute utilities like that. @lwhite1 was there anything related to that in the Table work? -- This is an automated message from the Apache Gi

[GitHub] [arrow] lidavidm merged pull request #34390: MINOR: [Docs][Java] Update status documentation for Java's CSV read support (as per ARROW-17786)

2023-03-02 Thread via GitHub
lidavidm merged PR #34390: URL: https://github.com/apache/arrow/pull/34390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]

[GitHub] [arrow] lidavidm commented on issue #34021: [Java] BufferAllocator allocated memory always increase

2023-03-02 Thread via GitHub
lidavidm commented on issue #34021: URL: https://github.com/apache/arrow/issues/34021#issuecomment-1451880878 Hmm, it appears doAction is missing that. If you want to file a new issue, that could be addressed. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow-ballista] Dandandan opened a new pull request, #696: Make task_manager pub

2023-03-02 Thread via GitHub
Dandandan opened a new pull request, #696: URL: https://github.com/apache/arrow-ballista/pull/696 # Which issue does this PR close? Closes #. # Rationale for this change # What changes are included in this PR? # Are there any user-facing changes?

[GitHub] [arrow-rs] tustvold commented on issue #3784: object_store: Why does builder take bucket?

2023-03-02 Thread via GitHub
tustvold commented on issue #3784: URL: https://github.com/apache/arrow-rs/issues/3784#issuecomment-1451884876 There a couple of reasons but the most compelling reason is so that the abstraction can then be mapped onto stores that don't have a similar namespacing concept, e.g. local filesys

[GitHub] [arrow-rs] tustvold commented on pull request #3786: minor: make Parquet CLI input args consistent

2023-03-02 Thread via GitHub
tustvold commented on PR #3786: URL: https://github.com/apache/arrow-rs/pull/3786#issuecomment-1451890324 I think it is fine for the more complex tools to have a different argument pattern from those that take a single input file -- This is an automated message from the Apache Git Service

[GitHub] [arrow] rok commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
rok commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451894336 > No, I mean complex numbers, which are increasingly used in AI workflows. Interesting! As Joris states there is an independent effort to enable that. > > `shape`, `dim_names` and

[GitHub] [arrow-rs] tustvold commented on pull request #3787: feat: add a function to parse a string without timezone to local timestamp

2023-03-02 Thread via GitHub
tustvold commented on PR #3787: URL: https://github.com/apache/arrow-rs/pull/3787#issuecomment-1451905983 Thank you for this, I would like to take some time to think about this API. I think we might want a way to specify the destination timezone for the function, perhaps adding the ability

[GitHub] [arrow-ballista] Dandandan opened a new issue, #697: Make `task_manager` module pub

2023-03-02 Thread via GitHub
Dandandan opened a new issue, #697: URL: https://github.com/apache/arrow-ballista/issues/697 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [.

[GitHub] [arrow] wgtmac commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
wgtmac commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1451951570 > https://github.com/apache/arrow-adbc/tree/main/java/driver/jdbc can do it but needs assistance generating the SQL > > https://github.com/apache/arrow/blob/main/java/adapter/j

[GitHub] [arrow] lidavidm commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
lidavidm commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1451961662 Ah. Interesting. Sorry, I misunderstood. There is stuff like that in the JDBC driver but not exposed for public use. What is the use case? -- This is an automated message f

[GitHub] [arrow] wgtmac commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
wgtmac commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1451989813 To provide standard JDBC access to a database or engine that speaks arrow vector natively. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [arrow-rs] alamb commented on a diff in pull request #3715: Add `into_primitive_dict_builder` to `DictionaryArray`

2023-03-02 Thread via GitHub
alamb commented on code in PR #3715: URL: https://github.com/apache/arrow-rs/pull/3715#discussion_r1123221345 ## arrow-array/src/array/dictionary_array.rs: ## @@ -941,4 +982,62 @@ mod tests { let a = DictionaryArrayfrom_iter(["32"]); let _ = DictionaryArray

[GitHub] [arrow] lwhite1 commented on issue #34338: [Java] BaseAllocator.DEBUG should be opt-in as HistoricalLog is immensely expensive and breaks profiling

2023-03-02 Thread via GitHub
lwhite1 commented on issue #34338: URL: https://github.com/apache/arrow/issues/34338#issuecomment-1452005098 I think this is a good change. The current setup can lead new users to the impression that Arrow Java is slow, when it's really just a matter of configuration. -- This is an auto

[GitHub] [arrow-rs] alamb commented on a diff in pull request #3788: refactor: assorted `FlightSqlServiceClient` improvements

2023-03-02 Thread via GitHub
alamb commented on code in PR #3788: URL: https://github.com/apache/arrow-rs/pull/3788#discussion_r1123226359 ## arrow-flight/src/sql/client.rs: ## @@ -83,13 +75,23 @@ impl FlightSqlServiceClient { /// Creates a new HTTPs FlightSql Client that connects via TCP to a server

[GitHub] [arrow] felipecrv commented on a diff in pull request #34408: GH-34361: [C++] work-in-progress: Add skeleton of the new APIs for handling null checks correctly for all types

2023-03-02 Thread via GitHub
felipecrv commented on code in PR #34408: URL: https://github.com/apache/arrow/pull/34408#discussion_r1123232448 ## cpp/src/arrow/array/data.h: ## @@ -229,15 +256,88 @@ struct ARROW_EXPORT ArrayData { void SetNullCount(int64_t v) { null_count.store(v); } - /// \brief Ret

[GitHub] [arrow-rs] alamb commented on a diff in pull request #3783: Return Buffers from ArrayData::buffers instead of slice (#1799)

2023-03-02 Thread via GitHub
alamb commented on code in PR #3783: URL: https://github.com/apache/arrow-rs/pull/3783#discussion_r1123232616 ## arrow-data/src/data/buffers.rs: ## @@ -0,0 +1,97 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See th

[GitHub] [arrow] felipecrv commented on a diff in pull request #34408: GH-34361: [C++] work-in-progress: Add skeleton of the new APIs for handling null checks correctly for all types

2023-03-02 Thread via GitHub
felipecrv commented on code in PR #34408: URL: https://github.com/apache/arrow/pull/34408#discussion_r1123232448 ## cpp/src/arrow/array/data.h: ## @@ -229,15 +256,88 @@ struct ARROW_EXPORT ArrayData { void SetNullCount(int64_t v) { null_count.store(v); } - /// \brief Ret

[GitHub] [arrow] lidavidm commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
lidavidm commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1452019973 Ok, cool. (So this is like an embeddable engine? I assume otherwise you could use Flight SQL.) Your best bet is to copy or refactor the code that is part of the JDBC driver.

[GitHub] [arrow-rs] alamb commented on a diff in pull request #3781: Make LocalFileSystem::put atomic (#3780)

2023-03-02 Thread via GitHub
alamb commented on code in PR #3781: URL: https://github.com/apache/arrow-rs/pull/3781#discussion_r1123243987 ## object_store/src/lib.rs: ## @@ -840,6 +840,12 @@ mod tests { crate::Error::NotFound { .. } )); +let files = flatten_list_stream(storag

[GitHub] [arrow] lidavidm commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
lidavidm commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1452020890 This is also different from this original issue, since this issue was about writing Arrow data to a database via JDBC -- This is an automated message from the Apache Git Service.

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #3783: Return Buffers from ArrayData::buffers instead of slice (#1799)

2023-03-02 Thread via GitHub
tustvold commented on code in PR #3783: URL: https://github.com/apache/arrow-rs/pull/3783#discussion_r1123246737 ## arrow-data/src/data/buffers.rs: ## @@ -0,0 +1,97 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See

[GitHub] [arrow] extabgrad commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

2023-03-02 Thread via GitHub
extabgrad commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1452026817 > This proposal currently states: `Elements in a fixed shape tensor extension array are stored in row-major/C-contiguous order.`. We can amend that to be more general. Could you state w

[GitHub] [arrow] wgtmac commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
wgtmac commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1452028953 Did you mean this? https://github.com/apache/arrow/blob/main/java/flight/flight-sql-jdbc-driver/src/main/java/org/apache/arrow/driver/jdbc/ArrowFlightJdbcVectorSchemaRootResultSet.java

[GitHub] [arrow-rs] crepererum commented on a diff in pull request #3788: refactor: assorted `FlightSqlServiceClient` improvements

2023-03-02 Thread via GitHub
crepererum commented on code in PR #3788: URL: https://github.com/apache/arrow-rs/pull/3788#discussion_r1123256026 ## arrow-flight/src/sql/client.rs: ## @@ -83,13 +75,23 @@ impl FlightSqlServiceClient { /// Creates a new HTTPs FlightSql Client that connects via TCP to a s

[GitHub] [arrow-datafusion] alamb merged pull request #5434: minor: Port more window tests to sqlogictests

2023-03-02 Thread via GitHub
alamb merged PR #5434: URL: https://github.com/apache/arrow-datafusion/pull/5434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow] lidavidm commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
lidavidm commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1452032629 Yes, code like that. (I suppose the Avatica dependency means we may not want to add it to arrow-jdbc, but it's already public - does it sort of work? We can talk about refactoring i

[GitHub] [arrow-datafusion] alamb commented on issue #3463: Enable parquet filter pushdown by default

2023-03-02 Thread via GitHub
alamb commented on issue #3463: URL: https://github.com/apache/arrow-datafusion/issues/3463#issuecomment-1452036106 Update here is I am working on a larger benchmarking story, part of which would give us more confidence to merge changes like this in. I hope to have that done early next wee

[GitHub] [arrow] wgtmac commented on issue #20730: [Java] Arrow-to-JDBC

2023-03-02 Thread via GitHub
wgtmac commented on issue #20730: URL: https://github.com/apache/arrow/issues/20730#issuecomment-1452038801 Yes, it seems like what I am searching for. I need some time to read into it before a concrete conclusion. Thanks for your help! -- This is an automated message from the Apache Git

[GitHub] [arrow] eitsupi commented on issue #34409: [R] Named lists cannot be serialized to a map column

2023-03-02 Thread via GitHub
eitsupi commented on issue #34409: URL: https://github.com/apache/arrow/issues/34409#issuecomment-1452040990 Maybe related to #15033? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow-rs] alamb commented on a diff in pull request #3788: refactor: assorted `FlightSqlServiceClient` improvements

2023-03-02 Thread via GitHub
alamb commented on code in PR #3788: URL: https://github.com/apache/arrow-rs/pull/3788#discussion_r1123267191 ## arrow-flight/src/sql/client.rs: ## @@ -83,13 +75,23 @@ impl FlightSqlServiceClient { /// Creates a new HTTPs FlightSql Client that connects via TCP to a server

[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #5452: fix: build union schema with child has same column name but qualifier…

2023-03-02 Thread via GitHub
jackwener commented on code in PR #5452: URL: https://github.com/apache/arrow-datafusion/pull/5452#discussion_r1123270256 ## datafusion/expr/src/logical_plan/builder.rs: ## @@ -1101,7 +1101,7 @@ pub fn union(left_plan: LogicalPlan, right_plan: LogicalPlan) -> Result

[GitHub] [arrow-datafusion] ursabot commented on pull request #5434: minor: Port more window tests to sqlogictests

2023-03-02 Thread via GitHub
ursabot commented on PR #5434: URL: https://github.com/apache/arrow-datafusion/pull/5434#issuecomment-1452048327 Benchmark runs are scheduled for baseline = f68214dc6553ede5f23ba54cac6150f4c1e18e8c and contender = a95e0ec2fd929aae1c2f67148243eb4825d81a3b. a95e0ec2fd929aae1c2f67148243eb482

[GitHub] [arrow-rs] alamb commented on pull request #3690: Allow precision loss on multiplying decimal arrays

2023-03-02 Thread via GitHub
alamb commented on PR #3690: URL: https://github.com/apache/arrow-rs/pull/3690#issuecomment-1452049023 FWIW @jackwener has identified that this PR will fix https://github.com/apache/arrow-datafusion/issues/5396d downstream in DataFusion -- This is an automated message from the Apache Git

[GitHub] [arrow] lafiona opened a new pull request, #34413: GH-34256: [Release] Update release scripts with main as new default branch

2023-03-02 Thread via GitHub
lafiona opened a new pull request, #34413: URL: https://github.com/apache/arrow/pull/34413 ### Rationale for this change This is a follow up task after switching the `apache/arrow` default branch name from `master` to `main`. Usages of `master` were evaluated to determine whether they ca

[GitHub] [arrow] github-actions[bot] commented on pull request #34413: GH-34256: [Release] Update release scripts with main as new default branch

2023-03-02 Thread via GitHub
github-actions[bot] commented on PR #34413: URL: https://github.com/apache/arrow/pull/34413#issuecomment-1452052435 * Closes: #34256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [arrow] github-actions[bot] commented on pull request #34413: GH-34256: [Release] Update release scripts with main as new default branch

2023-03-02 Thread via GitHub
github-actions[bot] commented on PR #34413: URL: https://github.com/apache/arrow/pull/34413#issuecomment-1452052480 :warning: GitHub issue #34256 **has no components**, please add labels for components. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [arrow-adbc] lidavidm commented on pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper

2023-03-02 Thread via GitHub
lidavidm commented on PR #480: URL: https://github.com/apache/arrow-adbc/pull/480#issuecomment-1452056230 I'll merge this, there's an open PR to fix ruby -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #5452: fix: build union schema with child has same column name but qualifier…

2023-03-02 Thread via GitHub
jackwener commented on code in PR #5452: URL: https://github.com/apache/arrow-datafusion/pull/5452#discussion_r1123274283 ## datafusion/expr/src/logical_plan/builder.rs: ## @@ -1782,4 +1782,24 @@ mod tests { table_scan(Some(table_name), &schema, None) } + +#[

[GitHub] [arrow-adbc] lidavidm merged pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper

2023-03-02 Thread via GitHub
lidavidm merged PR #480: URL: https://github.com/apache/arrow-adbc/pull/480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]

[GitHub] [arrow-rs] alamb commented on pull request #3401: Rewrite FlightSQL client in terms of the mid level client (WIP)

2023-03-02 Thread via GitHub
alamb commented on PR #3401: URL: https://github.com/apache/arrow-rs/pull/3401#issuecomment-1452058142 This has bitrotted too much -- I will redo the code if we want to pursue the idea -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   3   4   >