[GitHub] [arrow-julia] tpgillam opened a new issue, #327: DST ambiguities in ZonedDateTime not supported

2022-06-28 Thread GitBox
tpgillam opened a new issue, #327: URL: https://github.com/apache/arrow-julia/issues/327 It seems like the ArrowTypes representation of ZonedDateTime doesn't include enough information to resolve ambiguities around DST, e.g.: ```julia julia> zdt = ZonedDateTime(DateTime(2020, 11, 1

[GitHub] [arrow-testing] pitrou commented on pull request #56: ARROW-11417: Add integration files for buffer compression

2022-06-28 Thread GitBox
pitrou commented on PR #56: URL: https://github.com/apache/arrow-testing/pull/56#issuecomment-1168754007 @liukun4515 The compression file was generated using Arrow C++ IIRC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #28: Add basic driver validation suite

2022-07-01 Thread GitBox
lidavidm opened a new pull request, #28: URL: https://github.com/apache/arrow-adbc/pull/28 This is a set of self-contained tests that can be linked in and called from another test suite. Right now they don't deal with any actual data. -- This is an automated message from the Apache Git Se

[GitHub] [arrow-adbc] lidavidm merged pull request #27: Make AdbcConnectionNew 2-adic for consistency

2022-07-05 Thread GitBox
lidavidm merged PR #27: URL: https://github.com/apache/arrow-adbc/pull/27 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #29: Install the headers along with the driver manager

2022-07-05 Thread GitBox
lidavidm opened a new pull request, #29: URL: https://github.com/apache/arrow-adbc/pull/29 Right now we only install the shared library. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [arrow-adbc] lidavidm merged pull request #29: Install the headers along with the driver manager

2022-07-05 Thread GitBox
lidavidm merged PR #29: URL: https://github.com/apache/arrow-adbc/pull/29 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #30: Refactor the Derby driver into a JDBC driver

2022-07-06 Thread GitBox
lidavidm opened a new pull request, #30: URL: https://github.com/apache/arrow-adbc/pull/30 Also factors out a helper module. Currently the only feature is to bind Arrow data to a JDBC prepared statement. Ideally this would be migrated upstream into arrow-jdbc, though! -- This is an autom

[GitHub] [arrow-adbc] lidavidm merged pull request #30: Refactor the Derby driver into a JDBC driver

2022-07-07 Thread GitBox
lidavidm merged PR #30: URL: https://github.com/apache/arrow-adbc/pull/30 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-nanoarrow] paleolimbot opened a new pull request, #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
paleolimbot opened a new pull request, #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1 This PR imports the preliminary work from https://github.com/paleolimbot/nanoarrow (with reviews from @lidavidm and @pitrou and input from the dev mailing list via the [initial design document]

[GitHub] [arrow-nanoarrow] lidavidm commented on a diff in pull request #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
lidavidm commented on code in PR #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1#discussion_r917010603 ## .github/workflows/build-and-test.yaml: ## @@ -0,0 +1,104 @@ +on: Review Comment: Add the Apache license here? ## .gitignore: ## @@ -0,0 +1,6

[GitHub] [arrow-nanoarrow] paleolimbot commented on a diff in pull request #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
paleolimbot commented on code in PR #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1#discussion_r917019272 ## .github/workflows/build-and-test.yaml: ## @@ -0,0 +1,104 @@ +on: Review Comment: Done! ## .gitignore: ## @@ -0,0 +1,6 @@ +build Review C

[GitHub] [arrow-nanoarrow] paleolimbot commented on a diff in pull request #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
paleolimbot commented on code in PR #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1#discussion_r917020377 ## README.md: ## @@ -1 +1,65 @@ -# arrow-nanoarrow + +# nanoarrow + +[![Codecov test coverage](https://codecov.io/gh/paleolimbot/nanoarrow/branch/main/graph/badge

[GitHub] [arrow-nanoarrow] lidavidm commented on pull request #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
lidavidm commented on PR #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1#issuecomment-1179232441 @paleolimbot feel free to merge and continue setting things up -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow-adbc] lidavidm merged pull request #31: Add test suite for Java ADBC drivers

2022-07-08 Thread GitBox
lidavidm merged PR #31: URL: https://github.com/apache/arrow-adbc/pull/31 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #28: Add C driver validation suite

2022-07-08 Thread GitBox
lidavidm merged PR #28: URL: https://github.com/apache/arrow-adbc/pull/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #1: Add files from paleolimbot/nanoarrow

2022-07-08 Thread GitBox
paleolimbot merged PR #1: URL: https://github.com/apache/arrow-nanoarrow/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow

[GitHub] [arrow-adbc] lidavidm opened a new issue, #32: Cleanly separate C/Java code

2022-07-11 Thread GitBox
lidavidm opened a new issue, #32: URL: https://github.com/apache/arrow-adbc/issues/32 We should have a structure like ``` / adbc.h c/ driver_manager driver validation java/ ... ``` (also, the Java "testsuite" package should be ren

[GitHub] [arrow-adbc] lidavidm opened a new issue, #33: Implement database metadata calls in Java

2022-07-11 Thread GitBox
lidavidm opened a new issue, #33: URL: https://github.com/apache/arrow-adbc/issues/33 For C/Java parity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #35: Cleanly separate C/Java code

2022-07-11 Thread GitBox
lidavidm opened a new pull request, #35: URL: https://github.com/apache/arrow-adbc/pull/35 Fixes #32. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

[GitHub] [arrow-adbc] lidavidm merged pull request #34: Clarify bulk ingestion semantics

2022-07-11 Thread GitBox
lidavidm merged PR #34: URL: https://github.com/apache/arrow-adbc/pull/34 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #35: Cleanly separate C/Java code

2022-07-11 Thread GitBox
lidavidm merged PR #35: URL: https://github.com/apache/arrow-adbc/pull/35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm closed issue #32: Cleanly separate C/Java code

2022-07-11 Thread GitBox
lidavidm closed issue #32: Cleanly separate C/Java code URL: https://github.com/apache/arrow-adbc/issues/32 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #36: Implement metadata methods for JDBC

2022-07-12 Thread GitBox
lidavidm opened a new pull request, #36: URL: https://github.com/apache/arrow-adbc/pull/36 Not all of the metadata is currently mapped and there are some more cases to consider in the future. Fixes #33. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow-adbc] lidavidm merged pull request #36: Implement metadata methods for JDBC

2022-07-12 Thread GitBox
lidavidm merged PR #36: URL: https://github.com/apache/arrow-adbc/pull/36 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm closed issue #33: Implement database metadata calls in Java

2022-07-12 Thread GitBox
lidavidm closed issue #33: Implement database metadata calls in Java URL: https://github.com/apache/arrow-adbc/issues/33 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [arrow-adbc] lidavidm opened a new issue, #37: Reorganize and complete Python bindings

2022-07-12 Thread GitBox
lidavidm opened a new issue, #37: URL: https://github.com/apache/arrow-adbc/issues/37 The existing bindings aren't fully complete. We should have two packages: a low-level package with no/minimal dependencies that mirrors the C API closely; and eventually, a high-level package that i

[GitHub] [arrow-nanoarrow] paleolimbot commented on pull request #2: Minor typo fix in README

2022-07-13 Thread GitBox
paleolimbot commented on PR #2: URL: https://github.com/apache/arrow-nanoarrow/pull/2#issuecomment-1183393758 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #2: Minor typo fix in README

2022-07-13 Thread GitBox
paleolimbot merged PR #2: URL: https://github.com/apache/arrow-nanoarrow/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow

[GitHub] [arrow-adbc] lidavidm merged pull request #38: Add contributing info

2022-07-13 Thread GitBox
lidavidm merged PR #38: URL: https://github.com/apache/arrow-adbc/pull/38 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #39: Refactor Java APIs for clarity

2022-07-14 Thread GitBox
lidavidm opened a new pull request, #39: URL: https://github.com/apache/arrow-adbc/pull/39 - `AdbcDriver#connect` -> `AdbcDriver#open` - Make append vs create explicit in `bulkIngest` - Add `AdbcStatement#executeQuery` as a convenience method -- This is an automated message from the

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #40: Implement prepared statements, refactor for consistency

2022-07-14 Thread GitBox
lidavidm opened a new pull request, #40: URL: https://github.com/apache/arrow-adbc/pull/40 - Implement prepared statements - Define method needed for partitioned data (though no driver implements it yet) - Refactor test suite to test both Derby and Postgres (Postgres needs more CI wor

[GitHub] [arrow-adbc] lidavidm merged pull request #39: Refactor Java APIs for clarity

2022-07-15 Thread GitBox
lidavidm merged PR #39: URL: https://github.com/apache/arrow-adbc/pull/39 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #40: [Java] Implement prepared statements, refactor for consistency

2022-07-15 Thread GitBox
lidavidm merged PR #40: URL: https://github.com/apache/arrow-adbc/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-julia] palday opened a new issue, #328: `fromarrow` dispatch for `::Type{Union{Missing, T}}` is broken when `T` is parametric

2022-07-19 Thread GitBox
palday opened a new issue, #328: URL: https://github.com/apache/arrow-julia/issues/328 xref: https://github.com/JuliaCloud/AWSS3.jl/issues/263 The fix seems to be change `::Type{Union{Missing, T}}` to `::Type{<:Union{Missing, T}}`. -- This is an automated message from the Apache Gi

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #41: [Python] Complete minimal bindings for ADBC

2022-07-20 Thread GitBox
lidavidm opened a new pull request, #41: URL: https://github.com/apache/arrow-adbc/pull/41 Also refactors the bindings to not depend on PyArrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow-adbc] lidavidm closed issue #37: Reorganize and complete Python bindings

2022-07-20 Thread GitBox
lidavidm closed issue #37: Reorganize and complete Python bindings URL: https://github.com/apache/arrow-adbc/issues/37 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [arrow-adbc] lidavidm merged pull request #41: [Python] Complete minimal bindings for ADBC

2022-07-20 Thread GitBox
lidavidm merged PR #41: URL: https://github.com/apache/arrow-adbc/pull/41 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #42: [Docs] Update README

2022-07-21 Thread GitBox
lidavidm merged PR #42: URL: https://github.com/apache/arrow-adbc/pull/42 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new issue, #43: ADBC/Ibis pain points

2022-07-21 Thread GitBox
lidavidm opened a new issue, #43: URL: https://github.com/apache/arrow-adbc/issues/43 - Need way to query driver and server version - Need handling of multiple databases within a connection -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [arrow-testing] wjones127 opened a new pull request, #80: ARROW-17100: Add example of Arrow 2.0 DataPageV2 compression issue

2022-07-21 Thread GitBox
wjones127 opened a new pull request, #80: URL: https://github.com/apache/arrow-testing/pull/80 We used to always set `is_compressed=false` in page headers regardless of whether there was actual compression. Check you can read this file if you want to suppose files written by Arrow C++ 2.0.

[GitHub] [arrow-testing] wjones127 commented on pull request #80: ARROW-17100: Add example of Arrow 2.0 DataPageV2 compression issue

2022-07-21 Thread GitBox
wjones127 commented on PR #80: URL: https://github.com/apache/arrow-testing/pull/80#issuecomment-1191717690 cc @pitrou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [arrow-testing] pitrou commented on pull request #80: ARROW-17100: Add example of Arrow 2.0 DataPageV2 compression issue

2022-07-21 Thread GitBox
pitrou commented on PR #80: URL: https://github.com/apache/arrow-testing/pull/80#issuecomment-1191734043 Some nits: * avoid creating a subdir for a single file? * add a README.md in `data/parquet` to start describing the files being added? A bit like in https://github.com/apache/parqu

[GitHub] [arrow-testing] pitrou merged pull request #80: ARROW-17100: Add example of Arrow 2.0 DataPageV2 compression issue

2022-07-21 Thread GitBox
pitrou merged PR #80: URL: https://github.com/apache/arrow-testing/pull/80 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apac

[GitHub] [arrow-testing] pitrou commented on pull request #80: ARROW-17100: Add example of Arrow 2.0 DataPageV2 compression issue

2022-07-21 Thread GitBox
pitrou commented on PR #80: URL: https://github.com/apache/arrow-testing/pull/80#issuecomment-1191800075 Thanks for the update @wjones127 :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #44: [C] Fix compatibility issues noticed with Ibis

2022-07-21 Thread GitBox
lidavidm opened a new pull request, #44: URL: https://github.com/apache/arrow-adbc/pull/44 - The SQLite catalog is called "main" - Fix a couple bugs and note areas of improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [arrow-adbc] lidavidm merged pull request #44: [C] Fix compatibility issues noticed with Ibis

2022-07-22 Thread GitBox
lidavidm merged PR #44: URL: https://github.com/apache/arrow-adbc/pull/44 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm commented on pull request #45: [C][Java][Python] Add GetInfo

2022-07-25 Thread GitBox
lidavidm commented on PR #45: URL: https://github.com/apache/arrow-adbc/pull/45#issuecomment-1194001552 CC @hannes @krlmlr if either of you have comments - I noticed this was missing while working on the Ibis backend -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow-adbc] GavinRay97 opened a new issue, #46: Question around fixed hierarchy for schema

2022-07-25 Thread GitBox
GavinRay97 opened a new issue, #46: URL: https://github.com/apache/arrow-adbc/issues/46 I noticed that the ADBC metadata information assumes a fixed hierarchy: https://github.com/apache/arrow-adbc/blob/2485d7c3da217a7190f86128d769a7d0445755ab/java/core/src/main/java/org/apache/arrow/adbc/

[GitHub] [arrow-adbc] lidavidm commented on issue #46: Question around fixed hierarchy for schema

2022-07-25 Thread GitBox
lidavidm commented on issue #46: URL: https://github.com/apache/arrow-adbc/issues/46#issuecomment-1194305728 Hmm, real-world edge cases are always fun, thanks for poking around. (Admittedly we should've looked more closely at these.) We were debating whether to keep the hierarchy or s

[GitHub] [arrow-adbc] tokoko commented on a diff in pull request #47: [Java] Add basic Flight SQL driver

2022-07-26 Thread GitBox
tokoko commented on code in PR #47: URL: https://github.com/apache/arrow-adbc/pull/47#discussion_r929794347 ## java/driver/flight-sql/src/main/java/org/apache/arrow/adbc/driver/flightsql/FlightSqlDriver.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow-adbc] lidavidm commented on a diff in pull request #47: [Java] Add basic Flight SQL driver

2022-07-26 Thread GitBox
lidavidm commented on code in PR #47: URL: https://github.com/apache/arrow-adbc/pull/47#discussion_r929911902 ## java/driver/flight-sql/src/main/java/org/apache/arrow/adbc/driver/flightsql/FlightSqlDriver.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #3: Add asf.yaml

2022-07-26 Thread GitBox
paleolimbot merged PR #3: URL: https://github.com/apache/arrow-nanoarrow/pull/3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #4: Implement bitmap helpers

2022-07-26 Thread GitBox
paleolimbot opened a new issue, #4: URL: https://github.com/apache/arrow-nanoarrow/issues/4 Dealing with bitmaps is hard and we should provide some tools to do the finicky hard-to-get-right math: - `ArrowBitmapIsNull(const void* bitmap, int64_t i)` - `ArrowBitmapSetNull(void* bitma

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #5: Implement `ArrowArray*` helpers

2022-07-26 Thread GitBox
paleolimbot opened a new issue, #5: URL: https://github.com/apache/arrow-nanoarrow/issues/5 Now that we have buffer holders (`struct ArrowBuffer`) we can implement an owning `struct ArrowArray`. I envision the API something like the `ArrowSchema*` helpers: - `ArrowArrayInit(struct Ar

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #6: Implement metadata building utility

2022-07-26 Thread GitBox
paleolimbot opened a new issue, #6: URL: https://github.com/apache/arrow-nanoarrow/issues/6 Now that we have growable buffers we can implement a more natural way to build key/value metadata. It might just take one function: ``` ArrowMetadataBuild(struct ArrowBuffer* buffer, const c

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #7: Fix memcheck

2022-07-26 Thread GitBox
paleolimbot opened a new issue, #7: URL: https://github.com/apache/arrow-nanoarrow/issues/7 Right now the valgrind test isn't actually testing for memory leaks...we actually just need to: - Add `include(CTest)` in CMakeLists.txt if building tests - Change the current command to `ct

[GitHub] [arrow-nanoarrow] lidavidm commented on issue #7: Fix memcheck

2022-07-26 Thread GitBox
lidavidm commented on issue #7: URL: https://github.com/apache/arrow-nanoarrow/issues/7#issuecomment-1195873419 FWIW: Arrow does this sort of thing here https://github.com/apache/arrow/blob/master/cpp/valgrind.supp Can't claim to understand how it works, but it gets set up here: http

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
paleolimbot opened a new issue, #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8 This will require a bit of code but is also the thing that most people need help with when generating C arrays (looping over some row-based structure and generating a column-based structure). One way

[GitHub] [arrow-nanoarrow] lidavidm commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
lidavidm commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195959994 Looks reasonable to me. We may eventually want more conveniences, e.g. being able to append a range of values from a `int64_t*`, but this would work to start with. Also a `reserve

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
paleolimbot commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195962996 I'm not sure about the nested types...perhaps for the first PR I'll get the non-nested types in place and after that maybe it will be clearer whether the `struct ArrowArrayBuil

[GitHub] [arrow-nanoarrow] wesm commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
wesm commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195963131 I'm thinking that performance sensitive applications (especially those that might be migrating from C++ where anything that's in the Arrow builder headers is inlined) many of the buil

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
paleolimbot commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195967377 That's a good point...we need a system for that for the buffer builders too since those functions should be inlinable too (but it's inconvenient for readability if they're all

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
paleolimbot commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195970110 I think with the callable approach those functions will all exist (i.e., `ArrowInt32BuilderAppendInteger()`)...maybe they live in something like src/nanoarrow/array_builder_inl

[GitHub] [arrow-nanoarrow] lidavidm commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
lidavidm commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195974480 They could perhaps be Arrow type-oblivious macros that expand into a write into a particular buffer (and possibly a bounds-check/reserve call)? Then the builder is just a bag of 3

[GitHub] [arrow-nanoarrow] wesm commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
wesm commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195977036 I guess for the performance sensitive use case, I would focus on the reserve-then-unsafeappend usage pattern which we have in plenty of places in the Arrow codebase. -- This is an

[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-07-26 Thread GitBox
paleolimbot commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1195980100 Maybe the first step is not a builder class then...as per #5, a `struct ArrowArray` is already just a "bag of `struct ArrowBuffer`s" that can can be reserved/appended. Then we

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #49: [Docs] Update CONTRIBUTING.md

2022-07-29 Thread GitBox
lidavidm opened a new pull request, #49: URL: https://github.com/apache/arrow-adbc/pull/49 Clarify how we approach dependencies. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [arrow-adbc] mystic-lama opened a new pull request, #50: Updated relative path for parent pom

2022-07-30 Thread GitBox
mystic-lama opened a new pull request, #50: URL: https://github.com/apache/arrow-adbc/pull/50 Updated relative path for parent pom in validation module. This fixes the following build error [ERROR] Non-resolvable parent POM for org.apache.arrow.adbc:adbc-driver-validation:9.0.0-S

[GitHub] [arrow-adbc] mystic-lama commented on pull request #50: Updated relative path for parent pom

2022-07-31 Thread GitBox
mystic-lama commented on PR #50: URL: https://github.com/apache/arrow-adbc/pull/50#issuecomment-1200448725 @lidavidm minor PR for your attention :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow-adbc] lidavidm commented on pull request #50: Updated relative path for parent pom

2022-07-31 Thread GitBox
lidavidm commented on PR #50: URL: https://github.com/apache/arrow-adbc/pull/50#issuecomment-1200457556 Thanks! Wonder why this doesn't happen for me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow-adbc] lidavidm merged pull request #50: Updated relative path for parent pom

2022-07-31 Thread GitBox
lidavidm merged PR #50: URL: https://github.com/apache/arrow-adbc/pull/50 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] mystic-lama commented on pull request #50: Updated relative path for parent pom

2022-07-31 Thread GitBox
mystic-lama commented on PR #50: URL: https://github.com/apache/arrow-adbc/pull/50#issuecomment-1200567107 Mine is a new setup so kind-of catches these issues. It a minor fluke -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [arrow-adbc] lidavidm merged pull request #49: [Docs] Update CONTRIBUTING.md

2022-08-01 Thread GitBox
lidavidm merged PR #49: URL: https://github.com/apache/arrow-adbc/pull/49 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-nanoarrow] paleolimbot opened a new pull request, #9: Attempt memcheck + suppressions

2022-08-02 Thread GitBox
paleolimbot opened a new pull request, #9: URL: https://github.com/apache/arrow-nanoarrow/pull/9 Fixes #7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[GitHub] [arrow-nanoarrow] pitrou commented on a diff in pull request #9: Attempt memcheck + suppressions

2022-08-02 Thread GitBox
pitrou commented on code in PR #9: URL: https://github.com/apache/arrow-nanoarrow/pull/9#discussion_r935729152 ## .github/workflows/build-and-test.yaml: ## @@ -106,7 +106,7 @@ jobs: - name: Run tests with valgrind run: | cd build - valgrind --

[GitHub] [arrow-nanoarrow] paleolimbot commented on a diff in pull request #9: Attempt memcheck + suppressions

2022-08-02 Thread GitBox
paleolimbot commented on code in PR #9: URL: https://github.com/apache/arrow-nanoarrow/pull/9#discussion_r935736791 ## .github/workflows/build-and-test.yaml: ## @@ -106,7 +106,7 @@ jobs: - name: Run tests with valgrind run: | cd build - valgri

[GitHub] [arrow-nanoarrow] paleolimbot merged pull request #9: Attempt memcheck + suppressions

2022-08-02 Thread GitBox
paleolimbot merged PR #9: URL: https://github.com/apache/arrow-nanoarrow/pull/9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow

[GitHub] [arrow-nanoarrow] paleolimbot closed issue #7: Fix memcheck

2022-08-02 Thread GitBox
paleolimbot closed issue #7: Fix memcheck URL: https://github.com/apache/arrow-nanoarrow/issues/7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues

[GitHub] [arrow-adbc] lidavidm merged pull request #45: [C][Java][Python] Add GetInfo

2022-08-02 Thread GitBox
lidavidm merged PR #45: URL: https://github.com/apache/arrow-adbc/pull/45 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm closed issue #43: ADBC/Ibis pain points

2022-08-02 Thread GitBox
lidavidm closed issue #43: ADBC/Ibis pain points URL: https://github.com/apache/arrow-adbc/issues/43 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: iss

[GitHub] [arrow-nanoarrow] paleolimbot opened a new pull request, #10: Implement bitmap setters, getters, and element-wise builder

2022-08-02 Thread GitBox
paleolimbot opened a new pull request, #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10 Fixes #4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [arrow-adbc] lidavidm merged pull request #47: [Java] Add basic Flight SQL driver

2022-08-02 Thread GitBox
lidavidm merged PR #47: URL: https://github.com/apache/arrow-adbc/pull/47 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-nanoarrow] lidavidm commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-02 Thread GitBox
lidavidm commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1203133106 I guess the general comment is: as Wes suggests, many of these should be defined inline in the header so they can get inlined into the caller, right? -- This is an automated messag

[GitHub] [arrow-nanoarrow] wesm commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-02 Thread GitBox
wesm commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1203166422 Also, keep in mind that in C, unlike C++, it takes more work to guarantee that functions will actually be inlined in all compilers. You need something like ``` #ifdef _MSC_VER

[GitHub] [arrow-nanoarrow] pitrou commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-02 Thread GitBox
pitrou commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1203176462 > Also, keep in mind that in C, unlike C++, it takes more work to guarantee that functions will actually be inlined in all compilers. But it's also most of the time pointless. Th

[GitHub] [arrow-nanoarrow] wesm commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-02 Thread GitBox
wesm commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1203178890 > I would be in favor of inline functions over macros, which have a non-trivial debugging and maintainability cost. > > For the record, CPython has started migrating from macros ove

[GitHub] [arrow-nanoarrow] pitrou commented on issue #8: Implement `struct ArrowArrayBuilder`

2022-08-03 Thread GitBox
pitrou commented on issue #8: URL: https://github.com/apache/arrow-nanoarrow/issues/8#issuecomment-1203706719 > One way to do this is a bunch of callables that are something like the reverse of SQLite's result interface So this is for building arrays whose type is not strictly known a

[GitHub] [arrow-adbc] pitrou commented on a diff in pull request #51: [Python] Set up package with Poetry

2022-08-03 Thread GitBox
pitrou commented on code in PR #51: URL: https://github.com/apache/arrow-adbc/pull/51#discussion_r936684861 ## python/adbc_driver_manager/requirements-dev.txt: ## @@ -1,19 +1,124 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agre

[GitHub] [arrow-adbc] lidavidm commented on a diff in pull request #51: [Python] Set up package with Poetry

2022-08-03 Thread GitBox
lidavidm commented on code in PR #51: URL: https://github.com/apache/arrow-adbc/pull/51#discussion_r936686336 ## python/adbc_driver_manager/requirements-dev.txt: ## @@ -1,19 +1,124 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license ag

[GitHub] [arrow-adbc] lidavidm commented on pull request #51: [Python] Set up package with Poetry

2022-08-03 Thread GitBox
lidavidm commented on PR #51: URL: https://github.com/apache/arrow-adbc/pull/51#issuecomment-1204017482 I also grabbed https://pypi.org/project/adbc-driver-manager/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [arrow-nanoarrow] paleolimbot opened a new issue, #11: Inline performance-sensitive functions and their dependencies

2022-08-03 Thread GitBox
paleolimbot opened a new issue, #11: URL: https://github.com/apache/arrow-nanoarrow/issues/11 A number of functions should be defined as `inline` with their definition visible to the compiler when a client types `#include "nanoarrow.h"`. Buffer appenders, element-wise builder, and bitmap fu

[GitHub] [arrow-nanoarrow] lidavidm commented on issue #11: Inline performance-sensitive functions and their dependencies

2022-08-03 Thread GitBox
lidavidm commented on issue #11: URL: https://github.com/apache/arrow-nanoarrow/issues/11#issuecomment-1204065088 I don't think you need to split up the headers, just ```c /// \brief Docstring... inline ArrowErrorCode ArrowBufferAppend() { // Definition... } ```

[GitHub] [arrow-nanoarrow] paleolimbot commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-03 Thread GitBox
paleolimbot commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1204069688 I made #11 to set up a PR for the inlining process...that's definitely important but requires a rather different set of changes (splitting up header files, rearranging stuff, defi

[GitHub] [arrow-nanoarrow] pitrou commented on issue #11: Inline performance-sensitive functions and their dependencies

2022-08-03 Thread GitBox
pitrou commented on issue #11: URL: https://github.com/apache/arrow-nanoarrow/issues/11#issuecomment-1204071180 Note that inline functions (or macros) require you to be very careful if you want to maintain some ABI compatibility. I don't know if that's a goal of nanoarrow, but worth remembe

[GitHub] [arrow-adbc] lidavidm opened a new issue, #52: [Python] Implement DBAPI

2022-08-03 Thread GitBox
lidavidm opened a new issue, #52: URL: https://github.com/apache/arrow-adbc/issues/52 Will make implementing Ibis backends easier since we can reuse more of the code for SQLAlchemy-based backends -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [arrow-adbc] lidavidm opened a new issue, #53: [Python] Driver distribution

2022-08-03 Thread GitBox
lidavidm opened a new issue, #53: URL: https://github.com/apache/arrow-adbc/issues/53 We don't want to/can't futz with shared libraries when distributing packages. Instead, since drivers have an entrypoint, just expose the entrypoint in Python? e.g. `adbc_driver_sqlite` will bundle t

[GitHub] [arrow-nanoarrow] lidavidm commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-03 Thread GitBox
lidavidm commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1204106882 I don't think we need to split up header files or rearrange things, though, and I don't think we need to inline the buffer functions - it could just be moving the definitions from `b

[GitHub] [arrow-nanoarrow] lidavidm commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-03 Thread GitBox
lidavidm commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1204107639 But if it's easier to do things in stages then we can just review the current PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow-nanoarrow] paleolimbot commented on pull request #10: Implement bitmap setters, getters, and element-wise builder

2022-08-03 Thread GitBox
paleolimbot commented on PR #10: URL: https://github.com/apache/arrow-nanoarrow/pull/10#issuecomment-1204110572 You convinced me! I didn't know you could split up an `inline` declaration and definition and that solves the conundrum I was having. -- This is an automated message from the Ap

[GitHub] [arrow-nanoarrow] paleolimbot opened a new pull request, #12: Add metadata builder functions

2022-08-03 Thread GitBox
paleolimbot opened a new pull request, #12: URL: https://github.com/apache/arrow-nanoarrow/pull/12 Fixes #6. This PR adds functions `ArrowMetadataBuilderAppend()` (for blindly but efficiently adding a key/value pair to the end of some metadata) and `ArrowMetadataBuilderSet()` (to les

<    1   2   3   4   5   6   7   8   9   10   >