[GitHub] [arrow-julia] simsurace opened a new issue #303: Question on `Date` encoding

2022-03-09 Thread GitBox
simsurace opened a new issue #303: URL: https://github.com/apache/arrow-julia/issues/303 I'm trying to use Arrow to send data between a Julia (Arrows.jl) and a Rust (Polars) app. However, when I write a table containing Date, it is read by Polars as Extension("JuliaLang.Date", Date32,

[GitHub] [arrow-julia] kou opened a new issue #307: dev/release/release.sh misses a feature that removes old releases and RCs

2022-03-16 Thread GitBox
kou opened a new issue #307: URL: https://github.com/apache/arrow-julia/issues/307 See also: * "Current and older releases" section in https://infra.apache.org/release-download-pages.html * https://github.com/apache/arrow/blob/35b3567/dev/release/post-02-upload.sh#L52-L62 *

[GitHub] [arrow-julia] jonalm opened a new issue #308: Issues with changes in scopedenums introduced in 2.2.1

2022-03-17 Thread GitBox
jonalm opened a new issue #308: URL: https://github.com/apache/arrow-julia/issues/308 Context: The Arrow.jl 2.2.1 release changed the behavior of `@scopedenums` used in the `FlatBuffer` submodule, to remedy a type piracy issue. https://github.com/apache/arrow-julia/pull/267 Thi

[GitHub] [arrow-julia] quinnj closed issue #214: Can't roundtrip `NamedTuple`s of `Nanosecond`s twice

2022-03-18 Thread GitBox
quinnj closed issue #214: URL: https://github.com/apache/arrow-julia/issues/214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...

[GitHub] [arrow-julia] NHDaly opened a new issue #310: Arrow Table doesn't implement `==` for value equality

2022-03-28 Thread GitBox
NHDaly opened a new issue #310: URL: https://github.com/apache/arrow-julia/issues/310 I was surprised by this when I was writing a unit test that a function returned the expected structure. For example: ```julia julia> t = let io = IOBuffer(); Arrow.write(io, (v1=collect(1:10),

[GitHub] [arrow-julia] kou opened a new issue #311: Release process misses "add a new release to ASF's report database"

2022-03-30 Thread GitBox
kou opened a new issue #311: URL: https://github.com/apache/arrow-julia/issues/311 We can use https://reporter.apache.org/addrelease.html?arrow for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow-julia] kou opened a new issue, #313: RC verification script name is wrong

2022-04-09 Thread GitBox
kou opened a new issue, #313: URL: https://github.com/apache/arrow-julia/issues/313 https://github.com/apache/arrow-julia/blob/main/dev/release/README.md says `dev/release/verify.sh` but it should be `dev/release/verify_rc.sh`. -- This is an automated message from the Apache Git Service.

[GitHub] [arrow-julia] kou opened a new issue, #314: Verify release CI jobs are failed for a RC1 commit

2022-04-09 Thread GitBox
kou opened a new issue, #314: URL: https://github.com/apache/arrow-julia/issues/314 e.g.: https://github.com/apache/arrow-julia/runs/5957064732?check_suite_focus=true We can use a large RC number such as `100` to avoid this. -- This is an automated message from the Apache Git Servi

[GitHub] [arrow-julia] kou closed issue #307: dev/release/release.sh misses a feature that removes old releases and RCs

2022-04-15 Thread GitBox
kou closed issue #307: dev/release/release.sh misses a feature that removes old releases and RCs URL: https://github.com/apache/arrow-julia/issues/307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow-julia] kou closed issue #314: Verify release CI jobs are failed for a RC1 commit

2022-04-15 Thread GitBox
kou closed issue #314: Verify release CI jobs are failed for a RC1 commit URL: https://github.com/apache/arrow-julia/issues/314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow-julia] kou closed issue #313: RC verification script name is wrong

2022-04-15 Thread GitBox
kou closed issue #313: RC verification script name is wrong URL: https://github.com/apache/arrow-julia/issues/313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [arrow-julia] kou closed issue #311: Release process misses "add a new release to ASF's report database"

2022-04-15 Thread GitBox
kou closed issue #311: Release process misses "add a new release to ASF's report database" URL: https://github.com/apache/arrow-julia/issues/311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow-julia] bachdavi opened a new issue, #319: (de)serialization of (U)Int128

2022-04-28 Thread GitBox
bachdavi opened a new issue, #319: URL: https://github.com/apache/arrow-julia/issues/319 Hey đź‘‹ `(U)Int128` are currently serialized by setting the `bit_width=128`. That works perfectly fine with Julia, but as soon as serialized arrow batches are read from another language, such as P

[GitHub] [arrow-julia] pcjentsch opened a new issue, #320: Versions in footer and message do not agree, this causes issues reading Arrow files with other libraries (such as `arrow-rs`).

2022-04-29 Thread GitBox
pcjentsch opened a new issue, #320: URL: https://github.com/apache/arrow-julia/issues/320 here version is set as V5 https://github.com/apache/arrow-julia/blob/a3f6da7c1d59f8321315f4955a3b45c48d38aab4/src/write.jl#L374 and here as V4 https://github.com/apache/arrow-julia/blob/a3

[GitHub] [arrow-julia] quinnj closed issue #320: Versions in footer and message do not agree, this causes issues reading Arrow files with other libraries (such as `arrow-rs`).

2022-05-03 Thread GitBox
quinnj closed issue #320: Versions in footer and message do not agree, this causes issues reading Arrow files with other libraries (such as `arrow-rs`). URL: https://github.com/apache/arrow-julia/issues/320 -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [arrow-julia] ericphanson opened a new issue, #323: [ArrowTypes] add `@arrow_record`?

2022-05-17 Thread GitBox
ericphanson opened a new issue, #323: URL: https://github.com/apache/arrow-julia/issues/323 @kleinschmidt and I came up with ```julia using Arrow: ArrowTypes macro arrow_record(T) name = :(Symbol("JuliaLang.", @__MODULE__, ".", string(parentmodule($T), '.', nameof($T

[GitHub] [arrow-julia] markmbaum opened a new issue, #324: filtering DataFrame loaded from feather file triggers deleteat! error

2022-06-01 Thread GitBox
markmbaum opened a new issue, #324: URL: https://github.com/apache/arrow-julia/issues/324 I'm loading a table from feather file in a straightforward way ``` df = filename |> Arrow.Table |> DataFrame ``` and attempting to filter it based on the values of one column ``` filter

[GitHub] [arrow-adbc] lidavidm merged pull request #1: Initial import

2022-06-01 Thread GitBox
lidavidm merged PR #1: URL: https://github.com/apache/arrow-adbc/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm merged pull request #2: Add .asf.yaml to enable issues

2022-06-01 Thread GitBox
lidavidm merged PR #2: URL: https://github.com/apache/arrow-adbc/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm merged pull request #3: Get everything building again

2022-06-02 Thread GitBox
lidavidm merged PR #3: URL: https://github.com/apache/arrow-adbc/pull/3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm opened a new issue, #4: Scope for "1.0"

2022-06-02 Thread GitBox
lidavidm opened a new issue, #4: URL: https://github.com/apache/arrow-adbc/issues/4 IMO: - Prepared statements - Substrait support - Basic data ingestion - Basic database metadata (listing tables, etc) Along with: - Drivers for DuckDB and Flight SQL (and SQLite and/or Po

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #5: Refactor error handling

2022-06-02 Thread GitBox
lidavidm opened a new pull request, #5: URL: https://github.com/apache/arrow-adbc/pull/5 It's hard for the driver manager to allocate/free errors since it would have to manually intercept all calls. Also, it makes it hard to provide error messages when there is no driver. Move the error

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #7: [WIP] Bulk data ingestion

2022-06-06 Thread GitBox
lidavidm opened a new pull request, #7: URL: https://github.com/apache/arrow-adbc/pull/7 Sketch out an API for bulk data ingestion. Reuses the AdbcStatement structure. Based on #6. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #8: Set up basic CI pipeline

2022-06-07 Thread GitBox
lidavidm opened a new pull request, #8: URL: https://github.com/apache/arrow-adbc/pull/8 This only builds the driver manager (without tests). TODOs - [ ] Set up helper scripts, Conda environments - [ ] Can we share the main Arrow project's caches? - [ ] Build tests as well

[GitHub] [arrow-adbc] lidavidm merged pull request #8: Set up pre-commit CI pipeline

2022-06-07 Thread GitBox
lidavidm merged PR #8: URL: https://github.com/apache/arrow-adbc/pull/8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm commented on pull request #9: Fix pre-commit CI pipeline

2022-06-07 Thread GitBox
lidavidm commented on PR #9: URL: https://github.com/apache/arrow-adbc/pull/9#issuecomment-1149072070 Hmm, the pre-commit action isn't allowed. Time to do it manually… -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [arrow-adbc] lidavidm merged pull request #9: Fix pre-commit CI pipeline

2022-06-07 Thread GitBox
lidavidm merged PR #9: URL: https://github.com/apache/arrow-adbc/pull/9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #10: Fix build for ld.bfd in anticipation of setting up CI

2022-06-07 Thread GitBox
lidavidm opened a new pull request, #10: URL: https://github.com/apache/arrow-adbc/pull/10 Builds on #7. The driver manager and drivers define the same symbols. Normally, this means that if the driver manager loads a driver, and then the driver attempts to resolve ADBC functions

[GitHub] [arrow-adbc] lidavidm merged pull request #5: Refactor error handling

2022-06-09 Thread GitBox
lidavidm merged PR #5: URL: https://github.com/apache/arrow-adbc/pull/5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm merged pull request #6: Refactor initialization for consistency

2022-06-09 Thread GitBox
lidavidm merged PR #6: URL: https://github.com/apache/arrow-adbc/pull/6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm merged pull request #7: Bulk data ingestion

2022-06-09 Thread GitBox
lidavidm merged PR #7: URL: https://github.com/apache/arrow-adbc/pull/7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.

[GitHub] [arrow-adbc] lidavidm merged pull request #10: Fix build for ld.bfd in anticipation of setting up CI

2022-06-09 Thread GitBox
lidavidm merged PR #10: URL: https://github.com/apache/arrow-adbc/pull/10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #11: Set up basic C++ CI pipeline

2022-06-09 Thread GitBox
lidavidm merged PR #11: URL: https://github.com/apache/arrow-adbc/pull/11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #12: Sketch out ADBC Java APIs

2022-06-10 Thread GitBox
lidavidm opened a new pull request, #12: URL: https://github.com/apache/arrow-adbc/pull/12 Next steps: - CI setup - Java driver manager - Flight SQL driver - JNI ADBC driver -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow-adbc] lidavidm opened a new issue, #13: Potential DBI-inspired APIs

2022-06-13 Thread GitBox
lidavidm opened a new issue, #13: URL: https://github.com/apache/arrow-adbc/issues/13 - Function to `SELECT *` from a table without providing a query (makes it easier to provide non-query-engine based backends, e.g. a Parquet file backend) -- This is an automated message from the Apache G

[GitHub] [arrow-adbc] lidavidm opened a new issue, #14: Implement table/column reflection

2022-06-13 Thread GitBox
lidavidm opened a new issue, #14: URL: https://github.com/apache/arrow-adbc/issues/14 See https://github.com/lidavidm/arrow/issues/9 > Similar to AdbcConnectionGetTables, there should be a way to query the columns available in a table with their names, types, NULL-ness etc. >> I wa

[GitHub] [arrow-adbc] lidavidm commented on issue #14: Implement table/column reflection

2022-06-13 Thread GitBox
lidavidm commented on issue #14: URL: https://github.com/apache/arrow-adbc/issues/14#issuecomment-1153984889 Also see https://github.com/lidavidm/arrow/issues/6 where we need to clarify the signature for GetTables -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow-adbc] lidavidm merged pull request #12: Sketch out ADBC Java APIs

2022-06-13 Thread GitBox
lidavidm merged PR #12: URL: https://github.com/apache/arrow-adbc/pull/12 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #15: Fix MacOS build

2022-06-13 Thread GitBox
lidavidm opened a new pull request, #15: URL: https://github.com/apache/arrow-adbc/pull/15 Also renames the CMake options so they don't clash with the Arrow CMake config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #16: Fix Windows build

2022-06-14 Thread GitBox
lidavidm opened a new pull request, #16: URL: https://github.com/apache/arrow-adbc/pull/16 On top of #15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [arrow-adbc] lidavidm merged pull request #15: Fix MacOS build

2022-06-14 Thread GitBox
lidavidm merged PR #15: URL: https://github.com/apache/arrow-adbc/pull/15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm merged pull request #16: Fix Windows build

2022-06-14 Thread GitBox
lidavidm merged PR #16: URL: https://github.com/apache/arrow-adbc/pull/16 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #17: Add skeleton of Python bindings

2022-06-14 Thread GitBox
lidavidm opened a new pull request, #17: URL: https://github.com/apache/arrow-adbc/pull/17 These bindings are structured as a low-level module that mostly mirrors the ADBC API, and a TBD high-level module that will implement PEP 249 (except with Turbodbc-style extensions). This P

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #18: Add table/column reflection

2022-06-15 Thread GitBox
lidavidm opened a new pull request, #18: URL: https://github.com/apache/arrow-adbc/pull/18 Adds methods to get table schema/columns, and implements all the metadata methods for SQLite. The methods differ slightly from Flight SQL. Instead of returning a serialized schema with the tabl

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-15 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1156813679 CC @hannes, @krlmlr. This is roughly patterned off of Flight SQL (and seems similarish to DBI from quick look as well) I think we discussed an MSSQL style `information_schema` or ot

[GitHub] [arrow-adbc] lidavidm opened a new issue, #19: Create Hive driver based on Arrow adapter

2022-06-16 Thread GitBox
lidavidm opened a new issue, #19: URL: https://github.com/apache/arrow-adbc/issues/19 Removed in https://github.com/apache/arrow/pull/13382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [arrow-adbc] lidavidm opened a new issue, #20: Improve error handling

2022-06-16 Thread GitBox
lidavidm opened a new issue, #20: URL: https://github.com/apache/arrow-adbc/issues/20 We should fill out the error codes. Things to consider: - Flight/gRPC status codes: the gRPC status codes are nice because they're well-defined. - PEP 249 exception hierarchy - SQLSTATE standar

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #21: Improve error code table

2022-06-16 Thread GitBox
lidavidm opened a new pull request, #21: URL: https://github.com/apache/arrow-adbc/pull/21 Adds a wider set of error codes and describes what they're meant to be used for. Fixes #20. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow-adbc] lidavidm opened a new issue, #22: Set up linters, sanitizers

2022-06-16 Thread GitBox
lidavidm opened a new issue, #22: URL: https://github.com/apache/arrow-adbc/issues/22 We need to set up: - cpplint - clang-tidy - ASan/UBSan - Valgrind, possibly - flake8 for Cython -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [arrow-adbc] pitrou commented on issue #20: Improve error handling

2022-06-16 Thread GitBox
pitrou commented on issue #20: URL: https://github.com/apache/arrow-adbc/issues/20#issuecomment-1157797045 > Do we want to extend AdbcError with space for database-specific error codes? Probably? It seems that wouldn't hurt, and being possible to faithfully recreate errors is always

[GitHub] [arrow-adbc] lidavidm opened a new issue, #23: Clarify transaction semantics

2022-06-16 Thread GitBox
lidavidm opened a new issue, #23: URL: https://github.com/apache/arrow-adbc/issues/23 JDBC, ODBC, Flight SQL (implicitly): auto-commit PEP 249: manual commit We should define what the default is and add a function to set this on the connection. -- This is an automated message fr

[GitHub] [arrow-adbc] lidavidm commented on pull request #21: Improve error code table

2022-06-16 Thread GitBox
lidavidm commented on PR #21: URL: https://github.com/apache/arrow-adbc/pull/21#issuecomment-1157998986 Updates: now the error struct also includes space for vendor-specific codes and the SQLSTATE code from the SQL standards, for completeness. -- This is an automated message from the Apac

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-16 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158022230 Thanks. I wonder if we need that many entry points. If we assume a hierarchy like catalog -> schema -> table -> column, could we have one single entry point that returns a nested data

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-16 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158027202 Hmm, that sounds reasonable. So the schema would be something like this? ``` catalog_name: utf8 catalog_schemas: list>> ``` And if you filtered by (say) depth=catalog

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-16 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158037665 I'm not sure how difficult unpacking will be, and perhaps if filtering by catalog name we would only get ``` schema_name: utf8 schema_tables: list<...> ``` because the

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-16 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158040008 Hmm, the filters (at least currently, and in Flight SQL) can be patterns as well as fixed strings. If filtering by catalog name gives you the schemas directly, that starts to soun

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-16 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158246255 Fair enough, we can go with a static output schema for type stability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [arrow-testing] liukun4515 commented on pull request #56: ARROW-11417: Add integration files for buffer compression

2022-06-16 Thread GitBox
liukun4515 commented on PR #56: URL: https://github.com/apache/arrow-testing/pull/56#issuecomment-1158424874 @pitrou Hi, how do you generated the compression file? I'm doing compression work in rust version https://github.com/apache/arrow-rs/pull/1855 But the test failed when reading t

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-17 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159135921 Implemented for SQLite. It's a little tedious to implement for SQLite but isn't especially complicated, at least. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #24: Add basic transaction semantics

2022-06-17 Thread GitBox
lidavidm opened a new pull request, #24: URL: https://github.com/apache/arrow-adbc/pull/24 Clarify that autocommit is the default, add an option to disable it, and add an explicit commit. Though, this is all not currently implemented. Fixes #23. -- This is an automated messa

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-19 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159732527 I wonder if we also should offer a way to expose primary and foreign key, and perhaps unique constraints. This allows learning the data model from a remote database. I have devised a

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-19 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159733606 Since we have all the other metadata here, I think it's reasonable to add that information as well. So the 'tables' would now be something like: ``` table_name: utf8 table_typ

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-19 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159752806 Great! I think I can come up with a SQL query that returns this result for Postgres right from the database, but this is useful only if we support getting nested data from the database. -

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-20 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160481520 Is something like this what you were thinking? (This has less information than what Flight SQL provides but feels minimal, unless we also want to reflect the cascade/delete rules)

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-20 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160597922 This is similar to what MySQL does -- others (like Postgres, SQL Server, DuckDB (?)) would store the `fk_` columns in a detail table `constraint_column_usage` (we would use a list of struct

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-20 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160634749 Ah, something more like this then? ``` /// CONSTRAINT_SCHEMA is a Struct with fields: /// /// Field Name | Field Type| Comments ///

[GitHub] [arrow-adbc] krlmlr commented on pull request #18: Add table/column reflection

2022-06-20 Thread GitBox
krlmlr commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160714111 Yes, that looks about right -- and is still createable with *one single query* on DuckDB and perhaps Postgres. -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection

2022-06-21 Thread GitBox
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1161903416 Implemented for SQLite, though the constraint metadata ignores the column name filter for simplicity. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [arrow-julia] 0xYUANTI opened a new issue, #326: write fails on large dataframes

2022-06-21 Thread GitBox
0xYUANTI opened a new issue, #326: URL: https://github.com/apache/arrow-julia/issues/326 hi :) as indicated in the Tables.jl docs, for large dataframes (>10k columns), Tables.Schema stores its values in .storenames and .storedtypes instead of the type parameters. This needs to be

[GitHub] [arrow-adbc] lidavidm opened a new pull request, #25: Add basic CI for MacOS/Windows

2022-06-21 Thread GitBox
lidavidm opened a new pull request, #25: URL: https://github.com/apache/arrow-adbc/pull/25 Also fixes the tests not actually getting built/run! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow-adbc] lidavidm merged pull request #25: Add basic CI for MacOS/Windows

2022-06-21 Thread GitBox
lidavidm merged PR #25: URL: https://github.com/apache/arrow-adbc/pull/25 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apach

[GitHub] [arrow-testing] lihalite opened a new pull request #2: ARROW-5397: [FlightRPC] Add TLS certificates for testing Flight

2019-06-10 Thread GitBox
lihalite opened a new pull request #2: ARROW-5397: [FlightRPC] Add TLS certificates for testing Flight URL: https://github.com/apache/arrow-testing/pull/2 This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow-testing] wesm merged pull request #2: ARROW-5397: [FlightRPC] Add TLS certificates for testing Flight

2019-06-10 Thread GitBox
wesm merged pull request #2: ARROW-5397: [FlightRPC] Add TLS certificates for testing Flight URL: https://github.com/apache/arrow-testing/pull/2 This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow-testing] crepererum opened a new pull request #3: ARROW-5589: Start arrow-ipc corpus

2019-06-13 Thread GitBox
crepererum opened a new pull request #3: ARROW-5589: Start arrow-ipc corpus URL: https://github.com/apache/arrow-testing/pull/3 See apache/arrow#4547 Issue: ARROW-5589 This is an automated message from the Apache Git Se

[GitHub] [arrow-testing] wesm commented on issue #3: ARROW-5589: Start arrow-ipc corpus

2019-06-13 Thread GitBox
wesm commented on issue #3: ARROW-5589: Start arrow-ipc corpus URL: https://github.com/apache/arrow-testing/pull/3#issuecomment-501753169 +1 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow-testing] wesm merged pull request #3: ARROW-5589: Start arrow-ipc corpus

2019-06-13 Thread GitBox
wesm merged pull request #3: ARROW-5589: Start arrow-ipc corpus URL: https://github.com/apache/arrow-testing/pull/3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow-testing] crepererum opened a new pull request #4: ARROW-5605: add arrow-ipc crash 74aec871d14bb6b07c72ea8f0e8c9f72cbe6b73c

2019-06-14 Thread GitBox
crepererum opened a new pull request #4: ARROW-5605: add arrow-ipc crash 74aec871d14bb6b07c72ea8f0e8c9f72cbe6b73c URL: https://github.com/apache/arrow-testing/pull/4 Issue: ARROW-5605 This is an automated message from the Apa

[GitHub] [arrow-testing] lihalite opened a new pull request #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java

2019-06-18 Thread GitBox
lihalite opened a new pull request #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java URL: https://github.com/apache/arrow-testing/pull/5 Turns out Netty can only read PKCS#1 format private keys. Th

[GitHub] [arrow-testing] lihalite commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java

2019-06-18 Thread GitBox
lihalite commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java URL: https://github.com/apache/arrow-testing/pull/5#issuecomment-503266771 Part of https://github.com/apache/arrow/pull/4608 ---

[GitHub] [arrow-testing] wesm merged pull request #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java

2019-06-19 Thread GitBox
wesm merged pull request #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java URL: https://github.com/apache/arrow-testing/pull/5 This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow-testing] wesm commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java

2019-06-19 Thread GitBox
wesm commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java URL: https://github.com/apache/arrow-testing/pull/5#issuecomment-503593725 @lihalite since I squashed you may have to update the submodule hash in your PR

[GitHub] [arrow-testing] lihalite commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java

2019-06-19 Thread GitBox
lihalite commented on issue #5: ARROW-5643: [FlightRPC] Add PKCS#1 format keys for testing Flight in Java URL: https://github.com/apache/arrow-testing/pull/5#issuecomment-503602182 Thanks, will do! This is an automated messag

[GitHub] wesm commented on issue #23: Build wheels for 0.9.0

2018-03-21 Thread GitBox
wesm commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-374970680 LGTM, but do we want to bundle the required Boost shared libraries with the Windows and macOS wheels? --

[GitHub] cpcloud commented on issue #23: Build wheels for 0.9.0

2018-03-21 Thread GitBox
cpcloud commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-374983807 @wesm Might be a good idea. How will that interact with existing boost installations? This is an

[GitHub] wesm commented on issue #23: Build wheels for 0.9.0

2018-03-21 Thread GitBox
wesm commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-374986924 Conceivably there would be the possibility of ABI conflicts, but that would be true if we were statically linking also. We could go the namespaced Boost route like i

[GitHub] cpcloud commented on issue #23: Build wheels for 0.9.0

2018-03-21 Thread GitBox
cpcloud commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-375068142 @xhochy @kou I need some help with this. There's a ton of stuff here that I'm unfamiliar with especially the details of yum and apt-get packages. I don't have acc

[GitHub] kou opened a new pull request #24: Package .deb and .rpm for 0.9.0

2018-03-21 Thread GitBox
kou opened a new pull request #24: Package .deb and .rpm for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/24 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub an

[GitHub] kou commented on issue #23: Build wheels for 0.9.0

2018-03-21 Thread GitBox
kou commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-375198224 I created pull request for .deb and .rpm as #24. I'll merge #24 after I confirmed that it works on Travis CI. Can you rebase and remove cpp-linux staffs after that?

[GitHub] kou commented on issue #24: Package .deb and .rpm for 0.9.0

2018-03-22 Thread GitBox
kou commented on issue #24: Package .deb and .rpm for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/24#issuecomment-375204453 I've confirmed that it works on Travis CI. This is an automated message from the Apache Git S

[GitHub] kou commented on issue #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
kou commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-375204488 Done. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] kou closed pull request #24: Package .deb and .rpm for 0.9.0

2018-03-22 Thread GitBox
kou closed pull request #24: Package .deb and .rpm for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/24 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (fro

[GitHub] xhochy commented on issue #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
xhochy commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-375291195 Taking a look at this now This is an automated message from the Apache Git Service. To respond to t

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176430349 ## File path: cpp-linux/debian/rules ## @@ -23,8 +23,11 @@ override_dh_auto_configure: -DCMAKE_BUILD_TYPE=$(BUILD_TYPE

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176430659 ## File path: cpp-linux/yum/arrow.spec.in ## @@ -60,13 +60,17 @@ mkdir cpp/build cd cpp/build %cmake3 .. \ -DCMAKE_BUILD_TY

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176430659 ## File path: cpp-linux/yum/arrow.spec.in ## @@ -60,13 +60,17 @@ mkdir cpp/build cd cpp/build %cmake3 .. \ -DCMAKE_BUILD_TY

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176430763 ## File path: cpp-linux/yum/arrow.spec.in ## @@ -220,6 +225,13 @@ Documentation for Apache Arrow GLib. %endif %changelog +<<

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176431502 ## File path: scripts/python-wheels-windows.bat ## @@ -80,23 +79,25 @@ pushd parquet-cpp\build cmake -G "%GENERATOR%" ^ -

[GitHub] xhochy commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
xhochy commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176431729 ## File path: scripts/python-wheels-windows.bat ## @@ -80,23 +79,25 @@ pushd parquet-cpp\build cmake -G "%GENERATOR%" ^ -D

[GitHub] cpcloud commented on a change in pull request #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on a change in pull request #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#discussion_r176435008 ## File path: scripts/python-wheels-windows.bat ## @@ -80,23 +79,25 @@ pushd parquet-cpp\build cmake -G "%GENERATOR%" ^ -

[GitHub] cpcloud commented on issue #23: Build wheels for 0.9.0

2018-03-22 Thread GitBox
cpcloud commented on issue #23: Build wheels for 0.9.0 URL: https://github.com/apache/arrow-dist/pull/23#issuecomment-375323642 @xhochy looks like upgrading bison might broken thrift generation: ``` /Users/travis/build/apache/arrow-dist/arrow/parquet-cpp/build/thrift_ep-prefix

<    1   2   3   4   5   6   7   8   9   10   >