Re: [ARROW C++] ARROW_LOG

2022-07-05 Thread David Li
It is still better to use something like spdlog or GLog (the library wrapped by Arrow's log) directly; or even if you use OpenTelemetry, we do not export the headers so you would need to depend on it explicitly (and OTel's logging is still under development in any case). And the Arrow exporter i

Re: [ARROW C++] ARROW_LOG

2022-07-05 Thread Weston Pace
At the moment that log is used primarily for Arrow developers and is not likely to be terribly useful beyond that. It is not, as far as I know, very extensible. I think you can only configure it to log to stderr or to a single file. However, it could be made extensible if someone were motivated

Arrow sync call July 6 at 12:00 US/Eastern, 16:00 UTC

2022-07-05 Thread Ian Cook
Hi all, Our biweekly sync call is tomorrow at 12:00 noon Eastern time. The Zoom meeting URL for this and other biweekly Arrow sync calls is: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Alternatively, enter this information into the Zoom website or app to join the call: Mee

Re: accessing Substrait protobuf Python classes from PyArrow

2022-07-05 Thread Weston Pace
If this is to be used within pyarrow then I think it makes sense as something to do. Can you expand a bit more on what you are trying to do? Why do you need to modify the Substrait plan in pyarrow? > This came up in a data-source UDF scenario where the implementation is a > Python stream factor

[ARROW C++] ARROW_LOG

2022-07-05 Thread José Almeida
Hello guys, I am planning to add a log system into a c++ application that uses arrow flight sql, and I saw that arrows provides some logging methods (cpp/src/arrow/util/logging.h). I was wondering if there is any instruction/documentation related to this log, if there is a special configuration th

Re: [C#] Adding compression and decompression support

2022-07-05 Thread Micah Kornfield
I'm not an expert in the C# but I imagine there should be high quality LZ4 implementations in C#. There is [1] which is referenced on the LZ4 page for isntances The approach we took in java is to put compressors/decompressors in a separate sub-component, with the interface contained in the main pr

Re: Preparing for version 9.0.0 release

2022-07-05 Thread Raul Cumplido Dominguez
Hi, An update around the next release. The plan is to start preparing the 9.0.0 release around the 25th of July, which means we probably will be cutting a release in ~3 weeks from now. This is the link to the confluence page that was made to track the progress for the 9.0.0 release [1]. There

Re: accessing Substrait protobuf Python classes from PyArrow

2022-07-05 Thread Yaron Gvili
@Li, yes though in a new way. This came up in a data-source UDF scenario where the implementation is a Python stream factory, i.e., a Python callable returning a callable. I think it would be easier to get the factory and then the stream using Python code than using Cython or C++. Yaron.

[RESULT][VOTE] Accept donation of Flight SQL JDBC driver

2022-07-05 Thread David Li
The vote passes with 5 binding votes and 3 non-binding votes. Thanks all - I will update the form and all that should be left are the software grant and then the final vote from the incubator side. On Sun, Jul 3, 2022, at 14:55, Micah Kornfield wrote: > +1 Binding. > > On Sat, Jul 2, 2022 at 5:

Re: accessing Substrait protobuf Python classes from PyArrow

2022-07-05 Thread Li Jin
Yaron, do we need to parse the subtrait protobuf in Python so that we can get the UDFs and register them with Pyarrow? On Mon, Jul 4, 2022 at 1:24 PM Yaron Gvili wrote: > This rewriting of the package is basically what I had in mind; the `_ep` > was just to signal a private package, which cannot

Re: Can we add out of ASF packages to our APT/Yum repositories?

2022-07-05 Thread Jacob Wujciak
+1 (non-binding) On Mon, Jul 4, 2022 at 6:39 AM Sutou Kouhei wrote: > Hi, > > I'm developing the C API for DataFusion: > https://github.com/datafusion-contrib/datafusion-c/ > > DataFusion itself is developed by ASF but the C API isn't > developed by ASF. Because it's developed under > https://gi