Re: acero speed versus numpy

2023-08-22 Thread Spencer Nelson
> How many rows match your timestamp criteria? Usually between 1,000 and 5,000. I agree that filtering could be way more costly - it probably is. I just thought the expression is more complex and worth explaining in more detail. > Acero will not "fuse" the kernel and has no expression

Re: acero speed versus numpy

2023-08-22 Thread Weston Pace
How many rows match your timestamp criteria? In other words, how many rows are you applying the function to? If there is an earlier exact match filter on a timestamp that only matches 1 (or a few rows) then I are you sure the expression evaluation (and not the filtering) is the costly spot? >

reading parquet table from std::stringstream

2023-08-22 Thread Lewis, Michael L
Hi, Is there a way to read a parquet table, if the data is in a string_view ? Basically, I'm looking to do the reverse of, parquet::arrow::WriteTable. I can see FileReader has a ::ReadTable method, but can't seem to find a way of constructing a FileReader given some buffer. Anyone know how?