Re: Python Flight example with query command

[Wes McKinney]

It does not implement it. We don’t have FlightSQL implemented in Python yet
either but I expect this will happen within the next 12 months.
Contributions for this always welcome

On Tue, May 18, 2021 at 12:51 AM Tanveer Ahmad - EWI 
wrote:

> Hi David,
>
>
> Thank you for the reply.
>
>
> I have found that Arrow Datafusion<
> https://github.com/apache/arrow-datafusion/tree/3be087a78846beffdbc4a9f80c73938fa18d24a7/datafusion-examples/examples#distributed>
> project offers something similar for what I am looking for. Do you think
> this project implements FlightSQL proposal?
>
>
> Regards,
> Tanveer Ahmad
> 
> From: David Li 
> Sent: Saturday, May 15, 2021 3:10:53 PM
> To: dev@arrow.apache.org
> Subject: Re: Python Flight example with query command
>
> Hey Tanveer,
>
> Something like this should work:
>
> $ python examples/flight/client.py put localhost:1234 foo.csv
> File Name: foo.csv
> Table rows= 1
>a  b
> 0  1  2
> $ python examples/flight/client.py get localhost:1234 -p foo.csv
> Ticket: 
> 
>a  b
> 0  1  2
>
> Note that Flight itself does not implement SQL query functionality or
> anything of the sort. It is a common misconception, I think
> exacerbated since Flight is often discussed in the context of products
> like Dremio which implement such functionality on top of Flight. But
> really, Flight itself is just a 'dumb pipe' for Arrow data for
> building such systems.
>
> You may be interested in the FlightSQL proposal which defines at least
> an interface for database systems to make themselves available over
> Flight and for clients to generically query them. However that
> proposal has been stalled for a while.
>
> Best,
> David
>
> On 2021/05/15 12:15:26, Tanveer Ahmad - EWI  wrote:
> > Hi all,
> >
> >
> > For Python Flight example<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_arrow_tree_master_python_examples_flight=DwIBaQ=XYzUhXBD2cD-CornpT4QE19xOJBbRy-TBPLK0X9U2o8=0FbbJetCCSYzJEnEDCQ1rNv76vTL6SUFCukKhvNosPs=-rzSkXp2WuWkj32SFKyE8CQ6Z6ViEXHg1AnYkaBmj4E=Q0ypcfj8NSFdgVXa_c8NxxQ9dpVkQ1EQXE157BX3R7g=
> >, I can start server (python server.py -> Serving on
> grpc+tcp://localhost:5005) and client can put (python client.py put
> localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005
> -p mycsv.csv) command retrieves data with -p (path) option.
> >
> >
> > I am wondering how to query (like python client.py get localhost:5005 -c
> "select * from ? limit 10") using -c, command this data , which I had
> already put on server through put command.
> >
> >
> > Thanks.
> >
> > Regards,
> > Tanveer Ahmad
> >
> >
>

Re: Python Flight example with query command

[Tanveer Ahmad - EWI]

Hi David,


Thank you for the reply.


I have found that Arrow 
Datafusion<https://github.com/apache/arrow-datafusion/tree/3be087a78846beffdbc4a9f80c73938fa18d24a7/datafusion-examples/examples#distributed>
 project offers something similar for what I am looking for. Do you think this 
project implements FlightSQL proposal?


Regards,
Tanveer Ahmad

From: David Li 
Sent: Saturday, May 15, 2021 3:10:53 PM
To: dev@arrow.apache.org
Subject: Re: Python Flight example with query command

Hey Tanveer,

Something like this should work:

$ python examples/flight/client.py put localhost:1234 foo.csv
File Name: foo.csv
Table rows= 1
   a  b
0  1  2
$ python examples/flight/client.py get localhost:1234 -p foo.csv
Ticket: 

   a  b
0  1  2

Note that Flight itself does not implement SQL query functionality or
anything of the sort. It is a common misconception, I think
exacerbated since Flight is often discussed in the context of products
like Dremio which implement such functionality on top of Flight. But
really, Flight itself is just a 'dumb pipe' for Arrow data for
building such systems.

You may be interested in the FlightSQL proposal which defines at least
an interface for database systems to make themselves available over
Flight and for clients to generically query them. However that
proposal has been stalled for a while.

Best,
David

On 2021/05/15 12:15:26, Tanveer Ahmad - EWI  wrote:
> Hi all,
>
>
> For Python Flight 
> example<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_arrow_tree_master_python_examples_flight=DwIBaQ=XYzUhXBD2cD-CornpT4QE19xOJBbRy-TBPLK0X9U2o8=0FbbJetCCSYzJEnEDCQ1rNv76vTL6SUFCukKhvNosPs=-rzSkXp2WuWkj32SFKyE8CQ6Z6ViEXHg1AnYkaBmj4E=Q0ypcfj8NSFdgVXa_c8NxxQ9dpVkQ1EQXE157BX3R7g=
>  >, I can start server (python server.py -> Serving on 
> grpc+tcp://localhost:5005) and client can put (python client.py put 
> localhost:5005 mycsv.csv) and also get (python client.py get localhost:5005 
> -p mycsv.csv) command retrieves data with -p (path) option.
>
>
> I am wondering how to query (like python client.py get localhost:5005 -c 
> "select * from ? limit 10") using -c, command this data , which I had already 
> put on server through put command.
>
>
> Thanks.
>
> Regards,
> Tanveer Ahmad
>
>

Re: Python Flight example with query command

[David Li]

Hey Tanveer,

Something like this should work:

$ python examples/flight/client.py put localhost:1234 foo.csv
File Name: foo.csv
Table rows= 1
   a  b
0  1  2
$ python examples/flight/client.py get localhost:1234 -p foo.csv
Ticket: 

   a  b
0  1  2

Note that Flight itself does not implement SQL query functionality or
anything of the sort. It is a common misconception, I think
exacerbated since Flight is often discussed in the context of products
like Dremio which implement such functionality on top of Flight. But
really, Flight itself is just a 'dumb pipe' for Arrow data for
building such systems.

You may be interested in the FlightSQL proposal which defines at least
an interface for database systems to make themselves available over
Flight and for clients to generically query them. However that
proposal has been stalled for a while.

Best,
David

On 2021/05/15 12:15:26, Tanveer Ahmad - EWI  wrote: 
> Hi all,
> 
> 
> For Python Flight 
> example, 
> I can start server (python server.py -> Serving on grpc+tcp://localhost:5005) 
> and client can put (python client.py put localhost:5005 mycsv.csv) and also 
> get (python client.py get localhost:5005 -p mycsv.csv) command retrieves data 
> with -p (path) option.
> 
> 
> I am wondering how to query (like python client.py get localhost:5005 -c 
> "select * from ? limit 10") using -c, command this data , which I had already 
> put on server through put command.
> 
> 
> Thanks.
> 
> Regards,
> Tanveer Ahmad
> 
> 

Python Flight example with query command

[Tanveer Ahmad - EWI]

Hi all,


For Python Flight 
example, I 
can start server (python server.py -> Serving on grpc+tcp://localhost:5005) and 
client can put (python client.py put localhost:5005 mycsv.csv) and also get 
(python client.py get localhost:5005 -p mycsv.csv) command retrieves data with 
-p (path) option.


I am wondering how to query (like python client.py get localhost:5005 -c 
"select * from ? limit 10") using -c, command this data , which I had already 
put on server through put command.


Thanks.

Regards,
Tanveer Ahmad