polestar1988 opened a new issue, #37976:
URL: https://github.com/apache/arrow/issues/37976
### Describe the usage question you have. Please include as many useful
details as possible.
I have implemented my application using Apache arrow flight in golang, In
the server side I created arrow record and send to the client side :
this is my DoGet function :
```
func (s *Server) DoGet(ticket *flight.Ticket, server
flight.FlightService_DoGetServer) error {
queryResult, err := queryDatasource(dataSource, query)
if err != nil {
return err
}
defer queryResult.Close()
pool := memory.NewGoAllocator()
// ArrowSchema Parse schema to get column types and column names and
returns *arrow.Schema
schema, err := queryResult.ArrowSchema()
if err != nil {
return err
}
b := array.NewRecordBuilder(pool, schema)
defer b.Release()
//WriteValues writes the values inside the *array.RecordBuilder
err = queryResult.WriteValues(b)
if err != nil {
return err
}
rec := b.NewRecord()
if err != nil{
return err
}
defer rec.Release()
rw := flight.NewRecordWriter(server, ipc.WithSchema(rec.Schema()))
defer rw.Close()
if err := rw.Write(rec); err != nil {
return err
}
return nil
}
```
and in the client side :
```
//Retrieve data
doGetClient, err := client.DoGet(ctx, doGetTicket)
if err != nil {
return fmt.Errorf("failed to get data: %v", err)
}
reader, err := flight.NewRecordReader(doGetClient, nil)
if err != nil {
fmt.Printf("Creating record reader err: %v", err)
return err
}
defer reader.Release()
var record arrow.Record
for {
rec, err := reader.Read()
if err != nil {
if err == io.EOF {
break
}
return err
}
```
}
As you know as the data transferring is limited by 4Mb in grpc , and I faced
the error :
"received message larger than max (5000000 vs 4194304)"
how can I create chunks of data and send it to the client side and in client
side how can I put back the received chunks together and use it?
### Component(s)
Go
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
