Hi folks,
I read some data from a PostgreSQL database, convert it into
RecordBatches and try to send the data to a client. But I fail to
properly understand the usage of Apache Arrow C/GLib.
My information sources are the [C++ docs][1], [the Apache Arrow C/GLib
reference manual][2] and [the C/GLib Github files][3].
By following the usage description of Apache Arrow C++ and experimenting
with the wrapper classes in C, I build this minimal example of writing
out a RecordBatch into a buffer and (after theoretically sending and
receiving the buffer) trying to read that buffer back into a
RecordBatch. But it fails and i would be glad, if you could point out my
mistakes!
I omitted the error catching for readability. The code errors out at
creation of the GArrowRecordBatchStreamReader. If i use the arrowbuffer
or the buffer from the top in creating the InputStream, the error reads:
```[record-batch-stream-reader][open]: IOError: Expected IPC message of
type schema but got record batch```.
If i use the testBuffer the error complains about an invalid IPC stream,
so the data is just corrupt.
```
void testRecordbatchStream(GArrowRecordBatch *rb){
GError *error = NULL;
// Write Recordbatch
GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(300,
&error);
GArrowBufferOutputStream *bufferStream =
garrow_buffer_output_stream_new(buffer);
long written =
garrow_output_stream_write_record_batch(GARROW_OUTPUT_STREAM(bufferStream),
rb, NULL, &error);
// Use buffer as plain bytes
void *data = garrow_buffer_get_data(GARROW_BUFFER(buffer));
size_t length = garrow_buffer_get_size(GARROW_BUFFER(buffer));
// Read plain bytes and test serialize function
GArrowBuffer *testBuffer = garrow_buffer_new(data, length);
GArrowBuffer *arrowbuffer = garrow_record_batch_serialize(rb, NULL,
&error);
// Read RecordBatch from buffer
GArrowBufferInputStream *inputStream =
garrow_buffer_input_stream_new(arrowbuffer);
GArrowRecordBatchStreamReader *sr =
garrow_record_batch_stream_reader_new(GARROW_INPUT_STREAM(inputStream),
&error);
GArrowRecordBatch *rb2 = garrow_record_batch_reader_read_next(sr,
&error);
printf("Received RB: \n%s\n", garrow_record_batch_to_string(rb2,
&error));
}
```
[1]: https://arrow.apache.org/docs/cpp/index.html
[2]: https://arrow.apache.org/docs/c_glib/arrow-glib/
[3]: https://github.com/apache/arrow/tree/master/c_glib
