alamb commented on code in PR #1318:
URL:
https://github.com/apache/datafusion-ballista/pull/1318#discussion_r2348618253
##########
ballista/executor/src/flight_service.rs:
##########
@@ -185,16 +189,68 @@ impl FlightService for BallistaFlightService {
) -> Result<Response<Self::DoActionStream>, Status> {
let action = request.into_inner();
- let _action = decode_protobuf(&action.body).map_err(|e|
from_ballista_err(&e))?;
-
- Err(Status::unimplemented("do_action"))
+ match action.r#type.as_str() {
+ // Block transfer will transfer arrow ipc file block by block
+ // without decoding or decompressing, this will provide less
resource utilization
+ // as file are not decoded nor decompressed/compressed. Usually
this would transfer less data across
Review Comment:
FWIW the idea of sending uncompressed streams over gRPC makes a lot of sense
to me -- gRCP already has the ability to gzip communications at a higher so
there is no need to always force compression.
Not compressing will likely perform better in environments with fast
networks (aka data centers) as you have pointed out. For environments with
slower networks, you could still get the benefits of compression by enabling
`gzip` compression in the grpc layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
