OK, I don't know enough about sockets or networking to know what
hypothetical performance is possible with 16 concurrent packet streams
going through a single port (was the 5GB/s based on a single-threaded
or multithreaded benchmark? i.e. did it simulate the the equivalent
number / size / concurrency of packets that the Flight benchmark is
doing). If the CPU cores aren't being saturated then I guess IO is
blocking in some way. It might be best to involve folks from the gRPC
community who are more expert in this domain.

To me, > 20 GBit/sec seems like acceptable throughput, considering
that networking faster than 10 gigabit is relatively exotic. I don't
think that optimizing for > 10GBit network was even a short term goal
for Flight. For faster networks I would guess we're going to be
getting into RDMA for moving IPC payloads rather than going through
TCP

On Sun, Feb 24, 2019 at 12:23 PM Antoine Pitrou <anto...@python.org> wrote:
>
>
> Le 24/02/2019 à 18:35, Wes McKinney a écrit :
> > hi Antoine,
> >
> > All of the Flight traffic is going through a hard-coded single port
> >
> > https://github.com/apache/arrow/blob/master/cpp/src/arrow/flight/flight-benchmark.cc#L185
> >
> > What happens if you spin up a different server (and use a different
> > port) for each thread? I'm surprised no one else has mentioned this
> > yet
>
> Well that's not the expected usage model for a server, either :-)  If
> you run an HTTP server, for example, you don't expect to have to open
> different ports on the same machine (rather than only port 80 or 443) to
> get good scalability.
>
> Regards
>
> Antoine.

Reply via email to