Hi Philip, What exactly are you having trouble with? Flight is a protocol, so you'd be implementing your own metadata and data servers. (Flight doesn't provide you server implementations, just the means to build them and suggested conventions to follow.) The 'metadata' server would implement GetFlightInfo, and would need some way of knowing about the data servers, their locations, and the available datasets (Flight doesn't implement this for you). The 'data' server would implement DoGet.
What do you mean by 'no direct access'? It sounds like the clients do have access to the server in this scheme. There's also not a defined convention for clients to distribute writes across servers. -David On Wed, Jan 11, 2023, at 14:38, Philip Carinhas wrote: > I’d like to setup a sharded Flight server with one metadata server, and > several data servers. I’m not finding documentation on how to do this. In > particular we want to: > > 1. Setup a single flight metadata > > 2. Setup several sharded data Flight servers > 3. Setup clients that distribute data to sharded data servers: > a. All data is pushed to servers by clients, no direct access. This > assumes that data is placed in memory on the data servers? >
