As a follow up I decided to create a little sample API that would be exposed to the clients (the controller).
////////////////////////////////////// //Table information list<string> clusterList() list<string> tableList(1:string cluster) TableDescriptor describe(1:string table) TableSchema schema(1:string table) ////////////////////////////////////// // Takes the place of fetch and query // A non blocking call that starts the query and // return a handle to the query. QuerySession query(1:Query query) // A blocking call that will fetch the first query result, // and subsequent would iterate through the results. QueryResult fetchQueryResult(1:QuerySession session) // Closes the query and releases resources, there will have // to be an automatic session clean up on the server side // incase close is not called. void closeQuerySession(1:QuerySession session) // Tries to query the query in progress bool cancelQuery(1:QuerySession session) // Gets the status of a running query, current // result counts, percentage complete, etc QueryStatus queryStatus(1:QuerySession session) ////////////////////////////////////// // Term, TermDoc, and TermPosition iteration TermInfoResultsSession termInfo(1:TermInfo info) TermInfoResult fetchNextTermInfoResult(1:TermInfoResultsSession session) void closeTermInfoSession(1:TermInfoResultsSession session) ////////////////////////////////////// // Mutation session, I am less sure of this section. // But with this API, we could support transaction like semantics. MutateSession mutateSession() void mutate(1:Mutation mutate) void close(1:MutateSession session) On Fri, Sep 14, 2012 at 9:56 AM, Aaron McCurry <[email protected]> wrote: > In recent weeks I have had several discussions with developers over the > complexity of the RPC and how we need to make it easier to use. I agree, > but I have struggling to come up with a way to make things easier to use > while still having all the components necessary to allow all the different > functions to operate. > > To date the Blur shard server and the Blur controller server have shared > the same Thrift service API, perhaps that should change. A lot of the > extra pieces to the API have come from this fact. Basically the > controllers needs to be able to pass different information to shard servers > than the clients need to pass to the Blur controllers. > > The change that I am proposing, split the API into the 3 services. > > - User / Client API (for the controller) > - Shard Server API > - Admin API > > One pro to this design is that changes to the internal API (Shard Server) > could be changed and have less of an impact to the clients if any. > > The only drawback to this approach is that the controller has to be > deployed, even for testing. Currently you can just deploy a single shard > server and have the same features as a whole cluster. We could get around > this by allowing the controller API (and the admin API) to be run in the > same process as the shard server so that the same small scale testing could > occur. This then begs the question, do we need separate controller > processes? Should we allow for all 3 API types to always be in every > process? or should we keep things the way they are? or should we just > allow users to configure things they way they want? > > Aaron >
