You bring a good point, for submitting/managing jobs you could use a REST API. But still the protocol compatibility is needed for accessing HDFS to read/write data.
A On Tue, Oct 7, 2008 at 7:00 PM, Steve Loughran <[EMAIL PROTECTED]> wrote: > > I'm going to make another suggestion, which is : are there long-haul/high > stability APIs which we want to offer alongside the unstable/higher > performance IPC protocols. So IPC could be used within the cluster, but > something else would run for talking to the cluster from the outside > -assumes no trust of the caller and requires rigorous authentication > -less brittle to changes in the serialization formats > -designed with firewalls and timeouts in mind > A few years back, people would have said SOAP! or SOA, but something that > uses JSON over REST is the other option. Push up the files using WebDAV, > then use a RESTy interface to the JobTrackers to submit work > > There's engineering effort here, but this would be the API used by things > like IDE clients, and other applications to get work done >
