+1. How about we rename engine.api to runtime.api and engine to runtime.library?
Bikas -----Original Message----- From: Hitesh Shah [mailto:[email protected]] Sent: Tuesday, September 24, 2013 10:08 AM To: [email protected] Subject: Re: Merge branch TEZ-398 to master @Vinod, All of the re-factor was targeted on the engine apis. These APIs are for developers to write implementations of Inputs/Outputs/Processors. At this point in time, almost all implementations of Inputs/Outputs/Processors (I/O/P), all the ones that are commonly used have been ported, have been switched to using the new engine apis. >From an example point of view, the MRRSleepJob and OrderedWordCount both use the latest version of the tez apis ( though they do not have to change to use the new I/O/P as only the internal implemetations of these I/O/Ps have changed ). But yes, there is still some work left to create a few simple "Hello World" type implementations for others to use as a reference to write their own implementations. Any volunteers? thanks -- Hitesh On Sep 24, 2013, at 9:46 AM, Vinod Kumar Vavilapalli wrote: > Is there a simple example that uses these new APIs and run on top of TEZ? Something like Distributed Shell in YARN? > > Thanks, > +Vinod > > On Sep 23, 2013, at 11:55 PM, Siddharth Seth wrote: > >> Quite a bit of work has gone into the TEZ-398 branch over the past >> 2-3 weeks to clean up the tez-engine APIs, so that it is easier to >> write custom Inputs, Outputs and Processors. >> >> In it's current state, regular MR jobs and MRR* chains are running. >> Also, all the unit tests pass. We should merge this back into master, >> and continue further development there. I'll merge the changes >> tomorrow or day after if there are no objections. >> >> https://issues.apache.org/jira/browse/TEZ-398 has a document >> detailing some of these changes (although a little stale now) and a >> list of JIRAs for the changes that have gone in. The pending jiras >> under this umbrella jira are not blockers for the API changes - and can continue on the main branch. >> >> Thanks >> - Sid > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable > law. If the reader of this message is not the intended recipient, you > are hereby notified that any printing, copying, dissemination, > distribution, disclosure or forwarding of this communication is > strictly prohibited. If you have received this communication in error, > please contact the sender immediately and delete it from your system. > Thank You.<signature.asc> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
