AvroMultipleOutput ignores schemas (other than default)

2013-01-30 Thread Johannes Schulte
Hi! Maybe I'm the only one ever used this :D. Adding namedOutputs with AvroMultipleOutputs.addNamedOutput just adds them to a static map which is of course not available on the cluster during reduce execution. The unit tests pass though since the Instance of AvroMultipleOutputs is the same in

static schema validation

2013-01-30 Thread Aaron Kimball
Does Avro have an API to allow you to tell whether two schemas are a match, statically? i.e., schema1.canRead(schema2) /** return true iff schema1 can be used as a reader schema for schema2 */ From my (admittedly cursorary) scan of the docs + source, it seems like there isn't something quite

Re: Is Avro/Trevni strictly read-only?

2013-01-30 Thread Aaron Kimball
Hi ccleve, I'd definitely urge you to try out Kiji -- we who work on it think it's a pretty good fit for this specific use case. If you've got further questions about Kiji and how to use it, please send them to me, or ask the kiji user mailing list: http://www.kiji.org/getinvolved#Mailing_Lists

Re: Is Avro/Trevni strictly read-only?

2013-01-30 Thread Russell Jurney
Aaron - is there a way to create a Kiji table from Pig? I'm in the habit of not specifying schemas with Voldemort and MongoDB, just storing a Pig relation and the schema is set in the store. If I can arrange that somehow, I'm all over Kiji. Panthera is a fork :/ On Wed, Jan 30, 2013 at 3:20 PM,

Re: avro-rpc: running multiple threads on the netty server

2013-01-30 Thread Philip Zeyliger
Hi Matt, I do this with some frequency. The trick is to use Jetty manually to create an appropriate jetty.Server, and then start it. The snippets below should help. -- Philip import javax.servlet.Servlet; import org.apache.avro.ipc.ResponderServlet; import

Re: avro-rpc: running multiple threads on the netty server

2013-01-30 Thread Matt Corgan
Thanks Philip, I plan to try the http approach as well for clients that can't use Netty ..sent from my phone On Jan 30, 2013 7:05 PM, Philip Zeyliger phi...@cloudera.com wrote: Matt, I misread your question. The multiple-thread approach is with the Jetty/HTTP mechanism, not the Netty one.