Re: how to load custom Writable class from sequence file?

2013-09-24 Thread Yang
thanks for bringing up scalding. I actually didn't know that, and meant to use scala as an "easier and quick -and -dirty java". but yes, scalding seems more suited for this. On Tue, Sep 24, 2013 at 2:22 AM, Dmitriy Ryaboy wrote: > I assume by scala you mean scalding? > If so, yeah, scalding sho

Re: how to load custom Writable class from sequence file?

2013-09-24 Thread Dmitriy Ryaboy
John, any chance that still applies to trunk? Looks like there was some work left in that one. Seems like a good idea though... On Tue, Sep 24, 2013 at 6:21 AM, John Meagher wrote: > There's a patch available to allow using any available javax.script > language to do the conversion from any Java

Re: how to load custom Writable class from sequence file?

2013-09-24 Thread John Meagher
There's a patch available to allow using any available javax.script language to do the conversion from any Java object type in the sequence file to pig types. See https://issues.apache.org/jira/browse/PIG-1777 On Tue, Sep 24, 2013 at 5:22 AM, Dmitriy Ryaboy wrote: > I assume by scala you mean sc

Re: how to load custom Writable class from sequence file?

2013-09-24 Thread Dmitriy Ryaboy
I assume by scala you mean scalding? If so, yeah, scalding should be much easier for working with custom data types. Pig doesn't handle generic "objects" well. You have to write converters to and from, like the ones we created in ElephantBird for Protocol Buffers and Thrift (and a bunch of writabl

Optimize Date/Time Calculations

2013-09-24 Thread Justin
Hello, I should preface this by saying that I'm new to Pig. Also, this is a test and I'm still in the early stages of validating my logic/the pipeline. I'm trying to gather some metrics against a time series Hive table. Here's what I have right now. DEFINE UnixToISO org.apache.pig.piggybank.eva