Re: Duplicate jar are getting into PigContext

Oh great! Let me get it fixed. On Wed, Mar 26, 2014 at 10:11 PM, Dotan Patrich wrote: > Hi Cheolsoo, > > Thank you very much for the reply and interest in this. > I actually did opened a jira issue for this a few weeks ago - > PIG-3798 > > Thanks,

Re: Duplicate jar are getting into PigContext

Hi Cheolsoo, Thank you very much for the reply and interest in this. I actually did opened a jira issue for this a few weeks ago - PIG-3798 Thanks, Dotan On Wed, Mar 26, 2014 at 8:50 PM, Cheolsoo Park wrote: > Hi Dotan, > > Very sorry for the l

Re: Recordings from Pig user meetup at Linkedin, Mar 14

Thank you Mark, greatly appreciated! Jarcec On Wed, Mar 26, 2014 at 05:56:53PM -0700, Mark Wagner wrote: > Hi Jarek, > > I'm looking into it with the people in charge of recording here at > LinkedIn. I'll update this thread with any new information I get. > > Sorry for the delay and thanks for

Re: Recordings from Pig user meetup at Linkedin, Mar 14

Hi Jarek, I'm looking into it with the people in charge of recording here at LinkedIn. I'll update this thread with any new information I get. Sorry for the delay and thanks for your patience! -Mark On Wed, Mar 26, 2014 at 5:46 PM, Jarek Jarcec Cecho wrote: > Thank you for the feedback Cheolsoo

Re: Recordings from Pig user meetup at Linkedin, Mar 14

Thank you for the feedback Cheolsoo. I think that having the recordings would be generally useful, so If you do not mind please ask him. Jarcec On Wed, Mar 26, 2014 at 10:54:23AM -0700, Cheolsoo Park wrote: > Hi Jarcec, > > I don't know what's the plan, but I can ask Mark Wanger who was the hos

Re: generic union types in piggybank

Hi Cheolsoo, Thanks for your reply! (Liang and I work together.) The restriction to "simple" union types is still there in the latest code; see lines 83-95, here: https://github.com/apache/pig/blob/trunk/src/org/apache/pig/impl/util/avro/AvroStorageSchemaConversionUtilities.java I know that ele

Re: Hadoop 2.2 Windows Version

Hi Sekhar, There isn't a full tar available for a Windows distro yet, so you'd have to build from source. Have you already reviewed the BUILDING.txt file in the root of the source tree to verify that you have all of the pre-requisites on your build machine? If so, then feel free to post the erro

Re: Duplicate jar are getting into PigContext

Hi Dotan, Very sorry for the late reply. >> Debuging it, I can see that the same jar from the local file system is being added multiple times to the PigContext skipJars member. >From a brief look, skipJars is updated by JobControlCompiler

Re: line feeds

Thanks for the feedback! I kind of figured the answer was "use a different load/store func", and I'll just do that. I half-posted this message as a warning to other people to avoid PigStorage for all but the most simple data :-) will On Wed, Mar 26, 2014 at 2:37 PM, Cheolsoo Park wrote: > Hi

Re: line feeds

Hi Will, You're right that PigStorage doesn't handle \n. PigStorage is really a dummy reference implementation of Load/StoreFunc, so I'd not recommend to use it in production. In particular, when you have complex data structures and special characters in data, advanced file formats work far better

Re: Pig0.12 gets confused about schema after a nested FOREACH

Hi Jamin, >> Out of bound access. Trying to access non-existent column: 8. Schema activityID:chararray,reqHost:chararray,rspPylByt:long pylByt:long,reqTime:double,reqDur:double,rspTime:double,rspDur:double has 8 column(s). Did you try to disable ColumnMapKeyPrune optimization? You can do it by ad

Re: generic union types in piggybank

Hi Liang, Does the new builtin AvroStorage work for you? I don't use Avro myself, so I cannot test it out. But it looks like that restriction is removed in the new AvroStorage. Here is the relevant code- https://github.com/apache/pig/blob/trunk/src/org/apache/pig/impl/util/avro/AvroTupleWrapper.j

Re: pig-0.12.0+PIG-3285: Encounter "NoClassDefFoundError: org.cloudera.htrace.Trace" during reading hbase table in pig grunt

Hi, Did you confirm that "org.cloudera.htrace.Trace" was actually added at runtime? As far as I understand, PIG-3285 adds dependency jars only if they're present in classpath on the client side. You can perhaps turn on DEBUG logs and look for this line

Re: Recordings from Pig user meetup at Linkedin, Mar 14

Hi Jarcec, I don't know what's the plan, but I can ask Mark Wanger who was the host of the meet-up. Thanks, Cheolsoo On Tue, Mar 25, 2014 at 8:58 AM, Jarek Jarcec Cecho wrote: > Sadly I was not able to attend the last bay area user meetup at Linkedin > that was held on March 14. I'm very inter

line feeds

I was debugging some warnings in a script I had: FIELD_DISCARDED_TYPE_CONVERSION_FAILED ACCESSING_NON_EXISTENT_FIELD I got it down to basically these two lines: --foo was stored using PigStorage foo = LOAD '' AS (key:chararray, value:map[chararray]); STORE foo INTO '...'; The problem is some

Hadoop 2.2 Windows Version

Hi, I have tried to prepare the Hadoop 2.2 version tar using windows sdk and maven but i am not able to create the tar which can be used to work on windows environment. Can someone let me know where can i get the Hadoop 2.2 version tar file to run Hadoop on windows? Any Link? Thanks Sekhar