Re: New PigEditor for Eclipse with autocomplete

2011-07-26 Thread Romain Rigaux
Thank you for trying! I also updated it today and the keywords are now case insensitive. Yes, basically the Pig grammar is not complete yet, so some instructions are not seen as valid. Right now %declare does not accept back quote and needs a semi colon. I will fix some cases like this tomorrow.

Re: Ramdom behavior of PIG ???

2011-07-26 Thread Dmitriy Ryaboy
They better, that's the combination I'm running in production :). D On Tue, Jul 26, 2011 at 12:08 PM, Corbin Hoenes wrote: > Dmitriy, > > Does the HBaseStorage class from Pig 0.8.1 and HBase 90.3 work together? > We > just upgraded our HBase cluster and a developer found some issues and we > a

Re: Ramdom behavior of PIG ???

2011-07-26 Thread Corbin Hoenes
Dmitriy, Does the HBaseStorage class from Pig 0.8.1 and HBase 90.3 work together? We just upgraded our HBase cluster and a developer found some issues and we aren't sure if they are related to the upgrade. On Tue, Jul 26, 2011 at 12:16 PM, Dmitriy Ryaboy wrote: > Vincent, can you try replaci

Re: Checking for '\n' and EOF in input

2011-07-26 Thread Dmitriy Ryaboy
It sounds like you need to write your own recordReader (and associated inputFormat) D On Tue, Jul 26, 2011 at 11:12 AM, wrote: > Hello, > > I have a custom loader function to read in a parsed schema from some log > files, but it seems there is a problem with some of the log files and I need > t

Re: Ramdom behavior of PIG ???

2011-07-26 Thread Dmitriy Ryaboy
Vincent, can you try replacing the HBase classes with those from trunk? A couple of fixes went in that might address that. Also, make sure you are running 0.90.3 D On Tue, Jul 26, 2011 at 10:40 AM, Vincent Barat wrote: > Hi, > > I'm using PIG 0.8.1 with HBase 0.90 and the following script someti

Checking for '\n' and EOF in input

2011-07-26 Thread Brett_Meyer
Hello, I have a custom loader function to read in a parsed schema from some log files, but it seems there is a problem with some of the log files and I need to detect if the end of a line in the log does not end with '\n' or is EOF when loading from the file. I'm currently running Pig 0.8.0 wi

Ramdom behavior of PIG ???

2011-07-26 Thread Vincent Barat
Hi, I'm using PIG 0.8.1 with HBase 0.90 and the following script sometime returns an empty set, and sometimes work ! start_sessions = LOAD 'startSession' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('meta:sid meta:infoid meta:imei meta:timestamp') AS (sid:chararray, infoid:chararr

Ramdom behavior of PIG ???

2011-07-26 Thread Vincent Barat
Hi, I'm using PIG 0.8.1 with HBase 0.90 and the following script somethime returns an empty set, and sometimes work ! start_sessions = LOAD 'startSession' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('meta:sid meta:infoid meta:imei meta:timestamp') AS (sid:chararray, infoid:charar

Exception running penny data sampler

2011-07-26 Thread Doug Daniels
Hi, I'm trying to run the data sampler tool from the penny library, and am getting a ClassNotFoundException for a netty class. I'm using the trunk version of pig, with the patch from PIG-2013 applied. I'm running a simple script that uses pig test data from test/org/apache/pig/test/data/Input

Re: RDBMS and Pig

2011-07-26 Thread Dmitriy Ryaboy
I don't think there ever was a general-purpose SQLLOADER. It's just explaining the concept -- that if you want to, you can write one, and you aren't limited to reading from HDFS. D On Tue, Jul 26, 2011 at 7:21 AM, Juan Martin Pampliega wrote: > I found about in the PigLatin manual. > http://pi

Re: RDBMS and Pig

2011-07-26 Thread Juan Martin Pampliega
I found about in the PigLatin manual. http://pig.apache.org/docs/r0.8.1/piglatin_ref1.html#Backward+Compatibility But maybe it is deprecated as there is no reference in the Java Doc. On Tue, Jul 26, 2011 at 10:11 AM, Jacob Perkins wrote: > You might also take a look at > > > http://pig.apache.or

return dict from Python UDF?

2011-07-26 Thread Norbert Burger
Hi all - I'm having some difficulty returning a dict/map from a Python UDF. The following code returns "java.lang.String cannot be cast to org.python.core.PyObject". What am I missing? Thanks, Norbert Here's my test Pig script: REGISTER 'pigfuncs.py' USING jython AS pigfuncs; cvps = LOAD 'test.

RowCount

2011-07-26 Thread Grant Ingersoll
I've been doing the following to count rows: x = foreach (group foo all) generate COUNT($1); Is that the current best practice? If so, would there be interest in a patch that simply did: x = rowcount(foo) ? I find myself doing sanity checks on scripts a fair amount and am starting to wish for

Re: RDBMS and Pig

2011-07-26 Thread Jacob Perkins
You might also take a look at http://pig.apache.org/docs/r0.8.1/api/index.html?org/apache/pig/piggybank/storage/DBStorage.html which is going to require that you 'register' the piggybank jar. I'm not entirely sure that's what you're looking for either though. I'm curious, in the same java doc I

Re: RDBMS and Pig

2011-07-26 Thread Juan Martin Pampliega
>From what I read, data from a RDBMS is read by the command: LOAD 'sql://mytable' USING SQLLOADER(); I couldn't find any info on how the connection parameters to the database are passed and which databases are supported. On Tue, Jul 26, 2011 at 7:40 AM, Mance Rylan wrote: > Hi, > > I have googl

RDBMS and Pig

2011-07-26 Thread Mance Rylan
Hi, I have googled a lot about if I can have Pig interact with an RDBMS. Is there any way to have Pig load data from an rdbms? perform some operations and then store data on Hadoop? Thanks, Mance

Re: New PigEditor for Eclipse with autocomplete

2011-07-26 Thread Gianmarco
Nice! It strangely gives me an error on this line: %declare jarFile `ls dist/ | tail -n 1` it says "Missing EOF at %" Cheers, -- Gianmarco De Francisci Morales On Tue, Jul 26, 2011 at 09:43, Russell Jurney wrote: > Bravo! Three cheers! > > Russell Jurney > twitter.com/rjurney > russell.jur.

Re: New PigEditor for Eclipse with autocomplete

2011-07-26 Thread Russell Jurney
Bravo! Three cheers! Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com On Jul 25, 2011, at 6:06 PM, Romain Rigaux wrote: > Hi, > > I just posted a new Pig Editor for Eclipse: > http://romainr.github.com/PigEditor/ > > The goal is to have it help you like Eclipse for