set operations on bags?

2009-12-12 Thread James Kebinger
Are there set operations on bags beyond the DIFF operator? I'd like to compare bags to find elements in both of them (intersection). I can imagine union and set addition and subtraction being useful here too. thanks james

CSV format loader

2009-12-08 Thread James Kebinger
Hi all, I realized a week or two ago that PigStorage(',') wasn't adequate to parse files that had commas embedded in properly CSV quoted fields. I went ahead and built a CSV parser for pig 0.3 that deals with embedded quotes (but not embedded newlines). Its up on github: http://github.com/jkebinge

Re: comment character in pig scripts?

2009-12-04 Thread James Kebinger
terrific. I should have tried that next. thanks! On Fri, Dec 4, 2009 at 6:59 PM, Jianyong Dai wrote: > Use double dash in the beginning of the line. > --a = load '1.txt' > > Daniel > > > James Kebinger wrote: > >> Please forgive my ignorance, but is the

comment character in pig scripts?

2009-12-04 Thread James Kebinger
Please forgive my ignorance, but is there a comment character in Pig scripts? It occurs to me I've never seen an example with a comment in it, and leading # or ; characters don't appear to work as comment characters. thanks!

NullPointerException calling illustrate

2009-11-15 Thread James Kebinger
Hello all, I'm encountering an error when I call illustrate - it blows up with a null pointer and the following exception: java.lang.NullPointerException at org.apache.pig.pen.util.DisplayExamples.ShortenField(DisplayExamples.java:205) at org.apache.pig.pen.util.DisplayExamples.MakeArray(D