Sent: Tuesday, August 25, 2009 12:52 PM
To: pig-dev@hadoop.apache.org; Dmitriy Ryaboy
Cc: pi.so...@gmail.com
Subject: Re: switching to different parser in Pig
Jflex is covered by GPL, but code generated by it is not. Only the code that
is generated by Jflex goes into pig.jar.
We can't checkin Jfle
ser, can we have a plan to move the other parsers in Pig to
>> the same technology?
>>
>> Thanks,
>> Santhosh
>>
>> PS: I am assuming we are not moving to Antlr.
>>
>>
>> -Original Message-
>> From: Alan Gates [mailto:ga...@yahoo-inc.com]
&
pi.so...@gmail.com
Subject: Re: switching to different parser in Pig
Santosh,
Am I missing something about Jflex licensing? I thought that it being
GPL, we can't package it with apache-licensed software, which prevents
it from being a viable option (regardless of technical merits)
-Dmitriy
On T
to
> the same technology?
>
> Thanks,
> Santhosh
>
> PS: I am assuming we are not moving to Antlr.
>
>
> -Original Message-
> From: Alan Gates [mailto:ga...@yahoo-inc.com]
> Sent: Tuesday, February 24, 2009 10:17 AM
> To: pig-dev@hadoop.apache.org; pi.so...@
ology?
Thanks,
Santhosh
PS: I am assuming we are not moving to Antlr.
-Original Message-
From: Alan Gates [mailto:ga...@yahoo-inc.com]
Sent: Tuesday, February 24, 2009 10:17 AM
To: pig-dev@hadoop.apache.org; pi.so...@gmail.com
Subject: Re: switching to different parser in Pig
Sorry, af
Yes.
And one thing I should have mentioned was Chris W's thoughts along the lines
that it would be very nice to expose the logical plan to something like
Cascading so that a global restructuring could be done across more than just
Pig programs. It works the other way as well, with it becoming pos
Sorry, after I sent that email yesterday I realized I was not very
clear. I did not mean to imply that antlr didn't have good
documentation or good error handling. What I wanted to say was we
want all three of those things, and it didn't appear that antlr
provided all three, since it does
(1) Lack of good documentation which makes it hard to and time consuming
to learn javacc and make changes to Pig grammar
<== ANTLR is very very well documented.
http://www.pragprog.com/titles/tpantlr/the-definitive-antlr-reference
http://media.pragprog.com/titles/tpantlr/toc.pdf
http://www.antlr.o
Hi
I got this info from javacc mailing lists. This may prove helpful:
-Original Message- From: Ken Beesley
[mailto:ken@xrce.x
We looked into antlr. It appears to be very similar to javacc, with
the added feature that the java code it generates is humanly
readable. That isn't why we want to switch off of javacc. Olga
listed the 3 things we want out of a parser that javacc isn't giving
us (lack of docs, no easy c
Should be pretty close but we may need to cleanup the interface a bit. Then
the new parser module can be switched in easily.
BTW, have we already got the solution for the new parser generator?
Pi
On Fri, Feb 20, 2009 at 9:03 PM, Ted Dunning wrote:
>
> Probably nearly the same effect as you su
Probably nearly the same effect as you suggest. Are the concepts at the
logical plan layer similar to those expressed in pig latin? Or has a
significant transformation occurred by then?
On Fri, Feb 20, 2009 at 1:59 AM, pi song wrote:
> Sounds good but how about exposing the logical plan layer
Sounds good but how about exposing the logical plan layer instead? Wouldn't
that yield the same effect? From python for example you still can construct
a logical plan and give to Pig to execute.
On Wed, Feb 18, 2009 at 10:07 AM, Ted Dunning wrote:
> 2009/2/17 Alan Gates
>
> > [not commenting on
2009/2/17 Alan Gates
> [not commenting on the switch, only on the exposure of AST's] Is that
> correct?
>
Nearly so.
> So whether we switch parsing technologies or not is not of interest to you,
> only the interfaces we expose?
>
I would think that switching parsing technologies would encoura
Ted,
If understand your comments correctly you aren't chiming in on whether
we should switch parsers, just that you would like there to be a
published interface of what pig latin syntax trees look like so you
could generate them in other tools and then feed them into pig. Is
that correct
Hey,
Just chiming in to say that Hive uses ANTLR; Ashish (athu...@facebook.com)
can provide more detailed feedback on their experiences with ANTLR.
Later,
Jeff
On Sat, Feb 14, 2009 at 5:16 AM, Ted Dunning wrote:
> Not even close.
>
> Take, for example,
>
> 1) the problem of using the output of
Not even close.
Take, for example,
1) the problem of using the output of a Pig query as the list of files used
as input.
2) Or running some query in an iterative fashion until convergence is
reached.
3) Or running a pig query, doing a matrix computation on the result and then
running another pi
Due to my limited knowledge, I don't quite understand why building ast from
outside Pig would be helpful. Isn't Pig Latin already good enough to
interface to the world?
In terms of parser generator, has anyone considered ANTLR? I had spent a few
weeks on it a while ago. It is quite well-documented
This sounds like a great idea !
Would be great if other means of generating ast's for pig was possible.
Regards,
Mridul
Ted Dunning wrote:
In general, it would be really, really nice if it were easy to build
abstract Pig syntax trees outside of the normal parser.
For instance, I find the fact
In general, it would be really, really nice if it were easy to build
abstract Pig syntax trees outside of the normal parser.
For instance, I find the fact that pig is not a full scale scripting
language incredibly confining. I would love to be able to build a DSL in
groovy that let me use groovy
Pig Developers,
Pig currently uses javacc for parsing pig commands. We have found
several shortcomings with using javacc. In particular,
(1) Lack of good documentation which makes it hard to and time consuming
to learn javacc and make changes to Pig grammar
(2) No easy way to customize error ha
21 matches
Mail list logo
