[
https://issues.apache.org/jira/browse/PIG-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269089#comment-13269089
]
[email protected] commented on PIG-2167:
----------------------------------------------------
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. > This review took a little time as I got interrupted in the middle. :)
bq. > Great job overall.
bq. > Some comments are about refactoring a little part of the code your
modifying.
bq. > Let me know what you think.
Thanks for the review julien. I made patch v3 based on Dmitriy's review
comments but was not able to upload in RB because git diff's are compatible
with RB. I have incorporated most of your review comments in this patch (v4).
Please check my comments below.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/AllExpressionVisitor.java,
line 98
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101421#file101421line98>
bq. >
bq. > iterate on entries() instead
The MultiMap class under org.apache.pig.impl.util package doesn't seem to
support iteration using entrySet.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/AllExpressionVisitor.java,
lines 100-103
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101421#file101421line100>
bq. >
bq. > All those for loops could be factored into a private
visitAll(Collection<LogicalExpressionPlan>)
Good catch. Added new private method and refactored all occurrences of for loop
to use this method.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/SchemaResetter.java,
lines 137-140
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101422#file101422line137>
bq. >
bq. > same comment.
bq. > You can factor out the for loops in those methods.
Added new private method.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOCube.java,
line 78
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101423#file101423line78>
bq. >
bq. > chain all exceptions
Modified.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java,
line 397
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101429#file101429line397>
bq. >
bq. > you can make this a javadoc comment
is javadoc generated for private methods as well? Since this is a private
method I kept it as a simple comment.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java,
line 608
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101429#file101429line608>
bq. >
bq. > add the name of the duplicate in the error message
Added.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestCubeOperator.java,
line 48
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101438#file101438line48>
bq. >
bq. > you van now use mock.Storage() for those tests.
bq. > see PIG-2650
Updated from patch v3 onwards.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestCubeOperator.java,
line 139
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101438#file101438line139>
bq. >
bq. > Use the message part of the assert statement to ensure a good error
message when it fails.
bq. >
bq. > Tuple t = it.next()
bq. > assertTrue(expected+" contains "+t, expected.contains(t))
Updated.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java,
line 522
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101429#file101429line522>
bq. >
bq. > same comment. I guess it is intended, but it looks strange. If the
constructor has side effect, maybe there should be another way
This inserts a project expression to logical expression plan (lEplan in this
case). I found this way in other files too. I could not find a better way to
attach a project expression to a logical expression plan. Do you have any other
thoughts/ways for doing this?
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanGenerator.g,
line 469
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101430#file101430line469>
bq. >
bq. > please add some comments to explain what you're doing here
Updated.
bq. On 2012-05-05 00:09:48, Julien Le Dem wrote:
bq. >
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AstValidator.g,
lines 259-273
bq. > <https://reviews.apache.org/r/4670/diff/2/?file=101428#file101428line259>
bq. >
bq. > do you know if there's a way to avoid duplication across .g files?
Since all these grammar files serves different purposes, I am not sure if it is
possible to avoid duplicates.
- Prasanth_J
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4670/#review7484
-----------------------------------------------------------
On 2012-04-30 04:05:03, Prasanth_J wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4670/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-30 04:05:03)
bq.
bq.
bq. Review request for pig and Dmitriy Ryaboy.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This is a review board for https://issues.apache.org/jira/browse/PIG-2167
bq.
bq.
bq. This addresses bug PIG-2167.
bq. https://issues.apache.org/jira/browse/PIG-2167
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/AllExpressionVisitor.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/SchemaResetter.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOCube.java
PRE-CREATION
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalRelationalNodesVisitor.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ProjectStarExpander.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AliasMasker.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AstPrinter.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AstValidator.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanGenerator.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryLexer.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParser.g
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLexer.pig
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLogicalPlanGenerator.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestParser.pig
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryLexer.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryParser.java
1325115
bq.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestCubeOperator.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4670/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests: All passed
bq.
bq. Pre-commit tests: All passed
bq. ant clean test-commit
bq.
bq.
bq. Thanks,
bq.
bq. Prasanth_J
bq.
bq.
> CUBE operation in Pig
> ---------------------
>
> Key: PIG-2167
> URL: https://issues.apache.org/jira/browse/PIG-2167
> Project: Pig
> Issue Type: New Feature
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Labels: gsoc2012, mentor
> Attachments: PIG-2167.1.patch, PIG-2167.2.patch, PIG-2167.3.patch,
> Pig-Cubing-Performance.png
>
>
> Computing aggregates over a cube of several dimensions is a common operation
> in data warehousing.
> The standard SQL syntax is "GROUP relation BY dim1, dim2, dim3 WITH CUBE" --
> which in addition to all dim1-2-3, produces aggregations for just dim1, just
> dim1 and dim2, etc. NULL is generally used to represent "all".
> A presentation by Arnab Nandi describes how one might implement efficient
> cubing in Map-Reduce here: http://pdf.cx/44wrk
> We can start with the naive solution which only works for algebraic measures,
> and work up from there.
> This is a candidate project for Google summer of code 2012. More information
> about the program can be found at
> https://cwiki.apache.org/confluence/display/PIG/GSoc2012
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
