[jira] [Updated] (PIG-2553) Pig shouldn't allow attempts to write multiple relations into same directory
[ https://issues.apache.org/jira/browse/PIG-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-2553: - Attachment: PIG-2553_2.patch Hey [~cheolsoo] attaching a patch. Please note I have made changes to UriUtil.java in removing unused imports. Rest is as we discussed, except that I did not find broken indentation (other than docs, not sure if that needs to be indented too?). Let me know if the indentation needs to be adjusted. Pig shouldn't allow attempts to write multiple relations into same directory Key: PIG-2553 URL: https://issues.apache.org/jira/browse/PIG-2553 Project: Pig Issue Type: Improvement Reporter: Dmitriy V. Ryaboy Assignee: Prashant Kommireddi Attachments: PIG-2553_1.patch, PIG-2553_2.patch, PIG-2553.patch We've seen multiple occasions where users accidentally try to store 2 or more different relations to the same destination directory. Currently, this passes the Pig planner and fails on MR side due to concurrent attempts to create the same part file on the reducer. This is extremely confusing to the user, and hard to debug. We should instead fail their scripts before they are even submitted, since we can identify the erroneous condition from the beginning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3158) Errors in the document Control Structures
[ https://issues.apache.org/jira/browse/PIG-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MIYAKAWA Taku updated PIG-3158: --- Attachment: PIG-3158.patch Thank you for your comments! I updated the patch. About the first two points, I reverted the changes. About the last point, I updated the list as: %default -param_file -param %declare. Errors in the document Control Structures --- Key: PIG-3158 URL: https://issues.apache.org/jira/browse/PIG-3158 Project: Pig Issue Type: Bug Components: documentation Reporter: MIYAKAWA Taku Assignee: MIYAKAWA Taku Labels: documentation Attachments: PIG-3158.patch, PIG-3158.patch This is a patch to fix errors in document Control Structures. # from ... import Pig statement is commented out in several example programs. # Several examples include invalid shebang lines #! /usr/bin/python. No space character is permitted between ! and the path. # The sentence ... a map of parameters (...) must be provided as was illustrated in the example above does not make a sence, because the example actually shows implicit binding. # An invalid comment line // In a jython script # 1. param and 2. Command line arguments in the section Passing Parameters to a Script are effectively heading lines, but not formatted in that way. # The python module sys is used but not imported in several example programs. # An example program in the section Automated Pig Latin Generation does not actually generate Pig Latin statements. The program does parameter binding. The patch removes the program and merges the sections Automated Pig Latin Generation, Conditional Compilation and Parallel Execution. # An example program in PigProgressNotificationListener Object lacks the first line of the class definition. # An input file of the example program idmapreduce.java is indicated to be placed on the local working directory, but the file must be placed on the home direcotry on the HDFS. # From the statement Unlike user defined functions (UDFs), which only allow quoted strings as its parameters, the which clause should be removed because UDFs can actually take types of parameters other than strings. # In several example programs, smart quotation marks (‘ and ’) are wrongly used to quote strings. The patch replaces them by straight quotation marks ( ' ). # In several examples, endash marks (U+2013) are wrongly used as hyphens. The patch replaces them by hyphens ( - ). # Removes a meaningless statement: If parameters are specified using the preprocessor statements, the script should include either %declare or %default. # A parameter precedence list says parameters defined in a script have the lowest precedence, but actually parameters in parameter files have the lowest precedence. # The last example of the page declares a parameter with a command output, but it wrongly uses quotation marks, not back ticks. # Improves indentation of example programs. # Updates a link: [PigServer|http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html] # Fixes typos: #* Compile is a static function on the Pig -object- +class+ #* a Java Properties object -and- or +a+ file containing a list of properties #* The example -above- +in the previous section+ shows how to make use of this call #* PigStats as it -is today will- +was before has+ become SimplePigStats #* -pig.java- +Pig+ Object #* BoundScript -.java- Object #* PigStats -.java- Object #* PigProgressNotificationListener -.java- Object #* a fully substituted Pig script +is+ produced -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3158) Errors in the document Control Structures
[ https://issues.apache.org/jira/browse/PIG-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MIYAKAWA Taku updated PIG-3158: --- Attachment: (was: PIG-3158.patch) Errors in the document Control Structures --- Key: PIG-3158 URL: https://issues.apache.org/jira/browse/PIG-3158 Project: Pig Issue Type: Bug Components: documentation Reporter: MIYAKAWA Taku Assignee: MIYAKAWA Taku Labels: documentation Attachments: PIG-3158.patch This is a patch to fix errors in document Control Structures. # from ... import Pig statement is commented out in several example programs. # Several examples include invalid shebang lines #! /usr/bin/python. No space character is permitted between ! and the path. # The sentence ... a map of parameters (...) must be provided as was illustrated in the example above does not make a sence, because the example actually shows implicit binding. # An invalid comment line // In a jython script # 1. param and 2. Command line arguments in the section Passing Parameters to a Script are effectively heading lines, but not formatted in that way. # The python module sys is used but not imported in several example programs. # An example program in the section Automated Pig Latin Generation does not actually generate Pig Latin statements. The program does parameter binding. The patch removes the program and merges the sections Automated Pig Latin Generation, Conditional Compilation and Parallel Execution. # An example program in PigProgressNotificationListener Object lacks the first line of the class definition. # An input file of the example program idmapreduce.java is indicated to be placed on the local working directory, but the file must be placed on the home direcotry on the HDFS. # From the statement Unlike user defined functions (UDFs), which only allow quoted strings as its parameters, the which clause should be removed because UDFs can actually take types of parameters other than strings. # In several example programs, smart quotation marks (‘ and ’) are wrongly used to quote strings. The patch replaces them by straight quotation marks ( ' ). # In several examples, endash marks (U+2013) are wrongly used as hyphens. The patch replaces them by hyphens ( - ). # Removes a meaningless statement: If parameters are specified using the preprocessor statements, the script should include either %declare or %default. # A parameter precedence list says parameters defined in a script have the lowest precedence, but actually parameters in parameter files have the lowest precedence. # The last example of the page declares a parameter with a command output, but it wrongly uses quotation marks, not back ticks. # Improves indentation of example programs. # Updates a link: [PigServer|http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html] # Fixes typos: #* Compile is a static function on the Pig -object- +class+ #* a Java Properties object -and- or +a+ file containing a list of properties #* The example -above- +in the previous section+ shows how to make use of this call #* PigStats as it -is today will- +was before has+ become SimplePigStats #* -pig.java- +Pig+ Object #* BoundScript -.java- Object #* PigStats -.java- Object #* PigProgressNotificationListener -.java- Object #* a fully substituted Pig script +is+ produced -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (PIG-3172) Partition filter push down does not happen when there is a non partition key filter
Rohini Palaniswamy created PIG-3172: --- Summary: Partition filter push down does not happen when there is a non partition key filter Key: PIG-3172 URL: https://issues.apache.org/jira/browse/PIG-3172 Project: Pig Issue Type: Bug Affects Versions: 0.10.1 Reporter: Rohini Palaniswamy A = LOAD 'job_confs' USING org.apache.hcatalog.pig.HCatLoader(); B = FILTER A by grid == 'cluster1' and dt '2012_12_01' and dt '2012_11_20'; C = FILTER B by params#'mapreduce.job.user.name' == 'userx'; D = FOREACH B generate dt, grid, params#'mapreduce.job.user.name' as user, params#'mapreduce.job.name' as job_name, job_id, params#'mapreduce.job.cache.files'; dump D; The query gives the below warning and ends up scanning the whole table instead of pushing the partition key filters grid and dt. [main] WARN org.apache.pig.newplan.PColFilterExtractor - No partition filter push down: Internal error while processing any partition filter conditions in the filter after the load -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (PIG-3173) Partition filter push down does not happen partition keys condition include a AND and OR construct
Rohini Palaniswamy created PIG-3173: --- Summary: Partition filter push down does not happen partition keys condition include a AND and OR construct Key: PIG-3173 URL: https://issues.apache.org/jira/browse/PIG-3173 Project: Pig Issue Type: Bug Reporter: Rohini Palaniswamy Assignee: Rohini Palaniswamy A = load 'db.table' using org.apache.hcatalog.pig.HCatLoader(); B = filter A by (region=='usa' AND dt=='201302051800') OR (region=='uk' AND dt=='201302051800'); C = foreach B generate name, age; DUMP C; gives the below warning and scans the whole table. 2013-02-06 22:22:16,233 [main] WARN org.apache.pig.newplan.PColFilterExtractor - No partition filter push down: You have an partition column (region ) in a construction like: (pcond and ...) or (pcond and ...) where pcond is a condition on a partition column. 2013-02-06 22:22:16,233 [main] WARN org.apache.pig.newplan.PColFilterExtractor - No partition filter push down: You have an partition column (datestamp ) in a construction like: (pcond and ...) or (pcond and ...) where pcond is a condition on a partition column. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3138) Decouple PigServer.executeBatch() from compilation of batch
[ https://issues.apache.org/jira/browse/PIG-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574795#comment-13574795 ] Prashant Kommireddi commented on PIG-3138: -- Hey [~cheolsoo], would this require a +1 from another commit to make it in? Decouple PigServer.executeBatch() from compilation of batch --- Key: PIG-3138 URL: https://issues.apache.org/jira/browse/PIG-3138 Project: Pig Issue Type: Improvement Reporter: Prashant Kommireddi Assignee: Prashant Kommireddi Fix For: 0.12 Attachments: PIG-3138_1.patch, PIG-3138_hotfix.patch, PIG-3138.patch executeBatch() currently does parsing and building of LogicalPlan in addition to the actual execution. It will be beneficial to separate out parsing/building from execution - that will allow us to get a handle on load/store and other operators before execution of batch. Useful for folks using PigServer API. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3138) Decouple PigServer.executeBatch() from compilation of batch
[ https://issues.apache.org/jira/browse/PIG-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574796#comment-13574796 ] Prashant Kommireddi commented on PIG-3138: -- Typo - committer Decouple PigServer.executeBatch() from compilation of batch --- Key: PIG-3138 URL: https://issues.apache.org/jira/browse/PIG-3138 Project: Pig Issue Type: Improvement Reporter: Prashant Kommireddi Assignee: Prashant Kommireddi Fix For: 0.12 Attachments: PIG-3138_1.patch, PIG-3138_hotfix.patch, PIG-3138.patch executeBatch() currently does parsing and building of LogicalPlan in addition to the actual execution. It will be beneficial to separate out parsing/building from execution - that will allow us to get a handle on load/store and other operators before execution of batch. Useful for folks using PigServer API. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Pig 11 release notes - draft
Hey Bill, There are also a few new builtins for better Map support https://issues.apache.org/jira/browse/PIG-2600 A way to load pig statements at startup https://issues.apache.org/jira/browse/PIG-2456 -Prashant On Thu, Feb 7, 2013 at 3:11 PM, Rohini Palaniswamy rohini.adi...@gmail.comwrote: This is something Daniel mentioned in his talk in pig meetup and is missing in this list PIG-2482 - Integrate HCat DDL command into Pig Also there are a lot of UDFs added for DateTime datatype. A mention of that would be good. Regards, Rohini On Thu, Feb 7, 2013 at 2:27 PM, Rohini Palaniswamy rohini.adi...@gmail.comwrote: Boolean data type is available from 0.10 Regards, Rohini On Thu, Feb 7, 2013 at 1:24 PM, Bill Graham billgra...@gmail.com wrote: Hi, In the Pig 11 release I'd like to include some specifics in the RELEASE_NOTES.txt file. Julien compiled this list (which I've tweaked a bit). Please give it a read and let me know if we've missed anything big, or mis-stated anything. Maybe an absentee agenda item suggestion for those at the meetup? :) (I ran into some distractions this past week with cutting a release candidate, but doing so is still high on my list. Stay tuned...) - New RANK, CUBE and ROLLUP operators - New DateType and Boolean data types - Support for Groovy UDFs - Support for loading macros from jars - Support for custom PigReducerEstimators - Suoport for custom PigProgressNotificatonListeners - Support for schema-based Tuples for reduced memory footprint - Support for passing environment variables to streaming jobs - New cleanupOnSuccess method in StoreFunc interface - UDF timing utilities - UDF lifecycle improvements - Performance improvements to merge join - Performance improvements to local mode - Performance improvements to algebraics in-memory aggregation - Performance improvements to Spillable management - Performance improvements to HBaseStorage - Performance improvements to AvroStorage - Penny has been removed - 300+ bug fixes * * thanks, Bill
[jira] [Commented] (PIG-3138) Decouple PigServer.executeBatch() from compilation of batch
[ https://issues.apache.org/jira/browse/PIG-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574828#comment-13574828 ] Cheolsoo Park commented on PIG-3138: [~prkommireddi], yes, it does. Decouple PigServer.executeBatch() from compilation of batch --- Key: PIG-3138 URL: https://issues.apache.org/jira/browse/PIG-3138 Project: Pig Issue Type: Improvement Reporter: Prashant Kommireddi Assignee: Prashant Kommireddi Fix For: 0.12 Attachments: PIG-3138_1.patch, PIG-3138_hotfix.patch, PIG-3138.patch executeBatch() currently does parsing and building of LogicalPlan in addition to the actual execution. It will be beneficial to separate out parsing/building from execution - that will allow us to get a handle on load/store and other operators before execution of batch. Useful for folks using PigServer API. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (PIG-3174) Remove rpm and deb artifacts from build.xml
Alan Gates created PIG-3174: --- Summary: Remove rpm and deb artifacts from build.xml Key: PIG-3174 URL: https://issues.apache.org/jira/browse/PIG-3174 Project: Pig Issue Type: Task Components: build Affects Versions: 0.12 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.12 I propose that we remove the targets to build rpms and debs from build.xml and consequently quit publishing them as part of our releases. Bigtop publishes these packages now. And building them takes infrastructure that not every committer/PMC member has. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3174) Remove rpm and deb artifacts from build.xml
[ https://issues.apache.org/jira/browse/PIG-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-3174: Attachment: PIG-3174.patch Remove rpm and deb artifacts from build.xml --- Key: PIG-3174 URL: https://issues.apache.org/jira/browse/PIG-3174 Project: Pig Issue Type: Task Components: build Affects Versions: 0.12 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.12 Attachments: PIG-3174.patch I propose that we remove the targets to build rpms and debs from build.xml and consequently quit publishing them as part of our releases. Bigtop publishes these packages now. And building them takes infrastructure that not every committer/PMC member has. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3174) Remove rpm and deb artifacts from build.xml
[ https://issues.apache.org/jira/browse/PIG-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-3174: Status: Patch Available (was: Open) Remove rpm and deb artifacts from build.xml --- Key: PIG-3174 URL: https://issues.apache.org/jira/browse/PIG-3174 Project: Pig Issue Type: Task Components: build Affects Versions: 0.12 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.12 Attachments: PIG-3174.patch I propose that we remove the targets to build rpms and debs from build.xml and consequently quit publishing them as part of our releases. Bigtop publishes these packages now. And building them takes infrastructure that not every committer/PMC member has. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Subscription: PIG patch available
Issue Subscription Filter: PIG patch available (28 issues) Subscriber: pigdaily Key Summary PIG-3174Remove rpm and deb artifacts from build.xml https://issues.apache.org/jira/browse/PIG-3174 PIG-3169Remove temporary files that are not needed https://issues.apache.org/jira/browse/PIG-3169 PIG-3167Job stats are printed incorrectly for map-only jobs https://issues.apache.org/jira/browse/PIG-3167 PIG-3166Update eclipse .classpath according to ivy library.properties https://issues.apache.org/jira/browse/PIG-3166 PIG-3164Pig current releases lack a UDF endsWith.This UDF tests if a given string ends with the specified suffix. https://issues.apache.org/jira/browse/PIG-3164 PIG-3158Errors in the document Control Structures https://issues.apache.org/jira/browse/PIG-3158 PIG-3154TestPackage.testOperator fails in trunk https://issues.apache.org/jira/browse/PIG-3154 PIG-3142Fixed-width load and store functions for the Piggybank https://issues.apache.org/jira/browse/PIG-3142 PIG-3136Introduce a syntax making declared aliases optional https://issues.apache.org/jira/browse/PIG-3136 PIG-3123Simplify Logical Plans By Removing Unneccessary Identity Projections https://issues.apache.org/jira/browse/PIG-3123 PIG-3114Duplicated macro name error when using pigunit https://issues.apache.org/jira/browse/PIG-3114 PIG-3108HBaseStorage returns empty maps when mixing wildcard- with other columns https://issues.apache.org/jira/browse/PIG-3108 PIG-3105Fix TestJobSubmission unit test failure. https://issues.apache.org/jira/browse/PIG-3105 PIG-3098Add another test for the self join case https://issues.apache.org/jira/browse/PIG-3098 PIG-3088Add a builtin udf which removes prefixes https://issues.apache.org/jira/browse/PIG-3088 PIG-3069Native Windows Compatibility for Pig E2E Tests and Harness https://issues.apache.org/jira/browse/PIG-3069 PIG-3028testGrunt dev test needs some command filters to run correctly without cygwin https://issues.apache.org/jira/browse/PIG-3028 PIG-3027pigTest unit test needs a newline filter for comparisons of golden multi-line https://issues.apache.org/jira/browse/PIG-3027 PIG-3026Pig checked-in baseline comparisons need a pre-filter to address OS-specific newline differences https://issues.apache.org/jira/browse/PIG-3026 PIG-3025TestPruneColumn unit test - SimpleEchoStreamingCommand perl inline script needs simplification https://issues.apache.org/jira/browse/PIG-3025 PIG-3024TestEmptyInputDir unit test - hadoop version detection logic is brittle https://issues.apache.org/jira/browse/PIG-3024 PIG-3015Rewrite of AvroStorage https://issues.apache.org/jira/browse/PIG-3015 PIG-3010Allow UDF's to flatten themselves https://issues.apache.org/jira/browse/PIG-3010 PIG-2959Add a pig.cmd for Pig to run under Windows https://issues.apache.org/jira/browse/PIG-2959 PIG-2955 Fix bunch of Pig e2e tests on Windows https://issues.apache.org/jira/browse/PIG-2955 PIG-2880Pig current releases lack a UDF charAt.This UDF returns the char value at the specified index. https://issues.apache.org/jira/browse/PIG-2880 PIG-1914Support load/store JSON data in Pig https://issues.apache.org/jira/browse/PIG-1914 PIG-1151Date Conversion + Arithmetic UDFs https://issues.apache.org/jira/browse/PIG-1151 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225filterId=12322384
[jira] [Commented] (PIG-2830) Macros should work in Grunt
[ https://issues.apache.org/jira/browse/PIG-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575025#comment-13575025 ] Mike Kerzhner commented on PIG-2830: This issue has the following side effect: when a Pig script with a macro is embedded in Python, execution will result in an error. Debugging this type of issue is difficult and non-obvious. Can we bump the priority of this ticket? Macros should work in Grunt --- Key: PIG-2830 URL: https://issues.apache.org/jira/browse/PIG-2830 Project: Pig Issue Type: Improvement Components: grunt, parser Affects Versions: 0.10.0, 0.11, 0.10.1 Reporter: Russell Jurney Priority: Minor Labels: fun, grunt, happy, macro, pants Fix For: 0.12 It would be very helpful in writing Pig scripts if Grunt could load and use Macros in an interactive session. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3174) Remove rpm and deb artifacts from build.xml
[ https://issues.apache.org/jira/browse/PIG-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575046#comment-13575046 ] Bill Graham commented on PIG-3174: -- +1 Remove rpm and deb artifacts from build.xml --- Key: PIG-3174 URL: https://issues.apache.org/jira/browse/PIG-3174 Project: Pig Issue Type: Task Components: build Affects Versions: 0.12 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.12 Attachments: PIG-3174.patch I propose that we remove the targets to build rpms and debs from build.xml and consequently quit publishing them as part of our releases. Bigtop publishes these packages now. And building them takes infrastructure that not every committer/PMC member has. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Pig 11 release notes - draft
This is great, thanks guys. On Fri, Feb 8, 2013 at 12:36 PM, Prashant Kommireddi prash1...@gmail.comwrote: Hey Bill, There are also a few new builtins for better Map support https://issues.apache.org/jira/browse/PIG-2600 A way to load pig statements at startup https://issues.apache.org/jira/browse/PIG-2456 -Prashant On Thu, Feb 7, 2013 at 3:11 PM, Rohini Palaniswamy rohini.adi...@gmail.comwrote: This is something Daniel mentioned in his talk in pig meetup and is missing in this list PIG-2482 - Integrate HCat DDL command into Pig Also there are a lot of UDFs added for DateTime datatype. A mention of that would be good. Regards, Rohini On Thu, Feb 7, 2013 at 2:27 PM, Rohini Palaniswamy rohini.adi...@gmail.comwrote: Boolean data type is available from 0.10 Regards, Rohini On Thu, Feb 7, 2013 at 1:24 PM, Bill Graham billgra...@gmail.com wrote: Hi, In the Pig 11 release I'd like to include some specifics in the RELEASE_NOTES.txt file. Julien compiled this list (which I've tweaked a bit). Please give it a read and let me know if we've missed anything big, or mis-stated anything. Maybe an absentee agenda item suggestion for those at the meetup? :) (I ran into some distractions this past week with cutting a release candidate, but doing so is still high on my list. Stay tuned...) - New RANK, CUBE and ROLLUP operators - New DateType and Boolean data types - Support for Groovy UDFs - Support for loading macros from jars - Support for custom PigReducerEstimators - Suoport for custom PigProgressNotificatonListeners - Support for schema-based Tuples for reduced memory footprint - Support for passing environment variables to streaming jobs - New cleanupOnSuccess method in StoreFunc interface - UDF timing utilities - UDF lifecycle improvements - Performance improvements to merge join - Performance improvements to local mode - Performance improvements to algebraics in-memory aggregation - Performance improvements to Spillable management - Performance improvements to HBaseStorage - Performance improvements to AvroStorage - Penny has been removed - 300+ bug fixes * * thanks, Bill -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.*
