Re: Review Request 27820: Setup for Macros in DataFu. Basic setup, no automated testing. Need feedback.

> On Nov. 14, 2014, 2:40 a.m., Matthew Hayes wrote: > > datafu-pig/src/main/macros/nlp/tf_idf.pig > > Lines 72 (patched) > > > > > > Shouldn't this be SUM? As far as I can tell, it's OK that this is COUNT, if we're

Re: Contributing UDFs in Java to PiggyBank

Hi Ravi, You might also want to look at Apache DataFu to see if something similar doesn't exist already there. It also contains a collection of Pig UDF's. http://datafu.incubator.apache.org/docs/datafu/guide.html Regards,Eyal On Wednesday, November 23, 2016 10:50 AM, Adam Szita

[jira] [Commented] (PIG-4852) Add accumulator implementation for MaxTupleBy1stField

[ https://issues.apache.org/jira/browse/PIG-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385567#comment-15385567 ] Eyal Allweil commented on PIG-4852: --- Adam - What you're saying is of course true. But since

[jira] [Commented] (PIG-4943) Schema issue while storing multiple pig outputs using CSVExcelStorage

[ https://issues.apache.org/jira/browse/PIG-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364255#comment-15364255 ] Eyal Allweil commented on PIG-4943: --- It looks like this problem is described by [PIG-4689|https

[jira] [Updated] (PIG-4852) Add accumulator implementation for MaxTupleBy1stField

[ https://issues.apache.org/jira/browse/PIG-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4852: -- Labels: newbie performance (was: ) > Add accumulator implementation for MaxTupleBy1stFi

[jira] [Created] (PIG-4852) Add accumulator implementation for MaxTupleBy1stField

Eyal Allweil created PIG-4852: - Summary: Add accumulator implementation for MaxTupleBy1stField Key: PIG-4852 URL: https://issues.apache.org/jira/browse/PIG-4852 Project: Pig Issue Type

[jira] [Updated] (PIG-4808) PluckTuple overwrites regex if used more than once in the same script

[ https://issues.apache.org/jira/browse/PIG-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4808: -- Attachment: PIG-4808.patch This patch fixes the bug by removing the static modifier, and adds a new test

[jira] [Commented] (PIG-4803) Improve performance of regex-based builtin functions

[ https://issues.apache.org/jira/browse/PIG-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145309#comment-15145309 ] Eyal Allweil commented on PIG-4803: --- Definitely a bug in PluckTuple - I opened https://issues.apache.org

[jira] [Created] (PIG-4808) PluckTuple overwrites regex if used more than once in the same script

Eyal Allweil created PIG-4808: - Summary: PluckTuple overwrites regex if used more than once in the same script Key: PIG-4808 URL: https://issues.apache.org/jira/browse/PIG-4808 Project: Pig

[jira] [Updated] (PIG-4808) PluckTuple overwrites regex if used more than once in the same script

[ https://issues.apache.org/jira/browse/PIG-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4808: -- Status: Patch Available (was: Open) > PluckTuple overwrites regex if used more than once in the s

[jira] [Assigned] (PIG-4803) Improve performance of regex-based builtin functions

[ https://issues.apache.org/jira/browse/PIG-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil reassigned PIG-4803: - Assignee: Eyal Allweil > Improve performance of regex-based builtin functi

[jira] [Created] (PIG-4803) Improve performance of regex-based builtin functions

Eyal Allweil created PIG-4803: - Summary: Improve performance of regex-based builtin functions Key: PIG-4803 URL: https://issues.apache.org/jira/browse/PIG-4803 Project: Pig Issue Type

[jira] [Updated] (PIG-4803) Improve performance of regex-based builtin functions

[ https://issues.apache.org/jira/browse/PIG-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4803: -- Attachment: PIG-4803.patch This patch gives REPLACE a similar compiled pattern caching strategy

[jira] [Updated] (PIG-4803) Improve performance of regex-based builtin functions

[ https://issues.apache.org/jira/browse/PIG-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4803: -- Labels: perfomance regex (was: ) > Improve performance of regex-based builtin functi

[jira] [Commented] (PIG-2599) Mavenize Pig

[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130470#comment-15130470 ] Eyal Allweil commented on PIG-2599: --- I've worked with countless Maven projects, and it almost always works

[jira] [Commented] (PIG-4761) Add more information to front end error messages

[ https://issues.apache.org/jira/browse/PIG-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128262#comment-15128262 ] Eyal Allweil commented on PIG-4761: --- Any update? Any reason not to pull it? (sorry for nagging, it just

[jira] [Commented] (PIG-4761) Add more information to front end error messages

[ https://issues.apache.org/jira/browse/PIG-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092133#comment-15092133 ] Eyal Allweil commented on PIG-4761: --- Did they pass? How hard is it / long does it take to run them locally

[jira] [Commented] (PIG-4762) XMLLoader issue when the parent and child tags starts with the same name(See Description).

[ https://issues.apache.org/jira/browse/PIG-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070790#comment-15070790 ] Eyal Allweil commented on PIG-4762: --- On Pig 12 and 14, the first XML seems to be loaded fine for me. Were

[jira] [Commented] (PIG-4761) Add more information to front end error messages

[ https://issues.apache.org/jira/browse/PIG-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068588#comment-15068588 ] Eyal Allweil commented on PIG-4761: --- Hey guys, can someone look at this? It's very small, but should

[jira] [Updated] (PIG-4761) Add more information to front end error messages

[ https://issues.apache.org/jira/browse/PIG-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4761: -- Attachment: PIG-4761.patch Passes ant clean test-commit. Only consists of changes to FrontendException

[jira] [Updated] (PIG-4761) Add more information to front end error messages

[ https://issues.apache.org/jira/browse/PIG-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4761: -- Patch Info: Patch Available > Add more information to front end error messa

[jira] [Commented] (PIG-4617) XML loader is not working fine with pig 0.14 version

[ https://issues.apache.org/jira/browse/PIG-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055946#comment-15055946 ] Eyal Allweil commented on PIG-4617: --- There doesn't seem to have been a change to the XML loader in Pig

[jira] [Created] (PIG-4761) Add more information to front end error messages

Eyal Allweil created PIG-4761: - Summary: Add more information to front end error messages Key: PIG-4761 URL: https://issues.apache.org/jira/browse/PIG-4761 Project: Pig Issue Type: Improvement

[jira] [Commented] (PIG-1798) nested foreach - alias for expression does not work

[ https://issues.apache.org/jira/browse/PIG-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002226#comment-15002226 ] Eyal Allweil commented on PIG-1798: --- If this is a duplicate of [PIG-1858], and it's been closed, does

[jira] [Resolved] (PIG-1556) Need a clean way to kill Pig jobs.

[ https://issues.apache.org/jira/browse/PIG-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil resolved PIG-1556. --- Resolution: Fixed Fix Version/s: 0.9.1 This looks like it was solved long ago - see https

[jira] [Resolved] (PIG-2031) NPE in TOP

[ https://issues.apache.org/jira/browse/PIG-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil resolved PIG-2031. --- Resolution: Fixed The issue, as it's described, no longer occurs. NPE in TOP

[jira] [Commented] (PIG-2031) NPE in TOP

[ https://issues.apache.org/jira/browse/PIG-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585712#comment-14585712 ] Eyal Allweil commented on PIG-2031: --- Is this issue still relevant? I think the current

[jira] [Updated] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4365: -- Attachment: PIG-4365-3.patch Implementing Rohini's suggestions for code reuse in the new test within

[jira] [Commented] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580197#comment-14580197 ] Eyal Allweil commented on PIG-4365: --- Any comments? As far as I can see, no one has written

[jira] [Commented] (PIG-2678) Create abstract PigProgressNotificationListener

[ https://issues.apache.org/jira/browse/PIG-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558164#comment-14558164 ] Eyal Allweil commented on PIG-2678: --- This is really old. Is it still relevant

[jira] [Commented] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557831#comment-14557831 ] Eyal Allweil commented on PIG-4365: --- I posted the latest patch to the review board, here

Review Request 34537: PIG-4365 TOP udf should implement Accumulator interface

/TestTOP.java 1680807 trunk/test/org/apache/pig/test/TestAccumulator.java 1680807 Diff: https://reviews.apache.org/r/34537/diff/ Testing --- Thanks, Eyal Allweil

[jira] [Updated] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4365: -- Attachment: PIG-4365-2.patch New patch. I changed it according to Rohini's comments; it returns null

[jira] [Commented] (PIG-4450) Make Pig Eclipse Setup Easier

[ https://issues.apache.org/jira/browse/PIG-4450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525210#comment-14525210 ] Eyal Allweil commented on PIG-4450: --- Following these instructions (and those in the wiki

[jira] [Work started] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on PIG-4365 started by Eyal Allweil. - TOP udf should implement Accumulator interface

[jira] [Updated] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4365: -- Fix Version/s: 0.15.0 Affects Version/s: 0.15.0 Release Note: TOP udf implements

[jira] [Assigned] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil reassigned PIG-4365: - Assignee: Eyal Allweil TOP udf should implement Accumulator interface

[jira] [Updated] (PIG-4365) TOP udf should implement Accumulator interface

[ https://issues.apache.org/jira/browse/PIG-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-4365: -- Attachment: PIG-4365.patch I think the implementation of Accumulator is pretty straightforward - I

Using Pig web documentation in Pig-Eclipse plugin

Hi guys, I'm working on the next version of Pig-Eclipse, which will contain (among other things) hover information. For example, for macros, it will display the macro definition line - DEFINE my_macro(A, sortkey) RETURNS C For UDF's it will attempt to find the javadoc (the jar has to be loaded

[jira] [Commented] (PIG-2949) JsonLoader only reads arrays of objects

[ https://issues.apache.org/jira/browse/PIG-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278482#comment-14278482 ] Eyal Allweil commented on PIG-2949: --- Will there be a 0.13.1 or 0.14.1 release? If so

[jira] [Commented] (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14261063#comment-14261063 ] Eyal Allweil commented on PIG-366: -- Hi [~spas]] - I wasn't able to get PigPen to work

[jira] [Updated] (PIG-2949) JsonLoader only reads arrays of objects

[ https://issues.apache.org/jira/browse/PIG-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-2949: -- Patch Info: Patch Available JsonLoader only reads arrays of objects

[jira] [Commented] (PIG-3615) Update the way that JsonLoader/JsonStorage deal with BigDecimal

[ https://issues.apache.org/jira/browse/PIG-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213454#comment-14213454 ] Eyal Allweil commented on PIG-3615: --- Are you sure? I am looking at https://github.com

[jira] [Commented] (PIG-3615) Update the way that JsonLoader/JsonStorage deal with BigDecimal

[ https://issues.apache.org/jira/browse/PIG-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212752#comment-14212752 ] Eyal Allweil commented on PIG-3615: --- Daniel, you committed all of the older patch, the one

[jira] [Commented] (PIG-2949) JsonLoader only reads arrays of objects

[ https://issues.apache.org/jira/browse/PIG-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14209537#comment-14209537 ] Eyal Allweil commented on PIG-2949: --- David's fix seems to work, but maybe we want

[jira] [Assigned] (PIG-2949) JsonLoader only reads arrays of objects

[ https://issues.apache.org/jira/browse/PIG-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil reassigned PIG-2949: - Assignee: Eyal Allweil JsonLoader only reads arrays of objects

[jira] [Updated] (PIG-3615) Update the way that JsonLoader/JsonStorage deal with BigDecimal

[ https://issues.apache.org/jira/browse/PIG-3615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-3615: -- Attachment: PIG-3615.patch How about if we only do half the change? We change the way JsonLoader works

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203931#comment-14203931 ] Eyal Allweil commented on PIG-3399: --- Ok, having set PIG_HOME (to /tmp), pulling to head

[jira] [Updated] (PIG-3413) JsonLoader fails the pig job in case of malformed json input

[ https://issues.apache.org/jira/browse/PIG-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eyal Allweil updated PIG-3413: -- Attachment: PIG-3413.patch This is a patch based on trunk which includes the fix for ignoring malformed

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200143#comment-14200143 ] Eyal Allweil commented on PIG-3399: --- I don't have that file. I didn't set PIG_HOME on my

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197874#comment-14197874 ] Eyal Allweil commented on PIG-3399: --- I'm still getting errors - different ones, though

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198992#comment-14198992 ] Eyal Allweil commented on PIG-3399: --- It's here - https://gist.github.com/eyala

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14199320#comment-14199320 ] Eyal Allweil commented on PIG-3399: --- Here it is: Testsuite

[jira] [Commented] (PIG-3399) Missing required library: 'build/ivy/lib/Pig/javacc-4.2.jar' pig

[ https://issues.apache.org/jira/browse/PIG-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195962#comment-14195962 ] Eyal Allweil commented on PIG-3399: --- Neither works for me on trunk, but copying the jar

[jira] [Commented] (PIG-3413) JsonLoader fails the pig job in case of malformed json input

[ https://issues.apache.org/jira/browse/PIG-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195963#comment-14195963 ] Eyal Allweil commented on PIG-3413: --- I'm having trouble building trunk - probably because

[jira] [Commented] (PIG-3413) JsonLoader fails the pig job in case of malformed json input

[ https://issues.apache.org/jira/browse/PIG-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194518#comment-14194518 ] Eyal Allweil commented on PIG-3413: --- Still interested in a patch? It does look pretty

[jira] [Commented] (PIG-3350) JsonLoader() is not working properly for maps

[ https://issues.apache.org/jira/browse/PIG-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194321#comment-14194321 ] Eyal Allweil commented on PIG-3350: --- If I try to run this in one script, it fails, saying

[jira] [Commented] (PIG-2836) Namespace in Pig macros collides with Pig scripts

[ https://issues.apache.org/jira/browse/PIG-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14185081#comment-14185081 ] Eyal Allweil commented on PIG-2836: --- I didn't try 0.13 - I had trouble setting it up, I

[jira] [Commented] (PIG-2836) Namespace in Pig macros collides with Pig scripts

[ https://issues.apache.org/jira/browse/PIG-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184534#comment-14184534 ] Eyal Allweil commented on PIG-2836: --- Here is a script that demonstrates this problem. I've

[jira] [Commented] (PIG-2836) Namespace in Pig macros collides with Pig scripts

[ https://issues.apache.org/jira/browse/PIG-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179692#comment-14179692 ] Eyal Allweil commented on PIG-2836: --- From what version is this fixed? I'm using 0.11.1

[jira] [Commented] (PIG-2175) Switch Pig wiki to use confluence

[ https://issues.apache.org/jira/browse/PIG-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14108804#comment-14108804 ] Eyal Allweil commented on PIG-2175: --- Thanks! My username is eyal. Switch Pig wiki to use

[jira] [Commented] (PIG-2175) Switch Pig wiki to use confluence

[ https://issues.apache.org/jira/browse/PIG-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14108421#comment-14108421 ] Eyal Allweil commented on PIG-2175: --- Who should have edit permissions for the wiki