[ https://issues.apache.org/jira/browse/PIG-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163040#comment-13163040 ]
Dmitriy V. Ryaboy commented on PIG-1150: ---------------------------------------- An accumulative version has been checked into the LinkedIn DataFu library (a contrib from Cloudera). You can use that. It's still not as efficient as one would like, I might have some news on that front once internal testing at Twitter is done :). > VAR() Variance UDF > ------------------ > > Key: PIG-1150 > URL: https://issues.apache.org/jira/browse/PIG-1150 > Project: Pig > Issue Type: New Feature > Affects Versions: 0.5.0 > Environment: UDF, written in Pig 0.5 contrib/ > Reporter: Russell Jurney > Assignee: Dmitriy V. Ryaboy > Attachments: var.patch > > > I've implemented a UDF in Pig 0.5 that implements Algebraic and calculates > variance in a distributed manner, based on the AVG() builtin. It works by > calculating the count, sum and sum of squares, as described here: > http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm > Is this a worthwhile contribution? Taking the square root of this value > using the contrib SQRT() function gives Standard Deviation, which is missing > from Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira