I have had both positive and negative results with PIG. The positive results were that I was able to express large recommendation computations in a very concise way. That was really helpful.
My negative results have been to do with the brittle nature of PIG vis a vis the version of the underlying hadoop system. That problem may have abated somewhat as everybody in the world except me and Amazon's EMR has pretty much piled up on version 20. I also know little about how Pig would interface well with other components. I know that I have had difficulty in the past injecting outside information into Pig, but that has been improved. I also know that "Pigs eat anything", but have no clear idea how well this would play out with, say, our vector formats and vectorizers. Ankur, what recent experience do you have? How well do PIG scripts play with other programs any more? On Sun, Feb 21, 2010 at 11:41 PM, Ankur C. Goel <gan...@yahoo-inc.com>wrote: > I had Sean's opinion on this and he was not too comfortable with the Idea > of having things in different languages in Mahout. However, given the > benefits of PIG, I feel otherwise. I may be biased here due to my own > experience of being able to do more in lesser time in Pig then in M/R, so I > thought let me ask how folks feel. > > Ted, I believe you have some PIG experience yourself so any thoughts on > this ? > -- Ted Dunning, CTO DeepDyve
