Thanks for the quick reply Dmitriy. Does it mean that its tough to have non-equi join type of joins between 2 datasets in PIG? Isn't it a common scenario in production systems?
On Tue, Dec 14, 2010 at 6:59 AM, Dmitriy Ryaboy <dvrya...@gmail.com> wrote: > Rajesh, that's not a map-reduce friendly computation, as it is essentially > a > cross. > Which is how you would implement something like this -- and it would be > awfully slow or just not computable for very large datasets: cross, then > filter. > > -Dmitriy > > On Mon, Dec 13, 2010 at 5:04 PM, Rajesh Balamohan < > rajesh.balamo...@gmail.com> wrote: > > > Hi Folks, > > > > I have 2 datasets (T1, T2) to be joinned. > > > > I need to join T1 with T2 based on some criteria. COGROUP does it based > on > > == condition. > > > > ex: COGROUP T1 by f1, T2 by f2 (but I need to filter T2.f2 > T1.f1). > > > > Is there a way to specify such conditions in PIG?. > > > > ~Rajesh.B > > > -- ~Rajesh.B
