Re: single output file per partition?

2013-08-21 Thread Igor Tatarinov
nfirm first >> that "cluster by X into 1 bucket" didn't work? I would have thought that >> would have done it. >> >> >> >> >> On Tue, Aug 20, 2013 at 2:29 PM, Igor Tatarinov wrote: >> >>> What's the best way to enforce a s

Re: single output file per partition?

2013-08-21 Thread Sanjay Subramanian
hem all but let me confirm first that "cluster by X into 1 bucket" didn't work? I would have thought that would have done it. On Tue, Aug 20, 2013 at 2:29 PM, Igor Tatarinov mailto:i...@decide.com>> wrote: What's the best way to enforce a single output file per

Re: single output file per partition?

2013-08-21 Thread Stephen Sprague
hat "cluster by X into 1 bucket" didn't work? I would have thought that >> would have done it. >> >> >> >> >> On Tue, Aug 20, 2013 at 2:29 PM, Igor Tatarinov wrote: >> >>> What's the best way to enforce a single output file per partiti

Re: single output file per partition?

2013-08-21 Thread Igor Tatarinov
"cluster by X into 1 bucket" didn't work? I would have thought that > would have done it. > > > > > On Tue, Aug 20, 2013 at 2:29 PM, Igor Tatarinov wrote: > >> What's the best way to enforce a single output file per partition? >> >> INSERT OVE

Re: single output file per partition?

2013-08-21 Thread Stephen Sprague
hi igor, lots of ideas there! I can't speak for them all but let me confirm first that "cluster by X into 1 bucket" didn't work? I would have thought that would have done it. On Tue, Aug 20, 2013 at 2:29 PM, Igor Tatarinov wrote: > What's the best way to enfo

single output file per partition?

2013-08-20 Thread Igor Tatarinov
What's the best way to enforce a single output file per partition? INSERT OVERWRITE TABLE PARTITION (x,y,z) SELECT ... FROM ... WHERE ... It tried adding CLUSTER BY x,y,z at the end thinking that sorting will force a single reducer per partition but that didn't work. I still got mult