[ https://issues.apache.org/jira/browse/PIG-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967212#comment-15967212 ]
liyunzhang_intel edited comment on PIG-5171 at 4/13/17 7:00 AM: ---------------------------------------------------------------- [~nkollar]: it is a problem of mr. I guess if change SecondarySort_7 like( change desc to asc) , it will pass because in spark it implements secondary sort in desc while in mr it does not in current case. {code} { 'num' => 7, 'java_params' => ['-Dpig.accumulative.batchsize=5'], 'pig' => q\register :FUNCPATH:/testudf.jar; a = load ':INPATH:/singlefile/studenttab10k' as (name:chararray, age:int, gpa:float); b = group a by age parallel 10; c = foreach b { d = order a by gpa, name asc; generate group, org.apache.pig.test.udf.evalfunc.AllFirstLetter(d.gpa), org.apache.pig.test.udf.evalfunc.AllFirstLetter(d.name); }; store c into ':OUTPATH:';\, }, {code} Can you help verify it? if it is a mr bug, file jira and close current jira. was (Author: kellyzly): [~nkollar]: it is a problem of mr. I guess if change SecondarySort_7 like( change desc to asc) , it will pass because in spark it implements secondary sort in desc while in mr it does not. {code} { 'num' => 7, 'java_params' => ['-Dpig.accumulative.batchsize=5'], 'pig' => q\register :FUNCPATH:/testudf.jar; a = load ':INPATH:/singlefile/studenttab10k' as (name:chararray, age:int, gpa:float); b = group a by age parallel 10; c = foreach b { d = order a by gpa, name asc; generate group, org.apache.pig.test.udf.evalfunc.AllFirstLetter(d.gpa), org.apache.pig.test.udf.evalfunc.AllFirstLetter(d.name); }; store c into ':OUTPATH:';\, }, {code} Can you help verify it? if it is a mr bug, file jira and close current jira. > SecondarySort_7 is failing with spark exec type > ----------------------------------------------- > > Key: PIG-5171 > URL: https://issues.apache.org/jira/browse/PIG-5171 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: Nandor Kollar > Fix For: spark-branch > > > different output produced -- This message was sent by Atlassian JIRA (v6.3.15#6346)