[ https://issues.apache.org/jira/browse/HADOOP-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559378#comment-16559378 ]
genericqa commented on HADOOP-15607: ------------------------------------ | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 46s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 27s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s{color} | {color:green} hadoop-aliyun in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 55m 46s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-15607 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933319/HADOOP-15607.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 2719c77de48c 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15 17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8d3c068 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14954/testReport/ | | Max. process+thread count | 336 (vs. ulimit of 10000) | | modules | C: hadoop-tools/hadoop-aliyun U: hadoop-tools/hadoop-aliyun | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14954/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream > ------------------------------------------------------------------------- > > Key: HADOOP-15607 > URL: https://issues.apache.org/jira/browse/HADOOP-15607 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 3.1.0, 2.10.0, 2.9.1, 3.2.0, 3.0.3 > Reporter: wujinhu > Assignee: wujinhu > Priority: Major > Attachments: HADOOP-15607.001.patch, HADOOP-15607.002.patch, > HADOOP-15607.003.patch, HADOOP-15607.004.patch > > > When I generated data with hive-tpcds tool, I got exception below: > 2018-07-16 14:50:43,680 INFO mapreduce.Job: Task Id : > attempt_1531723399698_0001_m_000052_0, Status : FAILED > Error: com.aliyun.oss.OSSException: The list of parts was not in ascending > order. Parts list must specified in order by part number. > [ErrorCode]: InvalidPartOrder > [RequestId]: 5B4C40425FCC208D79D1EAF5 > [HostId]: 100.103.0.137 > [ResponseError]: > <?xml version="1.0" encoding="UTF-8"?> > <Error> > <Code>InvalidPartOrder</Code> > <Message>The list of parts was not in ascending order. Parts list must > specified in order by part number.</Message> > <RequestId>5B4C40425FCC208D79D1EAF5</RequestId> > <HostId>xx.xx.xx.xx</HostId> > <ErrorDetail>current PartNumber 3, you given part number 3is not in > ascending order</ErrorDetail> > </Error> > at > com.aliyun.oss.common.utils.ExceptionFactory.createOSSException(ExceptionFactory.java:99) > at > com.aliyun.oss.internal.OSSErrorResponseHandler.handle(OSSErrorResponseHandler.java:69) > at > com.aliyun.oss.common.comm.ServiceClient.handleResponse(ServiceClient.java:248) > at > com.aliyun.oss.common.comm.ServiceClient.sendRequestImpl(ServiceClient.java:130) > at > com.aliyun.oss.common.comm.ServiceClient.sendRequest(ServiceClient.java:68) > at com.aliyun.oss.internal.OSSOperation.send(OSSOperation.java:94) > at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:149) > at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:113) > at > com.aliyun.oss.internal.OSSMultipartOperation.completeMultipartUpload(OSSMultipartOperation.java:185) > at com.aliyun.oss.OSSClient.completeMultipartUpload(OSSClient.java:790) > at > org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.completeMultipartUpload(AliyunOSSFileSystemStore.java:643) > at > org.apache.hadoop.fs.aliyun.oss.AliyunOSSBlockOutputStream.close(AliyunOSSBlockOutputStream.java:120) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) > at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101) > at > org.apache.hadoop.mapreduce.lib.output.TextOutputFormat$LineRecordWriter.close(TextOutputFormat.java:106) > at > org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.close(MultipleOutputs.java:574) > at org.notmysock.tpcds.GenTable$DSDGen.cleanup(GenTable.java:169) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:149) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1686) > > I reviewed code below, > {code:java} > blockId {code} > has thread synchronization problem > {code:java} > // code placeholder > private void uploadCurrentPart() throws IOException { > blockFiles.add(blockFile); > blockStream.flush(); > blockStream.close(); > if (blockId == 0) { > uploadId = store.getUploadId(key); > } > ListenableFuture<PartETag> partETagFuture = > executorService.submit(() -> { > PartETag partETag = store.uploadPart(blockFile, key, uploadId, > blockId + 1); > return partETag; > }); > partETagsFutures.add(partETagFuture); > blockFile = newBlockFile(); > blockId++; > blockStream = new BufferedOutputStream(new FileOutputStream(blockFile)); > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org