[ https://issues.apache.org/jira/browse/CARBONDATA-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903875#comment-16903875 ]
Ajantha Bhat commented on CARBONDATA-3490: ------------------------------------------ segment 6 and segment 7 got the same temp store path > Concurrent data load failure with carbondata FileNotFound exception > ------------------------------------------------------------------- > > Key: CARBONDATA-3490 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3490 > Project: CarbonData > Issue Type: Bug > Reporter: Ajantha Bhat > Priority: Major > > Caused by: > org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: > Problem while copying file from local store to carbon store > at > org.apache.carbondata.core.util.CarbonUtil.copyCarbonDataFileToCarbonStorePath(CarbonUtil.java:2750) > at > org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.commitCurrentFile(AbstractFactDataWriter.java:283) > at > org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:393) > ... 11 more > Caused by: java.io.FileNotFoundException: > /tmp/carbon865982118689228_1/Fact/Part0/Segment_6/1/part-0-1_batchno0-0-6-1565329654844.carbondata > (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.<init>(FileInputStream.java:138) > at java.io.FileInputStream.<init>(FileInputStream.java:93) > at > org.apache.carbondata.core.datastore.filesystem.LocalCarbonFile.getDataInputStream(LocalCarbonFile.java:309) > at > org.apache.carbondata.core.datastore.filesystem.LocalCarbonFile.getDataInputStream(LocalCarbonFile.java:299) > at > org.apache.carbondata.core.datastore.impl.FileFactory.getDataInputStream(FileFactory.java:179) > at > org.apache.carbondata.core.datastore.impl.FileFactory.getDataInputStream(FileFactory.java:175) > at > org.apache.carbondata.core.util.CarbonUtil.copyLocalFileToCarbonStore(CarbonUtil.java:2781) > at > org.apache.carbondata.core.util.CarbonUtil.copyCarbonDataFileToCarbonStorePath(CarbonUtil.java:2746) > ... 13 more > problem: When two load is happening concurrently, one load is cleaning the > temp directory of the concurrent load > cause: temp directory to store the carbon files is created using system.get > nano time, due to this two load have same store location. when one load is > completed, it cleaned the temp directory. causing dataload failure for other > load. > solution: > use UUID instead of nano time while creating the temp directory to have each > load a unique directory. -- This message was sent by Atlassian JIRA (v7.6.14#76016)