[
https://issues.apache.org/jira/browse/KYLIN-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532442#comment-16532442
]
Chao Long commented on KYLIN-3431:
--
I have replaced FileInputStream/FileOutputStream with
FIles.newInputStream/newOutputStream, but when I run CI, I get error message
like this:
2018-07-04 06:10:41,485 INFO [main] threadpool.DefaultScheduler:192 : Shutting
down DefaultScheduler
2018-07-04 06:10:41,487 ERROR [Scheduler 107702879 Job
b036fdd2-73e2-4463-b1a6-841018297f1c-21] dao.ExecutableDao:219 : error update
job output id:b036fdd2-73e2-4463-b1a6-841018297f1c-00
java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:216)
at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
at java.nio.channels.Channels.writeFully(Channels.java:101)
at java.nio.channels.Channels.access$000(Channels.java:61)
at java.nio.channels.Channels$1.write(Channels.java:174)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at
org.apache.kylin.common.persistence.FileResourceStore.putResourceImpl(FileResourceStore.java:161)
at
org.apache.kylin.common.persistence.FileResourceStore.checkAndPutResourceImpl(FileResourceStore.java:177)
at
org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
at
org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
at
org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
at
org.apache.kylin.job.dao.ExecutableDao.writeJobOutputResource(ExecutableDao.java:88)
at
org.apache.kylin.job.dao.ExecutableDao.updateJobOutput(ExecutableDao.java:216)
at
org.apache.kylin.job.execution.ExecutableManager.addJobInfo(ExecutableManager.java:480)
at
org.apache.kylin.job.execution.ExecutableManager.addJobInfo(ExecutableManager.java:490)
at
org.apache.kylin.job.execution.AbstractExecutable.addExtraInfo(AbstractExecutable.java:403)
at
org.apache.kylin.job.execution.AbstractExecutable.setEndTime(AbstractExecutable.java:419)
at
org.apache.kylin.job.execution.AbstractExecutable.onExecuteFinished(AbstractExecutable.java:121)
at
org.apache.kylin.job.execution.AbstractExecutable.onExecuteFinishedWithRetry(AbstractExecutable.java:98)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:175)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
code block: FileResourceStore.java->putResourceImpl()
@Override
protected void putResourceImpl(String resPath, InputStream content, long ts)
throws IOException {
synchronized (FileResourceStore.class) {
File f = file(resPath);
f.getParentFile().mkdirs();
try (final OutputStream out = Files.newOutputStream(f.toPath())) {
IOUtils.copy(content, out);
f.setLastModified(ts);
}
}
}
After checking the code, I find method
DefaultSchedulerTest.java->testSchedulerStop() cause this problem. I also find
"FIle.newInputStream" use "channal" to do his work, and If a class implements
the "InterruptibleChannel" interface, then when the thread on the channel is
interrupted, the channel will be closed and the thread will throws
"ClosedByInterruptException" exception.
So can I catch this exception["ClosedByInterruptException"] in
method["putResourceImpl"] or any other suggestions provided?
> Avoid FileInputStream/FileOutputStream
> --
>
> Key: KYLIN-3431
> URL: https://issues.apache.org/jira/browse/KYLIN-3431
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: image-2018-07-04-16-41-54-234.png
>
>
> They rely on finalizers (before Java 11), which create unnecessary GC load.
> The alternatives, {{Files.newInputStream}}, are as easy to use and don't have
> this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
