[
https://issues.apache.org/jira/browse/TAJO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699227#comment-14699227
]
Jinho Kim commented on TAJO-1340:
---------------------------------
Here is my hang log. I will remove the stream seek in TAJO-1738
{noformat}
587 "TajoMasterClientProtocol-3 Server Worker #1" #81 prio=5 os_prio=0
tid=0x00007f93c4002000 nid=0x72ee runnable [0x00007f93a42f3000]
588 java.lang.Thread.State: RUNNABLE
589 at sun.nio.ch.NativeThread.current(Native Method)
590 at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:468)
591 - locked <0x00000000cc243248> (a java.lang.Object)
592 - locked <0x00000000cc243238> (a java.lang.Object)
593 at
org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63)
594 at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
595 at
org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159)
596 at
org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
597 at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
598 at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
599 - locked <0x00000000f6f0e3b8> (a java.io.BufferedOutputStream)
600 at java.io.DataOutputStream.flush(DataOutputStream.java:123)
601 at
org.apache.hadoop.hdfs.protocol.datatransfer.Sender.send(Sender.java:82)
602 at
org.apache.hadoop.hdfs.protocol.datatransfer.Sender.readBlock(Sender.java:113)
603 at
org.apache.hadoop.hdfs.RemoteBlockReader2.newBlockReader(RemoteBlockReader2.java:414)
604 at
org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:818)
605 at
org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:697)
606 at
org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:355)
607 at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:618)
608 - locked <0x00000000c3a36b50> (a
org.apache.hadoop.hdfs.DFSInputStream)
609 at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:844)
610 - locked <0x00000000c3a36b50> (a
org.apache.hadoop.hdfs.DFSInputStream)
611 at
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:908)
612 - locked <0x00000000c3a36b50> (a
org.apache.hadoop.hdfs.DFSInputStream)
613 at
org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:143)
614 at
org.apache.tajo.storage.FSDataInputChannel.read(FSDataInputChannel.java:54)
615 at
org.apache.tajo.tuple.offheap.OffHeapRowBlock.copyFromChannel(OffHeapRowBlock.java:141)
616 at
org.apache.tajo.storage.rawfile.DirectRawFileScanner.next(DirectRawFileScanner.java:123)
617 at
org.apache.tajo.storage.rawfile.DirectRawFileScanner.next(DirectRawFileScanner.java:136)
618 at org.apache.tajo.storage.MergeScanner.next(MergeScanner.java:103)
619 at
org.apache.tajo.engine.planner.physical.FullScanIterator.hasNext(FullScanIterator.java:39)
620 at
org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:249)
621 at
org.apache.tajo.master.exec.NonForwardQueryResultFileScanner.getNextRows(NonForwardQueryResultFileScanner.java:162)
622 at
org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getQueryResultData(TajoMasterClientService.java:566)
{noformat}
> Change the default output file format.
> --------------------------------------
>
> Key: TAJO-1340
> URL: https://issues.apache.org/jira/browse/TAJO-1340
> Project: Tajo
> Issue Type: Improvement
> Reporter: Hyunsik Choi
> Assignee: Jinho Kim
> Fix For: 0.11.0
>
>
> Currently, the default output file is CSV. Due to its nature, CSV has mainly
> three problems:
> * Its line or field delimiter can be duplicated to some character included
> in the result data.
> * Plan text file is likely to be larger than other file formats.
> * Its read and write performance is slow.
> We need to change the default output file format into other file formats. We
> also need to investigate which file format is the best for it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
