we use zip codec in default. Some of the same lines were omitted from the error stack: at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:272)
Thanks, Min On Mon, Nov 2, 2009 at 2:57 PM, Zheng Shao <zsh...@gmail.com> wrote: > Min, can you check the default compression codec in your hadoop conf? > The 8-byte file must be a compressed file using the codec which > represents 0-length file. > > It seems that codec was not able to decompress the stream. > > Zheng > > On Sun, Nov 1, 2009 at 10:49 PM, Min Zhou <coderp...@gmail.com> wrote: >> I think there may be a bug still in this release. >> >> hive>select stuff_status from auctions where auction_id='2591238417' >> and pt='20091027'; >> >> auctions is a table partitioned by date, it stored as a textfile w/o >> compression. The query above should return 0 rows. >> but when hive.exec.compress.output=true, hive will crash with a >> StackOverflowError >> >> java.lang.StackOverflowError >> at java.lang.ref.FinalReference.<init>(FinalReference.java:16) >> at java.lang.ref.Finalizer.<init>(Finalizer.java:66) >> at java.lang.ref.Finalizer.register(Finalizer.java:72) >> at java.lang.Object.<init>(Object.java:20) >> at java.net.SocketImpl.<init>(SocketImpl.java:27) >> at java.net.PlainSocketImpl.<init>(PlainSocketImpl.java:90) >> at java.net.SocksSocketImpl.<init>(SocksSocketImpl.java:33) >> at java.net.Socket.setImpl(Socket.java:434) >> at java.net.Socket.<init>(Socket.java:68) >> at sun.nio.ch.SocketAdaptor.<init>(SocketAdaptor.java:50) >> at sun.nio.ch.SocketAdaptor.create(SocketAdaptor.java:55) >> at sun.nio.ch.SocketChannelImpl.socket(SocketChannelImpl.java:105) >> at >> org.apache.hadoop.net.StandardSocketFactory.createSocket(StandardSocketFactory.java:58) >> at >> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1540) >> at >> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1662) >> at java.io.DataInputStream.read(DataInputStream.java:132) >> at >> org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:96) >> at >> org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:86) >> at >> org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:74) >> at java.io.InputStream.read(InputStream.java:85) >> at org.apache.hadoop.util.LineReader.backfill(LineReader.java:82) >> at org.apache.hadoop.util.LineReader.readLine(LineReader.java:112) >> at >> org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:134) >> at >> org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:39) >> at >> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:256) >> at >> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:272) >> >> Each mapper will produce a 8 bytes deflate file on hdfs(we set >> hive.merge.mapfiles=false), their hex representation is like below: >> >> 78 9C 03 00 00 00 00 01 >> >> This is the reason why FetchOperator:272 is called recursively, and >> caused a stack overflow error. >> >> Regards, >> Min >> >> >> On Mon, Nov 2, 2009 at 6:34 AM, Zheng Shao <zsh...@gmail.com> wrote: >>> I have made a release candidate 0.4.1-rc0. >>> >>> We've fixed several critical bugs to hive release 0.4.0. We need hive >>> release 0.4.1 out asap. >>> >>> Here are the list of changes: >>> >>> HIVE-884. Metastore Server should call System.exit() on error. >>> (Zheng Shao via pchakka) >>> >>> HIVE-864. Fix map-join memory-leak. >>> (Namit Jain via zshao) >>> >>> HIVE-878. Update the hash table entry before flushing in Group By >>> hash aggregation (Zheng Shao via namit) >>> >>> HIVE-882. Create a new directory every time for scratch. >>> (Namit Jain via zshao) >>> >>> HIVE-890. Fix cli.sh for detecting Hadoop versions. (Paul Huff via zshao) >>> >>> HIVE-892. Hive to kill hadoop jobs using POST. (Dhruba Borthakur via >>> zshao) >>> >>> HIVE-883. URISyntaxException when partition value contains special chars. >>> (Zheng Shao via namit) >>> >>> >>> Please vote. >>> >>> -- >>> Yours, >>> Zheng >>> >> >> >> >> -- >> My research interests are distributed systems, parallel computing and >> bytecode based virtual machine. >> >> My profile: >> http://www.linkedin.com/in/coderplay >> My blog: >> http://coderplay.javaeye.com >> > > > > -- > Yours, > Zheng > -- My research interests are distributed systems, parallel computing and bytecode based virtual machine. My profile: http://www.linkedin.com/in/coderplay My blog: http://coderplay.javaeye.com
