Chris D'Urso created AVRO-1486:
----------------------------------
Summary: avro-tools-1.7.6.jar fromtext to stdout fails because
status also printed to stdout
Key: AVRO-1486
URL: https://issues.apache.org/jira/browse/AVRO-1486
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.7.6
Reporter: Chris D'Urso
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge] # First the functioning case,
the creation of test.txt.avro
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test.txt
one 1
two 2
three 3
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test.txt | java -jar
~/lib/avro-tools-1.7.6.jar fromtext --codec snappy - test.txt.avro
Mar 24, 2014 6:13:50 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Writing line = 5
Writing line = 5
Writing line = 7
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ ls -l test.txt.avro
-rwxrwxrwx 1 b_ebayadvertising hdmi-ebayadvertising 104 Mar 24 18:13
test.txt.avro
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ strings test.txt.avro
avro.codec
snappy
avro.schema
"bytes"
f@5:B
one 1
two 2
three 3
f@5:B
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ wc -c test.txt.avro
104 test.txt.avro
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ # now what should be
equivalent is not so
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test.txt | java -jar
~/lib/avro-tools-1.7.6.jar fromtext --codec snappy - - > test2.txt.avro
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ # note that "Writing line"
is absent, and this is because its been written to stdout as well as the true
payload
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ strings test2.txt.avro
avro.codec
snappy
avro.schema
"bytes"
gD?Writing line = 5
Writing line = 5
Writing line = 7
one 1
two 2
three 3
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ # note that test.txt.avro
can be restored but not test2.txt.avro, plus test2 is bigger due to having
mixed stream data
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test.txt.avro | java
-jar ~/lib/avro-tools-1.7.6.jar totext - -
one 1
two 2
three 3
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test2.txt.avro | java
-jar ~/lib/avro-tools-1.7.6.jar totext - -
Exception in thread "main" org.apache.avro.AvroRuntimeException:
java.io.IOException: Invalid sync!
at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:210)
at org.apache.avro.tool.ToTextTool.run(ToTextTool.java:77)
at org.apache.avro.tool.Main.run(Main.java:84)
at org.apache.avro.tool.Main.main(Main.java:73)
Caused by: java.io.IOException: Invalid sync!
at
org.apache.avro.file.DataFileStream.nextRawBlock(DataFileStream.java:293)
at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:198)
... 3 more
[b_ebayadvertising@phx-ctd-hadoopas1 id_to_purge]$ cat test2.txt.avro | wc -c
155
--
This message was sent by Atlassian JIRA
(v6.2#6252)
