[
https://issues.apache.org/jira/browse/PIG-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529484#comment-13529484
]
Russell Jurney commented on PIG-3015:
-------------------------------------
I got a clean clone of trunk and applied the patch, then attempt to load the
file again:
{noformat}
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/avro-1.7.2.jar
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/avro-ipc-1.7.2.jar
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/avro-mapred-1.7.2.jar
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/avro-tools-1.7.2.jar
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/trevni-avro-1.7.2.jar
grunt> REGISTER /me/Software/pig-trunk/build/ivy/lib/Pig/json-simple-1.1.jar
grunt> REGISTER /me/Software/pig-trunk/contrib/piggybank/java/piggybank.jar
grunt>
grunt>
grunt> messages = LOAD '/me/Data/test_inbox' USING AvroStorage();
2012-12-11 15:32:36,989 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
2998: Unhandled internal error. org/apache/avro/io/DatumReader
2012-12-11 15:32:36,989 [main] ERROR org.apache.pig.tools.grunt.Grunt -
java.lang.NoClassDefFoundError: org/apache/avro/io/DatumReader
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:510)
at
org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1206)
at
org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1194)
at
org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:4766)
at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3183)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1581)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1554)
at org.apache.pig.PigServer.registerQuery(PigServer.java:526)
at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:991)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:535)
at org.apache.pig.Main.main(Main.java:154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.apache.avro.io.DatumReader
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 27 more
{noformat}
> Rewrite of AvroStorage
> ----------------------
>
> Key: PIG-3015
> URL: https://issues.apache.org/jira/browse/PIG-3015
> Project: Pig
> Issue Type: Improvement
> Components: piggybank
> Reporter: Joseph Adler
> Assignee: Joseph Adler
> Attachments: PIG-3015.patch
>
>
> The current AvroStorage implementation has a lot of issues: it requires old
> versions of Avro, it copies data much more than needed, and it's verbose and
> complicated. (One pet peeve of mine is that old versions of Avro don't
> support Snappy compression.)
> I rewrote AvroStorage from scratch to fix these issues. In early tests, the
> new implementation is significantly faster, and the code is a lot simpler.
> Rewriting AvroStorage also enabled me to implement support for Trevni (as
> TrevniStorage).
> I'm opening this ticket to facilitate discussion while I figure out the best
> way to contribute the changes back to Apache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
