The "doc" field should be at the level of the record, not the field. Maybe
that's the issue even though the exception is not clear.
For the first version, you can let Pig generate the schema and then evolve
it.
Bertrand
On Tue, Oct 15, 2013 at 7:29 PM, anup ahire wrote:
> Hello ,
>
> I am try
Hello ,
I am trying to store data into avro using AvroStorage() with following
schema. I have pig 0.11.
{"type":"record","name":"TUPLE_0","fields":[{"name":"Header","type":["null","string"],"doc":"autogenerated
from Pig Field Schema"}]}
I am getting following errors when I run the job.
Caused b
Thanks for the heads up and the solution.
I was trying to get pig working with avro in a prototype fashion so I
didn't have any version constraints.
On Tue, Oct 1, 2013 at 10:00 AM, Rohini Palaniswamy wrote:
> 0.23 has json-simple jar in the hadoop installation, so there is no
> problem. When
0.23 has json-simple jar in the hadoop installation, so there is no
problem. When you go to hadoop 2.x, you will hit the same problem as
json-simple is not in 2.x hadoop installation. I actually hit the error you
mentioned with 1.x while trying 2.x.
Regards,
Rohini
On Mon, Sep 30, 2013 at 6:09 P
I ended up just using the .23.9 hadoop release without any issue.
On Mon, Sep 30, 2013 at 8:54 PM, Rohini Palaniswamy wrote:
> It hits this error when json-simple-1.1.jar is not in classpath. You can
> get around that by adding it to PIG_CLASSPATH apart from registering the
> jar. The probl
It hits this error when json-simple-1.1.jar is not in classpath. You can
get around that by adding it to PIG_CLASSPATH apart from registering the
jar. The problem is with java classloading where it fails to load the
exception class(ParseException) thrown by a constructor of the class(
AvroStorage
Not my day I guess.
Trying with Hadoop 1.2.x
Getting :
Caused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments 'null'
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:618)
Caused by: java.l
Are the releases from the download page not compatible with 23.x? or 2.X
Says they are -
http://pig.apache.org/releases.html#1+April%2C+2013%3A+release+0.11.1+available
In any case I tried it with .23.9 and received a different error:
2013-09-19 15:13:56,044 [main] WARN
org.apache.pig.backend.ha
It sounds like you're using a version of Pig that wasn't compiled for
Hadoop 2.x/.23. Try recompiling with 'ant clean jar
-Dhadoopversion=23'.
-Mark
On Thu, Sep 19, 2013 at 9:23 AM, j.barrett Strausser
wrote:
> Running
>
> Hadoop-2.1.0-Beta
> Pig-0.11.1
> Hive-0.11.1
>
> 1. Created Avro backed t
Running
Hadoop-2.1.0-Beta
Pig-0.11.1
Hive-0.11.1
1. Created Avro backed table in Hive.
2. Loaded the table in Pig - records = Load '/path' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();
3. Can successfully describe the relation.
I registered the following on pig start :
REGISTER pigg
Np, I included it myself forever before someone pointed that out :)
Russell Jurney http://datasyndrome.com
On Jan 9, 2013, at 2:15 PM, Cheolsoo Park wrote:
> Hi Russell,
>
> You're absolute right. Jackson is not needed. Thanks for point that out!
>
> Cheolsoo
>
>
> On Wed, Jan 9, 2013 at 11:25
Hi Russell,
You're absolute right. Jackson is not needed. Thanks for point that out!
Cheolsoo
On Wed, Jan 9, 2013 at 11:25 AM, Russell Jurney wrote:
> Jackson is no longer needed, right? Or is it coming back in 0.11?
>
> Russell Jurney http://datasyndrome.com
>
> On Jan 9, 2013, at 10:26 AM, C
Jackson is no longer needed, right? Or is it coming back in 0.11?
Russell Jurney http://datasyndrome.com
On Jan 9, 2013, at 10:26 AM, Cheolsoo Park wrote:
> Hi Milind,
>
> Please try this:
>
> REGISTER build/ivy/lib/Pig/avro-1.7.1.jar
> REGISTER build/ivy/lib/Pig/json-simple-1.1.jar
> REGISTER
Hi Milind,
Please try this:
REGISTER build/ivy/lib/Pig/avro-1.7.1.jar
REGISTER build/ivy/lib/Pig/json-simple-1.1.jar
REGISTER build/ivy/lib/Pig/jackson-mapper-asl-1.8.8.jar
REGISTER build/ivy/lib/Pig/jackson-core-asl-1.8.8.jar
REGISTER contrib/piggybank/java/piggybank.jar
employee = LOAD '/home/
Environment:
Pig version: 0.11
Hadoop 0.23.6.0.1301071353
Script:
REGISTER /homes/immilind/HadoopLocal/Jars/avro-1.7.1.jar
REGISTER /homes/immilind/HadoopLocal/Jars/jackson-all-1.8.10.jar
REGISTER /homes/immilind/HadoopLocal/Jars/jackson-core-asl-1.8.10.jar
REGISTER /homes/immilind/HadoopLocal
You can download, unzip and use Pig 0.10 without waiting for a
Cloudera release. It is a client-side tool. AvroStorage didn't work
very well until 0.10, I'm afraid.
Russell Jurney http://datasyndrome.com
On Jun 18, 2012, at 12:58 AM, Markus Resch wrote:
> Hey Russell,
>
> thanks a million for t
Hey Russell,
thanks a million for that hint. I tried registering all those jars as
well but it didn't work. What, from my point of view, is the most
annoying thing here is the fact that the data storage doesn't complain
about the missing jars but just works. So the following parts of the
script f
Oh, you maybe also need to load other jars? I load avro this way.
REGISTER /me/pig/build/ivy/lib/Pig/avro-1.5.3.jar
REGISTER /me/pig/build/ivy/lib/Pig/json-simple-1.1.jar
REGISTER /me/pig/contrib/piggybank/java/piggybank.jar
Russell Jurney http://datasyndrome.com
On Jun 15, 2012, at 1:50 AM,
In my experience, AvroStorage only really works reliably in Pig 0.10.
I believe CDH will have Pig 0.10 in v4.1. I suggest you upgrade to
v0.10 manually now, and if your data still won't load, please file a
JIRA.
Russell Jurney http://datasyndrome.com
On Jun 15, 2012, at 1:50 AM, Markus Resch wro
Hey all,
we're currently testing to switch over from CDH3 to CDH4.
When I try to read my Avro input data I get en Schema unknown Error:
bash-3.2$ pig
12/06/15 08:48:08 WARN pig.Main: Cannot write to log
file: /usr/lib/pig/pig_1339750088923.log
2012-06-15 08:48:09,415 [main] INFO
org.apache.pig.
20 matches
Mail list logo
