AvroStorage schema_uri pointing to local file doesn't work

2013-12-24 Thread Ruslan Al-Fakikh
I am running the pig script from, cannot find the file in the local file system. Why can't it use the schema file from front-end invocation? Does it mean that I am only limited to either HDFS locations for schema_uri or using embedding the schema string in AvroStorage parameters? Thanks in advance Ruslan Al-Fakikh

Re: STORE USING AvroStorage - ignores Pig field names, only using their position

2013-11-16 Thread Ruslan Al-Fakikh
including this last message to pig user list On Sun, Nov 17, 2013 at 7:40 AM, Ruslan Al-Fakikh wrote: > Russel, > > Actually this problem came from the situation when I had the same names in > pig relation schema and avro schema. And it turned out that AvroStorage > switches fiel

Re: STORE USING AvroStorage - ignores Pig field names, only using their position

2013-11-16 Thread Ruslan Al-Fakikh
On Saturday, November 16, 2013, Ruslan Al-Fakikh wrote: > >> Thanks, Russel! >> >> Do you mean that this is the expected behavior? Shouldn't AvroStorage map >> the pig fields by their names (not their field order) matching them to the >> names in the avro schema?

Re: STORE USING AvroStorage - ignores Pig field names, only using their position

2013-11-16 Thread Ruslan Al-Fakikh
Thanks, Russel! Do you mean that this is the expected behavior? Shouldn't AvroStorage map the pig fields by their names (not their field order) matching them to the names in the avro schema? Thanks, Ruslan Al-Fakikh On Sun, Nov 17, 2013 at 6:53 AM, Russell Jurney wrote: > Pig tup

STORE USING AvroStorage - ignores Pig field names, only using their position

2013-11-16 Thread Ruslan Al-Fakikh
* --{"b":"data_a","nonsense_name":"data_b"} --{"b":"data_a","nonsense_name":"data_b"} AvroStorage is build from the latest piggybank code. Using AvroStorage "debug": 5 parameter didn't help. $ pig -version Apache Pig version 0.11.0-cdh4.3.0 (rexported) compiled May 27 2013, 20:48:21 Any help would be appreciated. Thanks, Ruslan Al-Fakikh

Re: Avrodoc: browsable documentation for Avro schemas

2012-11-14 Thread Ruslan Al-Fakikh
Hey guys, Just a thought: For me as a user it would be more useful to operate on avro docs as I operate on javadocs which means: 1) Having an IDE plugin like "javadoc view" in Eclipse 2) Having some tools integrated to Maven or Ant to generate the html docs (I am not sure about this, but keeping it

Re: Avro file size is too big

2012-07-05 Thread Ruslan Al-Fakikh
"name" : "Impressions", "type" : [ "null", "int" ], "doc" : "" }, { "name" : "Clicks", "type" : [ "null", "int" ], "doc" : "&quo

Re: Avro file size is too big

2012-07-05 Thread Ruslan Al-Fakikh
sample files of each and/or code that will reproduce > this? > > Thanks, > > Doug > > On Wed, Jul 4, 2012 at 6:32 AM, Ruslan Al-Fakikh wrote: >> Hello, >> >> In my organization currently we are evaluating Avro as a format. Our >> concern is file size.

Re: Avro file size is too big

2012-07-05 Thread Ruslan Al-Fakikh
gt; > On Jul 4, 2012, at 6:33 AM, Ruslan Al-Fakikh wrote: > >> Hello, >> >> In my organization currently we are evaluating Avro as a format. Our >> concern is file size. I've done some comparisons of a piece of our >> data. >> Say we have sequence fil

Avro file size is too big

2012-07-04 Thread Ruslan Al-Fakikh
Hello, In my organization currently we are evaluating Avro as a format. Our concern is file size. I've done some comparisons of a piece of our data. Say we have sequence files, compressed. The payload (values) are just lines. As far as I know we use line number as keys and we use the default codec