Pig tuples have field order. Swap the order of the fields in your avro schema
and try again.
On Nov 16, 2013, at 6:19 PM, Ruslan Al-Fakikh metarus...@gmail.com wrote:
Hey guys,
When I store with AvroStorage, the names from Pig tuple fields are completely
ignored. The field values are
?
Thanks,
Ruslan Al-Fakikh
On Sun, Nov 17, 2013 at 6:53 AM, Russell Jurney
russell.jur...@gmail.comjavascript:_e({}, 'cvml',
'russell.jur...@gmail.com');
wrote:
Pig tuples have field order. Swap the order of the fields in your avro
schema and try again.
On Nov 16, 2013, at 6:19 PM
some
schema resolution as shown here:
http://avro.apache.org/docs/1.7.5/spec.html#Schema+Resolution
Thanks
On Sun, Nov 17, 2013 at 7:17 AM, Russell Jurney russell.jur...@gmail.com
wrote:
How can pig map from a to nonsence_name?
On Saturday, November 16, 2013, Ruslan Al-Fakikh wrote
. Is this a valid use case?
--
Sean
--
Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
be deleted, is there any code that will merge row sets
to get
rid of the unused space?
--
Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
?
Is it possible to update or delete records?
If records can be deleted, is there any code that will merge row sets to
get
rid of the unused space?
--
Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
https://issues.apache.org/jira/browse/PIG-3059
Russell Jurney http://datasyndrome.com
to a Lucene Index */
store input into 'input.lucene' using
LuceneIndexStorage('com.example.MyPigLuceneIndexOutputFormat');
There are also drivers for most NoSQLish databases...
Russell Jurney http://datasyndrome.com
On Dec 20, 2012, at 9:33 AM, Terry Healy the...@bnl.gov wrote:
I'm just getting
Fwiw, I do this in web apps all the time via the python avro lib and json.dumps
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
On Sep 18, 2012, at 12:38 PM, Doug Cutting cutt...@apache.org wrote:
On Tue, Sep 18, 2012 at 11:34 AM, Markus Strickler mar
This thread looks useful. Are you flushing too often?
http://apache-avro.679487.n3.nabble.com/avro-compression-using-snappy-and-deflate-td3870167.html
Russell Jurney http://datasyndrome.com
On Jul 4, 2012, at 6:33 AM, Ruslan Al-Fakikh metarus...@gmail.com wrote:
Hello,
In my organization
Consider Pig and AvroStorage.
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
On May 13, 2012, at 4:49 AM, Jacob Metcalf jacob_metc...@hotmail.com
wrote:
I have just spent several frustrating hours on getting an example MR job
using Avro working with Hadoop
:
}
]
}
On Tue, Apr 10, 2012 at 2:36 AM, Russell Jurney russell.jur...@gmail.com
wrote:
H unable to get this to work:
{
namespace: agile.data.avro,
name: Email,
type: record,
fields: [
{name:message_id, type: [string, null]},
{name:froms,type
Thanks Scott, looking at the raw data it seems to have been a truncated record
due to UTF problems.
Russell Jurney http://datasyndrome.com
On Mar 23, 2012, at 7:59 PM, Scott Carey scottca...@apache.org wrote:
It appears to be reading a union index and failing in there somehow. If it
did
Ok, now I have a followup question...
how does one recover from an exception writing an Avro? The incomplete
record is being written, which is crashing the reader.
On Fri, Mar 23, 2012 at 8:01 PM, Russell Jurney russell.jur...@gmail.comwrote:
Thanks Scott, looking at the raw data it seems
FWIW, there are avro libs for JavaScript and node on github.
Russell Jurney http://datasyndrome.com
On Feb 15, 2012, at 7:32 AM, Frank Grimes frankgrime...@gmail.com wrote:
Hi All,
Is there any way to send Avro data over HTTP encoded in JSON?
We want to integrate with Node.js and JSON
, Russell Jurney wrote:
I have the same bug. I read the code... there is no obvious fix. Arg.
On Feb 2, 2012, at 10:07 PM, Something Somethingmailinglists19@**
gmail.com mailinglist...@gmail.com wrote:
In my Pig script I have something like this...
%default MY_SCHEMA '/user/xyz/my
btw - the weird thing is... I've read the code. There isn't a filter for
.avro in there. Does Hadoop, or Avro itself (not that I can see it is
involved) do so?
On Fri, Feb 3, 2012 at 10:55 AM, Russell Jurney russell.jur...@gmail.comwrote:
Hmmm I applied it, but I still can't open files
/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg/avro/io.py,
line 233, in read_utf8
return unicode(self.read_bytes(), utf-8)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa0 in position 543:
invalid start byte
On Thu, Feb 2, 2012 at 2:06 PM, Russell Jurney russell.jur
A little bit more searching shows this:
http://www.harshj.com/2010/04/25/writing-and-reading-avro-data-files-using-python/
On Thu, Feb 2, 2012 at 2:48 PM, Russell Jurney russell.jur...@gmail.comwrote:
The jars being used are:
REGISTER /me/pig/build/ivy/lib/Pig/avro-1.5.3.jar
REGISTER /me
Further examination shows that the problematic emails I am encoding are
formatted in ISO-8859-1, not UTF-8. That is why I am getting character
problems. Looks like it is not an Avro problem after all. Thanks! :)
On Thu, Feb 2, 2012 at 2:49 PM, Russell Jurney russell.jur...@gmail.comwrote
:53 PM, Russell Jurney russell.jur...@gmail.comwrote:
Further examination shows that the problematic emails I am encoding are
formatted in ISO-8859-1, not UTF-8. That is why I am getting character
problems. Looks like it is not an Avro problem after all. Thanks! :)
On Thu, Feb 2, 2012 at 2
Cleaned up my environment by unsetting HADOOP_HOME, and removing some old
jacksons in my CLASSPATH and Pig's AvroStorage works again.
Woot!
On Thu, Feb 2, 2012 at 3:47 PM, Russell Jurney russell.jur...@gmail.comwrote:
Spoken too soon... this happens no matter what avros I load now. I can't
https://issues.apache.org/jira/browse/AVRO-981
I took Joe Crobak's advice and removed snappy as a dependency in the python
client for avro. With the patch in AVRO-981 applied, Avro installs, builds
and functions on Mac OS X.
--
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
Avro 1.53 doesn't have this issue? Does it use python-snappy?
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
On Dec 23, 2011, at 7:05 PM, Ken Krugler kkrugler_li...@transpac.com
wrote:
I installed brew, then ran 'brew install snappy', which worked.
But 'sudo
but not used
error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed
with exit status 1
--
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
errors, but once I installed
snappy (I use homebrew, so 'brew install snappy') python-snappy installs
fine.
HTH,
Joe
On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney
russell.jur...@gmail.comwrote:
I am unable to build Avro for Python on OS X 10.6.8 because python-snappy
fails to build
26 matches
Mail list logo
