[
https://issues.apache.org/jira/browse/PIG-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108491#comment-13108491
]
Damien Hardy commented on PIG-2289:
-----------------------------------
I get it working with :
DEFINE iplookup `wrapper.sh GeoIP`
ship ('wrapper.sh')
cache('/GeoIP/GeoIPcity.dat#GeoIP')
input (stdin using PigStreaming(','))
output (stdout using PigStreaming(','));
You can close this bug.
Thank you
> HBaseStorage do not care about delimiter in STORE
> -------------------------------------------------
>
> Key: PIG-2289
> URL: https://issues.apache.org/jira/browse/PIG-2289
> Project: Pig
> Issue Type: Bug
> Components: internal-udfs
> Affects Versions: 0.9.1, 0.10
> Environment: Hadoop, Hbase, zookeeper from cdh3u1
> Pig from github (version 0.9.1 then trunk:0.10)
> Reporter: Damien Hardy
>
> I want to store in Hbase a set of tupple generated by pig streaming (inspired
> by http://www.cloudera.com/blog/2009/06/analyzing-apache-logs-with-pig/ )
> Here is my script :
> set debug 'off'
> DEFINE iplookup `wrapper.sh GeoIP`
> ship ('wrapper.sh')
> cache('/GeoIP/GeoIPcity.dat#GeoIP');
> A = load 'log' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('default:body','-gt=_f:squid_t:201109161405
> -lte=_f:squid_t:201109161410 -loadKey') AS (rowkey, data);
> B = LIMIT A 10;
> C = FOREACH B {
> t =
> REGEX_EXTRACT(data,'([0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}):([0-9]+)
> ',1);
> generate rowkey, t;
> }
> D = STREAM C THROUGH iplookup AS (rowkey, ip, country_code, country, state,
> city);
> DESCRIBE D;
> -- DUMP D;
> STORE D INTO 'geoip_pig' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('location:ip
> location:country_code location:country location:state location:city') ;
> The "DESCRIBE D;" show :
> D: {rowkey: bytearray,ip: bytearray,country_code: bytearray,country:
> bytearray,state: bytearray,city: bytearray}
> as expected
> Store juste get the rowkey and put the rest of the tuple in the first column
> (location:ip) as you can see :
> hbase(main):033:0> get 'geoip_pig',
> "_f:squid_t:20110916140500_b:squid_s:200-1VPVjbVwywTpNtLA4mHl+A=="
> COLUMN CELL
>
>
> location:city timestamp=1316180980265, value=
> location:country timestamp=1316180980265, value=
> location:country_code timestamp=1316180980265, value=
> location:ip timestamp=1316180980265,
> value=90.9.213.170,FR,France,A9,Llupia
> location:state timestamp=1316180980265, value=
> 5 row(s) in 0.0150 seconds
> I tried also with option '-delim=,' without more effect.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
