[
https://issues.apache.org/jira/browse/PIG-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821205#comment-15821205
]
Adam Szita commented on PIG-4728:
---------------------------------
[~rohini], I agree on pulling hbase-procedure for testing only. New patch
[^PIG-4728-3.patch] attached with this change and I also fixed the
copyDependencies targets in build.xml to have proper hbase jars and other
required dependencies copied to lib dir.
[[email protected]], I've run the following simple Pig-HBase scenario on a
cluster (with Pig having [^PIG-4728-3.patch]):
Create HBase table, write it with Pig, check content from HBase shell and
finally load data back to Pig as well:
{code}
hbase(main):000:0>create 'ccys', 'ccy', 'country', 'rate'
{code}
{code}
grunt> cat ccymin.csv
52000 EUR HUN 100.999
0 GBP HUN 100.3713375
1 NOK HUN 102.9676184
2 NOK HUN 104.6205186
3 NOK HUN 102.4529758
4 HUF HUN 104.8512737
5 JPY HUN 101.3875097
6 USD HUN 101.5471822
7 DKK USA 102.922554
8 EUR HUN 103.3619292
grunt> A = LOAD 'ccymin.csv' using PigStorage() as (id: long, ccy: chararray,
country: chararray, rate: float);
grunt> B = FOREACH A GENERATE ccy, country, rate;
grunt> STORE B INTO 'hbase://ccys' USING
org.apache.pig.backend.hadoop.hbase.HBaseStorage('ccy:ccy country:country
rate:rate');
{code}
{code}
hbase(main):001:0> t = get_table 'ccys'
hbase(main):002:0> t.get 'HUF'
COLUMN CELL
ccy:ccy timestamp=1484130902361, value=HUN
country:country timestamp=1484130902361, value=104.85127
2 row(s) in 0.0670 seconds
{code}
{code}
grunt> C = LOAD 'hbase://ccys' USING
org.apache.pig.backend.hadoop.hbase.HBaseStorage('ccy:* country:*', '-loadKey
true');
(DKK,[ccy#USA],[country#102.922554])
(EUR,[ccy#HUN],[country#103.36193])
(GBP,[ccy#HUN],[country#100.37134])
(HUF,[ccy#HUN],[country#104.85127])
(JPY,[ccy#HUN],[country#101.38751])
(NOK,[ccy#HUN],[country#102.45297])
(USD,[ccy#HUN],[country#101.54718])
{code}
> Compilation against hbase 1.x fails with hbase-hadoop1-compat not found
> -----------------------------------------------------------------------
>
> Key: PIG-4728
> URL: https://issues.apache.org/jira/browse/PIG-4728
> Project: Pig
> Issue Type: Improvement
> Reporter: Ted Yu
> Assignee: Adam Szita
> Fix For: 0.17.0
>
> Attachments: PIG-4728-1.patch, PIG-4728-2.patch, PIG-4728-3.patch
>
>
> With the following change:
> {code}
> diff --git a/ivy/libraries.properties b/ivy/libraries.properties
> index c40befd..41ce9fb 100644
> --- a/ivy/libraries.properties
> +++ b/ivy/libraries.properties
> @@ -46,7 +46,7 @@ hadoop-common.version=2.6.0
> hadoop-hdfs.version=2.6.0
> hadoop-mapreduce.version=2.6.0
> hbase94.version=0.94.1
> -hbase95.version=0.98.12-${hbase.hadoop.version}
> +hbase95.version=1.1.2
> hsqldb.version=1.8.0.10
> hive.version=1.2.1
> httpcomponents.version=4.1
> {code}
> I ran 'ant compile'
> However, compilation failed with:
> {code}
> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:resolve] :: org.apache.hbase#hbase-hadoop1-compat;1.1.2:
> not found
> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
> {code}
> In hbase 1.x releases, hbase-hadoop1-compat module doesn't exist.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
