We need an unblock store function，but I cant find it in pig,is there an
unblock store function which I have not found?
we use pig for a long time,but we find pig is not Robust in multithreading
when I use pig to run big data.I get this error sometimes:
unable to store alias:table_middlexxx
after I rerun the same shell,It run successfully. I search this for many
times,but cant find the solution，please give me some advices.thanks.
I want to submit pig job using hdfs or hive user
conf.set("hadoop.job.ugi", "hdfs");
conf.set("HADOOP_USER_NAME", "hdfs");
conf.set("user", "hdfs");
but,every is wrong.how can I set the user to submit pig job
I have two pig udf jar:a.jar b.jar
a.jar needs a function from b.jar
I register a.jar and b.jar
but error happen： cant find class
I want to kill pig job using java.
now ,I can get pigserver by PigServer pigServer = new
PigServer(ExecType.MAPREDUCE, conf);
and I googled ,I can set pig name by
1、pigServer.getPigContext().getProperties().setProperty(PigContext.JOB_NAME,"my
job");
2、pigServer.setJobName("my job");
but,how ca
when I exec pig using java in eclipse,I always get error.my cluster is
right,Can someone please help in solving the issue? Regards
this is log:
2015-11-19 18:48:07.175 [pool-3-thread-3] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - There
is no log file to write t
when I use group all,I find that the order reverses. is there a setting I can
use not to Reverse order.
data:
(1,a)
(2,b)
(all,{(2,b),(1,a)})
BUT I want to get
(all,{(1,a),(2,b)})
Is there a setting? thanks
table1
,aa
14,
12,bb
table2
,aa
14,
12,bb
k = cogroup table1 by (a,b),table2 by (a,b);
k:
((,aa),{(,aa)},{})
((,aa),{},{(,aa)})
((14,),{(14,)},{(14,)})
((12,bb),{(12,bb)},{(12,bb)})
when I use cogroup,(,aa) null is in front,(14,) null is not in front. is there
a parameter I can get result lik
I have data like :
(lucy,{(34,,45),(34,,45),(34,,45),(34,,45),(34,,45),(34,,45),(34,,45),(34,,45),(34,,45)})
(lili,{(12,lili,23),(12,lili,23),(12,lili,23),(12,lili,34),(12,lili,23),(12,lili,89),(12,lili,23),(12,lili,23),(12,lili,23),(12,lili,34),(12,lili,23),(12,lili,89),(12,lili,23),(12,lili,23)})
you can check this:https://issues.apache.org/jira/browse/PIG-3900
but ,it is not solved.
At 2015-05-19 11:38:44, "李运田" wrote:
>table1 = load 'mytable' using org.apache.hcatalog.pig.HCatLoader();
>table2 = sample table1 0.01;
>sto
table1 = load 'mytable' using org.apache.hcatalog.pig.HCatLoader();
table2 = sample table1 0.01;
store table2 into 'table2' using org.apache.hcatalog.pig.HCatStorer();
table3 = filter table2 by a > 10;
store table3 into 'table3' using org.apache.hcatalog.pig.HCatStorer();
I find that table3's data
I USE
table1 = load 'school' using org.apache.hcatalog.pig.HCatLoader();
table2 = sample table 0.01;
every time I dump table2 ,I get different result, Is there one sample function
with seed? so the result is not changed every time.
thank you.
I have a big web ETL system ,which use pig as ETL tool. when I start tomcat , I
also start one pigserver,and all users use the only pigserver,perhaps when
these users use this system ,there will be some unknown errors ?
ounter.name.max
mapreduce.job.counters.groups.max"
can you give me some advice?
At 2015-04-24 21:56:20, "Alex Nastetsky" wrote:
>Have you looked at the RANK function?
>https://pig.apache.org/docs/r0.11.0/basic.html#rank
>
>On Fri, Apr 24, 2015 at 5:15 AM, 李运田 wrote:
>
>>
I have a big data about 10T ,I want to add column number to every column from 1
to COUNT(my data),,I use two functions in
http://stackoverflow.com/questions/9288578/how-can-i-add-row-numbers-for-rows-in-pig-or-hive
and http://www.aiuxian.com/article/p-139530.html
but I always get many file
e.jar
>……
>
>Daniel
>
>On 3/4/15, 1:18 AM, "李运田" wrote:
>
>>https://issues.apache.org/jira/browse/PIG-2532
>>this error is the same as mine。but I dont know how to added needed jars
>>to eclipse in script.
>>can you help me ?
>>
>>
>
s that for you. But in
>eclipse, you will need to do it by yourself.
>
>Daniel
>
>On 3/3/15, 4:40 AM, "李运田" wrote:
>
>>2015-03-03 19:54:43,072 INFO [main]
>>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
>>application appattempt_1
gt;
>
>
>
>When you run in command line, bin/pig script does that for you. But in
>eclipse, you will need to do it by yourself.
>
>Daniel
>
>On 3/3/15, 4:40 AM, "李运田" wrote:
>
>>2015-03-03 19:54:43,072 INFO [main]
>>org.apache.hadoop.mapreduce.v2.
gt;
>
>
>
>When you run in command line, bin/pig script does that for you. But in
>eclipse, you will need to do it by yourself.
>
>Daniel
>
>On 3/3/15, 4:40 AM, "李运田" wrote:
>
>>2015-03-03 19:54:43,072 INFO [main]
>>org.apache.hadoop.mapreduce.v2.
java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
在 2015-02-28 09:37:29，"李运田" 写道：
pigServer.registerQuery("tmp = load 'pig' using
org.apache.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("tmp = foreach tmp generat
java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
在 2015-02-28 09:37:29，"李运田" 写道：
pigServer.registerQuery("tmp = load 'pig' using
org.apache.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("tmp = foreach tmp generat
so I think something about
Hcatalog is wrong.......
At 2015-02-28 01:21:12, "Alan Gates" wrote:
What error message are you getting?
Alan.
李运田
February 26, 2015 at 18:58
I want to use hcatalog in eclipse to deal with tables in hive.
but I cant store table into hive:
so I think something about
Hcatalog is wrong...
At 2015-02-28 01:21:12, "Alan Gates" wrote:
What error message are you getting?
Alan.
李运田
February 26, 2015 at 18:58
I want to use hcatalog in eclipse to deal with tables in hive.
but I cant store table into
I want to use hcatalog in eclipse to deal with tables in hive.
but I cant store table into hive::
pigServer.registerQuery("tmp = load 'pig' using
org.apache.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("tmp = foreach tmp generate id;");
pigServer.registerQuery("store tmp into 'hive' using
I run pig in eclipse,
pigServer.registerQuery("b = load 'didi1' using
org.apache.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("a = order b by id;");
pigServer.getExamples("a");
it run well,but when I run this:pigServer.openIterator("a");
I get error ::
org.apache.pig.impl.logicalLayer.Fro
pigServer.registerQuery("a = load '" + inputTable+ "' using
org.apache.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("store a into '' using
org.apache.hcatalog.pig.HCatStorer();");
but,I always get error like this:
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.h
grunt> a = load 'table1419836128388' USING
org.apache.hcatalog.pig.HCatLoader();
grunt> dump a;
hcatalog服务已经起来了，但是还是错误。
I have started hcatalog server，I always get error。
2014-12-30 17:42:44,761 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil
- 1 map reduce job(s) faile
my eclipse and pig are in same linux.
this is my pig configuration in eclipse:
props.setProperty("fs.defaultFS", "hdfs://10.210.90.101:8020");
props.setProperty("hadoop.job.user", "hadoop");
props.setProperty("mapreduce.framework.name", "yarn");
props.setProperty("yarn.resourceman
I use hcatalog as metadata Storage .so I can use hive to create table and
manipulate data .I can also use pig to manipulate the table which Hive create.
now, I can use
"a = load 'tableHIVE' using org.apache.hcatalog.pig.HCatLoader();"
2014-12-23 15:31:45,791 [main] INFO
org.apache.hadoop.conf.
when I input the “hcat_server.sh start” command in
/usr/lib/hive-hcatalog/sbin.I ALWAYS get the error like this. do you have the
same problem.can you help me please.谢谢
[root@pdmdev00 sbin]# hcat_server.sh start
Started metastore server init, testing if initialized correctly...
Metastore startup
hi all.
I want to use pig in eclipse.my hadoop(yarn) cluster and eclipse are in the
same linux cluster .my pig configuration in eclipse：：
Properties props = new Properties();
props.setProperty("fs.defaultFS", "hdfs://10.210.90.*:8020");
props.setProperty("hadoop.job.user", "hado
31 matches
Mail list logo
