Oct 17, 2011 at 9:37 PM, kiranprasad
wrote:
Hi
How can I ignore the seperator character in middle of a column value.
eg : Seperator char is ‘|’.
The Record values are | seperated
xyz|1234|98798|”xyz|abc”|
Regards
Kiran.G
Hi
How can I ignore the seperator character in middle of a column value.
eg : Seperator char is ‘|’.
The Record values are | seperated
xyz|1234|98798|”xyz|abc”|
Regards
Kiran.G
After USING MultipleStorage() the files have been generated based on the
group, now
How can I append the Headers to all the files generated ?
-Original Message-
From: kiranprasad
Sent: Thursday, October 13, 2011 3:40 PM
To: user@pig.apache.org
Subject: Re: How to store each record in
Thank you All it is Working.
-Original Message-
From: Thomas Kappler
Sent: Thursday, October 13, 2011 12:33 PM
To: user@pig.apache.org
Subject: Re: How to store each record in a seperate file
On Thu, Oct 13, 2011 at 07:56, Ayon Sinha wrote:
Hi Kiranprasad,
What is your usecase? Are
.
Kiran.G
-Original Message-
From: Ayon Sinha
Sent: Thursday, October 13, 2011 11:26 AM
To: user@pig.apache.org
Subject: Re: How to store each record in a seperate file
Hi Kiranprasad,
What is your usecase? Are you sure you have picked the right tool for the
job? Pig/Hadoop is meant for
I wanna compare 2 files.
A.txt and B.txt
cat A;
(1,2,3)
(4,2,1)
(8,3,4)
(8,3,4)
(4,2,1)
(8,3,4)
(4,2,1)
cat B.txt;
1
2
3
now I wanna compare each A.$0 == B.$0 then write the result in separate
file.
-Original Message-
From: kiranprasad
Sent: Thursday, October 13, 2011 10:49 AM
To
arguments.
http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.html
You can definitely implement your own StoreFunc UDF.
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.
From: kiranp
Hi
After grouping a data set, how do I save each group in a separate file.
ex:
A = E:/data.txt' USING PigStorage(',');
B = GROUP A BY $0;
cat data.txt;
(1,2,3)
(4,2,1)
(8,3,4)
(4,3,3)
(7,2,5)
(8,4,3)
After grouping
(1,{(1,2,3)})
(4,{(4,2,1),(4,3,3)})
(7,{(7,2,5)})
(8,{(8,3,4),(8,4,3)})
How
for what files?
http://hadoop.apache.org/common/docs/r0.17.2/hdfs_user_guide.html#Fsck
Alex
On Tue, Oct 4, 2011 at 7:53 AM, kiranprasad
wrote:
I am getting the below exception when trying to execute PIG latin script.
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
I am getting the below exception when trying to execute PIG latin script.
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_201110042009_0005 A MAP_ONLYMessage: Job failed!
hdfs://10.0.0.61/tmp/temp1751671187/tmp-592386019,
Input(s):
Failed to read data from "
Even when I try to Store directly the loaded file it is generating only 5
output files.
The size of test.txt is 1 GB where as the putput folder size is 178 MB.
A= LOAD 'data/test.txt' USING PigStorage();
STORE A INTO 'output';
-Original Message-
From: kiranpr
Out of 35 files only 5 output files are generated.
The total number of records should be 10345 but in these 5 files only 2
records are there.
What could be the reason for generating only 5 output files.
-Original Message-
From: kiranprasad
Sent: Tuesday, September 20, 2011 12:52 PM
But when compared the number of records in output should be 12,600 but there
are only 2 records in Linux VM output folder.
Regards
Kiran.G
-Original Message-
From: Thejas Nair
Sent: Wednesday, September 21, 2011 10:55 PM
To: kiranprasad
Cc: user@pig.apache.org
Subject: Re: ERROR
DDB.txt' USING PigStorage();
grunt> B= FILTER A BY ($0 matches '9948.*');
grunt> STORE B INTO 'data/output2';
I think I missed some configurations !
Regards
Kiran.G
-Original Message-
From: kiranprasad
Sent: Wednesday, September 21, 2011 4:58 PM
To: Thejas
@pig.apache.org
Cc: kiranprasad
Subject: Re: ERROR 2118: Input path does not exist
The put command that Marek described can do that.
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#put
You will need to have hadoop client on that machine or move data to a
machine that has it. Copying 10GB of data
s (if your not executing with -x
local), put it there from your local drive with command:
hadoop fs -put
for ex, create dir and the put:
hadoop fs -mkdir /data
hadoop fs -put /data/test.txt /data/
Sincerely,
Marek M.
____
From: kiranprasad [kiranprasa...@imi
torage();
BACKSLASH before data!
-Original Message-----
From: kiranprasad [mailto:kiranprasa...@imimobile.com]
Sent: Monday, September 19, 2011 12:10 PM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
Hi
I am unable t run the below mentioned command : hadoop fs -
irst level structure.
hadoop fs -lsr /
To display all levels.
Sincerely,
Marek M.
________
From: kiranprasad [kiranprasa...@imimobile.com]
Sent: Saturday, September 17, 2011 8:46 AM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
When I do
PM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
What is the result of "hadoop fs -ls
hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt"
Regards,
--
Damien
Le 16/09/2011 17:04, kiranprasad a écrit :
Hi
I am getting the below mentioned exception afte
Hi
I am getting the below mentioned exception after I load a file and do Filter on
it.
The file(test.txt) is saved inside PIG home/data/ folder.
grunt> A= LOAD 'data/test.txt' USING PigStorage();
grunt> B= FOREACH A GENERATE $0;
grunt> DUMP B;
2011-09-17 01:17:43,408 [main] INFO org.apache
Let me know where am I doing wrong.
can I create a hadoop cluster on windows machine using Cygwin.
-Original Message-
From: kiranprasad
Sent: Thursday, September 15, 2011 10:21 AM
To: user@pig.apache.org
Subject: Re: Starting Hadoop File System process getting Stuck
Hi
I ve tried
, at 4:32 AM, kiranprasad wrote:
Hi
I am trying to start the PIG in hadoop mode, but it is getting stuck. Pls
help.
Below is where the process is getting stuck.
[kiranprasad.g@pig4 pig-0.8.1]$ bin/pig
2011-09-14 21:48:25,589 [main] INFO org.apache.pig.Main - Logging error
messages to: /home
Hi
I am trying to start the PIG in hadoop mode, but it is getting stuck. Pls help.
Below is where the process is getting stuck.
[kiranprasad.g@pig4 pig-0.8.1]$ bin/pig
2011-09-14 21:48:25,589 [main] INFO org.apache.pig.Main - Logging error
messages to: /home/kiranprasad.g/pig-0.8.1/pig_131601
Hi
I am using 3 VMs for Hadoop cluster and PIG. VM 1(master) is
namenode,JobTracker with diff ports, VM2 and VM3 are slaves
So in which system I need to install PIG and configure the PIG_CLASSPATH.
Hadoop is there in all 3 VMs.
Regards
Kiran.G
Hi
Does pig-0.8.1 works with hadoop-0.20.2-cdh3u0.
Iam getting below mentioned error.
ERROR 2999: Unexpected internal error. Failed to create DataStorage
Regards
Kiran.G
Hi
Iam using
Hadoop version : hadoop-0.20.2-cdh3u0 and PIG : pig-0.8.1
For Cluster I have 3 VMs(10.0.0.61-master, 10.0.0.62,10.0.0.63 - Slaves) and
another VM 10.0.0.64 in which I ve installed PIG from which I run the PIG
commands.
I ve updated the xmls , please find the below
mapred site.x
there. Why you have
different port in different config files?
Daniel
On Thu, Sep 8, 2011 at 2:07 AM, kiranprasad
wrote:
Hi
Iam getting the below exception when I do hadoop fs -ls /
Most commands print help when invoked w/o parameters.
[kiranprasad.g@pig1 hadoop-0.20.2-cdh3u0]$ bin/hadoop fs
HADOOP_HOME and HADOOP_CONF_DIR environment variables set
correctly?
Ashutosh
On Wed, Sep 7, 2011 at 21:58, kiranprasad
wrote:
Even after updating the core-site.xml file I am getting the same
exception,
Please help.
Regards
Kiran.G
IMImobile Plot 770, Rd. 44 Jubilee Hills, Hyderabad - 500033
M
-site.xml (10.0.0.61:9000) Make them consistent and then try.
Hope it helps,
Ashutosh
On Wed, Sep 7, 2011 at 07:16, kiranprasad
wrote:
Hi
Ive checked all the files are configured.
For this Iam using 4 VMs (10.0.0.61,10.0.0.62,10.0.0.**63,10.0.0.64)
1 VM(10.0.0.61) is for namenode , 2nd VM is for
Miglinski
Sent: Wednesday, September 07, 2011 5:09 PM
To: user@pig.apache.org
Subject: RE: ERROR 2999: Unexpected internal error. Failed to create
DataStorage
Check if you have configured /etc/hadoop/conf/* files properly.
Marek M.
-Original Message-
From: kiranprasad [mailto:kiranprasa
Marek M.
-Original Message-
From: kiranprasad [mailto:kiranprasa...@imimobile.com]
Sent: Wednesday, September 07, 2011 11:55 AM
To: user@pig.apache.org
Subject: ERROR 2999: Unexpected internal error. Failed to create DataStorage
Hi
Iam new to PIG, trying to set up HADOOP cluster.
The error
Hi
Iam new to PIG, trying to set up HADOOP cluster.
The error Iam getting is
[kiranprasad.g@pig1 pig-0.8.1]$ bin/pig
2011-09-07 19:45:50,606 [main] INFO org.apache.pig.Main - Logging error
messages to: /home/kiranprasad.g/pig-0.8.1/pig_1315404950603.log
2011-09-07 19:45:50,764 [main] INFO
ile(part-m-0001) instead of
multiple files
Hi,
STORE param INTO 'output/result' USING PigStorage(',');
If your data is comma delimited.
Marek M.
-----Original Message-
From: kiranprasad [mailto:kiranprasa...@imimobile.com]
Sent: Tuesday, September 06, 2011 12:02
Hi
I am new to PIG, I would like to know how to generate only single output file
by using STORE.
Regards
Kiran.G
34 matches
Mail list logo
