Re: Assigning resources to individual MR jobs of a Pig script

gt; > This is blocked by https://issues.apache.org/jira/browse/PIG-2597 and > assigned to me. > > I need to find time to get to this. Sorry! > Koji > > > > On Sunday, March 19, 2017, 10:35:55 AM EDT, Mohammad Tariq < > donta...@gmail.com> wrote:Yeah, I'm aware

Re: Assigning resources to individual MR jobs of a Pig script

rd, > Jeff Zhang > > > > > > On 3/18/17, 6:03 PM, "Mohammad Tariq" <donta...@gmail.com> wrote: > > >Hi Jeff, > > > >Thank you for the prompt response. However, I can't use Tez because of > >some > >reasons. > > > &

Re: Assigning resources to individual MR jobs of a Pig script

#enable-tez > > > > > Best Regard, > Jeff Zhang > > > > > > On 3/17/17, 3:24 AM, "Mohammad Tariq" <donta...@gmail.com> wrote: > > >Hi group, > > > >In any real world pig script we end up with multiple MR jobs(well, most of

Assigning resources to individual MR jobs of a Pig script

Hi group, In any real world pig script we end up with multiple MR jobs(well, most of the times). I was wondering if it's possible to allocate resources to individual MR jobs rather than assigning them at the script level itself. Tried looking at multiple places. Would really appreciate some

Re: Error in Pig Exection

Hi Kiran, Does hdfs://localhost:9000/user/hduser/pig/planet.osm exist on your HDFS? [image: http://] Tariq, Mohammad about.me/mti [image: http://] On Wed, Jun 22, 2016 at 1:08 PM, Kirandeep Kaur wrote: > Dear Sir > > I am working on pig

Re: store to defined filename

Hi there, You could do that with the help of MultipleOutputFormathttp://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/MultipleOutputFormat.htmlclass. It extends FileOutputFormat,and allows us to write the output data to different output files. *Warm regards,* *Mohammad Tariq

Re: Bulk load in hbase using pig

Could you please let us know how exactly you want to parse your logs? Warm Regards, Tariq cloudfront.blogspot.com On Wed, Feb 26, 2014 at 6:25 PM, David McNelis dmcne...@gmail.com wrote: The big question is how the log file needs to be parsed / formatting. I'd be inclined to write a UDF

Re: Reading Kafka directly from Pig?

Great job. +1 Warm Regards, Tariq cloudfront.blogspot.com On Wed, Aug 7, 2013 at 8:27 PM, Russell Jurney russell.jur...@gmail.comwrote: Cool stuff, a Pig Kafka UDF. Russell Jurney http://datasyndrome.com Begin forwarded message: *From:* David Arthur mum...@gmail.com *Date:* August 7,

Re: union

Hello Keren, There is nothing wrong in this. One dataset in Hadoop is usually one folder and not one file. Pig is doing what it is supposed to do and performing a union on both the files. You would have seen the content of both the files together while doing dump C. Since this is a map only job,

Re: union

Regards, Tariq cloudfront.blogspot.com On Thu, Jul 25, 2013 at 12:52 PM, Mohammad Tariq donta...@gmail.com wrote: Hello Keren, There is nothing wrong in this. One dataset in Hadoop is usually one folder and not one file. Pig is doing what it is supposed to do and performing a union on both

Pig giving priority to non Apache Hadoop

Hello list, Today I started Pig on my personal machine after a few weeks to give 0.11.1 a try. As soon as I issued bin/pig it threw this message on my terminal : apache@hadoop:/hadoop/projects/pig-0.11.1$ bin/pig 2013-06-26 06:05:45,121 [main] INFO org.apache.pig.Main - Apache Pig

Re: BinCond

Hello Soniya, It's like the ternary or the conditional operator available in java and works just like that. Here is the example provided in the reference manual : *Suppose we have relation A.* A = LOAD 'data' AS (f1:int, f2:int, B:bag{T:tuple(t1:int,t2:int)}); DUMP A;

Re: Coding standards of Pig

Hello Raj, You might find these links useful : http://wiki.apache.org/pig/PigPerformance http://wiki.apache.org/pig/PigPerformance And I don't have much idea as far as coding conventions are concerned as I haven't seen much on that. I had come across this small section on the reference manual

Re: Sequence File processing

+1 Best Regards, Tariq +91-9741563634 https://mtariq.jux.com/ On Tue, Dec 25, 2012 at 3:07 AM, Cheolsoo Park cheol...@cloudera.comwrote: Hi Srini, You can use STRSPLIT to split your value chararray and define schema in a FOREACH. For example, if the value consists of 3 integers (i.e.

Re: Limit number of Streaming Programs

Folks on the list need some time mate. I have specified a couple of links on the other thread of yours. Check it out and see if it helps. Best Regards, Tariq +91-9741563634 https://mtariq.jux.com/ On Tue, Dec 25, 2012 at 11:09 AM, Kshiva Kps kshiva...@gmail.com wrote: Hi, Is there any PIG

Re: Is Programming Pig book outdated?

Agree with Mr. Jagat. Regards, Mohammad Tariq On Fri, Nov 16, 2012 at 3:26 PM, Jagat Singh jagatsi...@gmail.com wrote: In open source community no book can be ever latest , so we have to live by this :) I would suggest you to start from this book and see the latest documentation

Re: accessing like array

load the data into a relation and use 'generate' to take only the required fields from this relation and put into some other relation..then store the 2nd relation into some file. Regards, Mohammad Tariq On Tue, Nov 6, 2012 at 7:43 PM, jamal sasha jamalsha...@gmail.com wrote: Hi, I

Unable to store data into HBase

in relation 'z'. Any help would be much appreciated. Many thanks. Regards, Mohammad Tariq

Re: Unable to store data into HBase

I don't think there is any problem with that as I am able to execute other queries, like loading data from an HBase table and storing it into another HBase table. Regards, Mohammad Tariq On Mon, Sep 3, 2012 at 1:57 PM, shashwat shriparv dwivedishash...@gmail.com wrote: What can conclude

Re: Unable to store data into HBase

), it is working perfectly fine. It seems there is some compatibility issue between Pig(0.10.0) and HBase(0.92.1). Any comments or suggestions? Regards, Mohammad Tariq On Mon, Sep 3, 2012 at 8:07 PM, chethan chethan...@gmail.com wrote: STORE raw_data INTO ‘hbase://sample_names’ USING

Re: FileAlreadyExistsException while running pig

Hello Haitao, Each time we run a MapReduce job, the job expects the output to be non-existent. If the output path is already there then FileAlreadyExists exception is thrown. And as we know that each Pig job is eventually a MapReduce job, it also expects the same. Regards, Mohammad

Re: DATA not storing as comma-separted

Hi Yogesh, Is 'load' working fine with PigStorage()?? Try to load something using PigStorage(',') and dump it to see if that is working. Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 4:41 PM, yogesh.kuma...@wipro.com wrote: Hello Dmitriy, I have also performed the cat

Re: DATA not storing as comma-separted

I have worked with pig-0.7.0 once and it was working fine. Try to see if there is anything interesting in the log files. Also, if possible, share 2-3 lines of your file..I'll give it a try on my machine. Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 5:20 PM, yogesh.kuma...@wipro.com

Re: DATA not storing as comma-separted

Also, it would be help to go to the MapReduce web UI and having a look at the details of the job corresponding to this query. Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 5:31 PM, Mohammad Tariq donta...@gmail.com wrote: I have worked with pig-0.7.0 once and it was working fine. Try

Re: DATA not storing as comma-separted

Hello Yogesh, Also add these lines, export PIG_CLASSPATH=/HADOOP_HOME/conf export HADOOP_CONF_DIR=/HADOOP_HOME/conf, and see if it works for you. Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 6:01 PM, yogesh.kuma...@wipro.com wrote: Hi mohammad, when I try the command Pig

Re: foreach in PIG is not working.

to load the file properly, and you are good to go. Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 11:36 PM, yogesh.kuma...@wipro.com wrote: Hi all, I loaded a file to pig by command from HDFS. A=load '/HADOOP/Yogesh/demo.txt' as (name:chararray, roll:int); its get loaded and when i do

Re: foreach in PIG is not working.

try this : A=load '/HADOOP/Yogesh/demo.txt' using PigStorage (' ') as (name:chararray, roll:int) Regards, Mohammad Tariq On Wed, Jul 25, 2012 at 11:47 PM, pablomar pablo.daniel.marti...@gmail.com wrote: are the commas in your file in the write places ? your DUMP A shows (Yogesh 12

Re: foreach in PIG is not working.

have shown in the previous mail. A=load '/HADOOP/Yogesh/demo.txt' using PigStorage (' ') as (name:chararray, roll:int) Pay attention to PigStorage (' ')..I haven't used comma there. Use it..It should work. Regards, Mohammad Tariq On Thu, Jul 26, 2012 at 12:01 AM, yogesh.kuma...@wipro.com

Re: foreach in PIG is not working.

; grunt dump b; Regards, Mohammad Tariq On Thu, Jul 26, 2012 at 12:06 AM, Mohammad Tariq donta...@gmail.com wrote: Don't use (',')...By default, LOAD looks for our data in a tab-delimited file using the default LOAD function i.e PigStorage. But the data in your file is space separated

Re: Is there a loader that loads a file as a line?

Hello Jonathan, Have a look at Hadoop's WholeFileInputFormat..Might fit into your requirements. Regards,     Mohammad Tariq On Fri, Jun 22, 2012 at 3:39 AM, Prashant Kommireddi prash1...@gmail.com wrote: I think you will need to implement a RecordReader/InputFormat of your own

Re: How pig get hadoop and hbase configuration?

Hello, Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder to hbase/lib folder...Some times due to incompatible jars this may happen..do it and see if it works for you. Regards,     Mohammad Tariq On Wed

Re: Re: How pig get hadoop and hbase configuration?

HBase is able to connect to ZooKeeper but the connection closes immediately. - This error means that your HMaster is not able to talk to your Namenode. Regards,     Mohammad Tariq On Wed, Jun 13, 2012 at 1:12 PM, lulynn_2008 lulynn_2...@163.com wrote: Hello, hadoop-core-*.jar and commons

Re: Re: How pig get hadoop and hbase configuration?

Could you send me your hadoop and hbase config files??? Regards,     Mohammad Tariq On Wed, Jun 13, 2012 at 1:18 PM, Mohammad Tariq donta...@gmail.com wrote: HBase is able to connect to ZooKeeper but the connection closes immediately. - This error means that your HMaster is not able to talk

Re: How to use TOP?

??Many thanks. Regards,     Mohammad Tariq On Tue, May 22, 2012 at 1:16 AM, Abhinav Neelam abhinavroc...@gmail.com wrote: Hey Mohammad, You need to have sorting requirements when you say 'top 5' records. Because relations/bags in Pig are unordered, it's natural to ask: 'top 5 by what parameter

Re: How to use TOP?

Yes, it would be better if I do it at the time of insertion.Just have to add one more column.Thanks again. Regards,     Mohammad Tariq On Tue, May 22, 2012 at 2:36 PM, Abhinav Neelam abhinavroc...@gmail.com wrote: Doing it in the pig script is not feasible because pig doesn't have any notion

How to use TOP?

); } Could anyone tell me how to achieve thatMany thanks. Regards,     Mohammad Tariq

Re: How to use TOP?

Hi Ruslan, Thanks for the response.I think I have made a mistake.Actually I just want the top 5 records each time.I don't have any sorting requirements. Regards,     Mohammad Tariq On Mon, May 21, 2012 at 9:31 PM, Ruslan Al-fakikh ruslan.al-fak...@jalent.ru wrote: Hey Mohammad, Here c

how to achieve foreach 'n'

��� �\55.55 2001.5 33.03 656.56 2.8260.76 � ��ȫ ��� �P156.14 200235.52 83.75 2.6 59.57 � ��ȫ ��� �D138.77 2002.5 21.51 105.76 2.6285.89 � ��ȫ ��� �871.89 200327.79 709.01 2.6385.44 Regards,     Mohammad Tariq

Re: how to achieve foreach 'n'

to do this.I am new to pig so finding it a bit tricky.Many thanks. Regards,     Mohammad Tariq On Thu, May 17, 2012 at 1:44 PM, Mohammad Tariq donta...@gmail.com wrote: Hello list,    I have loaded data from an Hbase table and the relation looks like this - 18.98   2000    1.21    193.46