Re: ANSWER PLEASE

ed. > > C. When Job A gets submitted, it doesn't consumes all the task slots. > > D. When Job A gets submitted, it consumes all the task slots. > Kai Voigt Am Germaniahafen 1 k...@123.org

Re:

) at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111) at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235) Thanks Jay Kai Voigt Am Germaniahafen 1 k...@123.org

Re: which file map and reduce in reading or writing?

xeonmailingl...@gmail.com: Hi, I am looking to YARN MapReduce internals, and I would like know if it possible to know which file a map/reduce function is reading or writing from inside a map or reduce function defined by the user, or simply by the client? Thanks, -- -- Kai

Re: Combiner doesn't take effect if there is no reducer phrase

understand here because the combiner is running on the Mapper node, so could some one please explain the behavior of Combiner?Thanks. Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel

Re: How to sort in a WordCount

this ? Thakns. laozh...@sina.cn Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel +49 160 96683050 Germany

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

? Cloudera vs Hortonworks vs Biginsights? Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel +49 160 96683050 Germany

Re: Map reduce Query

we take input in a mapper function and in some way pass it to reducer ? Regards, Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel +49 160 96683050

Re: Counters in MapReduce

Center for Cyber Security | Amrita Vishwa Vidyapeetham http://www.unmeshasreeveni.blogspot.in/ Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel +49 160 96683050

Re: Object in mapreduce

in mapper. Static is not working in mapreduce code -- Thanks Regards Unmesha Sreeveni U.B Junior Developer http://www.unmeshasreeveni.blogspot.in/ Kai Voigt Am Germaniahafen 1 k...@123.org 24143 Kiel

Re: Relationship between heap sizes and mapred.child.java.opt configuration

Chih-Hsien Wu chjaso...@gmail.com: I'm learning about Hadoop configuration. What is the connection between the datanode/ tasktracker heap sizes and the mapre.child.java.opts? Does one have to be exceeded to another? Kai Voigt Am Germaniahafen 1 k

Re: ALL HDFS Blocks on the Same Machine if Replication factor = 1

in another rack (if your cluster is configured to be rack-aware), to increase the distribution of the data. The third copy of the block gets stored onto a random host in that other rack. So your observations are correct. Kai -- Kai Voigt k...@123.org

Re: Not saving any output

in a db. I am using python (hadoop streaming). I am using mongo as backend server. Is it possible to run hadoop streaming jobs without specifying any output? What is the best way to deal with this. -- Kai Voigt k...@123.org

Re: Shuffle phase replication factor

, I am assuming that it writes intermediate data to HDFS. What replication factor is used for those temporary files? john -- Kai Voigt k...@123.org

Re: Can any one give Cloudera Manager Detailed Explanation link

on cloudera's web site. Kai Am 16.05.2013 um 14:09 schrieb Sathish Kumar sa848...@gmail.com: Hi All, We would like to know how the Cloudera Manager works (server and agent, db, etc..). Can anyone help me the web link or video link. Regards Sathish -- Kai Voigt k...@123.org

Re: Bloom Filter analogy in SQL

has been issued and approved by Sporting Index Ltd. Outbound email has been scanned for viruses and SPAM -- Kai Voigt k...@123.org

Re: 100K Maps scenario

-- Kai Voigt k...@123.org

Re: Reduce starts before map completes (at 23%)

light. Thanks Sai -- Kai Voigt k...@123.org

Re:

...@students.iitmandi.ac.in: Unsubscribe me How many more times, I have to mail u -- Kai Voigt k...@123.org

Re: How to take Whole Database From RDBMS to HDFS Instead of Table/Table

by Table ?. How do you guys did it? Is there some trick? Regards, samir. -- Kai Voigt k...@123.org

Re: aggregation by time window

Idea? What will be a key in a map phase? Suppose we have event at 10:07. How would you emit this to the multiple buckets? Thanks Oleg. On Mon, Jan 28, 2013 at 3:17 PM, Kai Voigt k...@123.org wrote: Quick idea: since each of your events will go into several buckets, you could use

Re: distributed cache

a copy to its local disk. So it won't need to do this again for future tasks on this node. After a job is done, all local copies and the HDFS copies of files in the distributed cache are cleaned up. Kai -- Kai Voigt k...@123.org

Re: a question on NameNode

. Kai -- Kai Voigt k...@123.org

Re: a question on NameNode

), the JobTracker will launch the task in parallel on another node. The node finishing first will be used to get the task's output. The slow task will be killed. Kai -- Kai Voigt k...@123.org

Re: Unit tests for Map and Reduce functions.

, and reducer Reduce function ? - Ravi -- Kai Voigt k...@123.org

Re: What's the basic idea of pseudo-distributed Hadoop ?

and reduce tasks in parallel (mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum), so you will actually see some concurrency on your one machine. Kai -- Kai Voigt k...@123.org

Re: Transfer large file 50Gb with DistCp from s3 to cluster

-tp34389118p34389118.html Sent from the Hadoop core-user mailing list archive at Nabble.com. -- Kai Voigt k...@123.org

Re: Hadoop or HBase

privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited. -- Kai Voigt k...@123.org

Re: Hadoop or HBase

difficult to take backup because it takes 1.5 days to take backup of data every time. Instead of that if we uses distributed file system we need not to do that. Thanks Regards, Kushal Agrawal kushalagra...@teledna.com -Original Message- From: Kai Voigt [mailto:k...@123.org

Re: Simple hadoop processes/testing on windows machine

to avoid additional hadoop windows registry or env params? - -- Kai Voigt k...@123.org

Re: Counting records

that is probably silly, but I still don't know the answer. To simplify (a fair bit) I want to count all the records that meet specific criteria. I would like to use MapReduce because I anticipate large sources and I want to get the performance and reliability that MapReduce offers. -- Kai Voigt k

Re: HDFS APPEND

Hello, Am 13.07.2012 um 22:19 schrieb abhiTowson cal: Does CDH4 have append option?? Yes!! Check out http://archive.cloudera.com/cdh4/cdh/4/hadoop/api/org/apache/hadoop/fs/FileSystem.html#append(org.apache.hadoop.fs.Path) Kai -- Kai Voigt k...@123.org

Re: Binary Files With No Record Begin and End

note that my files are not sequence file and just a custom binary file. -- Kai Voigt k...@123.org

Re: file checksum

this? tia -- --- Get your facts first, then you can distort them as you please.-- -- Kai Voigt k...@123.org

Re: map task execution time

and stop timestamps for every individual task. Since the JobTracker Java API is exposed, you can write your own application to fetch that data through your own code. Also, hadoop job on the command line can be used to read job statistics. Kai -- Kai Voigt k...@123.org

Re: How does Hadoop compile the program written in language other than Java ?

compile the program written in language other than Java. Thank you ! -- Lac Trung -- Kai Voigt k...@123.org

Re: How does Hadoop compile the program written in language other than Java ?

haven't understood how Hadoop run a mapreduce program in other language :D Vào 02:21 Ngày 04 tháng 3 năm 2012, Kai Voigt k...@123.org đã viết: Hi, the streaming API doesn't compile the streaming scripts. The PHP/Perl/Python/Ruby scripts you create as mapper and reducer will be called

Re: dfs.block.size

and set it when writing to the sequence file or does it need to be cluster wide setting in .xml files? Also, is there a way to check the block of a given file? -- Joseph Echeverria Cloudera, Inc. 443.305.9434 -- Kai Voigt k...@123.org

Re: Secondary namenode fsimage concept

machines contain the fsimage file . Namenode only contains the editlog file. Is it true? Thanks R.Shanmuganathan -- Kai Voigt k...@123.org

Re: Secondary namenode fsimage concept

the editlog file. Is it true? gt; gt; gt; gt; Thanks R.Shanmuganathan gt; gt; gt; gt; gt; gt; -- Kai Voigt k...@123.org -- Kai Voigt k...@123.org

Re: Secondary namenode fsimage concept

machines contain the fsimage file . Namenode only contains the editlog file. Is it true? gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; Thanks R.Shanmuganathan gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; gt; -- gt; Kai Voigt gt; k...@123.org

Re: phases of Hadoop Jobs

and reduce stage. Was this only for saving shuffling time? Or there are some other reasons. Best wishes! Chen On Mon, Sep 19, 2011 at 12:36 AM, Kai Voigt k...@123.org wrote: Hi Chen, the times when nodes running instances of the map and reduce nodes overlap. But map() and reduce

Re: Reducer to concatenate string values

the strings in order. waiting your response and thanks in advance. Regards, Daniel T. Yehdego Computational Science Program University of Texas at El Paso, UTEP dtyehd...@miners.utep.edu -- Kai Voigt k...@123.org

Re: phases of Hadoop Jobs

to? Thanks -- Nan Zhu School of Electronic, Information and Electrical Engineering,229 Shanghai Jiao Tong University 800,Dongchuan Road,Shanghai,China E-Mail: zhunans...@gmail.com -- Kai Voigt k...@123.org

Re: Hadoop with Netapp

things easier to handle, but it's a layer that will eat resources. Kai -- Kai Voigt k...@123.org

Re: Where is the hadoop-examples source code for the Sort example mapper/reducer?

/viewvc/hadoop/common/trunk/mapreduce/src/examples/org/apache/hadoop/examples/ But have not been able to track down the mapper/reducer implementation. -Sean Hogan -- Kai Voigt k...@123.org

Re: Where is the hadoop-examples source code for the Sort example mapper/reducer?

(link to the implementation of whatever sort they use and how it fits into MapReduce) Thanks. -Sean -- Kai Voigt k...@123.org

Re: Where is the hadoop-examples source code for the Sort example mapper/reducer?

multiple reducers will still be sorted. So I want to look for where the implementation of the shuffle/sort phase is located. Or find something on how Hadoop implements the MapReduce sort/shuffle phase. Thanks! -Sean -- Kai Voigt k...@123.org

Re: Question about RAID controllers and hadoop

. Department of Treasury, Office of Foreign Asset Controls (OFAC). As a recipient of this information you are obligated to comply with all applicable U.S. export laws and regulations. -- Kai Voigt k...@123.org

Re: Where is web interface in stand alone operation?

in standalone vs pseudo mode. Thank you. Cheers, A Df -- Kai Voigt k...@123.org

Re: Where is web interface in stand alone operation?

. Thank you. Cheers, A Df -- Harsh J -- Kai Voigt k...@123.org

Re: Can I number output results with a Counter?

-- Joseph Echeverria Cloudera, Inc. 443.305.9434 -- Joseph Echeverria Cloudera, Inc. 443.305.9434 -- Kai Voigt k...@123.org

Re: providing the same input to more than one Map task

, and that will be the ith element of my result vector. Or can you suggest another way to do it? Thanks, Alexandra Anghelescu -- Kai Voigt k...@123.org

Re: Best way to Merge small XML files

? -- Regards Shuja-ur-Rehman Baig http://pk.linkedin.com/in/shujamughal -- Kai Voigt k...@123.org