RE: Question about the behavior of HDFS.

Where ever you upload, it upload evenly to all machines. Namenode will not have data but has only the metadata From: ext bit1...@163.com [mailto:bit1...@163.com] Sent: Friday, December 19, 2014 9:19 AM To: user Subject: Question about the behavior of HDFS. Hi Hadoopers， I got a question about

OEV - NameNode crash Edits file for 1.0.3

Hi My NameNode is crashed. It gave a NullPointerException at FsDirectory.addChild 1) My Hadoop version 1.0.3 2) I patched a code for identifying the files throwing NullPointerException - I found those files. 3) Now I want to remove those file operations in the edits file

Tez part of YARN?

Hi I read many articles/diagrams as like Tez bundled with Hadoop 2.0 by default. Snipped from a article: Hadoop 2 has a specialized AM for MapReduce and another more generalized application framework called Tez that allows generic directed-acyclic-graphs (DAGs) of execution. My

High performance Count Distinct - NO Error

Hi I am looking for high performance count distinct solution on Hive Query. Regular count distinct is very slow but if I use probabilistic count distinct has more error percentage (if the number of records are small). Is there is any solution to have exact count distinct but using low memory

Hadoop Realtime Queries

Hi I want to perform realtime query on HDFS data. I tried hadoop/yarnt/hive, shark on spark, Tez, etc., But still I couldn't get subsecond performance on the large data that I have. I understand hadoop is not meant for this, but still want to achieve as max as possible 1) How can we

RE: Hadoop Realtime Queries

architecture design so that you could use HBase. Regards, Deepak From: Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:prabakaran.1.natara...@nsn.commailto:prabakaran.1.natara...@nsn.com] Sent: Thursday, July 31, 2014 3:32 AM To: user@hadoop.apache.orgmailto:user@hadoop.apache.org Subject

Hadoop and Hive Performance Tuning

Hi I am using hive queries on structured RC file. Can you please let me know, the key performance parameters that I have tune for better query performance (for Hadoop 2.3/ Yarn and Hive 0.13). Thanks and Regards Prabakaran.N aka NP nsn, Bangalore When I is replaced by We - even Illness

RE: Hadoop Realtime Queries

, 2014 at 6:20 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) prabakaran.1.natara...@nsn.commailto:prabakaran.1.natara...@nsn.com wrote: Hi, Thank you all for the reply. I want quick response for SQL queries . Thanks and Regards Prabakaran.N From: ext Bertrand Dechoux [mailto:decho

Multiple Part files

Hi After Map Reduce job, we are seeing multiple small part files in the output directory. We are using RC file format (snappy codec) 1) Do each part file will take 64MB block size? 2) How to merge these multiple RC format part files into one RC file? 3) What is the pros-cons of

RE: Hadoop with SAN

Hi I had a same question ☺ Yes, performance impact will be there. But we can reduce the impact. We have to create a RaidGroup with type RAID-0 and represent this RaidGroup with only one LUN (don’t share this RaidGroup to any other LUN). This is an alternative for JBOD in SAN storage. You

Hadoop SAN Storage reuse

Hi I know SAN storage is not recommended for Hadoop.But we don't want waste - already existing SAN Storage. How can we make use of SAN Storage for Hadoop - what are the best methods, can a Ethernet upgrade helps,...? Thanks Prabakaran.N

HDFS Quota Error

Hi When I run a query in Hive, I get below exception. I noticed the error No space left on device. Then I did hadoop fs -count -q /var/local/hadoop - which gave below output none infnone inf 69 275 288034318

RE: HDFS Quota Error

/cache/mapred/local On 22/05/14 09:04, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) wrote: Hi When I run a query in Hive, I get below exception. I noticed the error No space left on device. Then I did hadoop fs -count -q /var/local/hadoop - which gave below output none inf

RE: HDFS Quota Error

:34 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) prabakaran.1.natara...@nsn.commailto:prabakaran.1.natara...@nsn.com wrote: Hi When I run a query in Hive, I get below exception. I noticed the error “No space left on device”. Then I did “hadoop fs -count -q /var/local/hadoop” – which gave

RE: HDFS Quota Error

Just noted that inode is 100%. Any better solutions to solve this? Thanks and Regards Prabakaran.N aka NP nsn, Bangalore When I is replaced by We - even Illness becomes Wellness From: ext Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:prabakaran.1.natara...@nsn.com] Sent: Thursday

RE: HDFS Quota Error

. On Thu, May 22, 2014 at 2:54 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) prabakaran.1.natara...@nsn.commailto:prabakaran.1.natara...@nsn.com wrote: Just noted that inode is 100%. Any better solutions to solve this? Thanks and Regards Prabakaran.N aka NP nsn, Bangalore When I is replaced