how to write custom log loader and store in JSON format

Hi, I am new to pig and I have a log file in below format (Message,NIL,2015-07-01,22:58:53.66,E,xx.xxx.x.xxx,12,0xd6,BIZ,Componentname,0,0.0,key_1=valueKEY_2=KEY_3=VALUEKEY_4=AUKEY_5=COMPANYKEY_6=VALUEKEY_7=1222KEY_8=VALUEKEY_9=VALUEKEY_10=VALUEKEY_10=VALUE) for which I need

How to renew Kerberos ticket to run balancer for long time?

Hi, all Recently I have set up Kerberos security for a Hadoop cluster and added a few data nodes to it. While running hdfs balancer, I found that Kerberos ticket is expired and balancer stop. The Kerberos ticket has 1day lifetime with 7days max renewable lifetime. Are there any options to

Problem when configure the security in hadoop

Hi, I do the security configuration for Hadoop these days, the Kerberos works fine, but there maybe has some problems on sasl configuration. The following is the related configuration in hdfs-site.xml: property namedfs.http.policy/name valueHTTPS_ONLY/value

VCores used is greater than Total VCores

Hello, While running lot of Jobs on a YARN clusters, I noticed the following which looked little unusual to me: [image: Inline image 1] VCored Used VCores Total The hadoop Version used here is: 2.6.0.2.2.0.0-2041 Is it bug in YARN (scheduler/UI) ? Cheers, Subroto Sanyal