Re: NPE in TaskLogAppender
An update incase anyone else has this problem. It looks like an issue with competing log4j configurations that could be resolved by using Logger Repositories. I've updated a thread in the brisk user group http://groups.google.com/group/brisk-users/msg/619b5ebba2fb0e89 Has anyone else had this problem ? Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 15/08/2011, at 2:04 PM, aaron morton wrote: I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local. I keep running into this problem for Hive jobs INFO 13:52:39,923 Error from attempt_201108151342_0001_m_01_1: java.lang.NullPointerException at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67) at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264) at org.apache.hadoop.mapred.Child$4.run(Child.java:261) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.mapred.Child.main(Child.java:253) The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html Just for fun I tried… * setting mapred.acls.enabled to true * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to * There was no discernible increase in joy though. Any thoughts ? Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com
NPE in TaskLogAppender
I'm running the Cassandra Brisk server with Haddop core 20.203 on OSX, everything is local. I keep running into this problem for Hive jobs INFO 13:52:39,923 Error from attempt_201108151342_0001_m_01_1: java.lang.NullPointerException at org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:67) at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:264) at org.apache.hadoop.mapred.Child$4.run(Child.java:261) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) at org.apache.hadoop.mapred.Child.main(Child.java:253) The only info I've found online was http://www.mail-archive.com/common-user@hadoop.apache.org/msg12829.html Just for fun I tried… * setting mapred.acls.enabled to true * setting mapred.queue.default.acl-submit-job and mapred.queue.default.acl-administer-jobs to * There was no discernible increase in joy though. Any thoughts ? Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com
Re: Dumping Cassandra into Hadoop
Depends on what you mean by dumping into Hadoop. If you want to read them from a Hadoop Job then you can use either native Hadoop or Pig. See the contrib/word_count and contrib/pig examples. If you want to copy the data into a Hadoop File System install then I guess almost anything that can read from Cassandra and create a file should be OK. You can then copy it onto the HFS and read from there. Hope that helps. Aaron On 20 Oct 2010, at 04:01, Mark wrote: As the subject implies I am trying to dump Cassandra rows into Hadoop. What is the easiest way for me to accomplish this? Thanks. Should I be looking into pig for something like this?
