Can you provide the driver code for this job? Regards, Shahab
On Wed, Sep 17, 2014 at 10:28 AM, Blanca Hernandez < blanca.hernan...@willhaben.at> wrote: > Hi again, I changed the String objects with org.apache.hadoop.io.Text > objects (why is String not accepted?), and now I get another exception, so > I don´t really know if I solved something or I broke something: > > > > > > java.lang.Exception: java.lang.NullPointerException > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) > > Caused by: java.lang.NullPointerException > > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:988) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:675) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > > at > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:744) > > > > If I could debug it in my IDE, I think I could work faster, but I have the > problems already exposed. How am I testing now? Building a jar, copying it > on the server and running a Hadoop jar command (not very performance > approach…). > > > > Could you give me a hand on this? Any Windows + IntelliJ IDEa there? > Maaaany thanks! > > > > > > > > *Von:* Blanca Hernandez [mailto:blanca.hernan...@willhaben.at] > *Gesendet:* Mittwoch, 17. September 2014 15:27 > *An:* user@hadoop.apache.org > *Betreff:* ClassCastException on running map-reduce jobs + tests on > Windows (mongo-hadoop) > > > > Hi! > > > > I am getting some CCE and don´t really understand why… > > > > Here my mapper: > > > > public class AveragePriceMapper extends Mapper<String, BSONObject, String, > BSONObject>{ > > @Override > > public void map(final String key, final BSONObject val, final Context > context) throws IOException, InterruptedException { > > String id = “result_of_making_some_operations”; > > context.write(id, val); > > } > > } > > > > And in my configuration: > > > > config.setMapperOutputKey(String.class); > > config.setMapperOutputValue(BSONObject.class); > > > > > > On running my generated jar on the server, seems to work everything ok > until: > > > > 14/09/17 15:20:36 INFO mapred.MapTask: Processing split: > MongoInputSplit{URI=mongodb://user:pass@host:27017/my_db.my_collection, > authURI=null, min={ "_id" : { "$oid" : "541666d8e4b07265e257a42e"}}, max={ > }, query={ }, sort={ }, fields={ }, notimeout=false} > > 14/09/17 15:20:36 INFO mapred.MapTask: Map output collector class = > org.apache.hadoop.mapred.MapTask$MapOutputBuffer > > 14/09/17 15:20:36 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584) > > 14/09/17 15:20:36 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100 > > 14/09/17 15:20:36 INFO mapred.MapTask: soft limit at 83886080 > > 14/09/17 15:20:36 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600 > > 14/09/17 15:20:36 INFO mapred.MapTask: kvstart = 26214396; length = 6553600 > > 14/09/17 15:20:36 INFO mapred.LocalJobRunner: map task executor complete. > > 14/09/17 15:20:36 WARN mapred.LocalJobRunner: > job_local1701078621_0001java.lang.Exception: java.lang.ClassCastException: > class java.lang.String > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) > > Caused by: java.lang.ClassCastException: class java.lang.String > > at java.lang.Class.asSubclass(Class.java:3126) > > at > org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:885) > > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:981) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > > at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:675) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > > at > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:744) > > > > > > Did I miss something?? > > > > > > Another issue I am worry about: working on a Windows system makes > everything quite complicated with Hadoop. I have it installed and running, > the same as my mongoDB database (I am using the connector provided by > them). Running the same main class I am using in the hadooop jar call on > the server (in the example before), but from my IDE, I get this exception: > > > > PriviledgedActionException as:hernanbl cause:java.io.IOException: Failed > to set permissions of path: > \tmp\hadoop-hernanbl\mapred\staging\hernanbl1600842219\.staging to 0700 > > > > How could I make it run? > > > > > > Many thanks!! > > > > Best regards, > > > > Blanca >