Thanks for the reply. I will recheck and repeat the experiment using self-typed input. I am reinstalling Cloudera 5.2.
Ashok Kumar Harnal On 24 November 2014 at 21:38, Ted Dunning <ted.dunn...@gmail.com> wrote: > The error message that you got indicated that some input was textual and > needed to be an integer. > > Is there a chance that the type of some of your input is incorrect in your > sequence files? > > > > On Mon, Nov 24, 2014 at 3:47 PM, Ashok Harnal <ashokhar...@gmail.com> > wrote: > > > Thanks for reply. I did not compile mahout. Mahout 0.9 comes along with > > Cloudera 5.2. > > > > Ashok Kumar Harnal > > > > On 24 November 2014 at 18:42, <jayunit...@gmail.com> wrote: > > > > > Looks like maybe a mismatch between mahout version you compiled code > > > against and the mahout version installed in the cluster? > > > > > > > On Nov 24, 2014, at 8:08 AM, Ashok Harnal <ashokhar...@gmail.com> > > wrote: > > > > > > > > Thanks for reply. Here are the facts: > > > > > > > > 1. I am using mahout shell command and not a java program. So I am > not > > > > passing any arguments to map function. > > > > > > > > 2. I am using hadoop. Input training file is loaded in hadoop. It is > a > > > tab > > > > separated 'u1.base' file of MovieLens dataset. > > > > It is something like below. All users are there along with > whatever > > > > ratings they have given. > > > > > > > > 1 1 5 > > > > 1 2 3 > > > > 1 3 4 > > > > 1 4 3 > > > > 1 5 3 > > > > : > > > > : > > > > 2 1 4 > > > > 2 10 2 > > > > 2 14 4 > > > > : > > > > : > > > > > > > > 3. I use the following mahout command to build model: > > > > > > > > mahout parallelALS --input /user/ashokharnal/u1.base --output > > > > /user/ashokharnal/u1.out --lambda 0.1 --implicitFeedback true --alpha > > > > 0.8 --numFeatures 15 --numIterations 10 --numThreadsPerSolver 1 > > > > --tempDir /tmp/ratings > > > > > > > > 4. My test file is just two-lines tab-separated file as below: > > > > > > > > > > > > 1 1 > > > > 2 1 > > > > > > > > 5. This file is converted to sequence file using the following mahout > > > command: > > > > > > > > mahout seqdirectory -i /user/ashokharnal/ufind2.test -o > > > > /user/ashokharnal/seqfiles > > > > > > > > 6. I then run the following mahout command: > > > > > > > > mahout recommendfactorized --input /user/ashokharnal/seqfiles > > > > --userFeatures /user/ashokharnal/u1.out/U/ --itemFeatures > > > > /user/akh/u1.out/M/ --numRecommendations 1 --output > /tmp/reommendation > > > > --maxRating 1 > > > > > > > > 7. I am using CentOS 6.5 with Cloudera 5.2 installed. > > > > > > > > The error messages are as below: > > > > > > > > 14/11/24 18:06:48 INFO mapred.MapTask: Processing split: > > > > hdfs://master:8020/user/ashokharnal/seqfiles/part-m-00000:0+195 > > > > 14/11/24 18:06:49 INFO zlib.ZlibFactory: Successfully loaded & > > > > initialized native-zlib library > > > > 14/11/24 18:06:49 INFO compress.CodecPool: Got brand-new decompressor > > > [.deflate] > > > > 14/11/24 18:06:49 INFO mapred.LocalJobRunner: Map task executor > > complete. > > > > 14/11/24 18:06:49 WARN mapred.LocalJobRunner: > job_local1177125820_0001 > > > > java.lang.Exception: java.lang.RuntimeException: > > > > java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be > cast > > > > to org.apache.hadoop.io.IntWritable > > > > at > > > > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406) > > > > Caused by: java.lang.RuntimeException: java.lang.ClassCastException: > > > > org.apache.hadoop.io.Text cannot be cast to > > > > org.apache.hadoop.io.IntWritable > > > > at > > > > > > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:151) > > > > at > > > > > > org.apache.mahout.cf.taste.hadoop.als.MultithreadedSharingMapper.run(MultithreadedSharingMapper.java:60) > > > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) > > > > at > > > > > > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268) > > > > at > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text > > > > cannot be cast to org.apache.hadoop.io.IntWritable > > > > at > > > > > > org.apache.mahout.cf.taste.hadoop.als.PredictionMapper.map(PredictionMapper.java:44) > > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) > > > > at > > > > > > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper$MapRunner.run(MultithreadedMapper.java:268) > > > > 14/11/24 18:06:49 INFO mapred.JobClient: map 0% reduce 0% > > > > 14/11/24 18:06:49 INFO mapred.JobClient: Job complete: > > > job_local1177125820_0001 > > > > 14/11/24 18:06:49 INFO mapred.JobClient: Counters: 0 > > > > 14/11/24 18:06:49 INFO driver.MahoutDriver: Program took 2529 ms > > > > (Minutes: 0.04215) > > > > 14/11/24 18:06:49 ERROR hdfs.DFSClient: Failed to close inode 24733 > > > > > > > > > > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): > > > > No lease on > > > > > > /tmp/reommendation/_temporary/_attempt_local1177125820_0001_m_000000_0/part-m-00000 > > > > (inode 24733): File does not exist. Holder > > > > DFSClient_NONMAPREDUCE_157704469_1 does not have any open files. > > > > at > > > > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3319) > > > > at > > > > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3407) > > > > at > > > > > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3377) > > > > at > > > > > > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:673) > > > > at > > > > > > org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.complete(AuthorizationProviderProxyClientProtocol.java:219) > > > > at > > > > > > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:520) > > > > at > > > > > > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > > > > at > > > > > > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587) > > > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026) > > > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) > > > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) > > > > at java.security.AccessController.doPrivileged(Native Method) > > > > at javax.security.auth.Subject.doAs(Subject.java:415) > > > > at > > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007) > > > > > > > > at org.apache.hadoop.ipc.Client.call(Client.java:1411) > > > > at org.apache.hadoop.ipc.Client.call(Client.java:1364) > > > > at > > > > > > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) > > > > at com.sun.proxy.$Proxy16.complete(Unknown Source) > > > > at > > > > > > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:435) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > > at > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > > at > > > > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > > > > at > > > > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > > > > at com.sun.proxy.$Proxy17.complete(Unknown Source) > > > > at > > > > > > org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2180) > > > > at > > > org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2164) > > > > at > > > > > > org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:908) > > > > at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:926) > > > > at > > > > > > org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:861) > > > > at > > > org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2687) > > > > at > > > > > > org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2704) > > > > at > > > > > > org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) > > > > > > > > Sorry for bothering > > > > > > > > Ashok Kumar Harnal > > > > > > > > > > > > > > > > On 24 November 2014 at 15:50, Divyang Shah > > > <divyangas...@yahoo.com.invalid> > > > > wrote: > > > > > > > >> hello, problem is in map method argument you have passed is > not > > > >> matching with specified in job configuration. so, match both of > them. > > > >> > > > >> > > > >> On Sunday, 23 November 2014 8:31 AM, Ashok Harnal < > > > >> ashokhar...@gmail.com> wrote: > > > >> > > > >> > > > >> I use mahout 0.7 installed in Cloudera. After creating user-feature > > and > > > >> item-feature matrix in hdfs, I run the following command: > > > >> > > > >> mahout recommendfactorized --input /user/ashokharnal/seqfiles > > > >> --userFeatures $res_out_file/U/ --itemFeatures $res_out_file/M/ > > > >> --numRecommendations 1 --output $reommendation --maxRating 1 > > > >> > > > >> After some time, I get the following error: > > > >> > > > >> : > > > >> : > > > >> 14/11/23 08:28:20 INFO mapred.LocalJobRunner: Map task executor > > > complete. > > > >> 14/11/23 08:28:20 WARN mapred.LocalJobRunner: > job_local954305987_0001 > > > >> java.lang.Exception: java.lang.RuntimeException: > > > >> java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be > cast > > > to > > > >> org.apache.hadoop.io.IntWritable > > > >> at > > > >> > > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406) > > > >> Caused by: java.lang.RuntimeException: java.lang.ClassCastException: > > > >> org.apache.hadoop.io.Text cannot be cast to > > > >> org.apache.hadoop.io.IntWritable > > > >> at > > > >> > > > >> > > > > > > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:151) > > > >> at > > > >> > > > >> > > > > > > org.apache.mahout.cf.taste.hadoop.als.MultithreadedSharingMapper.run(MultithreadedSharingMapper.java:60) > > > >> at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) > > > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) > > > >> at > > > >> > > > >> > > > > > > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268) > > > >> at > > > >> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > > >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > >> at > > > >> > > > >> > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > >> at > > > >> > > > >> > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > >> at java.lang.Thread.run(Thread.java:744) > > > >> Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text > > > cannot > > > >> be cast to org.apache.hadoop.io.IntWritable > > > >> at > > > >> > > > >> > > > > > > org.apache.mahout.cf.taste.hadoop.als.PredictionMapper.map(PredictionMapper.java:44) > > > >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) > > > >> at > > > >> > > > >> > > > > > > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper$MapRunner.run(MultithreadedMapper.java:268) > > > >> > > > >> > > > >> Not sure what is wrong. > > > >> Request help. > > > >> > > > >> Ashok Kumar Harnal > > > >> > > > >> > > > >> > > > >> > > > >> -- > > > >> Visit my blog at: http://ashokharnal.wordpress.com/ > > > >> > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > -- > > > > Visit my blog at: http://ashokharnal.wordpress.com/ > > > > > > > > > > > -- > > Visit my blog at: http://ashokharnal.wordpress.com/ > > > -- Visit my blog at: http://ashokharnal.wordpress.com/
