Hi, I guess, you didn't change the conf files. Please overwrite the hama-default.xml (every nodes).
Avro RPC has been removed because its serialization function consumes high CPU, I/O, and creates a lot of object garbage. On Thu, Apr 25, 2013 at 9:25 AM, Leonidas Fegaras <[email protected]> wrote: > Hi Edward, > This shapshot fails in distributed mode: > > attempt_201304241631_0002_000000_0: 13/04/24 16:53:41 ERROR > sync.ZooKeeperSyncClientImpl: > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > NoNode for /bsp/job_201304241631_0002/peers > attempt_201304241631_0002_000000_0: 13/04/24 16:53:41 FATAL bsp.GroomServer: > Error running child > attempt_201304241631_0002_000000_0: java.lang.ClassNotFoundException: > org.apache.hama.bsp.message.AvroMessageManagerImpl > attempt_201304241631_0002_000000_0: at > java.net.URLClassLoader$1.run(URLClassLoader.java:202) > attempt_201304241631_0002_000000_0: at > java.security.AccessController.doPrivileged(Native Method) > attempt_201304241631_0002_000000_0: at > java.net.URLClassLoader.findClass(URLClassLoader.java:190) > attempt_201304241631_0002_000000_0: at > java.lang.ClassLoader.loadClass(ClassLoader.java:306) > attempt_201304241631_0002_000000_0: at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > attempt_201304241631_0002_000000_0: at > java.lang.ClassLoader.loadClass(ClassLoader.java:247) > attempt_201304241631_0002_000000_0: at java.lang.Class.forName0(Native > Method) > attempt_201304241631_0002_000000_0: at > java.lang.Class.forName(Class.java:247) > attempt_201304241631_0002_000000_0: at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820) > attempt_201304241631_0002_000000_0: at > org.apache.hama.bsp.message.MessageManagerFactory.getMessageManager(MessageManagerFactory.java:36) > attempt_201304241631_0002_000000_0: at > org.apache.hama.bsp.BSPPeerImpl.initilizeMessaging(BSPPeerImpl.java:294) > attempt_201304241631_0002_000000_0: at > org.apache.hama.bsp.BSPPeerImpl.<init>(BSPPeerImpl.java:185) > attempt_201304241631_0002_000000_0: at > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1258) > > It seems that Hama is encoding messages in Avro now. I looked at the Hama > jar files and there is no class AvroMessageManagerImpl. > Leonidas > > > > On 04/24/2013 03:25 AM, Edward J. Yoon wrote: >> >> Leonidas, >> >> Could you please test with >> http://people.apache.org/~edwardyoon/dist/0.7.0-SNAPSHOT/ and feedback >> me? >> >> On Tue, Apr 23, 2013 at 11:07 PM, Leonidas Fegaras <[email protected]> >> wrote: >>> >>> Yes, I think this is fine. I can test a pre-release of Hama 0.6.2 to make >>> sure that works well with MRQL. >>> I have also extended the MRQL make/ant files to work with Yarn. They will >>> be >>> part of the next patch. I have tested MRQL on Yarn in local mode only >>> because I don't have access to a Yarn cluster. >>> Leonidas >>> >>> >>> >>> On Apr 22, 2013, at 6:10 PM, Edward J. Yoon wrote: >>> >>>> Since Hama 0.6 version is more memory efficient than the old version, >>>> let's try to release based on Hama 0.6.* version. I want to evaluate >>>> MRQL's both MR version and BSP version, with large data sets on my >>>> cluster. I'll fix that problem soon and release Hama 0.6.2. What do >>>> you think? >>>> >>>> On Thu, Apr 18, 2013 at 6:22 AM, Edward J. Yoon <[email protected]> >>>> wrote: >>>>> >>>>> +1 >>>>> >>>>> On Thu, Apr 18, 2013 at 12:12 AM, Leonidas Fegaras >>>>> <[email protected]> >>>>> wrote: >>>>>> >>>>>> Edward, >>>>>> Unfortunately, the current MRQL doesn't work correctly with Hama >>>>>> 0.6.x. >>>>>> It >>>>>> works fine with Hama 0.5.0. >>>>>> (The splits generated by the FileInputFormat in Hama 0.6.0 cannot be >>>>>> smaller >>>>>> than a block, while Hama 0.6.1 doesn't work correctly with comma >>>>>> separated >>>>>> paths, which prevents joins). >>>>>> We can wait for the next Hama release (date?) or we can just release >>>>>> it >>>>>> as >>>>>> is for Hama 0.5.0. >>>>>> In either case, let's put a tentative release date: May 15, so we >>>>>> will >>>>>> have >>>>>> one month to write all guides and to setup a testbed. >>>>>> Do you agree to have our first release on May 15? >>>>>> Leonidas >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Apr 17, 2013, at 2:55 AM, Edward J. Yoon wrote: >>>>>> >>>>>>> I personally would recommend you release a first Apache MRQL (with a >>>>>>> well-described guide on how to get started or involved) that works >>>>>>> with open source Apache Hadoop 1.0 and Hama 0.6.x. >>>>>>> >>>>>>> On Sat, Apr 13, 2013 at 12:38 AM, Leonidas Fegaras >>>>>>> <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> I think the obvious person to manage the first release is me, if >>>>>>>> there >>>>>>>> is >>>>>>>> no >>>>>>>> other volunteer. >>>>>>>> I don't have any experience with release plans. Do we need to setup >>>>>>>> a >>>>>>>> timeline for future releases? >>>>>>>> Maybe we should develop a testbed first to be run on different >>>>>>>> cluster >>>>>>>> sizes >>>>>>>> before each official release. >>>>>>>> Leonidas >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Apr 11, 2013, at 8:42 PM, Edward J. Yoon wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> What are our plans for our first release under ASF? And who is >>>>>>>>> going >>>>>>>>> to do the release managing? >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Best Regards, Edward J. Yoon >>>>>>>>> @eddieyoon >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best Regards, Edward J. Yoon >>>>>>> @eddieyoon >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, Edward J. Yoon >>>>> @eddieyoon >>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, Edward J. Yoon >>>> @eddieyoon >>> >>> >> >> > -- Best Regards, Edward J. Yoon @eddieyoon
