Hi Jean Paul, I am also using the current version of the master on the incubator (i.e., 0.4.0) and Storm 0.9.3. If you want to confirm the setup, check the attached for some steps I wrote a while ago on how to get Samoa configured to play with a Storm configuration: (note that I didn't change much of the default settings)
My server has physical cpus so I am not running it in VMs, with a total of 65GB. Regarding the yahoo examples, there are probably applicable but refer to older versions of Samoa. So I would stick to the current ones from the Apache Incubator page of Samoa. Hope this helps! Nicolas On Sun, Apr 3, 2016 at 9:51 AM, Gianmarco De Francisci Morales <[email protected] > wrote: > I'm running the master version of SAMOA from github and Storm 0.9.6. > > In my case they are VMs, 4GB of RAM per core. > I think Nicolas has a different setup. > No changes on the Storm configuration. > > -- Gianmarco > > On Sun, Apr 3, 2016 at 10:47 AM, Jean Paul Barddal <[email protected]> > wrote: > > > Hi Gianmarco and Nicolas, > > > > Yes, it does. > > Can you tell me which version of SAMOA and Storm you have running in your > > cluster? > > I will try to set up a new cluster from scratch to see if I manage to get > > it to work. > > Also, can you tell me if your clusters’ nodes are VMs? And you many RAM > > each has? > > Finally, do you have any changes to your garbage collection on storm, or > > everything is as default? > > > > > > Nicolas, I honestly don’t recall where I saw this example using com.yahoo > > classes. > > Should I use others instead? > > > > Cheers, > > > > Jean > > > > > > > On Apr 3, 2016, at 7:40 PM, Gianmarco De Francisci Morales < > > [email protected]> wrote: > > > > > > Hi Jean Paul, > > > I cannot reproduce the issue on my setup, the default bagging works > fine > > on > > > my cluster. > > > Have you validated that your Storm cluster works for other > applications? > > > > > > -- Gianmarco > > > > > > On Thu, Mar 24, 2016 at 1:40 PM, Nicolas Kourtellis < > > [email protected]> > > > wrote: > > > > > >> Hi Jean, > > >> > > >> The example you mentioned running is using yahoo.labs code. > > >> Did you find these on the samoa - apache page? Can you point to the > > >> location? > > >> > > >> We may need to update those with the apache versions. > > >> > > >> In any case, this generator should now be on the master of apache > repo. > > >> > > >> Thanks, > > >> > > >> Nicolas > > >> > > >> On Tue, Mar 8, 2016 at 2:54 AM, Jean Paul Barddal < > [email protected]> > > >> wrote: > > >> > > >>> Nope. > > >>> I’ve tested the default configuration provided in examples in the > samoa > > >>> page, such as -s > > >>> (com.yahoo.labs.samoa.moa.streams.generators.RandomTreeGenerator -c 2 > > -o > > >> 10 > > >>> -u 10). > > >>> > > >>> Cheers, > > >>> > > >>> Jean > > >>> > > >>>> On Mar 8, 2016, at 12:18 AM, Gianmarco De Francisci Morales < > > >>> [email protected]> wrote: > > >>>> > > >>>> The Boosting is not very well tested, so I would expect some bugs > > >> there. > > >>>> Bagging is a bit more well tested. > > >>>> Are you using a dataset with a very large number of attributes? > > >>>> > > >>>> -- Gianmarco > > >>>> > > >>>> On Mon, Mar 7, 2016 at 2:55 AM, Jean Paul Barddal < > > [email protected] > > >>> > > >>>> wrote: > > >>>> > > >>>>> Hello Gianmarco, > > >>>>> > > >>>>> I’ve tried both Bagging and Boosting. > > >>>>> They end up in a GC error pretty quickly. > > >>>>> > > >>>>> Jean > > >>>>> > > >>>>> > > >>>>>> On Mar 7, 2016, at 1:03 AM, Gianmarco De Francisci Morales < > > >>>>> [email protected]> wrote: > > >>>>>> > > >>>>>> Hi Jean Paul, > > >>>>>> > > >>>>>> Which ensemble are you running? Bagging? > > >>>>>> And what's the ensemble size? > > >>>>>> > > >>>>>> Nicolas, if I remember correctly we do not have acking for Storm > > >>> enabled > > >>>>> in > > >>>>>> the current version on master. > > >>>>>> Maybe this is why not all instances are executed? > > >>>>>> If so, I could port the acking we have on the VHT branch on > master. > > >>>>>> > > >>>>>> Cheers, > > >>>>>> > > >>>>>> -- Gianmarco > > >>>>>> > > >>>>>> On Thu, Feb 25, 2016 at 4:23 PM, Jean Paul Barddal < > > >>> [email protected]> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> Hello again everyone, > > >>>>>>> > > >>>>>>> I managed to improve a little my situation. > > >>>>>>> Now, the problem only occurs with ensembles. > > >>>>>>> I am sending too screenshots of two issues I currently have. > > >>>>>>> If the amount of instances is to big (1 million or more), then I > > ran > > >>>>> into > > >>>>>>> a GC problem. > > >>>>>>> If less, sometimes the evaluation node does not execute all the > > >>>>> instances. > > >>>>>>> I’ve left the topology running over night and it was still stuck. > > >>>>>>> > > >>>>>>> http://i67.tinypic.com/10cqnvp.png > > >>>>>>> http://i66.tinypic.com/f9mwrc.png > > >>>>>>> > > >>>>>>> Cheers, > > >>>>>>> > > >>>>>>> Jean > > >>>>>>> > > >>>>>>> > > >>>>>>>> On Feb 23, 2016, at 8:53 AM, Jean Paul Barddal < > > >> [email protected]> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>> Dear Nicolas and Gianmarco, > > >>>>>>>> > > >>>>>>>> Thanks for the replies! > > >>>>>>>> > > >>>>>>>> The screenshot can be found at: > > http://i68.tinypic.com/25jdu9v.png > > >>>>>>>> > > >>>>>>>> Yes, I have tried a few examples from the guide. I’ve tried VHT > > and > > >>>>>>> Bagging over a RandomTreeGenerator. > > >>>>>>>> > > >>>>>>>> Sometimes it works, but then I get a GC (garbage collector > error) > > >>> too. > > >>>>>>>> > > >>>>>>>> Gianmarco, I’m running Storm 0.9.5 and SAMOA 0.0.4. > > >>>>>>>> > > >>>>>>>> Cheers, > > >>>>>>>> > > >>>>>>>> Jean > > >>>>>>>> > > >>>>>>>>> On Feb 22, 2016, at 6:11 PM, Gianmarco De Francisci Morales < > > >>>>>>> [email protected]> wrote: > > >>>>>>>>> > > >>>>>>>>> Also, which version of Storm are you using? > > >>>>>>>>> > > >>>>>>>>> -- Gianmarco > > >>>>>>>>> > > >>>>>>>>> On Mon, Feb 22, 2016 at 1:01 PM, Nicolas Kourtellis < > > >>>>>>> [email protected]> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Hi Jean, > > >>>>>>>>>> Thanks for your interest in SAMOA! > > >>>>>>>>>> I dont think the attachment came through. > > >>>>>>>>>> Maybe you can post it in an online storage and send a link? > > >>>>>>>>>> > > >>>>>>>>>> Regarding your question, have you tried the basic examples > from > > >> the > > >>>>>>> guide? > > >>>>>>>>>> E.g., the VHT in storm mode? > > >>>>>>>>>> > > >>>>>>>>>> Thanks, > > >>>>>>>>>> > > >>>>>>>>>> Nicolas > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On Mon, Feb 22, 2016 at 5:21 PM, Jean Paul Barddal < > > >>>>>>> [email protected]> > > >>>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Hello guys, > > >>>>>>>>>>> > > >>>>>>>>>>> I am trying to set up a SAMOA environment and I'm having some > > >>>>> issues. > > >>>>>>>>>>> I have the storm up and running and managed to download, > > compile > > >>> and > > >>>>>>> run > > >>>>>>>>>>> SAMOA in local configuration. > > >>>>>>>>>>> Now I'm trying to run it in cluster mode. > > >>>>>>>>>>> As you can see from the attached screnshot, the topology is > up > > >> and > > >>>>>>>>>>> running, but the spouts do not send any data to the bolts and > > so > > >>> on. > > >>>>>>>>>>> > > >>>>>>>>>>> Ideas? > > >>>>>>>>>>> > > >>>>>>>>>>> Cheers, > > >>>>>>>>>>> > > >>>>>>>>>>> Jean > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> -- > > >>>>>>>>>> Nicolas Kourtellis > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>> > > >>>>> > > >>> > > >>> > > >> > > >> > > >> -- > > >> Nicolas Kourtellis > > >> > > > > > -- Nicolas Kourtellis
I n s t a l l a t i o n o f s t o r m c l u s t e r : h t t p s : / / s t o r m . a p a c h e . o r g / d o c u m e n t a t i o n / S e t t i n g - u p - a - S t o r m - c l u s t e r . h t m l D o w n l o a d a s t a b l e d i s t r i b u t i o n f r o m h t t p s : / / s t o r m . a p a c h e . o r g / d o w n l o a d s . h t m l e . g . : w g e t h t t p : / / f t p . c i x u g . e s / a p a c h e / s t o r m / a p a c h e - s t o r m - 0 . 9 . 3 / a p a c h e - s t o r m - 0 . 9 . 3 . t a r . g z u n t a r t h e f i l e : t a r - x v f a p a c h e - s t o r m - 0 . 9 . 3 . t a r . g z S e t u p a p p r o p r i a t e l y t h e c o n f / s t o r m . y a m l f i l e w i t h i n t h e u n p a c k e d f o l d e r . A n e x a m p l e i s t h e f o l l o w i n g : s t o r m . z o o k e e p e r . s e r v e r s : - " 1 2 7 . 0 . 0 . 1 " s t o r m . l o c a l . d i r : " / v a r / s t o r m - l o g s " n i m b u s . h o s t : " 1 2 7 . 0 . 0 . 1 " s u p e r v i s o r . s l o t s . p o r t s : - 6 7 0 0 w o r k e r . c h i l d o p t s : " - X m x 2 0 0 0 m " s u p e r v i s o r . c h i l d o p t s : " - X m x 2 5 6 m " n i m b u s . c h i l d o p t s : " - X m x 5 1 2 m " C r e a t e f o l d e r ~ / . s t o r m C o p y t h e f i l e c o n f / s t o r m . y a m l i n t o t h e f o l d e r ~ / . s t o r m / S e t u p y o u r $ S T O R M _ H O M E t o p o i n t t o t h e f o l d e r o f s t o r m . e . g . e x p o r t S T O R M _ H O M E = / h o m e d i r e c t o r y / a p a c h e - s t o r m - 0 . 9 . 3 I n s t a l l a t i o n o f Z o o k e e p e r : h t t p : / / z o o k e e p e r . a p a c h e . o r g / d o c / r 3 . 3 . 3 / z o o k e e p e r S t a r t e d . h t m l # s c _ I n s t a l l i n g S i n g l e M o d e D o w n l o a d f r o m : w g e t h t t p : / / a p a c h e . r e d i r i s . e s / z o o k e e p e r / z o o k e e p e r - 3 . 4 . 6 / z o o k e e p e r - 3 . 4 . 6 . t a r . g z U n t a r : t a r - x v f z o o k e e p e r - 3 . 4 . 6 . t a r . g z G o i n t h e u n p a c k e d f o l d e r a n d c r e a t e c o n f / z o o . c f g f i l e a n d m o d i f y t h e d a t a D i r d i r e c t o r y . S t a r t z o o k e e p e r : b i n / z k S e r v e r . s h s t a r t S t a r t n i m b u s p r o c e s s ( g o i n t o t h e f o l d e r o f b i n / s t o r m ) . / s t o r m n i m b u s S t a r t s u p e r v i s o r p r o c e s s ( g o i n t o t h e f o l d e r o f b i n / s t o r m ) . / s t o r m s u p e r v i s o r S t a r t t h e U I f o r s t o r m ( g o i n t o t h e f o l d e r o f b i n / s t o r m ) . / s t o r m u i A f t e r y o u h a v e d o w n l o a d e d , u n p a c k e d a n d m v n - e d t h e s a m o a p a c k a g e , y o u c a n e x e c u t e t h e b i n / s a m o a c o m m a n d w i t h s t o r m a s t h e p r o c e s s i n g e n g i n e ( a n d c r o s s f i n g e r s ! ) .
