Ok, I find it works by adding
storm.scheduler: org.apache.storm.scheduler.resource.ResourceAwareScheduler
instead of adding
storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
suggested by the website..
ᐧ
On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang <wuy...@winlab.rutgers.edu>
wrote:
> When I redo the procedure, here is the more log
>
> *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom
> scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
> *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on
> initialization of server service-handler*
> *java.lang.ClassNotFoundException:
> “org.apache.storm.scheduler.resource.ResourceAwareScheduler”*
> * at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[?:1.8.0_151]*
> * at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_151]*
> * at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> ~[?:1.8.0_151]*
> * at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_151]*
> * at java.lang.Class.forName0(Native Method) ~[?:1.8.0_151]*
> * at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_151]*
> * at org.apache.storm.util$new_instance.invoke(util.clj:1027)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$mk_scheduler.invoke(nimbus.clj:127)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:215)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$exec_fn__1370__auto____11008.invoke(nimbus.clj:2451)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
> ~[storm-core-1.1.1.jar:1.1.1]*
> *2018-02-10 14:10:32.806 o.a.s.util main [ERROR] Halting process: ("Error
> on initialization")*
> *java.lang.RuntimeException: ("Error on initialization")*
> * at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$fn__11007$service_handler__11040.doInvoke(nimbus.clj:2448)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]*
> * at
> org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2536)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2569)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2592)
> ~[storm-core-1.1.1.jar:1.1.1]*
> * at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]*
> * at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]*
> * at org.apache.storm.daemon.nimbus.main(Unknown Source)
> ~[storm-core-1.1.1.jar:1.1.1]*
>
> I download a binary version of Storm 1.1.1 from http://www.apache.org/
> dyn/closer.lua/storm/apache-storm-1.1.1/apache-storm-1.1.1.tar.gz.
> ᐧ
>
> On Sat, Feb 10, 2018 at 2:02 PM, Wuyang Zhang <wuy...@winlab.rutgers.edu>
> wrote:
>
>> Hi Jerry,
>>
>> I just retry to use the resource aware scheduler in a single node with 4
>> slots without any further settings. The scheduler still kills the nimbus
>> process. Please find nimbus.log and storm.yaml attached.
>>
>> Without add the line
>>
>> storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
>>
>> in storm.yaml, everything works fine.
>>
>> I also tried
>>
>> conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
>>
>> The cluster can run, but I cannot see any log related to
>> DefaultResourceAware. It seems like that it doesn't really run.
>>
>> Please take a look at the log and see if I miss anything to enable them
>> to work?
>>
>> Best,
>> Wuyang
>> ᐧ
>>
>> On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qingshanyou...@gmail.com>
>> wrote:
>>
>>> Hi Jerry,
>>>
>>> Thank you for the reply.
>>>
>>> Please find the storm.yaml below:
>>>
>>> ########### These MUST be filled in for a storm configuration
>>> storm.zookeeper.servers:
>>> - "10.10.21.1"
>>> - "10.10.21.2"
>>> - "10.10.21.3"
>>> - "10.10.21.4"
>>> - "10.10.21.5"
>>> - "10.10.21.6"
>>> - "10.10.21.7"
>>>
>>> nimbus.seeds: ["10.10.21.1"]
>>>
>>> storm.zookeeper.port: 2181
>>> ###### These may optionally be filled in:
>>>
>>> worker.heap.memory.mb: 8192
>>>
>>> supervisor.scheduler.meta:
>>> tag: 10.10.20.1
>>>
>>> #storm.scheduler: storm.EmptyScheduler
>>> storm.scheduler: “org.apache.storm.scheduler.re
>>> source.ResourceAwareScheduler”
>>> supervisor.memory.capacity.mb: 2048.0
>>> supervisor.cpu.capacity: 4000.0
>>> supervisor.slots.ports:
>>> - 6700
>>> - 6701
>>> - 6702
>>> - 6703
>>>
>>> worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails
>>> -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
>>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M
>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
>>>
>>> On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <
>>> jerry.boyang.p...@gmail.com> wrote:
>>>
>>>> Wuyang,
>>>>
>>>> Can you provide your whole storm.yaml ?
>>>>
>>>> Best,
>>>>
>>>> Jerry
>>>>
>>>> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wuy...@winlab.rutgers.edu
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have the difficulty with the resource aware scheduler from the
>>>>> version 1.1.1.
>>>>>
>>>>> As suggested by the website, I add the following lines to storm.yaml:
>>>>>
>>>>> *storm.scheduler: “org.apache.storm.scheduler.re
>>>>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”*
>>>>> * supervisor.memory.capacity.mb: 2048.0*
>>>>> * supervisor.cpu.capacity: 400.0*
>>>>>
>>>>> Afterwards, I rerun the storm and the nimbus process has been killed.
>>>>> The last log I found from nimbus.log is:
>>>>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “
>>>>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”.
>>>>>
>>>>> I suppose something goes wrong when it runs the scheduler.
>>>>>
>>>>> Do I miss anything to enable this scheduler?
>>>>>
>>>>> Best,
>>>>> Wuyang
>>>>> ᐧ
>>>>>
>>>>
>>>>
>>>
>>
>
