Hi,
I was able to get rid hostnames from all places possible (places other
than faban and all machines involved) and replaced them with ip
addresses. The CmdService now atleast is getting setup on one of the
machines (Backend+Client). Now it is failing on the other machine
(Frontend).
Time Host Level Message
19:34:16 INFO Starting OlioDriver run using
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1L/
19:34:16 INFO START TIME : Fri Jul 06 19:34:16 CDT 2012
19:34:16 INFO Starting Registry.
19:34:16 sc-h03 INFO Registry started.
19:34:26 WARNING/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1L&exception=4>/
Agent(daemon)@128.105.102.201: No route to host. Will try remote shell
instead.
19:34:26 WARNING [/usr/bin/ssh, -n, 128.105.102.201,
/root/jayneel/cloudsuite/webserving/web-release/faban/bin/agent,
128.105.102.201, 128.105.102.203, 128.105.102.203,
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64,
-Dfaban.home=/root/jayneel/cloudsuite/webserving/web-release/faban/,
-Djava.security.policy=/root/jayneel/cloudsuite/webserving/web-release/faban/config/faban.policy,
-Djava.util.logging.config.file=/root/jayneel/cloudsuite/webserving/web-release/faban/config/logging.properties,
-Dfaban.registry.port=9998, -Dfaban.logging.port=9999, -Xmx3g, -Xms256m,
-XX:+DisableExplicitGC, faban.benchmarkName=OlioDriver,
faban.download=http://128.105.102.203:9980/]
stdout:
kernel.randomize_va_space = 0
19:34:46 INFO CmdService: Configured [email protected] on
server sc-h03
19:34:46 SEVERE/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1L&exception=7>/
Error accessing command agent on system 128.105.102.201
19:34:46 SEVERE CmdService setup failed. Exiting
19:34:52 sc-h03 INFO
[/root/jayneel/cloudsuite/webserving/web-release/faban/bin/fenxi,
process,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1L/,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1L//post/,
OlioDriver.1L]
stdout:
Creating database ... 3.80s
Parsing raw files ... 0.00s
Processing profile default_profile ...OK
Opening Database ... 0.72s
Executing queries ... 0.03s
Generating html pages ... 0.11s
19:34:53 INFO Killing benchmark
19:34:53 INFO killing [email protected]
19:34:53 INFO killing [email protected]
19:34:53 SEVERE/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1L&exception=13>/
Kill Failed for [email protected]
19:34:54 INFO OlioDriver Completed/Terminated
The exception says
java.rmi.ConnectIOException: Exception creating connection to:
128.105.102.201; nested exception is:
java.net.NoRouteToHostException: No route to host
Both machines are able to connect/ssh each other without passwords.
These machines are on the same network. I am not sure why the route not
being found to the frontend machine.
Jayneel
On 07/06/2012 01:41 AM, [email protected] wrote:
Yes, I have recompiled according to the step 11-17 on the setting up the
faban driver from webpage. I am still getting these messages of not able
to reconnect.
01:42:12 WARNING Retry connecting to
[email protected], count 1.
01:42:22 WARNING Retry connecting to
[email protected], count 2.
01:42:32 WARNING Retry connecting to
[email protected], count 3.
01:42:42 WARNING Retry connecting to
[email protected], count 4.
01:42:52 WARNING Retry connecting to
[email protected], count 5.
01:43:02 WARNING Retry connecting to
[email protected], count 6.
01:43:12 WARNING Retry connecting to
[email protected], count 7.
01:43:22 WARNING Retry connecting to
[email protected], count 8.
01:43:32 WARNING Retry connecting to
[email protected], count 9.
01:43:42 WARNING Retry connecting to
[email protected], count 10.
01:43:42 SEVERE Could not connect to [email protected]
01:43:42 SEVERE CmdService setup failed. Exiting
Jayneel
Have you recompile Faban following steps 11 to 17 from "Setting up the
Faban driver" installation guide?
Leonardo
On Fri, Jul 6, 2012 at 2:24 AM,<[email protected]> wrote:
Hi Binzhang,
I did change the hostnames to ip-address. It was still giving the
[email protected] connection retries.
I did see something peculiar in the message in the run after changing
all
reference of hostnames to ip-addresses. The java call stack printed in
the
end after the benchmark is killed still has reference to sc-h03. Do I
need
to do something specific in faban except from restart?
Jayneel
Killing benchmark
00:23:10 WARNING
[/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/java,
-Dfaban.home=/root/jayneel/cloudsuite/webserving/web-release/faban/,
-Djava.security.policy=/root/jayneel/cloudsuite/webserving/web-release/faban/config/faban.policy,
-Djava.util.logging.config.file=/root/jayneel/cloudsuite/webserving/web-release/faban/config/logging.properties,
-Dfaban.registry.port=9998, -Dfaban.logging.port=9999, -Xms32m,
-Xmx1024m,
-cp,
/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-codec-1.2.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-httpclient-3.1.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-logging.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabancommon.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabandriver.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabanagents.jar,
com.sun.faban.common.RegistryImpl]
stderr:
SocketHandler: connect failed to sc-h03:9999
Can't load log handler "java.util.logging.SocketHandler"
java.net.UnknownHostException: sc-h03
java.net.UnknownHostException: sc-h03
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:158)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at java.util.logging.SocketHandler.connect(SocketHandler.java:154)
at java.util.logging.SocketHandler.<init>(SocketHandler.java:112)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at java.lang.Class.newInstance0(Class.java:372)
at java.lang.Class.newInstance(Class.java:325)
at java.util.logging.LogManager$3.run(LogManager.java:384)
at java.security.AccessController.doPrivileged(Native Method)
at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:370)
at
java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:906)
at java.util.logging.LogManager.access$900(LogManager.java:147)
at
java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:987)
at java.util.logging.Logger.log(Logger.java:472)
at java.util.logging.Logger.doLog(Logger.java:498)
at java.util.logging.Logger.log(Logger.java:521)
at java.util.logging.Logger.info(Logger.java:1040)
at com.sun.faban.common.RegistryImpl.main(RegistryImpl.java:427)
I met the same problem and solved it by using the ip rather than the
hostname of the remote machine, say the "sc-h03.cs.wisc.edu".
It was weired since i added the entry in hosts file, and can ping and
ssh correctly with the hostname. I don't know the real reason, but
it worked. Maybe you could try it.
Regards,
Binzhang Fu
äºZ( 2012/7/6 3:25, Jayneel Gandhi å+^(TM)é?":
Hi Cansu,
Both machines are able to connect with ssh without password. The log
that I sent is after this setup was done.
Jayneel
On Jul 5, 2012, at 1:16 PM, Cansu Kaynak<[email protected]> wrote:
Please make sure that all machines can ssh to each other and then try
starting the experiment.
Cansu
On 07/05/2012 08:05 PM, Jayneel Gandhi wrote:
Hi Cansu,
Thanks for the suggestion. The problem was not a field left empty
but
one of the address provided was wrong in the configuration. I
corrected
that. But it lead to another issue. This was the log.
Time Host Level Message
12:05:32 INFO Starting OlioDriver run using
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1E/
12:05:32 INFO START TIME : Thu Jul 05 12:05:32 CDT 2012
12:05:32 INFO Starting Registry.
12:05:42 WARNING/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1E&exception=3>/
Agent(daemon)@sc-h01.cs.wisc.edu: No route to host. Will try
remote
shell instead.
12:05:48 WARNING [/usr/bin/ssh, -n, sc-h01.cs.wisc.edu,
/root/jayneel/cloudsuite/webserving/web-release/faban/bin/agent,
sc-h01.cs.wisc.edu, sc-h03.cs.wisc.edu, 128.105.102.203,
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64,
-Dfaban.home=/root/jayneel/cloudsuite/webserving/web-release/faban/,
-Djava.security.policy=/root/jayneel/cloudsuite/webserving/web-release/faban/config/faban.policy,
-Djava.util.logging.config.file=/root/jayneel/cloudsuite/webserving/web-release/faban/config/logging.properties,
-Dfaban.registry.port=9998, -Dfaban.logging.port=9999, -Xmx3g,
-Xms256m,
-XX:+DisableExplicitGC, faban.benchmarkName=OlioDriver,
faban.download=http://sc-h03.cs.wisc.edu:9980/]
stdout:
kernel.randomize_va_space = 0
12:06:12 WARNING Retry connecting to
[email protected],
count 1.
12:06:22 WARNING Retry connecting to
[email protected],
count 2.
12:06:32 WARNING Retry connecting to
[email protected],
count 3.
12:06:42 WARNING Retry connecting to
[email protected],
count 4.
12:06:52 WARNING Retry connecting to
[email protected],
count 5.
12:07:02 WARNING Retry connecting to
[email protected],
count 6.
12:07:12 WARNING Retry connecting to
[email protected],
count 7.
12:07:22 WARNING Retry connecting to
[email protected],
count 8.
12:07:32 WARNING Retry connecting to
[email protected],
count 9.
12:07:42 WARNING Retry connecting to
[email protected],
count 10.
12:07:42 SEVERE Could not connect to
[email protected]
12:07:42 SEVERE CmdService setup failed. Exiting
12:07:42 SEVERE/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1E&exception=17>/
FenXi process command [fenxi, process,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1E/,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1E//post/,
OlioDriver.1E] failed.
12:07:42 INFO Killing benchmark
12:07:42 WARNING
[/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/java,
-Dfaban.home=/root/jayneel/cloudsuite/webserving/web-release/faban/,
-Djava.security.policy=/root/jayneel/cloudsuite/webserving/web-release/faban/config/faban.policy,
-Djava.util.logging.config.file=/root/jayneel/cloudsuite/webserving/web-release/faban/config/logging.properties,
-Dfaban.registry.port=9998, -Dfaban.logging.port=9999, -Xms32m,
-Xmx1024m, -cp,
/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-codec-1.2.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-httpclient-3.1.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/commons-logging.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabancommon.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabandriver.jar:/root/jayneel/cloudsuite/webserving/web-release/faban/lib/fabanagents.jar,
com.sun.faban.common.RegistryImpl]
stderr:
SocketHandler: connect failed to sc-h03:9999
Can't load log handler "java.util.logging.SocketHandler"
java.net.UnknownHostException: sc-h03
java.net.UnknownHostException: sc-h03
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:158)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at java.util.logging.SocketHandler.connect(SocketHandler.java:154)
at java.util.logging.SocketHandler.<init>(SocketHandler.java:112)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at java.lang.Class.newInstance0(Class.java:372)
at java.lang.Class.newInstance(Class.java:325)
at java.util.logging.LogManager$3.run(LogManager.java:384)
at java.security.AccessController.doPrivileged(Native Method)
at
java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:370)
at
java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:906)
at java.util.logging.LogManager.access$900(LogManager.java:147)
at
java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:987)
at java.util.logging.Logger.log(Logger.java:472)
at java.util.logging.Logger.doLog(Logger.java:498)
at java.util.logging.Logger.log(Logger.java:521)
at java.util.logging.Logger.info(Logger.java:1040)
at com.sun.faban.common.RegistryImpl.main(RegistryImpl.java:427)
------------------------------------------------------------------------
I am not sure why the connection to sc-h03 is not happening. This is
the
run configuration.
/usr/lib/jvm/java-Xmx3g -Xms256m -XX:+DisableExplicitGCEnter
description
for this run heresc-h03.cs.wisc.edutrue0vmstat
10253030010/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1E/false100falsefalse63025130phpsc-h01.cs.wisc.edusc-h01.cs.wisc.edu:8080true0vmstat
10; mpstat 10; nicstat 10; iostat -x
10ApacheHttpdServiceNONEtrue/usr/local/nginx/logs/tmp/http_sessions/usr/local/nginx/conf/nginx.conf/root/jayneel/cloudsuite/webserving/web-release/app_dir/etc/php.inisc-h03.cs.wisc.edutrue0com.mysql.jdbc.Driversc-h03.cs.wisc.edu:mysql://dbHost/olio?user=olio&password=olio&relaxAutoCommit=true&sessionVariables=FOREIGN_KEY_CHECKS=0truefalse25MySQLServiceMysqlstatsfalse/var/mysqlolioolio/etc/my.cnfsc-h01.cs.wisc.edutrue0NONEfalse/root/jayneel/cloudsuite/webserving/web-release/filestoretrue0NONEMemcachedServiceMemcacheStats
-i 10true/usr/lib/memcached256
Both the machines are in the same network and can connect to each
other.
Do have some suggestion?
Thanks,
Jayneel
On 07/04/2012 05:28 PM, Cansu Kaynak wrote:
It seems like a configuration field was left empty or has an
inappropriate value while setting up the test run.
You can find the details about the cause of the failure if you
click
on the "exception" links on the page you posted.
After making sure that you filled out all the fields as described
in
the instructions, if you still get the same error, you can send
your
configuration parameters.
--
Cansu
On Jul 5, 2012, at 12:17 AM, Jayneel Gandhi wrote:
Hi,
I installed the web serving benchmark as specified on the
cloudsuite
webpage and tried to perform a benchmark run. The run failed
saying
"Benchmark Validation Failed". The stack providing some java
thread
stack.
Time Host Level Message
17:16:26 INFO Starting OlioDriver run using
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1A/
17:16:26 INFO START TIME : Wed Jul 04 17:16:26 CDT 2012
17:16:26 SEVERE/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1A&exception=2>/
Benchmark validation failed.
17:16:26 SEVERE/
exception
<http://sc-h03.cs.wisc.edu:9980/LogReader?runId=OlioDriver.1A&exception=3>/
FenXi process command [fenxi, process,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1A/,
/root/jayneel/cloudsuite/webserving/web-release/faban/output/OlioDriver.1A//post/,
OlioDriver.1A] failed.
17:16:26 INFO Killing benchmark
17:16:26 INFO OlioDriver Completed/Terminated
Do you guys have have some pointers that I can look into to fix
the
issue?
Thanks,
Jayneel
