hi,
i encountered a strange problem recently,
i have two hosts with GT4.0.2 installed,when i tried to submit two same jobs
with globusrun-ws to the two hosts.I got the different results.One of the
jobs finished successfully,and the other failed.
The rsl file's content and the output information are as follows:
=================================================================================
rsl file's content:
+++++++++++++++++++++++
<job>
<factoryEndpoint xmlns:gram="
http://www.globus.org/namespaces/2004/10/gram/job"; xmlns:wsa="
http://schemas.xmlsoap.org/ws/2004/03/addressing";>
<wsa:Address>
https://202.118.236.136:8443/wsrf/services/ManagedJobFactoryService
</wsa:Address>
<!--The only difference bettwen the two jobs is just the
destnation IP address-->
<!--I got host 1:202.118.236.136 and host 2:202.118.236.137--
>
<wsa:ReferenceProperties>
<gram:ResourceID>Fork</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<executable>/home/wtk1984/job/seqret</executable>
<directory>/home/wtk1984/job/</directory>
<argument>fasta::sodium.fasta</argument>
<argument>phylip::test1.phy</argument>
<argument>-feature</argument>
<argument>-firstonly</argument>
<stdout>/home/wtk1984/job/globusJob/test1.stdout</stdout>
<stderr>/home/wtk1984/job/globusJob/test1.stderr</stderr>
<fileStageIn>
<transfer>
<sourceUrl>gsiftp://202.118.236.132:2811/home/wtk1984/job/sodium.fasta</sourceUrl>
<destinationUrl>file:///home/wtk1984/job/aaa1204089258721.fasta</destinationUrl>
</transfer>
</fileStageIn>
<fileStageOut>
<transfer>
<sourceUrl>file:///home/wtk1984/job/test1.phy</sourceUrl>
<destinationUrl>gsiftp://202.118.236.132:2811/tmp/test.ans</destinationUrl>
</transfer>
</fileStageOut>
</job>
++++++++++++++++++++++++++++++
The failed job's output informations:
++++++++++++++++++++++++++++++
$ globusrun-ws -submit -f /tmp/job1.xml -S
Delegating user credentials...Done.
Submitting job...Done.
Job ID: uuid:ad3505de-ee8f-11dc-aef3-0002b0bc63c9
Termination time: 03/11/2008 10:49 GMT
Current job state: StageIn
Current job state: Failed
Destroying job...Done.
Cleaning up any delegated credentials...Done.
globusrun-ws: Job failed: Staging error for RSL element fileStageIn.
; nested exception is:
java.lang.ClassCastException:
com.sun.xml.internal.messaging.saaj.soap.ver1_1.Message1_1Impl cannot be
cast to org.apache.axis.Message
===============================================================================================
And i wonder what's wrong with the RFT,so i did a test about rft client.
I submited two file transfer jobs to the two hosts, and unfortunately both
of the jobs are failed!
i used the command, " rft -z host -f /tmp/rftjob.txt" to execute my job
and the failed output informations are as follows:
===============================================================================================
the content of rftjob.txt:
+++++++++++++++++
-rw-rw-rw- 1 wtk1984 wtk1984 3356 Mar 9 13:04 search.jsp
true
16000
16000
false
1
true
1
null
null
false
10
gsiftp://202.118.236.132:2811/tmp/rft.txt
gsiftp://202.118.236.137:2811/tmp/rftTest_Done.tmp
++++++++++++++++++++++++++++++++++++++++
and the content of the /etc/hosts:
127.0.0.1 localhost.localdomain localhost
202.118.236.132 server132 # The submitting host
202.118.236.136 impact
202.118.236.137 speed
++++++++++++++++++++++++++++++++++++++++
the failed output information:
+++++++++++++++++++++++++++++++++++++++++
2008-03-10 19:26:58,385 ERROR
delegation.DelegationUtil[main,getCertificateChainRP:558]
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Operation unauthorized (Mechanism level: Authorization failed. Expected
"/CN=host/localhost.localdomain" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
Exception in thread "main" Error during startup processing. Caused by .
Caused by AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: org.globus.common.ChainedIOException: Authentication failed
[Caused by: Operation unauthorized (Mechanism level: Authorization failed.
Expected "/CN=host/localhost.localdomain" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:Authentication failed.
Caused by GSSException: Operation unauthorized (Mechanism level:
Authorization failed. Expected "/CN=host/localhost.localdomain"
target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")
at org.globus.gsi.gssapi.GlobusGSSContextImpl.initSecContext(
GlobusGSSContextImpl.java:509)
at org.globus.gsi.gssapi.net.GssSocket.authenticateClient(
GssSocket.java:102)
at org.globus.gsi.gssapi.net.GssSocket.startHandshake(GssSocket.java
:140)
at org.globus.gsi.gssapi.net.GssSocket.getOutputStream(
GssSocket.java:161)
at org.apache.axis.transport.http.HTTPSender.writeToSocket(
HTTPSender.java:433)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:135)
at org.apache.axis.strategies.InvocationStrategy.visit(
InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
at org.apache.axis.client.Call.invoke(Call.java:2710)
at org.apache.axis.client.Call.invoke(Call.java:2386)
at org.apache.axis.client.Call.invoke(Call.java:2309)
at org.apache.axis.client.Call.invoke(Call.java:1766)
at
org.globus.delegationService.DelegationFactoryPortTypeSOAPBindingStub.getResourceProperty
(DelegationFactoryPortTypeSOAPBindingStub.java:1323)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:556)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:514)
at
org.globus.transfer.reliable.client.BaseRFTClient.delegateCredential(
BaseRFTClient.java:294)
at
org.globus.transfer.reliable.client.ReliableFileTransferClient.main(
ReliableFileTransferClient.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.globus.bootstrap.BootstrapBase.launch(BootstrapBase.java:95)
at org.globus.bootstrap.Bootstrap.main(Bootstrap.java:37)
{http://xml.apache.org/axis/}hostname:server132
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Operation unauthorized (Mechanism level: Authorization failed. Expected
"/CN=host/localhost.localdomain" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:144)
at org.apache.axis.strategies.InvocationStrategy.visit(
InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
at org.apache.axis.client.Call.invoke(Call.java:2710)
at org.apache.axis.client.Call.invoke(Call.java:2386)
at org.apache.axis.client.Call.invoke(Call.java:2309)
at org.apache.axis.client.Call.invoke(Call.java:1766)
at
org.globus.delegationService.DelegationFactoryPortTypeSOAPBindingStub.getResourceProperty
(DelegationFactoryPortTypeSOAPBindingStub.java:1323)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:556)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:514)
at
org.globus.transfer.reliable.client.BaseRFTClient.delegateCredential(
BaseRFTClient.java:294)
at
org.globus.transfer.reliable.client.ReliableFileTransferClient.main(
ReliableFileTransferClient.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.globus.bootstrap.BootstrapBase.launch(BootstrapBase.java:95)
at org.globus.bootstrap.Bootstrap.main(Bootstrap.java:37)
Caused by: org.globus.common.ChainedIOException: Authentication failed
[Caused by: Operation unauthorized (Mechanism level: Authorization failed.
Expected "/CN=host/localhost.localdomain" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
at org.globus.gsi.gssapi.net.GssSocket.startHandshake(GssSocket.java
:145)
at org.globus.gsi.gssapi.net.GssSocket.getOutputStream(
GssSocket.java:161)
at org.apache.axis.transport.http.HTTPSender.writeToSocket(
HTTPSender.java:433)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:135)
... 20 more
===============================================================================================
i thought there is something wrong with my /etc/hosts file, so i modified it
like this:
/etc/hosts
++++++++++++++++++++++++++++++++++++
127.0.0.1 localhost
202.118.236.132 server132 # The submitting host
202.118.236.136 impact
202.118.236.137 speed
++++++++++++++++++++++++++++++++++++++++
To my disappointment,It's not work, too.
And the failed information changed
from
"Expected "/CN=host/localhost.localdomain" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132"
to
"Expected "/CN=host/localhost" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")"
The details:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2008-03-10 19:46:17,929 ERROR
delegation.DelegationUtil[main,getCertificateChainRP:558]
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Operation unauthorized (Mechanism level: Authorization failed. Expected
"/CN=host/localhost" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
Exception in thread "main" Error during startup processing. Caused by .
Caused by AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: org.globus.common.ChainedIOException: Authentication failed
[Caused by: Operation unauthorized (Mechanism level: Authorization failed.
Expected "/CN=host/localhost" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:Authentication failed.
Caused by GSSException: Operation unauthorized (Mechanism level:
Authorization failed. Expected "/CN=host/localhost" target but
received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")
at org.globus.gsi.gssapi.GlobusGSSContextImpl.initSecContext(
GlobusGSSContextImpl.java:509)
at org.globus.gsi.gssapi.net.GssSocket.authenticateClient(
GssSocket.java:102)
at org.globus.gsi.gssapi.net.GssSocket.startHandshake(GssSocket.java
:140)
at org.globus.gsi.gssapi.net.GssSocket.getOutputStream(
GssSocket.java:161)
at org.apache.axis.transport.http.HTTPSender.writeToSocket(
HTTPSender.java:433)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:135)
at org.apache.axis.strategies.InvocationStrategy.visit(
InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
at org.apache.axis.client.Call.invoke(Call.java:2710)
at org.apache.axis.client.Call.invoke(Call.java:2386)
at org.apache.axis.client.Call.invoke(Call.java:2309)
at org.apache.axis.client.Call.invoke(Call.java:1766)
at
org.globus.delegationService.DelegationFactoryPortTypeSOAPBindingStub.getResourceProperty
(DelegationFactoryPortTypeSOAPBindingStub.java:1323)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:556)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:514)
at
org.globus.transfer.reliable.client.BaseRFTClient.delegateCredential(
BaseRFTClient.java:294)
at
org.globus.transfer.reliable.client.ReliableFileTransferClient.main(
ReliableFileTransferClient.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.globus.bootstrap.BootstrapBase.launch(BootstrapBase.java:95)
at org.globus.bootstrap.Bootstrap.main(Bootstrap.java:37)
{http://xml.apache.org/axis/}hostname:server132
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Operation unauthorized (Mechanism level: Authorization failed. Expected
"/CN=host/localhost" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:144)
at org.apache.axis.strategies.InvocationStrategy.visit(
InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
at org.apache.axis.client.Call.invoke(Call.java:2710)
at org.apache.axis.client.Call.invoke(Call.java:2386)
at org.apache.axis.client.Call.invoke(Call.java:2309)
at org.apache.axis.client.Call.invoke(Call.java:1766)
at
org.globus.delegationService.DelegationFactoryPortTypeSOAPBindingStub.getResourceProperty
(DelegationFactoryPortTypeSOAPBindingStub.java:1323)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:556)
at org.globus.delegation.DelegationUtil.getCertificateChainRP(
DelegationUtil.java:514)
at
org.globus.transfer.reliable.client.BaseRFTClient.delegateCredential(
BaseRFTClient.java:294)
at
org.globus.transfer.reliable.client.ReliableFileTransferClient.main(
ReliableFileTransferClient.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.globus.bootstrap.BootstrapBase.launch(BootstrapBase.java:95)
at org.globus.bootstrap.Bootstrap.main(Bootstrap.java:37)
Caused by: org.globus.common.ChainedIOException: Authentication failed
[Caused by: Operation unauthorized (Mechanism level: Authorization failed.
Expected "/CN=host/localhost" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]
at org.globus.gsi.gssapi.net.GssSocket.startHandshake(GssSocket.java
:145)
at org.globus.gsi.gssapi.net.GssSocket.getOutputStream(
GssSocket.java:161)
at org.apache.axis.transport.http.HTTPSender.writeToSocket(
HTTPSender.java:433)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java
:135)
... 20 more
================================================================================================
and i changed the "127.0.0.1 localhost" to "127.0.0.1 server132 localhost"
(server132 is the submitting host)
the jobs failed either.
and the output changed to :
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Number of transfers in this request: 1
Subscribed for overall status
Termination time to set: 60 minutes
Overall status of transfer:
Finished/Active/Failed/Retrying/Pending
0/0/1/0/0
Error:Error authenticating user at source/dest hostAuthentication failed
[Caused by: Operation unauthorized (Mechanism level: Authorization failed.
Expected "/CN=host/202.118.236.132" target but received
"/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")] [Caused
by: Authentication failed [Caused by: Operation unauthorized (Mechanism
level: Authorization failed. Expected "/CN=host/202.118.236.132" target but
received "/O=Grid/OU=GlobusTest/OU=simpleCA-server132/CN=host/server132")]]
All Transfers failed !
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
===============================================================================================
After that ,i was so angry,feeling helpless.
I know it's wrong but i have no choice.I changed the /etc/hosts again :
++++++++++++++++++++++++++++++
/etc/hosts
127.0.0.1 server localhost # server is not a host at all
202.118.236.132 server132 # The submitting host
202.118.236.136 impact
202.118.236.137 speed
+++++++++++++++++++++++++++++++++++++++++++++++++++++
oops!~~~ it works!!! And the jobs are successfully finished.
It is so weird and i can not figure out why,
can someone show me the details of how do the rft client translate host
names?
Although the rft works now, it has nothing to do with my globusrun-ws error!
I still got one job failed and the other successfully finished.the output
informations are the same to the former ones.
++++++++++++++++++++++++++++++++++++++++++++
$globusrun-ws -submit -f /tmp/job1.xml -S
Delegating user credentials...Done.
Submitting job...Done.
Job ID: uuid:c7654b9c-ee9b-11dc-8e1d-0002b0bc63c9
Termination time: 03/11/2008 12:16 GMT
Current job state: StageIn
Current job state: Failed
Destroying job...Done.
Cleaning up any delegated credentials...Done.
globusrun-ws: Job failed: Staging error for RSL element fileStageIn.
; nested exception is:
java.lang.ClassCastException:
com.sun.xml.internal.messaging.saaj.soap.ver1_1.Message1_1Impl cannot be
cast to org.apache.axis.Message
My project is due in several days.
Any advices is greatly appreciated.
thanks
