John Caruso et al,
Could xotcl-core[2] be the work around/solution to this SSL
ns_httpsget bug[1]?
cheers,
Torben
1. On July 15 2009 John Caruso wrote:
> We've run into a bug with AOLserver 4.5.1 / nsopenssl 3.0beta26. The
> bug is fully documented here:
>
> https://sourceforge.net/tracker/?
func=detail&aid=2822117&group_id=3152&atid=103152
>
>
> But the short version is that when using the nsopenssl client-side
> routines (e.g. ns_httpsget), the result may be truncated if the
> client starts reading before all of the data has been received. This
> bug ONLY occurs with an AOLserver client (any version) running
> against an AOLserver 4 / nsopenssl 3.0beta26 server. We've
> reproduced the bug on RHEL4, RHEL5, and Mac OS X.
>
> The bug is easily demonstrated by copying the file I've attached to
> this message (sslbug.tcl) to the top-level context of a web server
> running AOLserver 4.x/nsopenssl 3.0beta26 and then navigating to
> https://<server>/sslbug.tcl. If you comment out the ns_httpsget and
> use ns_httpget instead, you'll see that the bug disappears.
>
> We've done a lot of instrumenting of nsopenssl/AOLserver, but haven't
> been able to track down the root cause. It seems likely that it's
> related to data buffering, which seems like it would be occurring
> within AOLserver or Tcl...but the issue is definitely specific to
> SSL, which implies that it's something in nsopenssl 3.0beta26.
>
> Does anyone have any idea what might be causing this problem?
>
> - John
>
>
> -- AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[email protected]> with the body of "SIGNOFF AOLSERVER" in
> the email message. You can leave the Subject: field of your email
> blank.
2. On Feb 4, 2010, at 1:21 AM, Gustaf Neumann wrote:
Dear Brian and all,
xotcl-core of OpenACS contains a full set of HTTP client
support, including POST requests and https (via the
tcl tls package). Below is the description included in the
source ....
best regards
-gustaf neumann
===============================================
# Defined classes
# 1) HttpCore (common base class)
# 2) HttpRequest (for blocking requests + timeout support)
# 3) AsyncHttpRequest (for non-blocking requests + timeout support)
# 4) HttpRequestTrace (mixin class)
# 5) Tls (mixin class, applicable to various protocols)
#
######################
#
# 1 HttpRequest
#
# HttpRequest is a class to implement the client side
# for the HTTP methods GET and POST.
#
# Example of a GET request:
#
# set r [::xo::HttpRequest new -url http://www.openacs.org/]
#
# The resulting object $r contains all information
# about the requests, such as e.g. status_code or
# data (the response body from the server). For details
# look into the output of [$r serialize]. The result
# object $r is automatically deleted at cleanup of
# a connection thread.
#
# Example of a POST request with a form with var1 and var2
# (providing post_data causes the POST request).
#
# set r [::xo::HttpRequest new \
# -url http://yourhost.yourdomain/yourpath \
# -post_data [export_vars {var1 var2}] \
# -content_type application/x-www-form-urlencoded]
#
# More recently, we added timeout support for blocking http
# requests. By passing a timeout parameter, you gain control
# on the total roundtrip time (in milliseconds, ms):
#
# set r [::xo::HttpRequest new \
# -url http://www.openacs.org/ \
# -timeout 1500]
#
# Please, make sure that you use a recent distribution of tclthread
# ( > 2.6.5 ) to have the blocking-timeout feature working
# safely. This newly introduced feature makes use of advanced thread
# synchronisation offered by tclthread that needed to be fixed in
# tclthread <= 2.6.5. At the time of this writing, there was no
# post-2.6.5 release of tclthread, hence, you are required to
obtain a
# CVS snapshot, dating at least 2008-05-23. E.g.:
#
# cvs -z3 -d:pserver:[email protected]:/cvsroot/
tcl co \
# -D 20080523 -d thread2.6.5~20080523 thread
#
# Provided that the Tcl module tls (see e.g. http://
tls.sourceforge.net/)
# is available and can be loaded via "package require tls" into
# the aolserver, you can use both TLS/SSL secured or unsecured
requests
# in the synchronous/ asynchronous mode by using an
# https url.
#
# set r [::xo::HttpRequest new -url https://learn.wu-wien.ac.at/]
#
######################
#
# 2 AsyncHttpRequest
#
# AsyncHttpRequest is a subclass for HttpCore implementing
# asynchronous HTTP requests without vwait (vwait causes
# stalls on aolserver). AsyncHttpRequest requires to provide a
listener
# or callback object that will be notified upon success or
failure of
# the request.
#
# Asynchronous requests are much more complex to handle, since
# an application (a connection thread) can submit multiple
# asynchronous requests in parallel, which are likely to
# finish after the current request is done. The advantages
# are that the spooling of data can be delegated to a spooling
# thead and the connection thread is available for handling more
# incoming connections. The disadvantage is the higher
# complexity, one needs means to collect the received data.
#
#
# The following example uses the background delivery thread for
# spooling and defines in this thread a listener. This generic
# listener can be subclasses in applications.
#
# When using asynchronous requests, make sure to specify a listener
# for the callbacks and delete finally the request object in the
# bgdelivery thread.
#
# ::bgdelivery do ::xo::AsyncHttpRequest new \
# -url "https://oacs-dotlrn-conf2007.wu-wien.ac.at/conf2007/"; \
# -mixin ::xo::AsyncHttpRequest::SimpleListener
# -proc finalize {obj status value} { my destroy }
#
######################
#
# 3 HttpRequestTrace
#
# HttpRequestTrace can be used to trace one or all requests.
# If activated, the class writes protocol data into
# /tmp/req-<somenumber>.
#
# Use
#
# ::xo::HttpCore instmixin add ::xo::HttpRequestTrace
#
# to activate trace for all requests,
# or mixin the class into a single request to trace it.
#
Am 03.02.10 14:50, schrieb Fenton, Brian:
Good news - I found a public webservice at http://rpc.geocoder.us/
service/soap/ and ns_http works great with that with my AOLserver
version.
So, does anybody have any advice re using an SSL version? Are
sockets the way to go to do this?
thanks
Brian
________________________________________
From: AOLserver Discussion [[email protected]] On Behalf
Of Fenton, Brian [[email protected]]
Sent: 03 February 2010 11:58
To: [email protected]
Subject: Re: [AOLSERVER] differences between ns_httppost and
sockets?
Hi Jeff,
many thanks for that pointer! That makes a lot of sense.
Unfortunately it's not working for me with that particular
StockQuote web service. Now I know the web service is quite slow
but it works fine with the sockets approach. I tried different
timeouts but no joy. I wonder is it my AOLserver's (4.0.10)
version of ns_http? I can't seem to find any free web services to
test against - that webserviceX.NET is very slow.
My other question is - I'm hoping to do an SSL version once I have
the basic code working. I don't think there's a HTTPS version of
ns_http, is there? At least there is an SSL version of ns_sockopen
(ns_openssl_sockopen). Would that be the way to go?
Here's my code:
set result ""
set soap_request {<?xml version="1.0" encoding="utf-8"?
><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema";
xmlns:soap="http://schemas.xmlsoap.org/soap/
envelope/"><soap:Body><GetQuote xmlns="http://
www.webserviceX.NET/"><symbol>GOOG</symbol></GetQuote></
soap:Body></soap:Envelope>}
set type "text/xml"
set rqset [ns_set new rqset]
ns_set put $rqset SOAPAction "http://www.webserviceX.NET/GetQuote";
if {[catch {set rqid [ns_http queue POST "http://
www.webserviceX.NET/stockquote.asmx" $soap_request $rqset]} err ]} {
set result $err
ns_log Notice "Brian error result=$result"
} else {
set status [ns_http wait $rqid result 60]
ns_log Notice "Brian after wait rqid=$rqid status=$status
result=$result"
}
ns_return 200 text/plain $result
Brian
________________________________________
From: AOLserver Discussion [[email protected]] On Behalf
Of Jeff Rogers [[email protected]]
Sent: 03 February 2010 07:12
To: [email protected]
Subject: Re: [AOLSERVER] differences between ns_httppost and
sockets?
Hi Brian,
ns_httppost does url-encoding of the name-value pairs you pass in as
qsset. For SOAP you need to just post the xml message, not as a
name-value pair. If you sniffed the actual request you're sending
with
ns_httppost it would be something like
Message=<?xml version="1.0" .....
Try using ns_http queue instead.
set rqid [ns_http queue POST http://www.webserviceX.NET/
stockquote.asmx
$soap_request $rqset]
ns_http wait $rqid result
http://panoptic.com/wiki/aolserver/Ns_http for reference.
-J
Fenton, Brian wrote:
Hi
I`m having some difficulties that hopefully somebody here could
assist me with. I`m hand-coding a request to a SOAP service and I
can`t get it working with AOLserver`s ns_httppost command, but it
works fine using sockets. I`m wondering could it be some headers
or encoding that ns_httppost does? Or maybe a timeout? I`ve seen
some articles on the web that suggest putting \r\n between the
headers and the body (which you can see in the sockets example) -
does ns_httppost automatically do that for me? If not how would I
do it?
I`m using AOLserver 4.0.10
Here`s the code using ns_httppost that doesn`t work - just
returns empty string:
set result ""
set soap_request {<?xml version="1.0" encoding="utf-8"?
><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema";
xmlns:soap="http://schemas.xmlsoap.org/soap/
envelope/"><soap:Body><GetQuote xmlns="http://
www.webserviceX.NET/"><symbol>GOOG</symbol></GetQuote></
soap:Body></soap:Envelope>}
set type "text/xml"
set rqset [ns_set new rqset]
ns_set put $rqset SOAPAction "http://www.webserviceX.NET/GetQuote";
set qsset [ns_set new qsset]
ns_set put $qsset Message $soap_request
set cmd {set page [ns_httppost "http://www.webserviceX.NET/
stockquote.asmx" $rqset $qsset $type ]}
if {[catch $cmd errmsg]} {
set result $errmsg
} else {
set result $page
}
ns_return 200 text/plain $result
And here`s the sockets program (based on some fine code written
by Tom Jackson) - this works perfectly (abeit it takes a while to
come back):
set SOAP {<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema";
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/";>
<soap:Body>
<GetQuote xmlns="http://www.webserviceX.NET/";>
<symbol>GOOG</symbol>
</GetQuote>
</soap:Body>
</soap:Envelope>}
set length [string length $SOAP]
set headers ""
set fds [ns_sockopen www.webserviceX.NET 80]
set rid [lindex $fds 0]
set wid [lindex $fds 1]
puts $wid "POST /stockquote.asmx HTTP/1.1
Host: www.webservicex.net
Content-Type: text/xml; charset=utf-8
Content-Length: $length
SOAPAction: \"http://www.webserviceX.NET/GetQuote\";
$SOAP"
flush $wid
while {[set line [string trim [gets $rid]]] != ""} {
lappend headers $line
}
set page [read $rid]
close $rid
close $wid
ns_return 200 text/plain "
Sent:
POST /stockquote.asmx HTTP/1.1
Host: www.webservicex.net
Content-Type: text/xml; charset=utf-8
Content-Length: $length
SOAPAction: \"http://www.webserviceX.NET/GetQuote\";
$SOAP
Received:
[join $headers "\n"]\n\n$page"
Any suggestions?
Many thanks,
Brian
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email
to<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave
the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email
to<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave
the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email
to<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave
the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email
to<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave
the Subject: field of your email blank.
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
