Hi Scott
Thank you for the extensive explanation of your problem. I'm certainly
interested in reviewing your patch
Why using the usual command ::rivet::var_post (see
https://tcl.apache.org/rivet/manual3.2/var.html) to retrieve the posted
data didn't work for you?
-- Massimo
On 09/04/23 01:55, Scott Pitcher wrote:
Hi,
I've been writing a system for our website using Rivet as the
foundation. I'd written a proc for running CGI programs from the client
request, or, passing the request over. This works well but I'd recently
started to incorporate some fossil archives in the website. The fossil
CGI method worked except when I tried to clone the repo from a shell.
Fossil would report an error code and clone would fail.
I examined the website logs and it Fossil makes a POST request to the
CGI program. With wireshark I could see around 100 or so binary bytes in
the body of the POST request, below the headers:
0000 50 4f 53 54 20 2f 66 6f 73 73 2f 75 6e 69 6d 61 POST
/foss/unima 0010 6b 65 73 63 72 69 70 74 73 2e 63 67 69 2f 69 6e
kescripts.cgi/in 0020 64 65 78 20 48 54 54 50 2f 31 2e 30 0d 0a 48
6f dex HTTP/1.0..Ho 0030 73 74 3a 20 77 77 77 2e 73 65 72 76 65 72
32 2e st: www.server2. 0040 73 76 70 74 73 0d 0a 55 73 65 72 2d 41
67 65 6e svpts..User-Agen 0050 74 3a 20 46 6f 73 73 69 6c 2f 32 2e
32 30 20 28 t: Fossil/2.20 ( 0060 32 30 32 32 2d 31 31 2d 31 36 20
31 38 3a 34 36 2022-11-16 18:46 0070 3a 33 32 20 5b 32 31 30 65 38
39 61 30 35 39 5d :32 [210e89a059] 0080 29 0d 0a 43 6f 6e 74 65 6e
74 2d 54 79 70 65 3a )..Content-Type: 0090 20 61 70 70 6c 69 63 61
74 69 6f 6e 2f 78 2d 66 application/x-f 00a0 6f 73 73 69 6c 0d 0a
43 6f 6e 74 65 6e 74 2d 4c ossil..Content-L 00b0 65 6e 67 74 68 3a
20 31 30 30 0d 0a 0d 0a 00 00 ength: 100...... 00c0 00 61 78 da 05
c1 41 0a 80 20 10 00 c0 bb af 58 .axÚ.ÁA.. ..À»¯X 00d0 e8 1c ec 6e
ba e9 d1 4c ff 21 21 11 94 86 45 ef è.ìnºéÑLÿ!!...Eï 00e0 6f e6 ee
79 bf 32 6c e7 51 ea 3b 7e a5 3f 47 ab oæîy¿2lçQê;~¥?G« 00f0 c0 8c
88 c0 c8 4c 44 02 64 b5 4c ac b6 b3 d5 02 À..ÀÈLD.dµL¬¶³Õ. 0100 13
90 1a 20 a1 5d 30 08 e9 e0 8c 8e 5e 4b 8c 22 ... ¡]0.éà..^K." 0110
c6 27 f4 f3 6c 53 10 71 d6 38 b7 92 fa 01 36 4f Æ'ôólS.qÖ8·.ú.6O
0120 18 9a
I was able to retrieve the headers using ::rivet::apache_table array_get
headers_in. I wasn't able to retrieve the body of the POST request. I
tried ::rivet::raw_post but it returns an empty string each time.
I checked out the Rivet source and delved into src/mod_rivet_ng and the
Rivet_RawPost() function, and traced down to apache_request.h and
ApacheRequest_get_raw_post().
The problem is that Rivet_RawPost() uses Tcl_NewStringObj() to pass the
body back to Tcl, and given that the first 3 bytes are NUL, we end with
an empty string. I changed it to Tcl_NewByteArrayObj() and added a
length parameter which was passed by address all the way down to
ApacheRequest_get_raw_post():
data = TclWeb_GetRawPost(private->req, &length); if (!data) {
data = ""; } retval = Tcl_NewByteArrayObj((const
unsigned char *)data, length); Tcl_SetObjResult(interp, retval);
... and added a raw_length field to the ApacheRequest structure:
const char* temp_dir; char* raw_post; /* Raw post data. */ int
raw_length; request_rec* r; int nargs; } ApacheRequest;
The ApacheRequest_parse_urlencoded() function will fill this field in
just after filling in the raw_post pointer, and util_read needed a
length pointer parameter as well to get that working:
int ApacheRequest_parse_urlencoded(ApacheRequest *req) { ..... if ((rc =
util_read(req, &data, &length)) != OK) { return rc; } if (data) {
req->raw_post = (char*) data; /* Give people a way of getting at the raw
data. */ req->raw_length = length; split_to_parms(req, data); } } return
OK; }
I went to the effort because it's a simple change and I think it
completes raw_post , if that function can return the complete body.
That's just my opinion. It's important to me because I have a website
structure that can handle content, both HTTP and HTTPS at the same time,
as well as user base with login and differentiation between local and
remote access. It's been developed so we can include private content for
our business only and host files for customers both public (e.g.
documentation) and private (firmware and software downloads).
I really wanted to get fossil working under it as it streamlines that
part of the development for us. I was going to give up and host the
fossil archives separately but I persisted and came up with this. I
still haven't had time to pass the POST request into fossil but I'll do
so when I have time this weekend, but I do expect it to work, one way or
another.
Anyway. That's my situation and if it's at all any use to anyone, I'd be
more than happy to give you a patch. It's a really a minor set of
changes but they give raw_post a bit more utility.
Further: I'd really like Rivet to have a kind of "run_cgi" method, but
I've coded one in tcl at the moment and that's good for testing. At some
stage I'll write a Rivet command for doing the same which might make
environment setup and processing of the CGI output, headers etc, a bit
more efficient.
Kind regards, Scott Pitcher
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
