Re: Completely transform a request
I'm doing some testing here on the latest build from trunk. Will let you know ASAP whether this is going to be possible from solely within a connection input filter or whether you will need other "hooks" to pull it off. In the meantime... if someone else is more familiar with connection input filters and already knows there is no way to do this given the current design and implementation it would cut some corners to hear "it can't be done". You MAY have found a bug in connection input filtering. Maybe not. It's worth a look to see if that's the case. Kevin Kiley In a message dated 7/31/2007 5:53:10 AM Pacific Standard Time, [EMAIL PROTECTED] writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > People can get kinda short and blunt over here but be advised that the > only bad discussion about technology is not having one at all and, > in general, the "constructive criticism" is all well-intentioned. Well, then we might just have to continue discussing this technology. I'd repost my original question, and I kindly ask everyone just to forget all the OpenPGP stuff. I want to Completely Transform a Request. 100% transformation. Based on a certain logic, If an incoming request matches one of my action triggers, then I want to apply a transformation to the 100% of the incoming request. I know I can do that when I just want to modify brigade-by-brigade. But I need to read the WHOLE request before doing so. Even the METHOD line. Even the headers. Even the body. All of it. Then, completely transform that into another request, and have Apache process it. With the current input filtering framework, at the connection level, I should be able to do it. But I can't. If you NEED an example of what I'd like to transform, and into WHAT i want to transform it, see this post: What I'd like to transform: http://www.mail-archive.com/dev@httpd.apache.org/msg37206.html Into WHAT I want to transform it: a completely different request (i.e different method line, different headers and different body, and I can't do that in stages, I have to read the whole request first). Sincerely, - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGry75AlpOsGhXcE0RChjtAJ4kXkrjZKyJ5iG1Qtbtge2HEXFySQCdHej3 52H1qN3dNLZa7fL8/Bre8BI= =ooIU -END PGP SIGNATURE- ** Get a sneak peek of the all-new AOL at http://discover.aol.com/memed/aolcom30tour
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > People can get kinda short and blunt over here but be advised that the > only bad discussion about technology is not having one at all and, > in general, the "constructive criticism" is all well-intentioned. Well, then we might just have to continue discussing this technology. I'd repost my original question, and I kindly ask everyone just to forget all the OpenPGP stuff. I want to Completely Transform a Request. 100% transformation. Based on a certain logic, If an incoming request matches one of my action triggers, then I want to apply a transformation to the 100% of the incoming request. I know I can do that when I just want to modify brigade-by-brigade. But I need to read the WHOLE request before doing so. Even the METHOD line. Even the headers. Even the body. All of it. Then, completely transform that into another request, and have Apache process it. With the current input filtering framework, at the connection level, I should be able to do it. But I can't. If you NEED an example of what I'd like to transform, and into WHAT i want to transform it, see this post: What I'd like to transform: http://www.mail-archive.com/dev@httpd.apache.org/msg37206.html Into WHAT I want to transform it: a completely different request (i.e different method line, different headers and different body, and I can't do that in stages, I have to read the whole request first). Sincerely, - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGry75AlpOsGhXcE0RChjtAJ4kXkrjZKyJ5iG1Qtbtge2HEXFySQCdHej3 52H1qN3dNLZa7fL8/Bre8BI= =ooIU -END PGP SIGNATURE-
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > That's all good news. You're almost there. In case you want to take a look, you can check out the filter function from http://www.buanzo.com.ar/files/openpgp.conn.filter.in.c > You have the 'content' encryption/decryption part > worked out but you want to also make sure no one > between the client and the COS ( Content Origin Server ) > can ever "see" where someone is going or what they > are asking for. That's why it is encrypted, yes. It comes out from the browser (or local pgp-http proxy, as we were discussing earlier), passess through any proxy because it is still valid HTTP 1.1, and when it gets to the server, even before virtual host processing (so the unecrnypted Host: header can say anything, as long as it reaches the "correct" server to which the request was encrypted to), it gets decrypted, "replaced", and then processed, and served. The answer will be encrypted to the sender, of course, and signed with the server's key, so both server and client can privately communicate and also verify their identities. It's using PGP for HTTP. > Your "POST /HTTP_OPENPGP_DECRYPT HTTP/1.1" > approach will, of course, always reveal the actual server Well, no, the encrypted Host: header might point the server to one of its virtual hosts. See above. > Let's define what you are really trying to do and > maybe the reason you are having trouble implementing > it in Apache will be a little clearer. OK. > The word for this is "tunnelling". I was pretty sure tunneling, when talking about HTTP, had to do with the CONNECT method. But, if you are applying it here to simplify my concept, then, yes, I could accept it. Yes, as the HTTP request that holds the encrypted one IS valid HTTP and proxies will be able to read it, not decrypt the encapsulated request. Yes, this breaks a couple of things, but we don't proxy cache SSL, do we? I'm starting to feel I should develop httpp as you mentioned. Actually, it was my first damned idea. I've been reanalyzing this all over the past year and a half. So, we are tunneling an encrypted request using a capsule request. > You want to "tunnel" the real (encrypted) request > through the non-secure HTTP protocol using a > "fake" request that appears to be un-encrypted. I dropped that idea because writing a mozilla firefox addon to handle that was too big a pain in the back just to get a Proof of Concept application working. If the http experts here really consider I should be using a brand new "httpp" LOC, then I'll analyze, further discuss, and finally (probably) go for it. > It's classic tunnelling with a twist. > You want to both "tunnel" and "redirect". OK. > You SHOULD be able to do this, if you want, however > quirky some might think the approach is. I'm open to suggestions on how to properly implement an OpenPGP encryption/decryption layer to HTTP. I really am. I don't want to have all the answers. I had an idea, with benefits and disadvantages over SSL, that uses OpenPGP. I believe it is a good thing, and I'm open to any suggestions on how to correctly/properly implement it, because it might be useful to some, like banks. Take into account that no user and password would be required to login to a site by using this, and that even the OpenPGP signed http request can go through SSL isntead of using OpenPGP encryption, makes it flexible enough, and secure enough, to make things like phishing disappear. > By choosing to "tunnel", however, you are missing > the opportunity to "answer the phone". I don't agree. The request gets to Apache. Apache decrypts it, and answers the phone on the encapsulated request. If an input filter can't get me there then I have to 1) implement this using a different approach, or 2) make it a horrible hack to make it work in Apache, 3) drop Apache and use a proxy in front of apache that decrypts the request, then gives the unencrypted one to apache to serve. I'd really like to have 1 (and 3, because it would work for pre 2.0 Apache and other webservers). - From what you say, I CAN do 1 (mod-rewrite stuff, internal redirect, etc) easily. But the fact that a connection input filter can't COMPLETELY transform a request, INCLUDING the method line, then serve this new, transformed, request, sounds weird to me. > Since the initial request appears to be a normal > HTTP request then all of Apache's normal http input > handlers are going to kick in right at the front door. > Keep in mind that the first line of input for any > HTTP server is not considered a "Header" at all. > > It is called the "HTTP method line" and by the time > it is parsed by the server you have a whole bunch > of server variables that have been intialized and > the server now thinks it "knows where it's going". > > What FOLLOWS the "HTTP method line" are things > called "input headers". Quoting Nick's book: ? AP_FTYPE_CONNECTION filters operate on connections, at the TCP
Re: Completely transform a request
> Arturo wrote... > > Thanks for taking the time to discuss this with me. > I really appreciate it. > > I was able to read a whole "pgp-encrypted" request, > even a large 12+MB one using my code. I read the > content-length header, then read up to that quantity of > bytes, saving the brigades to a context brigade. > Of course, I just DECLINE when the request is not > a "POST /HTTP_OPENPGP_DECRYPT HTTP/1.1" one. That's all good news. You're almost there. >> Just send your headers normally, add the encrypted BODY, >> and let the Server side conn-filter do it's thing on >> the BODY data. >> >> Why isn't that "good enough?" > > Privacy. Ah... Ok... I get it now. You have the 'content' encryption/decryption part worked out but you want to also make sure no one between the client and the COS ( Content Origin Server ) can ever "see" where someone is going or what they are asking for. Your "POST /HTTP_OPENPGP_DECRYPT HTTP/1.1" approach will, of course, always reveal the actual server being contacted unless the target is just running as a portal in some "other" domain... so all you are really trying to mask is the actual DOCUMENT being requested and, perhaps, any associated QUERY parms. Right? > I really want to put headers and URI, and body if > applicable. That's why the special POST request URI > I'm using has minimal headers. The real headers, > the real body, the real URI is inside the > encrypted body. Ok... let's take a deep breath here. Let's define what you are really trying to do and maybe the reason you are having trouble implementing it in Apache will be a little clearer. The word for this is "tunnelling". You don't want to implement your own LOC ( Left of Colon ) protocol named "httpp" ( http + PGP ) like Secure Socket Layer does ( https = http + SSL ). You want to "tunnel" the real (encrypted) request through the non-secure HTTP protocol using a "fake" request that appears to be un-encrypted. The problem you are running into is that you want to let Apache's normal "http" protocol handler "answer the phone" and then "rip the rug out" and create the "real" request and discard the "dummy" one. It's classic tunnelling with a twist. You want to both "tunnel" and "redirect". No problem. You SHOULD be able to do this, if you want, however quirky some might think the approach is. By choosing to "tunnel", however, you are missing the opportunity to "answer the phone". Since the initial request appears to be a normal HTTP request then all of Apache's normal http input handlers are going to kick in right at the front door. I don't think just a simple "filter" is going to do all the job for you, in this case. Keep in mind that the first line of input for any HTTP server is not considered a "Header" at all. It is called the "HTTP method line" and by the time it is parsed by the server you have a whole bunch of server variables that have been intialized and the server now thinks it "knows where it's going". What FOLLOWS the "HTTP method line" are things called "input headers". So even if you figure out how to vaporize the inbound "fake" headers and replace the "buckets" with your own you are still going to have to do something else to pull off your "redirect" to the "secret URI" regardless of how you find out what it really is. Just to satisfy my own curiosity I worked up a module here that is, in fact, able to do this. It is by no means a working PGP demonstration but it does do a simple imitation of what you are trying to do. In other words, I posted myself some gobbly-gook which has an actual target URI in it and not only am I able to "filter" the gobbly-gook and turn it back into non-gobbly-gook I was able to simply do a standard "internal redirect" to the URI once I pulled it out of the non-gobbly-gook. It's called an "internal-redirect". Apache has been able to do this for a long time and you need to take a hard look at it and see if it will work for you. The trick is that you are going to have to do the same things that the existing Apache module mod_rewrite does. It's the same concept as mod_rewrite only it needs to happen a little later than usual. Take a close look at mod_rewrite to see all of the API calls it makes. You can make these same calls yourself from within your filter and make it appear as if mod_rewrite actually "ran" on your request. So I was able to send a "fake" (dummy) request into Apache, start filtering my posted "gobbly-gook", pull a real destination URI out of the "gobbly-gook" and then tell Apache to "redirect" to it. It seems to work fine. What this does NOT do is address your reported problem that you are having trouble vaporizing existing request HEADER buffers and then replacing them with your own. You may, in fact, have found a bug there but it may also simply be that you are "too late in the game" to pull this from within a BODY data input filter. So the vaporization of existing input req
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 William A. Rowe, Jr. wrote: > +1 :) > Actually, I gave the wrong RFC - take a look at > http://www.ietf.org/rfc/rfc2817 > Simply using an 'Upgrade: PGP' header would be enough to make the transition. [...] > same hooks to interpret the Upgrade header. WAY interesting. I'll see if I can come up with a scheme that utilizes it. Thanks for the pointer, William. - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq6FQAlpOsGhXcE0RCl4oAJ9wNSPQdBoomOsE62fWyDET6qIvEwCbB7sB Sp/SJIrGKmAweV3/xkt2lhU= =fGGb -END PGP SIGNATURE-
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > People can get kinda short and blunt over here but be advised that the > only bad discussion about technology is not having one at all and, > in general, the "constructive criticism" is all well-intentioned. I know, I really know! :) I was being thankful! :) I've been into development mailing lists since I was 14 years old. I started out with GNU/Linux in September 1994, back in the 1.x days. I've been contributing in many different ways (Nmap, Audacity, Samba-vscan, some translation work, beta-testing, giving talks, even my rock band licenses music using the creative commons by-sa license!). > Ultimately, I think that's the way to go. You should have that setup > available regardless of anything else you cook up. Yes, I will do that. Definitely! Anyway, I really wanted to learn how to program Mozilla Firefox extensions, and Apache modules. And I'm loving it. > Even if your "standards" are approved by all the various parties involved [...] > out for all the clients, probably. That's just the way it is. You're right. I need to provide solid tools in the meantime. > It's still impossible to get all major clients up to speed on HTTP/1.1 > RFCs alone, much less throw in something like this for them to implement. :) > I'm not trying to poo-poo your current plans. Yes, that's what I've understood. Thanks for taking the time to discuss this with me. I really appreciate it. > Well, if it's a request headed at an Apache server then sooner or later > you are > going to have deal with 'chunking'. If you are going to allow HUGE posts > up to > Apache with your OpenPGP then at some point it better be able to 'chunk' > or you'll have to force the Server to think you are, at all times, a > legacy HTTP > 1.0 client and then you can get conflict errors trying to use other headers. I was able to read a whole "pgp-encrypted" request, even a large 12+MB one using my code. I read the content-length header, then read up to that quantity of bytes, saving the brigades to a context brigade. Of course, I just DECLINE when the request is not a "POST /HTTP_OPENPGP_DECRYPT HTTP/1.1" one. >>...take into account I've already implemented reading and decrypting the >> request withing the conn filter. > > Ok. So help me out here. Why isn't that enough? Apache wants headers in their own bucket, and even in their own brigades. Each header. When I create a bucket containing the whole decrypted request (take into account I've destroyed all previous brigades, clean-ed up the last one) into the last brigade I've get_brigaded()ed, it just doesn't work. So, I started making tests. Eat the whole encrypted request, replacing it with a simple "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n" one. It didn't work. So I went piece by piece. One bucket for "GET / HTTP/1.1\r\n", another for "Host: localhost\r\n", then another for "\r\n". When I did that, I got this on the logs: [Wed Jul 18 19:55:08 2007] [error] [client 127.0.0.1] request failed: error reading the headers These are different tests: 127.0.0.1 - - [20/Jul/2007:18:16:15 -0300] "GET / HTTP/1.0\r\n" 400 293 127.0.0.1 - - [20/Jul/2007:18:16:48 -0300] "GET / HTTP/1.0" 400 293 127.0.0.1 - - [20/Jul/2007:18:24:42 -0300] "GET / HTTP/1.0\r\nHost: localhost\r\n" 400 293 > I think your problem at the moment might be in how you are implementing > things on the CLIENT side. For the moment, this is my client side code: cat hand-crafted-encrypted-request | nc localhost 80 The hand-crafted request is the one I showed earlier (the one with HTTP/1.1, content-length, host and connection headers). > Just send your headers normally, add the encrypted BODY, and let the > Server side conn-filter do it's thing on the BODY data. > > Why isn't that "good enough?" Privacy. I really want to put headers and URI, and body if applicable. That's why the special POST request URI I'm using has minimal headers. The real headers, the real body, the real URI is inside the encrypted body. > If there is any piece of information outside the "BODY" that you > are trying to "hide" from normal view then why don't you just add > it to the normal request headers as an "encrypted field"... without > trying to encrypt ALL of the headers and create the "Trojan Horse" > scenario on the Server side? I thought about it, only encrypting the required headers, body, and maybe the itself, or the query string if it is a GET request. But it would add LOTS of overhead. A simple 2 character string could become a 1024 bytes monster when using openpgp. Encrypting the whole request allows for less overhead, more privacy, less CPU usage on both sides (less number of encryptions per request, from N-encryptions/req, to one e/req). - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futuraban
Re: Completely transform a request
[EMAIL PROTECTED] wrote: >> That's why I thought bringing the concept over here was a good idea. >> I'm finally getting some constructive criticism! > It's an interesting idea. There have been years of work put into making > HTTP and Apache "extensible" for ideas just such as this one and regardless > of what anyone thinks of your idea if you can't implement it easily > right now > then the ball just isn't into the end-zone yet on the Server design. > > People can get kinda short and blunt over here but be advised that the > only bad discussion about technology is not having one at all and, > in general, the "constructive criticism" is all well-intentioned. +1 Actually, I gave the wrong RFC - take a look at http://www.ietf.org/rfc/rfc2817 Simply using an 'Upgrade: PGP' header would be enough to make the transition. All the code for rfc 2817 is already in mod_ssl... perhaps some of the more generic handling of 'Connection: Upgrade' could be moved up in the protocol layer, and mod_ssl and your mod_pgp/gpg module could both leverage the very same hooks to interpret the Upgrade header. Bill
Re: Completely transform a request
> That's why I thought bringing the concept over here was a good idea. > I'm finally getting some constructive criticism! It's an interesting idea. There have been years of work put into making HTTP and Apache "extensible" for ideas just such as this one and regardless of what anyone thinks of your idea if you can't implement it easily right now then the ball just isn't into the end-zone yet on the Server design. People can get kinda short and blunt over here but be advised that the only bad discussion about technology is not having one at all and, in general, the "constructive criticism" is all well-intentioned. > Not really. I have many paths. One of them is almost fully avoiding pgp from > inside a browsers plugin, and just write a proxy that implements the required > functionality, and the same in reverse at the server side, so it's more easily > plugged into any browser/httpd app. Ultimately, I think that's the way to go. You should have that setup available regardless of anything else you cook up. Even if your "standards" are approved by all the various parties involved the horse has left the barn with clients and user-agents and even if they wanted to implement your new scheme it would be years before it happened and then years after that before all the bugs were worked out for all the clients, probably. That's just the way it is. It's still impossible to get all major clients up to speed on HTTP/1.1 RFCs alone, much less throw in something like this for them to implement. I'm not trying to poo-poo your current plans. Let's assume you just want to see it work the way you are going and that it SHOULD be possible to do it with Apache's current design. > The problem with an encrypted request is that, well, it doesn't work on chunks. Well, if it's a request headed at an Apache server then sooner or later you are going to have deal with 'chunking'. If you are going to allow HUGE posts up to Apache with your OpenPGP then at some point it better be able to 'chunk' or you'll have to force the Server to think you are, at all times, a legacy HTTP 1.0 client and then you can get conflict errors trying to use other headers. But just for the sake of getting something to work... let's assume that you are just trying to package simple, short HTTP requests and they will all fit into one buffer read on the Server side. If you can't get that working then all the chunking in the world isn't going to make any difference later. >...take into account I've already implemented reading and decrypting the > request withing the conn filter. Ok. So help me out here. Why isn't that enough? I think your problem at the moment might be in how you are implementing things on the CLIENT side. If you are able to "run" the request in your Mozilla/Firefox plug-in and add the encrypted POST data to the request then why do you need the "Trojan Horse" thing? Just send your headers normally, add the encrypted BODY, and let the Server side conn-filter do it's thing on the BODY data. Why isn't that "good enough?" > If I had just to decrypt the body, it'd be quite easy. Yup. That's what filters are designed to do. WHY do you need to encrypt the headers at all? Help me understand. If there is any piece of information outside the "BODY" that you are trying to "hide" from normal view then why don't you just add it to the normal request headers as an "encrypted field"... without trying to encrypt ALL of the headers and create the "Trojan Horse" scenario on the Server side? Later... Kevin Kiley In a message dated 7/28/2007 11:31:33 AM Pacific Standard Time, [EMAIL PROTECTED] writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > Is this the way you actually plan on implementing "OpenPGP"? Not really. I have many paths. One of them is almost fully avoiding pgp from inside a browsers plugin, and just write a proxy that implements the required functionality, and the same in reverse at the server side, so it's more easily plugged into any browser/httpd app. In any case, Enigform, mod_openpgp, etc, are all things that help me find a solid base for implementing Openpgp into http. For example, the way request signing was implement is quite straight forward and http compliant. (see http://freshmeat.net/articles/view/2599). The problem with an encrypted request is that, well, it doesn't work on chunks. I could make it work on chunks, but that would include LOTS of overhead to the request. And I mean LOTS. Are you a GnuPG user? You probably know what I'm talking about if you are. > The Nestcape plugin just wakes up for every Navigation and then > rewrites the request into a POST buffer + encryption? Not Netscape, but Mozilla Firefox. > I really don't think that's going to fly in the long run. That's why I thought bringing the concept over here was a good idea. I'm finally getting some constructive criticism! > Even if you go this route... is it only go
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 William A. Rowe, Jr. wrote: > FWIW, this isn't a valid HTTP request. The bare minimum was > POST /HTTP_OPENPGP_DECRYPT HTTP/1.1 (the last bit was not optional.) You're right. I used the one from an original idea, this is what I'm inputting Apache today: POST /HTTP_OPENPGP_DECRYPT HTTP/1.1 Host: localhost Connection: close Content-Length: 1678 - -BEGIN_PGP_MESSAGE- Version: GnuPG v1.4.7 (GNU/Linux) hQIOA9YKl/p/3dcgEAf/erCrgwG8kB35bKerk3gMNqh0N2IUh2iPk3qgWsFurvOW 26nA4WU2ZmB3i5ZP4aaZwKZulsBhBA7IyX+lKbf6IyewLIaw0N/sgcoMBCOW0DmN hfJ0mgGFIGwl+uFyQoCwXk33H5j1lJidnC/AvRyqMHwxSOOXcFBuDoCVNXLiQAx8 lqKlLHtccLuG4fAAYfviqLmuK8vpFcbVURw96rh+KmWpMCY70U9JGiD/9jlcLKlo oYYkiLgJ+fDDxDwGAZ/6ryAN3tlPUyq2vLClqzbd/fgtAnTcnjhGeI3HzDUDr1aG TDPOAzpQM0ho385J4xv1ZfQRajSRY8AOcGz0s0pAgLX6wFy47IUKrsQeNMBy a/YBe4SGJyjyvDXxpUMhbftZMKDMLCL3qjfyy+v6S86i3dEI16/0a3J4ms4T7Zk5 3E08dzok+uvoLVDBJ7wpFhYACcguXogqQgkanwytW/CIzaXz43BEJnrRXXzPuzx4 N1cR2yQFqiuR+S6ycEo/qEL2XNM3rJc0ReQEPyMHzTwZhNPDXl1Zc2hjE/HjNeQy sQ70D1+KQHwFWK1w+PDNamoAM30bRmaE+HcpcowHiOi/uGMOxi5RcYRi7Ap+6yps 5inK/AGWMFGx4+zdsO+uSpmShR44O+SX6WOOBajgHHNLqZLvn1YnPdtsNkhmeLLA BNLpAT5uSv0sMBSnRq//0HhcgjRlQX9JiZzJdr1PxM7x061wTYwuWRLwWepuALG6 23Ywtmdsm+TKSn5MdDYFJFzmVKBP8lEB9yy8KeFgAWupqlm0/aXlz47ZEAds+5wi vkO5Oujm5kfR4E+hUbd0OQtvzvUnTGeh959g5P29UjR25bKWa2vgbj5ecZmE50+t QEHJYojLqZIK2JaG7E+IF5xJzZsnSJMm/UL7xrYE8rqLMHe+oz7Uj+1Ue0Nv/jOp xMFZSF/rLZsOzB4HAmLTN/RiW2K/M5YpFPmRxWHnJOeLxKgmAMY4ZG6m5/40ePlQ lKN64J6b/dOAYnEJYp/DvjZXX0t379QNzgTcsI3tQhIEsM/Dgcqe6Y3Za2JFPx74 KdvjhdpWBLCYSlnyLe5Dp69aLQMmMSNzSnj0BfWAQvKq/N4YNXk8nPo8G3oTO3hv yTcdpPVTZzTKNdUkmmC2dsEO6AXf7gdhHQrPTLXWeMfSED2O3L1p4AoQZi+cnWQI OxUklg8KoGuwKgJFIIi1aGo7aINbgfn12It9ovQA7yO459Yu6Ksd5W66cBbJbeyf pAiQTz4hu/7Hh1WOm7sIzOsglxI3C/gtG6xFBq9S6Nc13shGfY9WojVVGMUKRPt5 hmcD4bE595UcunoBb8VAKloZ15jD149fqc/evzgeMZIEpVloqd2dj98E6d0m5LPR +7NBnqaKrn+Z5lTA8z1mhMMv17pSi0XxczA/3Vs2Vn+/zpuupR7fdXZY1uiu6vGr 3SwZkRx6hJHdVA6y+J7OC5YOtBKUxTGc1N4oa1uUhhPmViwFURCuZqxqRbE= =OPnL - -END_PGP_MESSAGE- - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq5RiAlpOsGhXcE0RCg2VAJ91EmY5Kl4hjahItkMq9EIypk4UPQCcCbux sqQJ+zV5vY+Z7WkyOKH8k/A= =UDsO -END PGP SIGNATURE-
Re: Completely transform a request
Arturo 'Buanzo' Busleiman wrote: > Nick Kew wrote: >>> It's your question that I find hard to decipher. > > Yes, I'm sorry. Let me try and explain. > > For example, If I read this request with my connection input filter (which I > was able to do): > > =- cut here -= > POST /HTTP_OPENPGP_DECRYPT > Host: localhost FWIW, this isn't a valid HTTP request. The bare minimum was POST /HTTP_OPENPGP_DECRYPT HTTP/1.1 (the last bit was not optional.) Out of curiosity, is this how current browsers introduced PGP request signing support? Split headers? To remain compatible with other servers, it was trivial to simply provide a signature header. One issue though, you can't prevent minor bits from being flipped on the way thorough, so that would be a problem when passing through proxies. Second issue, you can't trust the Host: header as it wasn't crypted (or signed). Worst, you expected Connection: keep-alive to behave (it's actually redundant for HTTP/1.1) but then failed to pass Content-Length in an non-crypted form. The server would have ENOCLUE where the POST ended. That's likely to be part of your problem. This is a bad engineering design, as TOKiley points out. You really should look at Connection-Upgrade semantics from RFC 2816 for how this should have been handled. Is there an RFC on this (even draft)? Perhaps you misread it; if not, something is horribly broken in the review process under our AD :) Bill
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > Is this the way you actually plan on implementing "OpenPGP"? Not really. I have many paths. One of them is almost fully avoiding pgp from inside a browsers plugin, and just write a proxy that implements the required functionality, and the same in reverse at the server side, so it's more easily plugged into any browser/httpd app. In any case, Enigform, mod_openpgp, etc, are all things that help me find a solid base for implementing Openpgp into http. For example, the way request signing was implement is quite straight forward and http compliant. (see http://freshmeat.net/articles/view/2599). The problem with an encrypted request is that, well, it doesn't work on chunks. I could make it work on chunks, but that would include LOTS of overhead to the request. And I mean LOTS. Are you a GnuPG user? You probably know what I'm talking about if you are. > The Nestcape plugin just wakes up for every Navigation and then > rewrites the request into a POST buffer + encryption? Not Netscape, but Mozilla Firefox. > I really don't think that's going to fly in the long run. That's why I thought bringing the concept over here was a good idea. I'm finally getting some constructive criticism! > Even if you go this route... is it only going to work with Nestsape, > or something? That's not much of an "Open standard". Well, the browsers should implement the openpgp extensions, not a plugin, see this quote form an interview: TM: I understand you want it to become an accepted standard. How is the approval process going? Some guys from the IETF OpenPGP Working Group are helping me with some procedural details, but I expect to have it submitted to the IETF by July this year (2007). Hopefully, it can become an RFC. That would probably make Enigform and other plugins obsolete, because the browser itself could/should support it. I?d love to focus on the server-side, really. (taken from http://www.freesoftwaremagazine.com/blogs/interview_with_arturo_busleiman) > If you need to get in there BEFORE the intial headers are processed > look at the mod_ssl code. It does the job. OK, but take into account I've already implemented reading and decrypting the request withing the conn filter. > If you don't care about intercepting the header's for your "Trojan" > POST then a simple rewrite of mod_deflate should do the job. mod_deflate deflates the body, not the headers. That's not what I need. If I had just to decrypt the body, it'd be quite easy. I need to take a request body, decrypt it, and use that as the request. It's another concept. I might not be clear at explaining it. See my other message, with the example request. > If you really do need to "vanish" the entire original request including > the headers then you can still do that from within a non-connection > based filter but you are going to have to replace all the tables that > have already been created by Apache like request->headers_in > with your own tables. Yes, I've thought about that, too. But how can I change to which real URI the request was targeted to? I think I could use something like: 1: decrypt_body in conn filter, and tag the request 2: move headers from body to headers (in a post_read_request filter, maybe) 3: replace the "POST /HTTP_OPENPGP_DECRYPT" uri with the one that was included in the decrypted body. 4: finally, process this new request. I wanted to avoid all that, and just completely transform the request at the connection input filter stage, because no HTTP is processed by apache at that stage, yet. Sounded logical. "Get a whole request, decrypt it, then rewrite original request using the now decrypted text". > You could also just create an entire new 'sub-request' the moment > your filter wakes up and discard the original but be advised that > there are still some things that might not work as advertised for > a 'sub-request' that currenly work fine if the request is 'main'. > There are still some bugs lurking in that area. I thought about using sub-requests, too. Didn't work out. > Third alternative would be to look at the legacy "mod_gzip" code > for the Apache 1.3 series. It implements a "connection filter" just > like mod_ssl does but does NOT require EAPI or any rewrites > to standard Apache. The methods used in the legacy mod_gzip > will still work in the 2.0 series, if desired. I'll take a good look at that. Thanks for the suggestion. - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq4tQAlpOsGhXcE0RCmxdAJ9t+RLXiYxAWMMYV2yTc0vG+d36qgCfbKAH 3Yt+fZPDfOtFRNwmjmhiYDE= =7J2O -END PGP SIGNATURE-
Re: Completely transform a request
> I wrote about this last week, on dev@httpd.apache.org, with a thread whose subject > was "Introducing mod_openpgp": Yes, I saw that. It was your new question about "Posting" a "Secret" request and then trying to re-dump it into Apache as a "Trojan Horse" that had me confused. Is this the way you actually plan on implementing "OpenPGP"? The Nestcape plugin just wakes up for every Navigation and then rewrites the request into a POST buffer + encryption? I really don't think that's going to fly in the long run. Even if you go this route... is it only going to work with Nestsape, or something? That's not much of an "Open standard". As to your last question... Nick Kew's post is on the mark. Look hard and long at mod_ssl and mod_deflate. If you need to get in there BEFORE the intial headers are processed look at the mod_ssl code. It does the job. If you don't care about intercepting the header's for your "Trojan" POST then a simple rewrite of mod_deflate should do the job. If it can't then there's still something essentially wrong with the Apache 2.0 filtering design and it would be interesting to find out. If you really do need to "vanish" the entire original request including the headers then you can still do that from within a non-connection based filter but you are going to have to replace all the tables that have already been created by Apache like request->headers_in with your own tables. The memory isn't protected at that point and it's no problem to do this, but I can't think of any existing ( public ) Apache filter that does this yet. You could also just create an entire new 'sub-request' the moment your filter wakes up and discard the original but be advised that there are still some things that might not work as advertised for a 'sub-request' that currenly work fine if the request is 'main'. There are still some bugs lurking in that area. Third alternative would be to look at the legacy "mod_gzip" code for the Apache 1.3 series. It implements a "connection filter" just like mod_ssl does but does NOT require EAPI or any rewrites to standard Apache. The methods used in the legacy mod_gzip will still work in the 2.0 series, if desired. Later... Kevin In a message dated 7/28/2007 10:03:27 AM Pacific Standard Time, [EMAIL PROTECTED] writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > It is, in fact, possible to do what you are trying to do but before > anyone tells you how, in public, do you mind expaining, in public, > what the heck you are actually trying to do here? Hi :) I'm writing a draft on OpenPGP extensions to the HTTP Protocol. So far, I've implemented request signing, which enables apache to verify a request using openpgp. http://freshmeat.net/articles/view/2599 I plan on implementing encrypted requests, too. I'm also currently writing a session management extension. All client-side stuff is contained in a Firefox extension called Enigform. I wrote about this last week, on dev@httpd.apache.org, with a thread whose subject was "Introducing mod_openpgp": http://www.gossamer-threads.com/lists/apache/dev/333889 I hope the answer to "what the heck" is now clear. - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq3agAlpOsGhXcE0RCoxwAJ9KClqBRMRG3+2ASQXZ9uxUt5vC/gCfQAh6 YMYGFold1eI9tU+vsFHJc+Q= =lgFi -END PGP SIGNATURE- ** Get a sneak peek of the all-new AOL at http://discover.aol.com/memed/aolcom30tour
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Nick Kew wrote: > It's your question that I find hard to decipher. Yes, I'm sorry. Let me try and explain. For example, If I read this request with my connection input filter (which I was able to do): =- cut here -= POST /HTTP_OPENPGP_DECRYPT Host: localhost - - -BEGIN*PGP*MESSAGE- Version: GnuPG v1.4.7 (GNU/Linux) hQIOA9YKl/p/3dcgEAf/erCrgwG8kB35bKerk3gMNqh0N2IUh2iPk3qgWsFurvOW 26nA4WU2ZmB3i5ZP4aaZwKZulsBhBA7IyX+lKbf6IyewLIaw0N/sgcoMBCOW0DmN hfJ0mgGFIGwl+uFyQoCwXk33H5j1lJidnC/AvRyqMHwxSOOXcFBuDoCVNXLiQAx8 lqKlLHtccLuG4fAAYfviqLmuK8vpFcbVURw96rh+KmWpMCY70U9JGiD/9jlcLKlo oYYkiLgJ+fDDxDwGAZ/6ryAN3tlPUyq2vLClqzbd/fgtAnTcnjhGeI3HzDUDr1aG TDPOAzpQM0ho385J4xv1ZfQRajSRY8AOcGz0s0pAgLX6wFy47IUKrsQeNMBy a/YBe4SGJyjyvDXxpUMhbftZMKDMLCL3qjfyy+v6S86i3dEI16/0a3J4ms4T7Zk5 3E08dzok+uvoLVDBJ7wpFhYACcguXogqQgkanwytW/CIzaXz43BEJnrRXXzPuzx4 N1cR2yQFqiuR+S6ycEo/qEL2XNM3rJc0ReQEPyMHzTwZhNPDXl1Zc2hjE/HjNeQy sQ70D1+KQHwFWK1w+PDNamoAM30bRmaE+HcpcowHiOi/uGMOxi5RcYRi7Ap+6yps 5inK/AGWMFGx4+zdsO+uSpmShR44O+SX6WOOBajgHHNLqZLvn1YnPdtsNkhmeLLA BNLpAT5uSv0sMBSnRq//0HhcgjRlQX9JiZzJdr1PxM7x061wTYwuWRLwWepuALG6 23Ywtmdsm+TKSn5MdDYFJFzmVKBP8lEB9yy8KeFgAWupqlm0/aXlz47ZEAds+5wi vkO5Oujm5kfR4E+hUbd0OQtvzvUnTGeh959g5P29UjR25bKWa2vgbj5ecZmE50+t QEHJYojLqZIK2JaG7E+IF5xJzZsnSJMm/UL7xrYE8rqLMHe+oz7Uj+1Ue0Nv/jOp xMFZSF/rLZsOzB4HAmLTN/RiW2K/M5YpFPmRxWHnJOeLxKgmAMY4ZG6m5/40ePlQ lKN64J6b/dOAYnEJYp/DvjZXX0t379QNzgTcsI3tQhIEsM/Dgcqe6Y3Za2JFPx74 KdvjhdpWBLCYSlnyLe5Dp69aLQMmMSNzSnj0BfWAQvKq/N4YNXk8nPo8G3oTO3hv yTcdpPVTZzTKNdUkmmC2dsEO6AXf7gdhHQrPTLXWeMfSED2O3L1p4AoQZi+cnWQI OxUklg8KoGuwKgJFIIi1aGo7aINbgfn12It9ovQA7yO459Yu6Ksd5W66cBbJbeyf pAiQTz4hu/7Hh1WOm7sIzOsglxI3C/gtG6xFBq9S6Nc13shGfY9WojVVGMUKRPt5 hmcD4bE595UcunoBb8VAKloZ15jD149fqc/evzgeMZIEpVloqd2dj98E6d0m5LPR +7NBnqaKrn+Z5lTA8z1mhMMv17pSi0XxczA/3Vs2Vn+/zpuupR7fdXZY1uiu6vGr 3SwZkRx6hJHdVA6y+J7OC5YOtBKUxTGc1N4oa1uUhhPmViwFURCuZqxqRbE= =OPnL - - -END*PGP*MESSAGE- =- cut here -= When I decrypt it, I get this: =- cut here -= POST /pba/test.php HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 (example) Accept: text/html, blahblah Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://localhost/pba/index.html X-OpenPGP-Type: S X-OpenPGP-Sig-Fields: body X-OpenPGP-Sig: iD8DBQFGflnpw7MFlotPrwCeKb0qqa5Vt6eaPVaqHuUG2SVHz/c==B/eo X-OpenPGP-Digest-Algo: SHA1 X-OpenPGP-Version: GnuPG v1.4.7 (GNU/Linux) X-OpenPGP-Agent: Enigform 0.8.1 for Mozilla Firefox Cache-Control: max-age=0 Content-Type: application/x-www-form-urlencoded Content-Length: 15 variable=dsadas =- cut here -= And that's the request that should be served. > But from what you've said elsewhere, I wonder if mod_ssl or mod_deflate > might be a model for what you're doing? I've already analyzed mod_deflate, and some pieces of mod_ssl. In any case, I need to read the whole request before attempting to decrypt it, that's the nature of openpgp-encrypted data. Regarding mod_ssl, I believe that as it is stream-oriented, it encrypts by chunks, and decrypts in the same fashion, something like a direct replacement stage. - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq4QSAlpOsGhXcE0RCrPqAJwOV80dpnokNuxMabg8qjU1o6HcYgCfQpJm eBS3tHSfiRMko1qq42AtizU= =IjGO -END PGP SIGNATURE-
Re: Completely transform a request
On Sat, 28 Jul 2007 12:51:41 -0300 Arturo 'Buanzo' Busleiman <[EMAIL PROTECTED]> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Hi group, > > Sorry to bother here, but I didn't get any reply from modules-dev, so > this might be a slightly more complicated issue. If anyone would like > to help with this off-list, that would be great to avoid pestering > here :) It's your question that I find hard to decipher. But from what you've said elsewhere, I wonder if mod_ssl or mod_deflate might be a model for what you're doing? -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Completely transform a request
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 [EMAIL PROTECTED] wrote: > It is, in fact, possible to do what you are trying to do but before > anyone tells you how, in public, do you mind expaining, in public, > what the heck you are actually trying to do here? Hi :) I'm writing a draft on OpenPGP extensions to the HTTP Protocol. So far, I've implemented request signing, which enables apache to verify a request using openpgp. http://freshmeat.net/articles/view/2599 I plan on implementing encrypted requests, too. I'm also currently writing a session management extension. All client-side stuff is contained in a Firefox extension called Enigform. I wrote about this last week, on dev@httpd.apache.org, with a thread whose subject was "Introducing mod_openpgp": http://www.gossamer-threads.com/lists/apache/dev/333889 I hope the answer to "what the heck" is now clear. - -- Arturo "Buanzo" Busleiman - Consultor Independiente en Seguridad Informatica SHOW DE FUTURABANDA - Sabado 18 de Agosto 2007 (Speed King, Capital Federal) Entradas anticipadas a traves de www.futurabanda.com.ar - Punk Rock Melodico -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGq3agAlpOsGhXcE0RCoxwAJ9KClqBRMRG3+2ASQXZ9uxUt5vC/gCfQAh6 YMYGFold1eI9tU+vsFHJc+Q= =lgFi -END PGP SIGNATURE-
Re: Completely transform a request
It is, in fact, possible to do what you are trying to do but before anyone tells you how, in public, do you mind expaining, in public, what the heck you are actually trying to do here? What's in the posted body and why does it need to become the "secret (encrypted) request"? In a message dated 7/28/2007 8:52:38 AM Pacific Standard Time, [EMAIL PROTECTED] writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi group, Sorry to bother here, but I didn't get any reply from modules-dev, so this might be a slightly more complicated issue. If anyone would like to help with this off-list, that would be great to avoid pestering here :) My connection-level input filter needs to replace a request with another request. This "another request" is contained, in encrypted form, in the BODY of the POST request I'm going to replace. It's a 100% transformation. I'm using the ap_get_brigade() in a loop approach. I've been able to use ap_save_brigade, and move all the request's bucket brigades to a ctx bucket brigade, and use the last bucket-brigade of the request as starting point to insert the decrypted request. The problem is, Apache expects request headers in it's own bucket, and it's own bucket brigade. So, I can use that last bucket brigade to insert the first line of an HTTP request, but I can't add headers. I even get the 'error while reading headers" error. So, I've thought I'd just use that bucket brigade to insert the first line of request (GET /blabla HTTP/1.1\r\n), and "then", append mode bucket brigades. But HOW to actually do this is eluding me. Any ideas or different approaches to this problem? The source for the function can be obtained from: http://www.buanzo.com.ar/files/openpgp.conn.filter.in.c It's a ripoff from the whole module, so it's easier to read. Thanks! ** Get a sneak peek of the all-new AOL at http://discover.aol.com/memed/aolcom30tour
