[Web-SIG] Python pickle and web security.
Hello, I posted this on my blog the other day about people using pickle for sessions, but got no response. Do you guys think using pickles for sessions is an ok thing to do? ... Some python web frame works are using pickle to store session data. Pickle is a well known poor choice for secure systems. However it seems to be more widely known by those writing network applications, than those making web frameworks. Is your web framework using pickle for sessions despite the warnings in the python documentation about it being insecure? By using sessions with pickle people who can write to the database servers session table can execute code on the app server. Or people who can get data into the session file/memcache data store can execute data. This might be an issue if the database server is run by separate people than the app server. Or if the session table is compromised by an sql injection attack elsewhere. There are some more secure ways of storing pickled data. Pickle is deemed to be untrustworthy for data. In that it is not certain that code can not be snuck into the data that will be executed by pickle. So if some data from user input is put into the pickle, then it is possible that code could be run. There are some people who know more about how to exploit pickle, however the warning in the python documentation is this: ""Warning: The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.""" Cerealizer might be an alternative option... http://home.gna.org/oomadness/en/cerealizer/index.html Or maybe these other two. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415503 http://barnesc.blogspot.com/2006/01/rencode-reduced-length-encodings.html ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Sep 15, 2006, at 4:29 AM, René Dudfield wrote: > Hello, > > I posted this on my blog the other day about people using pickle for > sessions, but got no response. Do you guys think using pickles for > sessions is an ok thing to do? You don't want to accept pickles from an untrusted source, which typically means you don't want to accept pickles over the network. Even then, there are ways to use pickles securely. For example, you can, if you know what you're doing, arrange to prevent pickle from calling global objects or control specifically what global objects are callable. There is nothing wrong with using pickles to store data internally. As long as the pickles are generated by the application, there is no risk to the application reading them again, assuming that they are stored where they can't be tampered with. Saying pickle is inherently insecure is like saying Python is inherently insecure. You don't want to execute Python from an untrusted source. If someone can tamper with your Python code, then you have a serious security problem as well. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Fri, 2006-09-15 at 18:29 +1000, René Dudfield wrote: > Hello, > > I posted this on my blog the other day about people using pickle for > sessions, but got no response. Do you guys think using pickles for > sessions is an ok thing to do? Either encrypt the pickle or have a seeded (md5) signature so that you can verify that the pickle has not been tampered. I use pickles routinely, but with an md5 signature that combines a seed and the pickle. Someone cannot generate a valid signature without also knowing the seed. I am paranoid enough so that I only pickle dictionaries and then only extract and verify my list of expected keys after unpickling. I can't prove that's secure, but I am not losing sleep over it. Presumably someone who knew the seed could generate a valid signature *and* inject code into the pickle that got executed by the unpickle operation. > > > > > ... > > Some python web frame works are using pickle to store session data. > Pickle is a well known poor choice for secure systems. However it > seems to be more widely known by those writing network applications, > than those making web frameworks. > > Is your web framework using pickle for sessions despite the warnings > in the python documentation about it being insecure? > > By using sessions with pickle people who can write to the database > servers session table can execute code on the app server. Or people > who can get data into the session file/memcache data store can execute > data. > > This might be an issue if the database server is run by separate > people than the app server. Or if the session table is compromised by > an sql injection attack elsewhere. > > There are some more secure ways of storing pickled data. > > Pickle is deemed to be untrustworthy for data. In that it is not > certain that code can not be snuck into the data that will be executed > by pickle. So if some data from user input is put into the pickle, > then it is possible that code could be run. > > There are some people who know more about how to exploit pickle, > however the warning in the python documentation is this: > > ""Warning: > The pickle module is not intended to be secure against erroneous or > maliciously constructed data. Never unpickle data received from an > untrusted or unauthenticated source.""" > > > Cerealizer might be an alternative option... > http://home.gna.org/oomadness/en/cerealizer/index.html > > Or maybe these other two. > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415503 > http://barnesc.blogspot.com/2006/01/rencode-reduced-length-encodings.html > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/python%40venix.com -- Lloyd Kvam Venix Corp ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
Hi, I think my main point was about using pickle for sessions, not just using pickle by itself. Unlike loading other data, code gets run when you load a pickle. It is indeed like running python code. So if you do not trust where you store your pickles to run python code, then that is a problem. If the unpickle or pickle code is not bug free, then you can not trust that unpickling a pickle will not allow data to be made which can trick the unpickle escaping code. With the history of bugs with the unpickle code, I don't think relying on it is a good idea. For a list of pickle bugs you can search the python bug tracker. There are over 70 bugs listed including the open, closed, and deleted bugs. With 13 open bugs listed. One of the bugs was closed because: 'Closing due to lack of response. cPickle is such a complex module, without a test case the leak cannot be found.' I think that line says best about how much you should trust the C module pickle code that is 5753 lines long, and has not been audited. Will pickle *always* escape data you pass it correctly when it encodes it into a pickle? Will unpickle *always* unescape parts of the pickle correctly? If not then those pickles can run code. The risk of using pickle does not seem to be worth the convenience that it gives. With alternatives to pickle which do not execute code being available why not use them? By using pickle for session data you allow people the oportunity to put data into the pickle. For example say you store a given GET variable in the session. Combining that you allow people with pickle-sessions to put data into the pickle, and the risk that pickle might not encode/decode it correctly is the problem I see. However if allowing untrusted data to be placed into a pickle is ok, then this is not a problem. That only leaves the problem of allowing the data store of your sessions to be able to execute code where you load sessions. This means you allow execution of code from your data store to your session loading code. Which means if you use a separate database machine(quite common), or if you use a separate memcache server(not unheard of) you allow these machines to execute code on the session using machine. There's a reason why people use separate user accounts, and separate machines for doing different tasks. That reason is to limit what each user or machine can do. By using pickles for sessions those benefits are removed in some cases. Cheers, On 9/15/06, Jim Fulton <[EMAIL PROTECTED]> wrote: > > On Sep 15, 2006, at 4:29 AM, René Dudfield wrote: > > > Hello, > > > > I posted this on my blog the other day about people using pickle for > > sessions, but got no response. Do you guys think using pickles for > > sessions is an ok thing to do? > > You don't want to accept pickles from an untrusted source, which > typically means you don't want to accept pickles over the network. > Even then, there are ways to use pickles securely. For example, you > can, if you know what you're doing, arrange to prevent pickle from > calling global objects or control specifically what global objects > are callable. > > There is nothing wrong with using pickles to store data internally. > As long as the pickles are generated by the application, there is no > risk to the application reading them again, assuming that they are > stored where they can't be tampered with. > > Saying pickle is inherently insecure is like saying Python is > inherently insecure. You don't want to execute Python from an > untrusted source. If someone can tamper with your Python code, then > you have a serious security problem as well. > > Jim > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
That seems like a good way to stop the untrusted session store from being able to inject sessions in there. That could at least solve the problem of using pickles from untrusted session stores. Are you just using the basic python types? eg dict, string, list, numbers etc? If so, perhaps using another serialiser will remove some more risk if you cared. On 9/15/06, Python <[EMAIL PROTECTED]> wrote: > On Fri, 2006-09-15 at 18:29 +1000, René Dudfield wrote: > > Hello, > > > > I posted this on my blog the other day about people using pickle for > > sessions, but got no response. Do you guys think using pickles for > > sessions is an ok thing to do? > > Either encrypt the pickle or have a seeded (md5) signature so that you > can verify that the pickle has not been tampered. I use pickles > routinely, but with an md5 signature that combines a seed and the > pickle. > > Someone cannot generate a valid signature without also knowing the seed. > I am paranoid enough so that I only pickle dictionaries and then only > extract and verify my list of expected keys after unpickling. I can't > prove that's secure, but I am not losing sleep over it. > > Presumably someone who knew the seed could generate a valid signature > *and* inject code into the pickle that got executed by the unpickle > operation. > > > > > > > > > > > ... > > > > Some python web frame works are using pickle to store session data. > > Pickle is a well known poor choice for secure systems. However it > > seems to be more widely known by those writing network applications, > > than those making web frameworks. > > > > Is your web framework using pickle for sessions despite the warnings > > in the python documentation about it being insecure? > > > > By using sessions with pickle people who can write to the database > > servers session table can execute code on the app server. Or people > > who can get data into the session file/memcache data store can execute > > data. > > > > This might be an issue if the database server is run by separate > > people than the app server. Or if the session table is compromised by > > an sql injection attack elsewhere. > > > > There are some more secure ways of storing pickled data. > > > > Pickle is deemed to be untrustworthy for data. In that it is not > > certain that code can not be snuck into the data that will be executed > > by pickle. So if some data from user input is put into the pickle, > > then it is possible that code could be run. > > > > There are some people who know more about how to exploit pickle, > > however the warning in the python documentation is this: > > > > ""Warning: > > The pickle module is not intended to be secure against erroneous or > > maliciously constructed data. Never unpickle data received from an > > untrusted or unauthenticated source.""" > > > > > > Cerealizer might be an alternative option... > > http://home.gna.org/oomadness/en/cerealizer/index.html > > > > Or maybe these other two. > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415503 > > http://barnesc.blogspot.com/2006/01/rencode-reduced-length-encodings.html > > ___ > > Web-SIG mailing list > > Web-SIG@python.org > > Web SIG: http://www.python.org/sigs/web-sig > > Unsubscribe: > > http://mail.python.org/mailman/options/web-sig/python%40venix.com > -- > Lloyd Kvam > Venix Corp > > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Sat, 2006-09-16 at 12:23 +1000, René Dudfield wrote: > That seems like a good way to stop the untrusted session store from > being able to inject sessions in there. That could at least solve the > problem of using pickles from untrusted session stores. > > Are you just using the basic python types? eg dict, string, list, > numbers etc? If so, perhaps using another serialiser will remove some > more risk if you cared. Besides the basic types, date/time objects are often included. My use of md5 signatures was focused primarily on preventing unwanted data manipulation. I would agree that outside data should be acquired in formats that are simpler than pickles. I am pickling data that has been checked and accepted. > > > On 9/15/06, Python <[EMAIL PROTECTED]> wrote: > > On Fri, 2006-09-15 at 18:29 +1000, René Dudfield wrote: > > > Hello, > > > > > > I posted this on my blog the other day about people using pickle for > > > sessions, but got no response. Do you guys think using pickles for > > > sessions is an ok thing to do? > > > > Either encrypt the pickle or have a seeded (md5) signature so that you > > can verify that the pickle has not been tampered. I use pickles > > routinely, but with an md5 signature that combines a seed and the > > pickle. > > > > Someone cannot generate a valid signature without also knowing the seed. > > I am paranoid enough so that I only pickle dictionaries and then only > > extract and verify my list of expected keys after unpickling. I can't > > prove that's secure, but I am not losing sleep over it. > > > > Presumably someone who knew the seed could generate a valid signature > > *and* inject code into the pickle that got executed by the unpickle > > operation. > > > > > > > > > > > > > > > > > ... > > > > > > Some python web frame works are using pickle to store session data. > > > Pickle is a well known poor choice for secure systems. However it > > > seems to be more widely known by those writing network applications, > > > than those making web frameworks. > > > > > > Is your web framework using pickle for sessions despite the warnings > > > in the python documentation about it being insecure? > > > > > > By using sessions with pickle people who can write to the database > > > servers session table can execute code on the app server. Or people > > > who can get data into the session file/memcache data store can execute > > > data. > > > > > > This might be an issue if the database server is run by separate > > > people than the app server. Or if the session table is compromised by > > > an sql injection attack elsewhere. > > > > > > There are some more secure ways of storing pickled data. > > > > > > Pickle is deemed to be untrustworthy for data. In that it is not > > > certain that code can not be snuck into the data that will be executed > > > by pickle. So if some data from user input is put into the pickle, > > > then it is possible that code could be run. > > > > > > There are some people who know more about how to exploit pickle, > > > however the warning in the python documentation is this: > > > > > > ""Warning: > > > The pickle module is not intended to be secure against erroneous or > > > maliciously constructed data. Never unpickle data received from an > > > untrusted or unauthenticated source.""" > > > > > > > > > Cerealizer might be an alternative option... > > > http://home.gna.org/oomadness/en/cerealizer/index.html > > > > > > Or maybe these other two. > > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415503 > > > http://barnesc.blogspot.com/2006/01/rencode-reduced-length-encodings.html > > > ___ > > > Web-SIG mailing list > > > Web-SIG@python.org > > > Web SIG: http://www.python.org/sigs/web-sig > > > Unsubscribe: > > > http://mail.python.org/mailman/options/web-sig/python%40venix.com > > -- > > Lloyd Kvam > > Venix Corp > > > > > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/python%40venix.com -- Lloyd Kvam Venix Corp ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Sep 15, 2006, at 7:23 PM, René Dudfield wrote: > That seems like a good way to stop the untrusted session store from > being able to inject sessions in there. That could at least solve the > problem of using pickles from untrusted session stores. > > Are you just using the basic python types? eg dict, string, list, > numbers etc? If so, perhaps using another serialiser will remove some > more risk if you cared. Why do you assume the session store is untrusted? If someone can hack into my database, they can typically hack into my web application so its pretty weird to consider the backend session store to be "untrusted". I think this is why using pickle for sessions is pretty harmless as you're the one writing to them, not the user. While I can imagine a few situations where an untrusted session store might come into play, I'd generally imagine that the vast majority of the time one does trust their session storage as much as they trust that their application can't have its source code modified. Cheers, Ben ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Mon, 2006-09-18 at 10:27 -0700, Ben Bangert wrote: > Why do you assume the session store is untrusted? If someone can hack > into my database, they can typically hack into my web application so > its pretty weird to consider the backend session store to be > "untrusted". You are assuming that the pickle is stored in a secure database. If the pickle is in a cookie or some other client side storage, then it is definitely not to be trusted. -- Lloyd Kvam Venix Corp ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Sep 18, 2006, at 2:16 PM, Python wrote: > On Mon, 2006-09-18 at 10:27 -0700, Ben Bangert wrote: >> Why do you assume the session store is untrusted? If someone can hack >> into my database, they can typically hack into my web application so >> its pretty weird to consider the backend session store to be >> "untrusted". > > You are assuming that the pickle is stored in a secure database. > If the > pickle is in a cookie or some other client side storage, then it is > definitely not to be trusted. Right. Storing pickles in cookies is a very bad idea. Hopefully, no one is doing that. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Mon, 2006-09-18 at 14:24 -0400, Jim Fulton wrote: > On Sep 18, 2006, at 2:16 PM, Python wrote: > > > On Mon, 2006-09-18 at 10:27 -0700, Ben Bangert wrote: > >> Why do you assume the session store is untrusted? If someone can hack > >> into my database, they can typically hack into my web application so > >> its pretty weird to consider the backend session store to be > >> "untrusted". > > > > You are assuming that the pickle is stored in a secure database. > > If the > > pickle is in a cookie or some other client side storage, then it is > > definitely not to be trusted. > > Right. Storing pickles in cookies is a very bad idea. > Hopefully, no one is doing that. As it happens, I am not using cookies to store pickles, but I've considered it. What makes it "a very bad idea"? > > Jim > > -- > Jim Fultonmailto:[EMAIL PROTECTED]Python > Powered! > CTO (540) 361-1714 > http://www.python.org > Zope Corporation http://www.zope.com http://www.zope.org > > > -- Lloyd Kvam Venix Corp ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Python pickle and web security.
On Sep 18, 2006, at 2:34 PM, Python wrote: > On Mon, 2006-09-18 at 14:24 -0400, Jim Fulton wrote: >> On Sep 18, 2006, at 2:16 PM, Python wrote: >> >>> On Mon, 2006-09-18 at 10:27 -0700, Ben Bangert wrote: Why do you assume the session store is untrusted? If someone can hack into my database, they can typically hack into my web application so its pretty weird to consider the backend session store to be "untrusted". >>> >>> You are assuming that the pickle is stored in a secure database. >>> If the >>> pickle is in a cookie or some other client side storage, then it is >>> definitely not to be trusted. >> >> Right. Storing pickles in cookies is a very bad idea. >> Hopefully, no one is doing that. > > As it happens, I am not using cookies to store pickles, but I've > considered it. What makes it "a very bad idea"? Because, by default, a pickle can be constructed that will call more or less any importable object. You never want to load pickles from an untrusted source and, as you pointed out, cookies are an untrusted source. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
