Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Oct 29, 10:10 am, Chouser <[EMAIL PROTECTED]> wrote: > Is there something wrong with JSON I'm not seeing? Not at all. I use JSON myself and am totally fine with keeping it that way. I just thought the idea of reading Clojure data on the JS side was interesting/fun. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wed, Oct 29, 2008 at 9:53 AM, Tom Hickey <[EMAIL PROTECTED]> wrote: > > Chouser, does ClojureScript not have something like this in order to > do read? It doesn't. ClojureScript currently uses the standard Clojure reader and parts of the compiler. This means, for example, that you can't have a stand-along client-side repl. The repl I use for testing accepts a string from the user, sends it to the Java/Clojure server for compilation to JavaScript, which is then sent back to the browser for evaluation. It would certainly be possible to port the reader and compiler to JavaScript as well, but I'd question the utility of this. The browsers will always have faster and better support for reading JSON than for reading Clojure s-expressions. Is there something wrong with JSON I'm not seeing? --Chouser --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Oct 22, 9:41 am, Randall R Schulz <[EMAIL PROTECTED]> wrote: > On Wednesday 22 October 2008 06:27, .Bill Smith wrote: > > > Would you have S-expressions going from the server back to the client > > as well? > > If that client is a Web browser, then presumably it would require an > S-Expression reader written in JavaScript to decode them. > > That would be a nice thing to have. Is anybody aware of one? Douglas Crockford has an s-expression reader as part of his Little JavaScripter: http://javascript.crockford.com/little.html http://javascript.crockford.com/scheme.html Upgrading that to read the other Clojure data structures would be a start. I'm horrible when it comes to regular expressions, so I never too a crack at it. Chouser, does ClojureScript not have something like this in order to do read? Cheers, Tom --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Fri, Oct 24, 2008 at 5:16 PM, Adam Jones <[EMAIL PROTECTED]> wrote: > > > > On Oct 22, 6:17 am, Chouser <[EMAIL PROTECTED]> wrote: > > On Wed, Oct 22, 2008 at 4:30 AM, Brett Morgan <[EMAIL PROTECTED]> > wrote: > > > > > I understand the lisp way is to use the reader plus macros to interpret > the > > > incoming data stream. This is hella cool in that it seriously cuts down > on > > > the amount of development work I have to do. The reader is already > done, and > > > using macros to build the tree walker? And have them applied to a stm > core? > > > Very lightweight in comparison to what I'd do traditionally. Very cool. > > > > I think that if you use "read" rather than "load" or "eval" on the > > incoming s-expressions, you'll have a lot less to worry about. > > Without the eval step there's no need to try to block arbitrary > > function calls and such, because they'll never be evaluated in the > > first place -- any symbols that match function call names will simply > > be returned from the reader as symbols. > > > > If you then want to call macroexpand on them to help process the > > expressions (I've got no sense of whether this would be a useful > > approach or not) then the only code being run would be your own macro. > > There'd be no way for the incoming s-expressions to define new macros > > or functions. > > > > Perhaps you'd still want to audit the LispReader.java code for > > security vulnerabilities and/or run the reader in some sort of Java > > sandbox, but I wouldn't be surprised if neither of these is actually > > necessary. > > > > --Chouser > > > > This is exactly the suggestion I was going to make. Dealing with > untrusted code doesn't seem to be one of the main design > considerations of Clojure. That I know of, there are no provisions to > keep someone from re-defing existing functions, reaching out of their > current namespace, or jumping into those aspects of Java land that are > available. The safest way to handle data from the client is to keep it > as data and write code to parse and react to it, anything else is > setting up a creativity contest between yourself and anyone trying to > break the application. If companies like Microsoft and Oracle can't > win those with all of their money, a single individual doesn't stand a > chance. > My understanding of the lisp way, and I am but a newbie in the land of lisp, is to deal with untrusted code by writing an meta-circular interpreter for a language that is appropriate for the situation at hand. If you don't trust the people generating the code not re-def functions, take def out of the interpreter. Or leave it in, and have it guard predefined functions. My excitement with clojure is that i get to make lots of dsls really easily, at least in comparison to the heavy engineering approach that is required in the java world... > > -Adam > > > -- Brett Morgan http://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Oct 22, 6:17 am, Chouser <[EMAIL PROTECTED]> wrote: > On Wed, Oct 22, 2008 at 4:30 AM, Brett Morgan <[EMAIL PROTECTED]> wrote: > > > I understand the lisp way is to use the reader plus macros to interpret the > > incoming data stream. This is hella cool in that it seriously cuts down on > > the amount of development work I have to do. The reader is already done, and > > using macros to build the tree walker? And have them applied to a stm core? > > Very lightweight in comparison to what I'd do traditionally. Very cool. > > I think that if you use "read" rather than "load" or "eval" on the > incoming s-expressions, you'll have a lot less to worry about. > Without the eval step there's no need to try to block arbitrary > function calls and such, because they'll never be evaluated in the > first place -- any symbols that match function call names will simply > be returned from the reader as symbols. > > If you then want to call macroexpand on them to help process the > expressions (I've got no sense of whether this would be a useful > approach or not) then the only code being run would be your own macro. > There'd be no way for the incoming s-expressions to define new macros > or functions. > > Perhaps you'd still want to audit the LispReader.java code for > security vulnerabilities and/or run the reader in some sort of Java > sandbox, but I wouldn't be surprised if neither of these is actually > necessary. > > --Chouser This is exactly the suggestion I was going to make. Dealing with untrusted code doesn't seem to be one of the main design considerations of Clojure. That I know of, there are no provisions to keep someone from re-defing existing functions, reaching out of their current namespace, or jumping into those aspects of Java land that are available. The safest way to handle data from the client is to keep it as data and write code to parse and react to it, anything else is setting up a creativity contest between yourself and anyone trying to break the application. If companies like Microsoft and Oracle can't win those with all of their money, a single individual doesn't stand a chance. -Adam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
I was heading in the direction of generating javascript on the server for page generation, so it makes some sense to keep using that technique for communication as well. In this model the webclient is trusting the javascript the server is generating, so I have to be diligent in my code generation not to allow external untrusted input become inline code. This gives me the power of macros to generate the boilerplate js and html that is required to build out a full UI. =) On Thu, Oct 23, 2008 at 12:27 AM, .Bill Smith <[EMAIL PROTECTED]>wrote: > > Would you have S-expressions going from the server back to the client > as well? > > Bill > > > > -- Brett Morgan http://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wed, Oct 22, 2008 at 9:41 AM, Randall R Schulz <[EMAIL PROTECTED]> wrote: > > On Wednesday 22 October 2008 06:27, .Bill Smith wrote: >> Would you have S-expressions going from the server back to the client >> as well? > > If that client is a Web browser, then presumably it would require an > S-Expression reader written in JavaScript to decode them. > > That would be a nice thing to have. Is anybody aware of one? Or write a JSON writer in Clojure. That would probably run faster and be more fun to create. Although I think such things already exist. --Chouser --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wednesday 22 October 2008 06:27, .Bill Smith wrote: > Would you have S-expressions going from the server back to the client > as well? If that client is a Web browser, then presumably it would require an S-Expression reader written in JavaScript to decode them. That would be a nice thing to have. Is anybody aware of one? > Bill Randall Schulz --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
Would you have S-expressions going from the server back to the client as well? Bill --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wed, Oct 22, 2008 at 6:55 AM, Brett Morgan <[EMAIL PROTECTED]> wrote: > > > On Wed, Oct 22, 2008 at 9:54 PM, Parth Malwankar <[EMAIL PROTECTED]> > wrote: >> >> >> >> On Oct 22, 3:42 pm, Parth Malwankar <[EMAIL PROTECTED]> wrote: >> > On Oct 22, 1:30 pm, "Brett Morgan" <[EMAIL PROTECTED]> wrote: >> > >> > - Recently the #= reader macro was added. This makes the reader >> > do the evaluation before using the value. You may want to >> > disable this. E.g. >> > >> > user=> #=(+ 1 1) >> > 2 >> > >> > I am not sure how to disable this. There is a >> > similar thing #. in CL and it is important to disable it before >> > reading potentially unsafe expressions. Maybe Rich or someone >> > else can comment on how to disable this. >> > >> >> Oops. The example I meant to give was: >> user=> `(+ 1 1) >> (clojure/+ 1 1) >> user=> `#=(+ 1 1) >> 2 > > That's actually exactly the style of security breach capability that i was > worried about. Thank you =) > There will be a flag to disable #= while reading. Rich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wed, Oct 22, 2008 at 4:30 AM, Brett Morgan <[EMAIL PROTECTED]> wrote: > > I understand the lisp way is to use the reader plus macros to interpret the > incoming data stream. This is hella cool in that it seriously cuts down on > the amount of development work I have to do. The reader is already done, and > using macros to build the tree walker? And have them applied to a stm core? > Very lightweight in comparison to what I'd do traditionally. Very cool. I think that if you use "read" rather than "load" or "eval" on the incoming s-expressions, you'll have a lot less to worry about. Without the eval step there's no need to try to block arbitrary function calls and such, because they'll never be evaluated in the first place -- any symbols that match function call names will simply be returned from the reader as symbols. If you then want to call macroexpand on them to help process the expressions (I've got no sense of whether this would be a useful approach or not) then the only code being run would be your own macro. There'd be no way for the incoming s-expressions to define new macros or functions. Perhaps you'd still want to audit the LispReader.java code for security vulnerabilities and/or run the reader in some sort of Java sandbox, but I wouldn't be surprised if neither of these is actually necessary. --Chouser --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Wed, Oct 22, 2008 at 9:54 PM, Parth Malwankar <[EMAIL PROTECTED]>wrote: > > > > On Oct 22, 3:42 pm, Parth Malwankar <[EMAIL PROTECTED]> wrote: > > On Oct 22, 1:30 pm, "Brett Morgan" <[EMAIL PROTECTED]> wrote: > > > > - Recently the #= reader macro was added. This makes the reader > > do the evaluation before using the value. You may want to > > disable this. E.g. > > > > user=> #=(+ 1 1) > > 2 > > > > I am not sure how to disable this. There is a > > similar thing #. in CL and it is important to disable it before > > reading potentially unsafe expressions. Maybe Rich or someone > > else can comment on how to disable this. > > > > Oops. The example I meant to give was: > user=> `(+ 1 1) > (clojure/+ 1 1) > user=> `#=(+ 1 1) > 2 > That's actually exactly the style of security breach capability that i was worried about. Thank you =) > > Parth > > > > -- Brett Morgan http://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Oct 22, 3:42 pm, Parth Malwankar <[EMAIL PROTECTED]> wrote: > On Oct 22, 1:30 pm, "Brett Morgan" <[EMAIL PROTECTED]> wrote: > > - Recently the #= reader macro was added. This makes the reader > do the evaluation before using the value. You may want to > disable this. E.g. > > user=> #=(+ 1 1) > 2 > > I am not sure how to disable this. There is a > similar thing #. in CL and it is important to disable it before > reading potentially unsafe expressions. Maybe Rich or someone > else can comment on how to disable this. > Oops. The example I meant to give was: user=> `(+ 1 1) (clojure/+ 1 1) user=> `#=(+ 1 1) 2 Parth --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
On Oct 22, 1:30 pm, "Brett Morgan" <[EMAIL PROTECTED]> wrote: > Hi all, > > I am thinking about a potential architecture for a webapp where in the > server gets s expressions posted from an ajax web client. > > From a security standpoint, the s expressions are coming from an untrusted > computer, and thus are in need of careful vetting. > > With my java dev hat on, i'd move forward by building a lexer, a parser, and > a tree walker to interpret the incoming datastream, with careful > consideration to the various potential attacks a malicious user can submit. > > I understand the lisp way is to use the reader plus macros to interpret the > incoming data stream. This is hella cool in that it seriously cuts down on > the amount of development work I have to do. The reader is already done, and > using macros to build the tree walker? And have them applied to a stm core? > Very lightweight in comparison to what I'd do traditionally. Very cool. > > My concern is, what are the security considerations of this architectural > choice? Do I have to worry about people submitting malformed s expressions? > Submitting s expressions that contain data that expands out reader macros? > Do I have to watch for any particular bad code practices in constructing the > macros? How do I go about error recovery and reporting on bad input? > > Thanks in advance. > Hi Brett, Yes, being able to use the clojure reader directly is really neat. Some things you could do are: - use a separate namespace for evaluation the expressions and provide a clear interface between the core and the sexpressions that you get. - have a black or white list, and allow or reject the s-expression based on this. For example you might want to disallow namespace switching functions. E.g. below, I can't clobber defn as I am in the 'user' namespace and 'defn' is in the clojure ns. user=> (def defn :somethig-bad) java.lang.Exception: Name conflict, can't def defn because namespace: user refers to:#=(var clojure/defn) (NO_SOURCE_FILE:1) user=> Some other things you may want to reject expressions based on are java interop and file IO. This should basically be a find operation for a bunch of symbols on the list before you give it to the reader 'read' - Recently the #= reader macro was added. This makes the reader do the evaluation before using the value. You may want to disable this. E.g. user=> #=(+ 1 1) 2 I am not sure how to disable this. There is a similar thing #. in CL and it is important to disable it before reading potentially unsafe expressions. Maybe Rich or someone else can comment on how to disable this. I suppose a lot of this is dictated by what you want to do with the s-expressions. Parth > -- > > Brett Morganhttp://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
I was honestly thinking of using S expressions purely as a replacement for the current trend to use JSON for everything. JSON basically is S expressions, except using hashmaps as the base type. The advantage being that I get a parser and tree walker language for free. On Wed, Oct 22, 2008 at 9:04 PM, Christian Vest Hansen <[EMAIL PROTECTED] > wrote: > > If your untrusted S-expressions are expected to be pure functions on > some input to some output, then you can use the existing java > sandboxing features[1] to execute/read/whatever them in threads that > are locked down tight. Then after sanitizing the output (which should > be raw data), and you're done. > > That's what I think, at least. > > [1]: http://java.sun.com/javase/6/docs/api/java/lang/SecurityManager.html > > On Wed, Oct 22, 2008 at 10:30 AM, Brett Morgan <[EMAIL PROTECTED]> > wrote: > > Hi all, > > > > I am thinking about a potential architecture for a webapp where in the > > server gets s expressions posted from an ajax web client. > > > > From a security standpoint, the s expressions are coming from an > untrusted > > computer, and thus are in need of careful vetting. > > > > With my java dev hat on, i'd move forward by building a lexer, a parser, > and > > a tree walker to interpret the incoming datastream, with careful > > consideration to the various potential attacks a malicious user can > submit. > > > > I understand the lisp way is to use the reader plus macros to interpret > the > > incoming data stream. This is hella cool in that it seriously cuts down > on > > the amount of development work I have to do. The reader is already done, > and > > using macros to build the tree walker? And have them applied to a stm > core? > > Very lightweight in comparison to what I'd do traditionally. Very cool. > > > > My concern is, what are the security considerations of this architectural > > choice? Do I have to worry about people submitting malformed s > expressions? > > Submitting s expressions that contain data that expands out reader > macros? > > Do I have to watch for any particular bad code practices in constructing > the > > macros? How do I go about error recovery and reporting on bad input? > > > > Thanks in advance. > > > > -- > > > > Brett Morgan http://brett.morgan.googlepages.com/ > > > > > > > > > > > -- > Venlig hilsen / Kind regards, > Christian Vest Hansen. > > > > -- Brett Morgan http://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Re: Reader + Macros on untrusted S Expressions: Security considerations?
If your untrusted S-expressions are expected to be pure functions on some input to some output, then you can use the existing java sandboxing features[1] to execute/read/whatever them in threads that are locked down tight. Then after sanitizing the output (which should be raw data), and you're done. That's what I think, at least. [1]: http://java.sun.com/javase/6/docs/api/java/lang/SecurityManager.html On Wed, Oct 22, 2008 at 10:30 AM, Brett Morgan <[EMAIL PROTECTED]> wrote: > Hi all, > > I am thinking about a potential architecture for a webapp where in the > server gets s expressions posted from an ajax web client. > > From a security standpoint, the s expressions are coming from an untrusted > computer, and thus are in need of careful vetting. > > With my java dev hat on, i'd move forward by building a lexer, a parser, and > a tree walker to interpret the incoming datastream, with careful > consideration to the various potential attacks a malicious user can submit. > > I understand the lisp way is to use the reader plus macros to interpret the > incoming data stream. This is hella cool in that it seriously cuts down on > the amount of development work I have to do. The reader is already done, and > using macros to build the tree walker? And have them applied to a stm core? > Very lightweight in comparison to what I'd do traditionally. Very cool. > > My concern is, what are the security considerations of this architectural > choice? Do I have to worry about people submitting malformed s expressions? > Submitting s expressions that contain data that expands out reader macros? > Do I have to watch for any particular bad code practices in constructing the > macros? How do I go about error recovery and reporting on bad input? > > Thanks in advance. > > -- > > Brett Morgan http://brett.morgan.googlepages.com/ > > > > -- Venlig hilsen / Kind regards, Christian Vest Hansen. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
Reader + Macros on untrusted S Expressions: Security considerations?
Hi all, I am thinking about a potential architecture for a webapp where in the server gets s expressions posted from an ajax web client. >From a security standpoint, the s expressions are coming from an untrusted computer, and thus are in need of careful vetting. With my java dev hat on, i'd move forward by building a lexer, a parser, and a tree walker to interpret the incoming datastream, with careful consideration to the various potential attacks a malicious user can submit. I understand the lisp way is to use the reader plus macros to interpret the incoming data stream. This is hella cool in that it seriously cuts down on the amount of development work I have to do. The reader is already done, and using macros to build the tree walker? And have them applied to a stm core? Very lightweight in comparison to what I'd do traditionally. Very cool. My concern is, what are the security considerations of this architectural choice? Do I have to worry about people submitting malformed s expressions? Submitting s expressions that contain data that expands out reader macros? Do I have to watch for any particular bad code practices in constructing the macros? How do I go about error recovery and reporting on bad input? Thanks in advance. -- Brett Morgan http://brett.morgan.googlepages.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~--~~~~--~~--~--~---
