Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Am 06.06.2014 18:16, schrieb Vilem Novak: ...like restarting the discussion, which already came to some conclusions. And in what direction did that conclusion point? IMHO the strength of Blender is to be an open and powerful tool. We should concentrate on that. If someone else wants to start a Blender cloud service, it is his job to deal (pun intended) with the security implications. ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
In my experience non techy people will happily ignore the little warning we have (happens over and over to my clients and coworkers). I propose making a blocking popup like this: This file contains drivers and python scripts that have been disabled for security reasons. * Continue with disabled drivers and scripts * Reload with enabled drivers and scripts (trusted sources only) * Always open files with enabled drivers and scripts (trusted sources only) This will make it easier for people to understand what's happening and ensure it can't be ignored. Daniel Salazar patazstudio.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Rather than trying to sandbox python or limit functionality, why not just sandbox the environment blender is running in? One could set up a render server which clones a previously set up user with all the necessary software installed in the user's path and setroot it so it can't touch anything outside of it's home directory. Any render scripts could do whatever they want and when finished, the user could be archived or deleted. No need to worry about any malicious scripts because if they mess anything up it would only be in the temporary user's space and it would be deleted once the job is finished. On Wed, Jun 4, 2014 at 11:17 PM, Daniel Salazar - patazstudio.com zan...@gmail.com wrote: In my experience non techy people will happily ignore the little warning we have (happens over and over to my clients and coworkers). I propose making a blocking popup like this: This file contains drivers and python scripts that have been disabled for security reasons. * Continue with disabled drivers and scripts * Reload with enabled drivers and scripts (trusted sources only) * Always open files with enabled drivers and scripts (trusted sources only) This will make it easier for people to understand what's happening and ensure it can't be ignored. Daniel Salazar patazstudio.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
That sounds like a VM to me? Daniel Salazar patazstudio.com On Thu, Jun 5, 2014 at 1:07 AM, Dahlia Trimble dahliatrim...@gmail.com wrote: Rather than trying to sandbox python or limit functionality, why not just sandbox the environment blender is running in? One could set up a render server which clones a previously set up user with all the necessary software installed in the user's path and setroot it so it can't touch anything outside of it's home directory. Any render scripts could do whatever they want and when finished, the user could be archived or deleted. No need to worry about any malicious scripts because if they mess anything up it would only be in the temporary user's space and it would be deleted once the job is finished. On Wed, Jun 4, 2014 at 11:17 PM, Daniel Salazar - patazstudio.com zan...@gmail.com wrote: In my experience non techy people will happily ignore the little warning we have (happens over and over to my clients and coworkers). I propose making a blocking popup like this: This file contains drivers and python scripts that have been disabled for security reasons. * Continue with disabled drivers and scripts * Reload with enabled drivers and scripts (trusted sources only) * Always open files with enabled drivers and scripts (trusted sources only) This will make it easier for people to understand what's happening and ensure it can't be ignored. Daniel Salazar patazstudio.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Yes, that would be one way to do it. Perhaps a render appliance. On Thu, Jun 5, 2014 at 12:29 AM, Daniel Salazar - patazstudio.com zan...@gmail.com wrote: That sounds like a VM to me? Daniel Salazar patazstudio.com On Thu, Jun 5, 2014 at 1:07 AM, Dahlia Trimble dahliatrim...@gmail.com wrote: Rather than trying to sandbox python or limit functionality, why not just sandbox the environment blender is running in? One could set up a render server which clones a previously set up user with all the necessary software installed in the user's path and setroot it so it can't touch anything outside of it's home directory. Any render scripts could do whatever they want and when finished, the user could be archived or deleted. No need to worry about any malicious scripts because if they mess anything up it would only be in the temporary user's space and it would be deleted once the job is finished. On Wed, Jun 4, 2014 at 11:17 PM, Daniel Salazar - patazstudio.com zan...@gmail.com wrote: In my experience non techy people will happily ignore the little warning we have (happens over and over to my clients and coworkers). I propose making a blocking popup like this: This file contains drivers and python scripts that have been disabled for security reasons. * Continue with disabled drivers and scripts * Reload with enabled drivers and scripts (trusted sources only) * Always open files with enabled drivers and scripts (trusted sources only) This will make it easier for people to understand what's happening and ensure it can't be ignored. Daniel Salazar patazstudio.com ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On Sat, May 24, 2014 at 7:21 AM, Chad Fraleigh ch...@triularity.org wrote: Just wondering.. has anyone out there wrote a python interpreter *in* python that has security/sandbox functionality/hooks? Then it could offer the option (as another user selectable security level) of secure but slow, which might be adequate for simple or non-intensively called scripts. Since it would be a python script itself, it would/should be portable across the underlying implementation. Depending on the features of such a hypothetical library/module, if the parsed expression tree can be checked and determined to be simple (i.e. no field assignments, no arbitrary function calls, etc..) and known safe, it could be then be directly eval'd by the real python and not just emulated. Essentially a variation of one of the ideas previously mentioned in this thread. -Chad Someone has done just this: http://pypy.org Which also provides a sand-boxed execution environment to prevent scripts doing things you may want to prevent: http://pypy.readthedocs.org/en/latest/sandbox.html ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
This has been tried several times, and always proven breakable by malicious code. Python is not designed for this, people always found ways to escape from the “jail”. On 25/05/2014 10:19, Chad Fraleigh wrote: On Sun, May 25, 2014 at 12:34 AM, Campbell Barton ideasma...@gmail.comwrote: On Sat, May 24, 2014 at 7:21 AM, Chad Fraleigh ch...@triularity.org wrote: Just wondering.. has anyone out there wrote a python interpreter *in* python that has security/sandbox functionality/hooks? Then it could offer the option (as another user selectable security level) of secure but slow, which might be adequate for simple or non-intensively called scripts. Since it would be a python script itself, it would/should be portable across the underlying implementation. Depending on the features of such a hypothetical library/module, if the parsed expression tree can be checked and determined to be simple (i.e. no field assignments, no arbitrary function calls, etc..) and known safe, it could be then be directly eval'd by the real python and not just emulated. Essentially a variation of one of the ideas previously mentioned in this thread. -Chad Someone has done just this: http://pypy.org Which also provides a sand-boxed execution environment to prevent scripts doing things you may want to prevent: http://pypy.readthedocs.org/en/latest/sandbox.html But that seems to be just another implementation of python at the binary library level. What I meant was a python language interpreter that is written purely in python code itself, which will run on top of the existing python runtime that blender uses (not replace it). It would need to only support the parsing of scripts, access checking/hooks, and the minimum execution of the language itself. Since it would run on top of the real python, any standard modules needed would be done by delegating it to the underlying ones (so it would have a very minimalistic runtime). -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On Sun, May 25, 2014 at 6:19 PM, Chad Fraleigh ch...@triularity.org wrote: On Sun, May 25, 2014 at 12:34 AM, Campbell Barton ideasma...@gmail.comwrote: On Sat, May 24, 2014 at 7:21 AM, Chad Fraleigh ch...@triularity.org wrote: Just wondering.. has anyone out there wrote a python interpreter *in* python that has security/sandbox functionality/hooks? Then it could offer the option (as another user selectable security level) of secure but slow, which might be adequate for simple or non-intensively called scripts. Since it would be a python script itself, it would/should be portable across the underlying implementation. Depending on the features of such a hypothetical library/module, if the parsed expression tree can be checked and determined to be simple (i.e. no field assignments, no arbitrary function calls, etc..) and known safe, it could be then be directly eval'd by the real python and not just emulated. Essentially a variation of one of the ideas previously mentioned in this thread. -Chad Someone has done just this: http://pypy.org Which also provides a sand-boxed execution environment to prevent scripts doing things you may want to prevent: http://pypy.readthedocs.org/en/latest/sandbox.html But that seems to be just another implementation of python at the binary library level. What I meant was a python language interpreter that is written purely in python code itself, Thats what pypy is, but in practice its not so useful since its not intended to remain embedded in cpython (even if it would work somehow, pypy's sandbox runs in a subprocess, which adds complications). which will run on top of the existing python runtime that blender uses (not replace it). It would need to only support the parsing of scripts, access checking/hooks, and the minimum execution of the language itself. You could almost do this with the `ast` module, the problem is you can't know what the script actually does until it executes. Since it would run on top of the real python, any standard modules needed would be done by delegating it to the underlying ones (so it would have a very minimalistic runtime). Having a python interpreter written in python as you are suggesting is possible, but I think it would be a lot of work and very slow and I think even in this case it would be difficult to be secure unless you restrict the drivers to arithmetic and a handful of math functions. -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers -- - Campbell ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On Sun, May 25, 2014 at 7:54 PM, Vilem Novak pildano...@post.cz wrote: If I understand the outcome of this discussion right: Sandboxing of python in general isn't functional and is too complicated. After reading all of the possible solutions, there are 2 which I seem to be reasonable: 1.Do not render when some drivers in the scene/linked data can not be executed. (wow, how much of my time would that save...) Something like this seems reasonable. But I wouldn't attempt to make this specific to drivers or rendering. - An input error maybe missing texture. - An output error may be baking physics (or texures). A tool like this should be configurable and usable for studios, errors should be able to log to files, run a command or mail the administrator. See this thread for details: http://markmail.org/message/2qyeqgf7pyhobdt7 2.do a primitive, very restrictive check of the expressions(vars, operators, math funcs.) Rather not - this gives a false sense of security, adds a new classification of driver to manage, more code to maintain (and more bug reports when devs find holes in the checking tool). Also its driver specific. What about other scripts which execute on startup (Which may be needed for populating the driver name-space)? This could accompany a warning, that would show when the expression doesn't make it through this check - 'Driver won't execute with scripts disabled' - this would display even when scripts ARE enabled, because of the case of sending files to renderfarms or sharing them in any other way. I really expect this to be quite rare.. Any of these 2 solutions would be satisfactory in production , I think... Not to fill the mail list with a long discussion, can I add this as a TODO topic somewhere in the wiki? By the reaction, I assume this is a topic that really needs some solution... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers -- - Campbell ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused allready many problems. Last is ruining several days of rendertime on a renderfarm which has scripts blocked(as is by default!). Actually, it happened to me allready several times- setting up renders, render nodes, and forgetting about some drivers and the new feature. I realized so far none of the crowd-render farms for blender don't support scripts on (sheepit, burp). That it of course a logical choice. So the idea is - can there be some check to determine if a driver is actually a python script? If it's using any commands, and not only numerical / logical operators? And then, could such simple drivers be enabled? It would really save my life very often. Now I have to write a script that bakes all drivers before sending file to render farm... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers -- Paolo Acampora, palu...@gmail.com Thecnical Director at MAD entertainment, Naples (Italy) ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Had the same problem here with the #frame driver in the Cycles seed value. Renderfarm just ignored it, and I only noticed it after rendering. On Fri May 23 12:26:46 2014, Paolo Acampora wrote: This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused allready many problems. Last is ruining several days of rendertime on a renderfarm which has scripts blocked(as is by default!). Actually, it happened to me allready several times- setting up renders, render nodes, and forgetting about some drivers and the new feature. I realized so far none of the crowd-render farms for blender don't support scripts on (sheepit, burp). That it of course a logical choice. So the idea is - can there be some check to determine if a driver is actually a python script? If it's using any commands, and not only numerical / logical operators? And then, could such simple drivers be enabled? It would really save my life very often. Now I have to write a script that bakes all drivers before sending file to render farm... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
I think most of us can agree this is an annoying feature, but if it were to be removed now, some nasty folks might take it as an invitation. I don't like this feature as it is, but simply removing it isn't a solution. We need to think of a smart way to always allow render farms to run scripts and drivers by default. Another silly idea: what if we leave this feature on, but only for paths that include the word download in addition to the user-defined folders in the preferences? Or keep a history of trusted authors (computer name or ip) and check if the author of a blend has been trusted before? -Greg On 23 May 2014 13:43, Tobias Kummer supertoi...@gmx.net wrote: Had the same problem here with the #frame driver in the Cycles seed value. Renderfarm just ignored it, and I only noticed it after rendering. On Fri May 23 12:26:46 2014, Paolo Acampora wrote: This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused allready many problems. Last is ruining several days of rendertime on a renderfarm which has scripts blocked(as is by default!). Actually, it happened to me allready several times- setting up renders, render nodes, and forgetting about some drivers and the new feature. I realized so far none of the crowd-render farms for blender don't support scripts on (sheepit, burp). That it of course a logical choice. So the idea is - can there be some check to determine if a driver is actually a python script? If it's using any commands, and not only numerical / logical operators? And then, could such simple drivers be enabled? It would really save my life very often. Now I have to write a script that bakes all drivers before sending file to render farm... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On 23.05.2014 13:53, Greg Zaal wrote: Another silly idea: what if we leave this feature on, but only for paths that include the word download in addition to the user-defined folders in the preferences? Or keep a history of trusted authors (computer name or ip) and check if the author of a blend has been trusted before? Think of a path like download/../. This issue canont be fixed trivially. You cannot sandbox Python easily. The only way out of that mess would be for render farms to, e.g. put the render in some LXC (or other) container and deny it network access. Burn down the container after the work is done and you’re quite safe. This is nothing blender can fix. It is a kind of a limitation of the python language (note that PyPy, for example, offers ways to allow sandboxing, but many libraries won’t work with PyPy and embedding is totally different, so I assume that blender won’t move to PyPy in the next decade). regards, Jonas disclaimer: not a blender developer. just my five cents. -Greg On 23 May 2014 13:43, Tobias Kummer supertoi...@gmx.net wrote: Had the same problem here with the #frame driver in the Cycles seed value. Renderfarm just ignored it, and I only noticed it after rendering. On Fri May 23 12:26:46 2014, Paolo Acampora wrote: This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused allready many problems. Last is ruining several days of rendertime on a renderfarm which has scripts blocked(as is by default!). Actually, it happened to me allready several times- setting up renders, render nodes, and forgetting about some drivers and the new feature. I realized so far none of the crowd-render farms for blender don't support scripts on (sheepit, burp). That it of course a logical choice. So the idea is - can there be some check to determine if a driver is actually a python script? If it's using any commands, and not only numerical / logical operators? And then, could such simple drivers be enabled? It would really save my life very often. Now I have to write a script that bakes all drivers before sending file to render farm... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On another note: IMO, despite what others have said about detecting malicious Python code to be an intractable problem, I still believe that when you're not trying to solve the problem of detecting whether an arbitrary piece of Python code for arbitrary/general purposes, the problem becomes much simpler. Secondly, if you combine this with a 3-tier security rating/permissions scheme - i.e. 0 = Trust nothing scripted at all, 1 = Allow low risk scripting, 2 = Trust everything by default / allow anything to run as needed - instead of the current all or nothing boolean switch, this would probably be sufficient for most use cases (and the teams which do in fact use anything beefier are probably already using or can get hold of their own private computing resources where they can set things up to be security free). This solution I proposed a year ago already ( http://aligorith.blogspot.co.nz/2013/06/probabilistic-security-another-take-on.html ). As for why I think 1-line driver expressions are a limited-enough domain that we could successfully lock it down (Note that such restrictions only apply when considering which expressions can be considered low risk enough to allow them to run): Such 1-line expressions serve a very specific purposes - that of calculating a numerical value given the driver variables defined for that driver and optionally using builtin math functions to operate on these. Thus, I think it's well within reason to define the following limitations on what can/cannot be done: 1) if the expression contains attempts at multiple lines of code (i.e. semicolons -- this rules out doing import nastymodule; nastymodule.evil_stuff()), it can't be up to any good. 2) If any identifier names used reference anything other than one of the driver's variables or a builtin math function, that is already exceeding the goals that a simple, well behaved, and verifiably non malicious code should be doing. 2.1) Incidentally, this should also catch any attempts to reference/call custom functions where the really nasty stuff lives. 2.2) We'd definitely want to look out for any obvious known weakpoints - e.g. exec, eval, import 2.3) Variable identifiers are something we can also verify for problems. In fact, it's something we should be doing now if we're not doing it already, so that users don't accidentally define identifiers they can't actually use! 3) Any identifiers using underscores are up to no good. This rules out digging around in Python internals to hack out to things for doing nasty business 4) Function calls on strings are suspicious. Again the operative word here is low risk - simple operations for calculating numerical values shouldn't have a need for string manipulations. And definitely not for any string operations where you're trying to change encoding, or perform any kind of decryption. (For that matter, hash-like content is surely bad too, but detecting that might require a bit too much data + processing for us to use for something like this) ... This is not a complete/conclusive list of restrictions we could apply and check for. However, hopefully you get the idea: we're just looking to identify what we can treat as sufficiently low risk that it really can't do much harm. Anything higher risk obviously deserves human attention for manual overriding (or running on a system where absolute security is less of a priority). Regards, Joshua On Fri, May 23, 2014 at 11:51 PM, Joshua Leung aligor...@gmail.com wrote: IIRC, we are already only blocking the drivers where Python code is used. To be specific, if it's of type scripted expression it contains Python code of some sort. Thus, if you can set up your drivers in a way which doesn't require an expression to be evaluated, you could avoid these problems. For example: - if you've just got a single variable, then just use averaged value, or - if you're summing the results of two variables just use sum, - if you're applying some trig function to the value or just some scale factor, it's perfectly fine to do by using a suitable combination of FModifiers and/or keyframes on the driver FCurve) Of course, this still leaves some situations unable to be resolved (i.e. if you need a custom function to perform some logic given the variables, outputting a suitable value based on those). In which case, your only choice there is to find some way to ensure that the security is disabled on that file when you go to use/render it. Regards, Joshua On Fri, May 23, 2014 at 10:26 PM, Paolo Acampora palu...@gmail.comwrote: This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On Fri, May 23, 2014 at 9:53 PM, Greg Zaal gregzzm...@gmail.com wrote: I think most of us can agree this is an annoying feature, but if it were to be removed now, some nasty folks might take it as an invitation. I don't like this feature as it is, but simply removing it isn't a solution. We need to think of a smart way to always allow render farms to run scripts and drivers by default. blender --enable-autoexec (see blender --help) ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
IIRC, we are already only blocking the drivers where Python code is used. To be specific, if it's of type scripted expression it contains Python code of some sort. Thus, if you can set up your drivers in a way which doesn't require an expression to be evaluated, you could avoid these problems. For example: - if you've just got a single variable, then just use averaged value, or - if you're summing the results of two variables just use sum, - if you're applying some trig function to the value or just some scale factor, it's perfectly fine to do by using a suitable combination of FModifiers and/or keyframes on the driver FCurve) Of course, this still leaves some situations unable to be resolved (i.e. if you need a custom function to perform some logic given the variables, outputting a suitable value based on those). In which case, your only choice there is to find some way to ensure that the security is disabled on that file when you go to use/render it. Regards, Joshua On Fri, May 23, 2014 at 10:26 PM, Paolo Acampora palu...@gmail.com wrote: This is an issue at our studio as well, I don't see any rationale in these overly security concerns, it just prevents you to work. 2014-05-23 12:06 GMT+02:00 Vilem Novak pildano...@post.cz: Hello, I realize how important is the security when .blend files are distributed, but I thought, is there a way to exclude drivers from the relatively new strict blocking mechanism? To me as animator, it caused allready many problems. Last is ruining several days of rendertime on a renderfarm which has scripts blocked(as is by default!). Actually, it happened to me allready several times- setting up renders, render nodes, and forgetting about some drivers and the new feature. I realized so far none of the crowd-render farms for blender don't support scripts on (sheepit, burp). That it of course a logical choice. So the idea is - can there be some check to determine if a driver is actually a python script? If it's using any commands, and not only numerical / logical operators? And then, could such simple drivers be enabled? It would really save my life very often. Now I have to write a script that bakes all drivers before sending file to render farm... Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers -- Paolo Acampora, palu...@gmail.com Thecnical Director at MAD entertainment, Naples (Italy) ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
On Fri, May 23, 2014 at 9:58 PM, Joshua Leung aligor...@gmail.com wrote: On another note: IMO, despite what others have said about detecting malicious Python code to be an intractable problem, I still believe that when you're not trying to solve the problem of detecting whether an arbitrary piece of Python code for arbitrary/general purposes, the problem becomes much simpler. Secondly, if you combine this with a 3-tier security rating/permissions scheme - i.e. 0 = Trust nothing scripted at all, 1 = Allow low risk scripting, 2 = Trust everything by default / allow anything to run as needed - instead of the current all or nothing boolean switch, this would probably be sufficient for most use cases (and the teams which do in fact use anything beefier are probably already using or can get hold of their own private computing resources where they can set things up to be security free). This solution I proposed a year ago already ( http://aligorith.blogspot.co.nz/2013/06/probabilistic-security-another-take-on.html ). As for why I think 1-line driver expressions are a limited-enough domain that we could successfully lock it down (Note that such restrictions only apply when considering which expressions can be considered low risk enough to allow them to run): Such 1-line expressions serve a very specific purposes - that of calculating a numerical value given the driver variables defined for that driver and optionally using builtin math functions to operate on these. Thus, I think it's well within reason to define the following limitations on what can/cannot be done: 1) if the expression contains attempts at multiple lines of code (i.e. semicolons -- this rules out doing import nastymodule; nastymodule.evil_stuff()), it can't be up to any good. 2) If any identifier names used reference anything other than one of the driver's variables or a builtin math function, that is already exceeding the goals that a simple, well behaved, and verifiably non malicious code should be doing. 2.1) Incidentally, this should also catch any attempts to reference/call custom functions where the really nasty stuff lives. 2.2) We'd definitely want to look out for any obvious known weakpoints - e.g. exec, eval, import 2.3) Variable identifiers are something we can also verify for problems. In fact, it's something we should be doing now if we're not doing it already, so that users don't accidentally define identifiers they can't actually use! 3) Any identifiers using underscores are up to no good. This rules out digging around in Python internals to hack out to things for doing nasty business 4) Function calls on strings are suspicious. Again the operative word here is low risk - simple operations for calculating numerical values shouldn't have a need for string manipulations. And definitely not for any string operations where you're trying to change encoding, or perform any kind of decryption. (For that matter, hash-like content is surely bad too, but detecting that might require a bit too much data + processing for us to use for something like this) ... This is not a complete/conclusive list of restrictions we could apply and check for. However, hopefully you get the idea: we're just looking to identify what we can treat as sufficiently low risk that it really can't do much harm. Anything higher risk obviously deserves human attention for manual overriding (or running on a system where absolute security is less of a priority). After having looked into this already, I am strongly against having _ANY_ attempt to validate the expression based on checking the string, these are so trivial to overcome its really not worth the effort to attempt it, and the security it provides can be worked around after 10min looking online. Regards, Joshua On Fri, May 23, 2014 at 11:51 PM, Joshua Leung aligor...@gmail.com wrote: IIRC, we are already only blocking the drivers where Python code is used. To be specific, if it's of type scripted expression it contains Python code of some sort. Thus, if you can set up your drivers in a way which doesn't require an expression to be evaluated, you could avoid these problems. For example: - if you've just got a single variable, then just use averaged value, or - if you're summing the results of two variables just use sum, - if you're applying some trig function to the value or just some scale factor, it's perfectly fine to do by using a suitable combination of FModifiers and/or keyframes on the driver FCurve) Of course, this still leaves some situations unable to be resolved (i.e. if you need a custom function to perform some logic given the variables, outputting a suitable value based on those). In which case, your only choice there is to find some way to ensure that the security is disabled on that file when you go to use/render it. Regards, Joshua On Fri, May 23, 2014 at 10:26 PM, Paolo Acampora
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Another alternative for drivers specifically may be to consider moving these away from Python, and using a dedicated expression language like Disney's SeExpr instead. Either that, or some kind of custom DSL (probably Pythonic or compatible with the existing Python syntax used in nature), but with a much more limited set of valid acceptable input. The expression validation proposal was already effectively a step in that direction, except that we still use Python as the backend still. In any case, we may have to consider something along these lines down the track when we have the new depsgraph fully operational, since the GIL issues will likely mean that we could end up with Python driver opnodes getting backed up waiting for the thread dedicated to dealing running anything interfacing with Python to be available. On Sat, May 24, 2014 at 12:21 AM, Campbell Barton ideasma...@gmail.comwrote: On Fri, May 23, 2014 at 9:58 PM, Joshua Leung aligor...@gmail.com wrote: On another note: IMO, despite what others have said about detecting malicious Python code to be an intractable problem, I still believe that when you're not trying to solve the problem of detecting whether an arbitrary piece of Python code for arbitrary/general purposes, the problem becomes much simpler. Secondly, if you combine this with a 3-tier security rating/permissions scheme - i.e. 0 = Trust nothing scripted at all, 1 = Allow low risk scripting, 2 = Trust everything by default / allow anything to run as needed - instead of the current all or nothing boolean switch, this would probably be sufficient for most use cases (and the teams which do in fact use anything beefier are probably already using or can get hold of their own private computing resources where they can set things up to be security free). This solution I proposed a year ago already ( http://aligorith.blogspot.co.nz/2013/06/probabilistic-security-another-take-on.html ). As for why I think 1-line driver expressions are a limited-enough domain that we could successfully lock it down (Note that such restrictions only apply when considering which expressions can be considered low risk enough to allow them to run): Such 1-line expressions serve a very specific purposes - that of calculating a numerical value given the driver variables defined for that driver and optionally using builtin math functions to operate on these. Thus, I think it's well within reason to define the following limitations on what can/cannot be done: 1) if the expression contains attempts at multiple lines of code (i.e. semicolons -- this rules out doing import nastymodule; nastymodule.evil_stuff()), it can't be up to any good. 2) If any identifier names used reference anything other than one of the driver's variables or a builtin math function, that is already exceeding the goals that a simple, well behaved, and verifiably non malicious code should be doing. 2.1) Incidentally, this should also catch any attempts to reference/call custom functions where the really nasty stuff lives. 2.2) We'd definitely want to look out for any obvious known weakpoints - e.g. exec, eval, import 2.3) Variable identifiers are something we can also verify for problems. In fact, it's something we should be doing now if we're not doing it already, so that users don't accidentally define identifiers they can't actually use! 3) Any identifiers using underscores are up to no good. This rules out digging around in Python internals to hack out to things for doing nasty business 4) Function calls on strings are suspicious. Again the operative word here is low risk - simple operations for calculating numerical values shouldn't have a need for string manipulations. And definitely not for any string operations where you're trying to change encoding, or perform any kind of decryption. (For that matter, hash-like content is surely bad too, but detecting that might require a bit too much data + processing for us to use for something like this) ... This is not a complete/conclusive list of restrictions we could apply and check for. However, hopefully you get the idea: we're just looking to identify what we can treat as sufficiently low risk that it really can't do much harm. Anything higher risk obviously deserves human attention for manual overriding (or running on a system where absolute security is less of a priority). After having looked into this already, I am strongly against having _ANY_ attempt to validate the expression based on checking the string, these are so trivial to overcome its really not worth the effort to attempt it, and the security it provides can be worked around after 10min looking online. Regards, Joshua On Fri, May 23, 2014 at 11:51 PM, Joshua Leung aligor...@gmail.com wrote: IIRC, we are already only blocking the drivers where Python code is
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
I don't think any type of checking will be safe against a determined attacker. One could conceivably rename objects to contain malicious code, and then use these as RNA path in an expression. -m On Fri, May 23, 2014 at 8:57 AM, Vilem Novak pildano...@post.cz wrote: thanks for the reactions. From the proposed solution I think that most sane solution would be some limitation for the one-line expressions, assumably all of those which Joshua proposed. Maybe there is a simple way to put all these limitations into a simple string-checking operation, just check if expression does not have: anything else but driver vars, operators, math functions(this might be the complex part, to define what should be included in this.)... I mean- rather check if there's what is allowed, then you don't have to care what all should be forbidden, because that is everything else... Of course, this can again lead to similar situation - an artist does something not allowed, he is again stuck with not knowing what is wrong (e.g. on the renderfarm), but I assume it would be much less cases. I cannot currently imagine creative cases which would end like this. Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Imho doing something like this will only worsen the situation. Right now a lot of .blends fail entirely, leaving many users to wonder why. This sucks. If we were to allow *some* expressions, but not others, potentially only half a .blend will fail. This sucks even more. At least when everything breaks it's clear there is an underlying reason; whether that is a user preference as is the case here, a bug in Blender or an unsupported version or whatever. At least more experienced users may quickly realize they did not allow script execution for this .blend. If only, say, one out of ten drivers fail, I would imagine it being tempting to go look for the cause of this at the drivers themselves; perhaps someone accidentally made a typo in an expression? I'm not sure if this has been proposed before, but would it be possible to show a pop-up when opening a (external) .blend file for the first time? Something similar to what OSX does when opening a freshly downloaded app or document: -- This .blend was created on another machine and contains executable code that could be harmful. Do you wish to allow Blender to execute these scripts? [Yes | No] -- It wouldn't solve the issue for render farms, but would at least provide clearity (as well as a much improved sense of control!) to the user. Just my two cents. Cheers, Patrick From: pildano...@post.cz To: bf-committers@blender.org CC: bf-committers@blender.org Date: Fri, 23 May 2014 17:57:31 +0200 Subject: [Bf-committers] Do drivers have to be blocked as python scripts? thanks for the reactions. From the proposed solution I think that most sane solution would be some limitation for the one-line expressions, assumably all of those which Joshua proposed. Maybe there is a simple way to put all these limitations into a simple string-checking operation, just check if expression does not have: anything else but driver vars, operators, math functions(this might be the complex part, to define what should be included in this.)... I mean- rather check if there's what is allowed, then you don't have to care what all should be forbidden, because that is everything else... Of course, this can again lead to similar situation - an artist does something not allowed, he is again stuck with not knowing what is wrong (e.g. on the renderfarm), but I assume it would be much less cases. I cannot currently imagine creative cases which would end like this. Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
I'm happy this is getting some attention. Apart from the usability issues this sheds some light into another problem which is: Blender should not render crippled scenes ever. At least on background mode Blender needs to stop if drives aren't functioning and/or if textures are missing. The fact that it happily continues is making us loose valuable render time. Daniel Salazar patazstudio.com On Fri, May 23, 2014 at 1:50 PM, patrick boelens p_boel...@msn.com wrote: Imho doing something like this will only worsen the situation. Right now a lot of .blends fail entirely, leaving many users to wonder why. This sucks. If we were to allow *some* expressions, but not others, potentially only half a .blend will fail. This sucks even more. At least when everything breaks it's clear there is an underlying reason; whether that is a user preference as is the case here, a bug in Blender or an unsupported version or whatever. At least more experienced users may quickly realize they did not allow script execution for this .blend. If only, say, one out of ten drivers fail, I would imagine it being tempting to go look for the cause of this at the drivers themselves; perhaps someone accidentally made a typo in an expression? I'm not sure if this has been proposed before, but would it be possible to show a pop-up when opening a (external) .blend file for the first time? Something similar to what OSX does when opening a freshly downloaded app or document: -- This .blend was created on another machine and contains executable code that could be harmful. Do you wish to allow Blender to execute these scripts? [Yes | No] -- It wouldn't solve the issue for render farms, but would at least provide clearity (as well as a much improved sense of control!) to the user. Just my two cents. Cheers, Patrick From: pildano...@post.cz To: bf-committers@blender.org CC: bf-committers@blender.org Date: Fri, 23 May 2014 17:57:31 +0200 Subject: [Bf-committers] Do drivers have to be blocked as python scripts? thanks for the reactions. From the proposed solution I think that most sane solution would be some limitation for the one-line expressions, assumably all of those which Joshua proposed. Maybe there is a simple way to put all these limitations into a simple string-checking operation, just check if expression does not have: anything else but driver vars, operators, math functions(this might be the complex part, to define what should be included in this.)... I mean- rather check if there's what is allowed, then you don't have to care what all should be forbidden, because that is everything else... Of course, this can again lead to similar situation - an artist does something not allowed, he is again stuck with not knowing what is wrong (e.g. on the renderfarm), but I assume it would be much less cases. I cannot currently imagine creative cases which would end like this. Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Agree with Daniel. Perhaps a 'Halt on Error' checkbooks that prevent the rendering if there are any missing libraries, textures or disabled scripts. This should be helpful in a studio environment. M On May 23, 2014 1:13 PM, Daniel Salazar - patazstudio.com zan...@gmail.com wrote: I'm happy this is getting some attention. Apart from the usability issues this sheds some light into another problem which is: Blender should not render crippled scenes ever. At least on background mode Blender needs to stop if drives aren't functioning and/or if textures are missing. The fact that it happily continues is making us loose valuable render time. Daniel Salazar patazstudio.com On Fri, May 23, 2014 at 1:50 PM, patrick boelens p_boel...@msn.com wrote: Imho doing something like this will only worsen the situation. Right now a lot of .blends fail entirely, leaving many users to wonder why. This sucks. If we were to allow *some* expressions, but not others, potentially only half a .blend will fail. This sucks even more. At least when everything breaks it's clear there is an underlying reason; whether that is a user preference as is the case here, a bug in Blender or an unsupported version or whatever. At least more experienced users may quickly realize they did not allow script execution for this .blend. If only, say, one out of ten drivers fail, I would imagine it being tempting to go look for the cause of this at the drivers themselves; perhaps someone accidentally made a typo in an expression? I'm not sure if this has been proposed before, but would it be possible to show a pop-up when opening a (external) .blend file for the first time? Something similar to what OSX does when opening a freshly downloaded app or document: -- This .blend was created on another machine and contains executable code that could be harmful. Do you wish to allow Blender to execute these scripts? [Yes | No] -- It wouldn't solve the issue for render farms, but would at least provide clearity (as well as a much improved sense of control!) to the user. Just my two cents. Cheers, Patrick From: pildano...@post.cz To: bf-committers@blender.org CC: bf-committers@blender.org Date: Fri, 23 May 2014 17:57:31 +0200 Subject: [Bf-committers] Do drivers have to be blocked as python scripts? thanks for the reactions. From the proposed solution I think that most sane solution would be some limitation for the one-line expressions, assumably all of those which Joshua proposed. Maybe there is a simple way to put all these limitations into a simple string-checking operation, just check if expression does not have: anything else but driver vars, operators, math functions(this might be the complex part, to define what should be included in this.)... I mean- rather check if there's what is allowed, then you don't have to care what all should be forbidden, because that is everything else... Of course, this can again lead to similar situation - an artist does something not allowed, he is again stuck with not knowing what is wrong (e.g. on the renderfarm), but I assume it would be much less cases. I cannot currently imagine creative cases which would end like this. Regards Vilem ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Do drivers have to be blocked as python scripts?
Just wondering.. has anyone out there wrote a python interpreter *in* python that has security/sandbox functionality/hooks? Then it could offer the option (as another user selectable security level) of secure but slow, which might be adequate for simple or non-intensively called scripts. Since it would be a python script itself, it would/should be portable across the underlying implementation. Depending on the features of such a hypothetical library/module, if the parsed expression tree can be checked and determined to be simple (i.e. no field assignments, no arbitrary function calls, etc..) and known safe, it could be then be directly eval'd by the real python and not just emulated. Essentially a variation of one of the ideas previously mentioned in this thread. -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
