Dino Viehland wrote: > Thanks for the info... Darn, it sounds like you actually want sys to be > isolated... > > We're going to do a review over the hosting APIs internally next week. That > should result in the last large set of changes. One thought that has > occurred to me is the ability to provide management of the current sys > instance. This is where most of the state is tracked. That would presumably > be something like allowing you to create a new system state and provide the > current system state at runtime. You could then switch system states before > running a new script code. In the multi-threaded case you could store the > system state in a thread local variable. Therefore rather than tracking a > set of engines you'd track a list of SystemState objects (or in DLR speak > they may be some opaque state object).
It sounds very similar to what we are currently doing with the native implementation. We are swapping out PyThreadState pointers by means of PyEval_AcquireThread and PyEval_ReleaseThread (See the Python/C API Reference, if you're interested). There is one problem that we've encountered by doing this: that is when there is some external stimuli (for lack of a better word) on a particular script while the state is swapped out. Python code tries to get executed when it's not available and the system crashes. The two most common cases our customers have been wanting to do (and we really want to give them this capability) is to create GUI for their scripts and to do sockets. In the case of the GUI, if a user clicks a button and the script has been swapped out, then there's trouble. Same with sockets when you receive a transmission. Both of these cases work fine if there is exactly one script in the whole system, but as soon as you add another, it's no dice. I guess the question is how do you swap a state back in when someone else is initiating the switch? Now, it could just be that this won't be a problem with what you're proposing, but I'll need more information. > The one interesting thing here is how this plays w/ console output. In v2.0 > that's separate from SystemState so you'd also need to update those at the > same time. That's not a problem, as we're already doing this in the native implementation. > Any feedback on that proposal? > > > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tony > Djordjevski > Sent: Wednesday, May 02, 2007 9:59 AM > To: Discussion of IronPython > Subject: Re: [IronPython] Multiple engine instances in IP 2.0 and beyond (was > IronPython 2.0 Alpha 1 Released) > > Thanks for the reply Dino. And sorry, this is going to be a long post. > > I guess the best way to tell you what we need from IronPython is to tell > you how our product works today. I should also note that we're porting > the application from native C++ to .Net. We currently have python > support by way of CPython and the Boost Python library. > > Our product is an emulation/simulation application within the > manufacturing domain. Our customers are able to emulate their > manufacturing process by adding virtual machines (what we call > components) to a 3D world that represents their factory floor. Each > component can have various "behaviors", and we supply some common > behaviors which the customer can add to a component. For example, a > conveyor component would have behaviors that deal with conveyance, > sensors that detect parts moving along the conveyor, and signals used > for communication, among others. > > The scripts are important because they are a means for the customer to > create custom behaviors (you can almost think of them as plugins). > Using the above example of a conveyor, a part might move down the > conveyor, a sensor would detect that part and then fire a signal. That > signal could be "wired" up to a script behavior. The script in turn > might turn off the conveyor, or slow it down, or do throughput analysis, > or a number of other things. > > The customer might have a number of these same conveyors on the factory > floor all with the same script behavior(s). The scripts need to be > isolated insofar that when they do turn off the conveyor, as in the > example above, that they turn off the conveyor in the component they are > associated with. So I would have to say that the state of the script > "instance" needs to be separate from each other. > > The scripts don't run all the time. They run once during compilation, > as that's the behavior of CPython. So any statements at the main module > level get executed then. If the script has specific functions defined, > we call those python functions when particular simulation events take place. > > To answer some of your specific questions: > Q1. Do you need multiple system states so they get their own modules, > console, etc... do you need everything in sys isolated? > > A1. I would have to say yes, for the most part. Shallow copies of > imported modules are probably OK, but we aren't far enough into the > migration to give you a definitive answer. There is one case in our C++ > implementation that this caused a problem and we were forced to call the > module's init function for each interpreter behind the scenes. > > As far as everything in sys is concerned, maybe not but I can think of > at least one situation that it must be isolated. Currently, we redirect > sys.stdout and sys.stderr to a window in our application, but that > doesn't stop a user from redefining those manually in one of their > scripts. If they do, then it shouldn't affect any other script in the > application. Other things, like sys.version, don't have to be isolated. > > Q2. Do you need to guarantee the isolation even if .NET code is called > (e.g. they could smuggled data via a static field). > > A2. No and in our case I don't think that would be desirable. The > scripts themselves need to be able to interact with other objects within > the simulation world. > > Q3. If they do need some rather high level of isolation are app domains > good enough? Do you need to marshal a lot of data in/out? Or is the > effort to spin up and use app domains correctly? > > A4. I'm fairly new to .Net, so I'm not sure. I'll have to read up on > app domains, but my gut tells me that is not what we want here. > > > I hope this sheds more light, Dino. I tried to be as clear as possible, > but it was a bit difficult to keep this short(ish) and still convey our > needs. > > Let me know if I can elaborate on any other questions you might have, or > if anything I've wrote here is unclear. > > One thing to think about: eventually we want to make our simulation > engine multi-threaded. Script functions will probably be getting > executed in parallel at some point. > > Thanks, > Tony > > Dino Viehland wrote: >> I'm not actually the one working on the engine APIs so that's the reason >> I've tended to be vague. I'll talk to the people doing it and let you know >> what I hear. >> >> But the more info you can give us the better decision we'll be able to make. >> For example what do you actually need to be isolated? Do you need multiple >> system states so they get their own modules, console, etc... do you need >> everything in sys isolated? Do you need to guarantee the isolation even if >> .NET code is called (e.g. they could smuggled data via a static field). If >> they do need some rather high level of isolation are app domains good >> enough? Do you need to marshal a lot of data in/out? Or is the effort to >> spin up and use app domains correctly? >> >> Ultimately understanding the scenarios where this is being used might end up >> with us coming up with an even better solution. >> >> The reasons why I ask that is because there are already many options on >> isolation depending upon what you're trying to do. For example in the >> dynamic language support for ASP.NET we're executing the same code in >> multiple modules. That gives each thread it's own set of globals but could >> allow sharing through system state. It also means that the different >> requests get to share imported modules. That's not much isolation but it >> could be enough for a lot of scenarios. On the other extreme is full app >> domain isolation. >> >> The more info we get here the more likely we'll capture the right result. >> The real challenge here is it's actually impossible for us to offer full >> isolation w/o going to app domain isolation. So anything we create will be >> less; for example Michael Foord noted that the recursion enforcement is a >> global setting and it's hard to not make that global. So we just need to >> figure out the right "less" that is both useful, maintainable, and >> understandable. >> > _______________________________________________ > users mailing list > users@lists.ironpython.com > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > _______________________________________________ > users mailing list > users@lists.ironpython.com > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > _______________________________________________ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
