[Mono-list] Building C# with CMake?
We've just moved from using scons to CMake to build our (mostly C++) codebase that embeds Mono. Does anyone know whether we can build our embedded C# bits with CMake as well? ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
[Mono-dev] Mono.Security.StrongName.Verify Assemblies In Memory?
We ship assemblies around over the network a lot and would like to be able to use Mono.Security.StrongName to verify assemblies without writing them to disk. Would it be OK to add Mono.Security.StrongName.Verfiy(Stream) and refactor some of the internals of Mono.Security.StrongName to facilitate this (using Streams internally, instead of FileStreams). If we made these changes, could we get them merged back in to Mono? Cheers, Jim ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-list] Embedding Mono 1.2.6 on Windows with Visual Studio 2003?
Is anyone building Mono 1.2.6 dll and lib files for embedding with Visual Studio 2003? If so, could you share your Solution and Project files please? Thanks, Jim ___ Support the World Aids Awareness campaign this month with Yahoo! For Good http://uk.promotions.yahoo.com/forgood/ ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-dev] What would you like to see in Mono?
Hi Miguel, What would be the top feature you would like to see in Mono? 1) Explicit (GC_Free style) assembly unloading. 2) Assembly unloading via application domain unload. 3) Full bytecode verification. (But you knew that) ;-) Cheers, Jim/Babbage. ___ Yahoo! Photos NEW, now offering a quality print service from just 8p a photo http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] 1488 Ways To Break A Runtime
New archive here: http://homepage.ntlworld.com/james.purbrick/VerifierTests7.tar I've added tests for assignment compatibility and new tests for merging stack state based on the verification type compatibility rules. All the new tests assembly with mono ilasm and give the expected errors when run through peverify /IL (although peverify doesn't seem to like my method pointers, I'm not sure if I've got something wrong there). I _think_ these are all the tests that I need to fail verification for our current use of Mono in Second Life. I don't think we need tests for other opcodes or more elaborate tests for call due to our whitelisting of opcodes and allowable methods. I also think we might be able to get away without binary tests and meta data tests as all of the assemblies we load are generated by mono ilasm which shouldn't generate broken binaries or meta data (Ankit, does this sound right?). Anyway, I need to get back to work on SL, so now would be a good time to put the current set of tests in svn and for people to check the current tests, add the missing ones and to get mono to fail all of the tests when it verifies them. Thanks for all your help putting the tests together, Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] 1003 Ways To Break A Runtime
ILASM doesn't seem to like them. I tried .field public int32 i .field public class [mscorlib]System.TypedReference val2 Both ilasm .net and mono are able to compile this. Oops, sorry Ankit, my fault. It was a problem with the scripts. I've fixed them and uploaded a new version here: http://homepage.ntlworld.com/james.purbrick/VerifierTests4.tar The coercion tests all build with mono ilasm using this version. Ofcourse, the resulting assembly has PEVerify errors. Well, that's good at least :-) I'm working on moving all the old tests to the new template format ATM and I think I'm going to start numbering the tests in a set rather than trying to generate a unique name out of the opcodes and types which complicates the scripts and limits the substitutions that can be make within the tests themselves. So, we'll get invalid_coercion_47.cil rather than invalid_coercion_stloc_int32_float64.cil or whatever. Any objections? The only problem I can see is that the numbers might change between versions of the test generation scripts, but I don't think that's too bad. Cheers, Jim. ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] 1076 Ways To Break A Runtime
New archive here: http://homepage.ntlworld.com/james.purbrick/VerifierTests5.tar This version incorporates all of the old tests. Most are generated from templates, but there were the odd few that didn't warrant a template and couldn't be generated from an existing one, so I've just included those as cil files. All the generated tests are now named *generated.cil and I've added a clean_tests script to get rid of them without deleting the non-generated scripts. The new make_unary_test script makes heavy use of substitutions, so is very flexible, but also makes the entries in make_tests which call it more complex, not sure what the right balance is there. There are still a few more tests I'm going to add as listed in the README. Once those are added and all of the tests are detected as invalid by Mono (or some other verifier) I *think* it will be reasonably safe to run code output by my bytecode transformer and then verified in Second Life. I had to run dos2unix on the make_tests.sh or else it wouldn't execute. The version in the new archive should be OK. Do you have/plan scripts to assemble / execute / log the results of the tests? I think Lupus had a plan for integrating the tests in to the Mono build/test process. I am planning to write a noddy script to see which tests assemble with Mono/MS ilasm and see which are detected as invalid by MS PEVerify (if anyone else has done that already, let me know). Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] 1003 Ways To Break A Runtime
Hi Paolo/All,
I've started using templates parameterized by types
and opcodes as it makes the tests much easier to write
and check.
I've written tests for shifts, stack merging and
coercion this way and started converting the old tests
and now have 15 shell scripts which generate 1003
tests :-)
http://homepage.ntlworld.com/james.purbrick/VerifierTests3.tar
1) It looks like object values should always
merge as System.Object is always a supertype of
both?
No. Say you have:
class A {}
class B : A {}
class C : B {}
Merging B and C will result in a B type, not Object.
The spec says: the closest common supertype.
If you need to merge C and string, then the closest
supertype is Object.
But, if all objects are subtypes of Object, any 2
object types will merge, so its always possible to
merge 2 stacks with equal numbers of object types?
If so, why all the special rules about compatibility
of arrays and interfaces etc.? Are they not objects
for the purposes of stack merges? Are the type
compatibility rules also used elsewhere? (If so,
where?)
2) Should shift op tests be based on Table 6:
Shift Operations (which makes sense) or Table 5:
Integer Operations (which 3.58 and 3.59 refer to
under Correctness and Verifiability).
I'd say Table 6.
Done.
6) How do i generate by-ref and ref any fields in
CIL for the 1.6 coercion tests for stfld?
Use:
.field int32 val1
and
.field [mscorlib]System.TypedReference val2
They are not allowed, so I'm not sure ilasm will
compile them.
ILASM doesn't seem to like them.
7) When does coercion happen?
Coercion should happens every time there is a store,
so calls, stloc, starg, stfld, stsfld, stelem, box,
stobj, cpobj, stind.
OK, done. Are there other tests needed for these
opcodes? Do the slot type and stack type also have to
match according to the verification type compatibility
rules?
Cheers,
Jim.
___
Win a BlackBerry device from O2 with Yahoo!. Enter now.
http://www.yahoo.co.uk/blackberry
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] The State Of Mono Assembly Verification?
Hi Paolo/All, --- Paolo Molaro [EMAIL PROTECTED] wrote: I'd like this to be done a bit differently. Take the existing tests you made and change (for example for the 3.1 ones) add with BINARY_NUM_OP. Then a little script will copy and replace and create tests for each of a list of opcodes. This way, we only have to check N test case + the list of opcodes (+ the script, of course;-), instead of checking N * numopcodes tests. OK, I've uploaded a new set here: http://homepage.ntlworld.com/james.purbrick/VerifierTests2.tar Which includes a number of scripts: create_templates.sh creates a template from examplars create_test_sets.sh creates test sets from templates I think they do what you're after. Eventually we can get rid of the examplar tests and just use the templates, create_test_set scripts and individual tests where they make sense. Up to 593 tests now :-D This pattern should be used as much as possible: there are many tests that look the same but apply to a number of similar opcodes and we want the test to check all the cases for all the opcodes while keeping the test source readable. Yes I agree. I have a few questions regarding the remaining tests: 1) Should the 1.8.1.3 tests to check that illegal stack merge failures are caught use the Verification Type (in stack state) types along with the 1.8.1.2.3 rules for type compatibility? It looks like object values should always merge as System.Object is always a supertype of both? 2) Should shift op tests be based on Table 6: Shift Operations (which makes sense) or Table 5: Integer Operations (which 3.58 and 3.59 refer to under Correctness and Verifiability). 3) How do I test that boxing byref, byref-like and void types fails verification. 4) What are overlapped object reference fields and how do I define one for the ldfld and ldflda tests? 5) Which stack types can be stored in valuetype fields? 1.6 seems to suggest that all stack parameter types should fail when stored in a field using stfld. 6) How do i generate by-ref and ref any fields in CIL for the 1.6 coercion tests for stfld? 7) When does coercion happen? Just for stfld and method calls? Stloc and starg check the type on the stack and type of slot match using the 1.8.1.2.3 rules and the Verfication Type set of types from 1.8.1.2.1? Cheers, Jim/Babbage. ___ Win a BlackBerry device from O2 with Yahoo!. Enter now. http://www.yahoo.co.uk/blackberry ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] The State Of Mono Assembly Verification?
OK, I've uploaded a first batch of ~230 verifier tests based on ECMA-335 III here: http://homepage.ntlworld.com/james.purbrick/VerifierTests.tar I'd appreciate it if you could take a look and let me know if they look OK, especially the III,3.1 and III,3.3 tests which I'll mechanically copy and modify to make most of the remaining tests. Feel free to contribute tests of your own though, the README lists tests that are missing for the ECMA sections that I've started working on and lists rules I'm not too clear on. Most of these tests assemble with both Mono and MS ILASM except in the cases where the CIL is so borked that it doesn't make sense (I've left these tests in for completeness though). Happily most of the tests are caught by the Mono runtime which reports that invalid CIL has been found and aborts (Mono 1.1.13). There are cases that cause the runtime to crash and others which just run silently though. I hope they look OK, Cheers, Jim/Babbage. --- Paolo Molaro [EMAIL PROTECTED] wrote: On 02/02/06 Sebastien Pouliot wrote: [...] Excellent mail Sebastien. Just giving a summary for lazy people. *) We have plans to make mono execute untrusted code. *) The more contributions we get in this area, the faster we'll reach our common goal. *) Security is tricky, 1 single bug is enough to have no-security whatsoever. *) If someone waits for the complete secure code before contributing, he won't have any code to contribute, so better start sooner:-) *) We won't give any warranty until the code is complete and an audit has been done by multiple people with no bugs found. *) If you need an assurance for a subset of tests, we could give it, just remember that this doesn't make the complete test case secure. Example: we can guarantee that a subset of IL code is safe to execute, this is not hard and can be done in relatively short time. What matters if this IL subset if sufficient for your needs and that this doesn'ìt imply that things outside the IL code (such as the metadata etc are safe). lupus ___ Yahoo! Photos NEW, now offering a quality print service from just 8p a photo http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Security spokesperson
Hello Christopher, That's great news! I've obviously been spending quite a bit of time rummaging around in Mono and ECMA-335 III,1.8 and will help out as much as I can! Cheers, Jim. --- Sebastien Pouliot [EMAIL PROTECTED] wrote: Hello Christopher, On Fri, 2006-02-03 at 09:06 +0200, Christopher Bergström wrote: [snip] Sebastien Pouliot wrote: I wish that security was more popular and btw I'm seeking contributors for creating a security is cool marketing campaign ;-) I admit I didn't expect any answers but it's... [snip] Need a spokesperson for this? Let me be first in line. ... really cool :-) Another developer and I have started working on a custom patchset for our hardened box that still allows Mono to run, but IMHO a bit more secure. What's the first step? Some documentation? Like a Tips Tricks/Best practices page? Caveats because of missing code.. There's probably more but an article on the wiki (or linked from the wiki) would be nice start. IRC channel? #mono-security ? I'm on both #mono and #monodev but if people thinks it would be easier to talk about security elsewhere I don't have any issue participating in a #mono-security channel. I've got a hit-list I've started to research and work on from my end. Please share your interests :) I'd be excited to get this off the ground. Thanks -- Sebastien Pouliot [EMAIL PROTECTED] Blog: http://pages.infinit.net/ctech/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Does your mail provider give you FREE antivirus protection? Get Yahoo! Mail http://uk.mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] The State Of Mono Assembly Verification?
mono_method_verify () has been superseded by verification during JIT time. When JIT time verification is failed is there an exception thrown that an embedding app can use to take appropriate action? The latter is more correct and complete, though a few checks are still missing. Do you know exactly which checks are missing? I can currently only inject microthreading in to assemblies that use a subset of the full CIL opcodes anyway and so have to whitelist opcodes as I transform assemblies. If the missing checks only apply to opcodes like exception handling that I currently don't allow then the missing checks may not be a problem. There are several rules to enforce, we haven't yet scheduled a full audit to make all the checks and make sure they are correct. Mono development is driven by user needs and user contributions, so contributing fixes and features is the best way to get something done. Fine. We need bytecode verification and are happy to contribute to get it done. Our initial goal is to fully verify the subset of CIL that we currently use. We'll help with advice, code reviews, and actual code, since full verification is one of our mid/long term goals anyway. Allowing the gamut of languages that target the CLI to be used is one of our mid/long term goals too and we'll need full verification to allow that, so our goals are aligned. Plus it helps to know that some people actually need the feature. Well, we need the feature. I suggest starting with a comprehensive list of test cases that we can use as a test suite for the verification process. What's the best way to set up this test suite? Manually craft unverifiable assemblies for each verification check and then test that Mono rejects them? open CLI assembly verifiers I could use instead? There is no complete and tested verifier, so your best bet is to help us improve the mono one. Fine. Also note that a verifier is not enough to ensure secure execution: you need also the CAS runtime support that Sebastien has been working on (activated with the --security switch). Yes, I've been talking to Sebastien about this. We're currently using method call whitelists rather than CAS while we're only allowing the LSL language and library calls, but want to use CAS in the mid/long term when we allow arbitrary languages and open up parts of the .NET framework for use by scripts. Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] The State Of Mono Assembly Verification?
Hi Everyone, It's good to see a healthy debate about these issues :-) I second this. It would be very very useful for us if mono wouldn't g_assert but throw exceptions when the dll is invalid/broken/obfuscated/maliciously modified. Does that mean that you might be able to contribute some time to this too, Joachim? I believe it would be useful to many people - even if most don't realize it today. Until then Mono is restricted to run trusted code which, IMHO, limits it usefulness (I admit the limit is probably rather low as there are very few applications supporting partial trust today). Certainly being able to run untrusted code was a big reason for us to embed Mono as a scripting engine. We want a system that will run untrusted code and to have the performance to do some heavy lifting beyond the lightweight scripting that is currently possible in Second Life using the current LSL interpreter. Anyway the truth (please feel all free to prove me wrong ;-) is that security, especially runtime security, hasn't been very popular with contributors - in any form (e.g. code, samples, reviews, test cases...). It's probably a catch 22. While you can't run untrusted code on Mono, people who want to run untrusted code won't use Mono and so won't contribute to it. While the implementation of security features is incomplete it would be useful to make clear which untrusted uses are possible and to aim to slowly increase the gamut of untrusted uses. It would seem that complete bytecode verification might be a good starting point. Once a complete verifier exists presumably untrusted code that makes no framework calls can be used. In our case, where the only method calls that can be made are iCalls that we already trust, this minimal usage of untrusted code would be very useful. From that point implementing CAS or other security features in such a way that Mono X.Y can allow untrusted code to be loaded as long as conditions N and M are met would gradually increase the utility of Mono for untrusted applications. The question Under what conditions can Mono be used to run untrusted code? is the one that I've been trying to answer by talking to Sebastien, Miguel and this list and I think its a useful question to have an answer to. Having a number of partially implemented security features seems to be of little use, while having some completed features and an understanding of the conditions under which they can be used seems to be useful. Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] The State Of Mono Assembly Verification?
Sounds good! This is the way we'd ideally like to be able to use the verifier: call an API from the process which embeds mono to check the verifiability of an assembly. If the assembly passes the test we store it as an asset and allow it to be loaded. If it fails the test we delete it. It might also make contributing to complete the verifier, or reasoning about the completeness of the verifier easier if it is independent of other code like the JIT. Cheers, Jim. --- Miguel de Icaza [EMAIL PROTECTED] wrote: Hello, IMHO, verification should be kept separate from the JIT. The job of the JIT is to generate machine code _fast_, while the goal of the verifier is to be _secure_. Mixing the two would probably lead to a JIT which wasn't very fast, and it wasn't very secure either. 'We are missing some checks' is a far cry from security. What about this plan: * Introduce an API in the runtime that verifies an assembly. * The API can be invoked from a tool, we already have pedump --verify. * This API could be exposed to those that do not want to call an external process to verify. The API would not be part of the standard JIT processing time, thus we avoid the performance penalty at JIT time. Microsoft does this: their runtime does a few checks, but not all the checks that are done by peverify. Their runtime will happily run invalid code (storing one kind of pointer into a different kind of variable). I wonder when verification is done in the MS runtime for untrusted code though. Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Re: [Mono-devel-list] Limiting Memory Allocation
Hi Ben/Everyone, --- Ben Maurer [EMAIL PROTECTED] wrote: Look at heap-prof. My profiler traps memory freed by the gc. This is currently what I'm using to measure the memory used by different scripts. I basically wrap calls to the scripts in calls to the profiler which turn on and off memory allocation monitoring. There are 2 problems with this approach: 1) When I deserialize a script the single call to IFormatter.Deserialise allocates a lot of memory other than the memory used by the script and a lot of those objects don't seem to be GCed, so a lot of objects not allocated by the script end up being counted towards the script's memory usage. 2) I end up using a lot of memory to keep references to every object allocated by the script so that when a GC occurs I can iterate through the list and return memory to the script for every object in the list which has been collected. Can anyone think of ways round these problems or better ways to keep track of the memory allocated by scripts? Given that I use serialization to move scripts between appdomains and processes I wonder whether just measuring the size of a binary serialization of the script might be a close enough approximation of its memory usage. Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Re: [Mono-devel-list] Limiting Memory Allocation
Hi Ben/Everyone, --- Ben Maurer [EMAIL PROTECTED] wrote: Look at heap-prof. My profiler traps memory freed by the gc. This is currently what I'm using to measure the memory used by different scripts. I basically wrap calls to the scripts in calls to the profiler which turn on and off memory allocation monitoring. There are 2 problems with this approach: 1) When I deserialize a script the single call to IFormatter.Deserialise allocates a lot of memory other than the memory used by the script and a lot of those objects don't seem to be GCed, so a lot of objects not allocated by the script end up being counted towards the script's memory usage. 2) I end up using a lot of memory to keep references to every object allocated by the script so that when a GC occurs I can iterate through the list and return memory to the script for every object in the list which has been collected. Can anyone think of ways round these problems or better ways to keep track of the memory allocated by scripts? Given that I use serialization to move scripts between appdomains and processes I wonder whether just measuring the size of a binary serialization of the script might be a close enough approximation of its memory usage. Cheers, Jim. ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Re: [Mono-devel-list] Limiting Memory Allocation
At the moment no memory is allocated by the script, it's all allocated by calls in to unmanaged library functions or by Deserialization, so I could get the library calls to track the memory they allocate for scripts and then store that in the script, so when it's deserialized I still know how much memory the script is using. That works in the short term, but isn't a general solution for when we allow other languages which can dynamically allocate memory outside library calls and open up bits of the .NET framework which may allocate memory themselves. I don't want to have to transform the bytecode of all the .NET libraries to inject code around every newobj or newarr instruction. Also, is there enough information present at bytecode transormation time to work out how much memory is being allocted? The profiler approach is nice if only I could avoid including the extra Deserialization info and measuring the size of binary serializations seems simple and general too. If anyone else has any ideas how best to do this please let me know. Cheers, Jim. --- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, If you are talking about the memory allocated by the script itself, I think the best solution would be to explicitly generate code to track it, i.e. for each newobj or newarr IL opcode you emit, emit some statements to increase a counter or something. Zoltan On 1/18/06, Jim Purbrick [EMAIL PROTECTED] wrote: Hi Ben/Everyone, --- Ben Maurer [EMAIL PROTECTED] wrote: Look at heap-prof. My profiler traps memory freed by the gc. This is currently what I'm using to measure the memory used by different scripts. I basically wrap calls to the scripts in calls to the profiler which turn on and off memory allocation monitoring. There are 2 problems with this approach: 1) When I deserialize a script the single call to IFormatter.Deserialise allocates a lot of memory other than the memory used by the script and a lot of those objects don't seem to be GCed, so a lot of objects not allocated by the script end up being counted towards the script's memory usage. 2) I end up using a lot of memory to keep references to every object allocated by the script so that when a GC occurs I can iterate through the list and return memory to the script for every object in the list which has been collected. Can anyone think of ways round these problems or better ways to keep track of the memory allocated by scripts? Given that I use serialization to move scripts between appdomains and processes I wonder whether just measuring the size of a binary serialization of the script might be a close enough approximation of its memory usage. Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Creating AppDomains From Embedded Mono
Fixed! It looks as though the problem was with the environment as I suspected. Once I made the assemblies available to mono either by copying them to the working directory or by using mono_set_dirs() the CreateDomain crash went away. I'm not sure why it didn't work when I had MONO_PATH and MONO_CFG_DIR set, but anyway. The second crash which occured the first time I tried to use mono_assembly_open() or mono_domain_assembly_open() to load an assembly in to the new domain seems to have been caused by calling CreateDomain directly and then extracting the MonoDomain* from the MonoAppDomain: switching back to my orignal mechanism (which executed an assembly in the new domain which called mono_domain_get() via an icall to get the new MonoDomain*) seems to fix things. In the future an easy way to create domains from C and/or an easy way to get the MonoDomain* from a MonoAppDomain would be nice as using icalls and calling back and forth between managed and unmanaged code is a faff, but it's not essential. Robert, Paolo and Zoltan, thanks for all your help! Cheers, Jim. --- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, I tried reproducing the crash in CreateDomain, but it seems to work fine for me even on windows (using mono SVN). I will try 1.1.13 later. As for this new crash, I think you should call mono_domain_assembly_open () instead of mono_assembly_open (). Zoltan On 1/12/06, Jim Purbrick [EMAIL PROTECTED] wrote: Hi Robert, Setting MONO_CFG_DIR to C:\Apps\Mono-1.1.13\etc and MONO_PATH to C:\Apps\Mono-1.1.13\lib didn't seem to help. Calling mono_set_dirs(C:\\Apps\\Mono-1.1.13\\lib, C:\\Apps\\Mono-1.1.13\\etc) before calling mono_jit_init(root domain) helped a bit in that I could get rid of the System, PEAPI and Mono.CompilerServices.SymbolWriter DLLs that I'd had to copy to the debugsim.exe directory, but I still get the crash on the call to mono_assembly_open(Script.dll,...). Thanks for all your help, Cheers, Jim. --- Robert Jordan [EMAIL PROTECTED] wrote: Hi Jim, I had a regression (it was actually my mistatake) with 1.1.13 under Windows with my own mono embedding apps: I forgot to set MONO_CFG_DIR to point to mono's etc directory. This seems to prevent System.dll from beeing correctly loaded. You should either set both MONO_PATH and MONO_CFG_DIR or use mono_set_dirs (libdir, etcdir) before calling mono_jit_init (). HTH Robert I just noticed a warning saying that System.dll, which is referenced by another loaded assembly, couldn't be found and that the Mono-INFO messages (output as I have MONO_LOG_LEVEL set to debug as a Windows environment variable) suggest that mono isn't looking for assemblied in the MONO_PATH, which is set to C:\Apps\Mono-1.1.13\lib. If I copy System.dll from C:\Apps\Mono-1.1.13\lib\mono\1.0 to the same directory as debugsim.exe then I no longer get the warning about System.dll _AND_ the error message on the call to AppDomain.CreateDomain() changes to Unhandled exception at 0x7c964ed1 in debugsim.exe: 0xC008: An invalid HANDLE was specified. With the following stack trace: ntdll.dll!7c964ed1() ntdll.dll!7c964ed1() ntdll.dll!7c9268ad() ntdll.dll!7c91056d() ntdll.dll!7c90e9c0() ntdll.dll!7c91901b() ntdll.dll!7c94243c() msvcrt.dll!77c2c2de() ntdll.dll!7c91056d() msvcrt.dll!77c2c2de() msvcrt.dll!77c2c2e3() ntdll.dll!7c90104b() mono.dll!1005a60e() mono.dll!10079c8a() mono.dll!10078dc9() mono.dll!1007a813() mono.dll!1007a327() mono.dll!1007a4e0() mono.dll!1007a5e0() debugsim.exe!load_class(_MonoDomain * domain=0x03839ae8) So, have I just set up MONO_PATH incorrectly so the embedded mono VM can't find the assemblies it needs when it makes the AppDomain.CreateDomain() and mono_assembly_open() calls? That would fit my theory that it's something to do with my new Windows set up that is causing the problem. Currently my MONO_PATH is set to C:\Apps\Mono-1.1.13\lib in the User variables section of the environment. Does that sound right? Should I see Mono-INFO messages about probing directories on the MONO_PATH if everything is working properly? I don't remember seeing warnings about unfound assemblies or having to copy assemblies to the debugsim.exe directory before. Thanks again for all your help, Cheers, Jim. --- Jim Purbrick [EMAIL PROTECTED] wrote: Thanks Zoltan, I've got it working on Linux too and it used to work on Windows until my hard drive
Re: [Mono-dev] Creating AppDomains From Embedded Mono
Hi Robert/Lupus/Everyone,
I've tried Robert's approach (which cleans my code up,
but is vulnerable to changes in _MonoAppDomain as
Paolo said), but I still get the same crash when
making the mono_runtime_invoke() call to
AppDomain.CreateDomain().
At this point I'm struggling with the limited
debugging I can do in VS 2003 when the code calls in
to glib or the mono. The error I get is Unhandled
exception at 0x10059acc in debugsim.exe: 0xC005:
Access violation reading location 0x0007. And the
call stack is:
mono.dll!10059acc()
libglib-2.0-0.dll!0032bb3e()
mono.dll!10059bad()
libglib-2.0-0.dll!0032baf9()
mono.dll!10059736()
mono.dll!1002b3ba()
mono.dll!1002b7a0()
mono.dll!10067527()
debugsim.exe!createDomain(const char *
name=0x0302edf8) Line 31 + 0x11C++
Which doesn't tell me a lot. I wonder whether the
problem is something to do with my new Windows
configuration as the code worked fine with mono
1.1.9.2 in my old Windows installation, but now I get
this problem with mono 1.1.9.2, 1.1.12.1 and 1.1.13.
Could some differences in the environment be causing
the problem? The value of MONO_PATH or some other
environment variable? This is just wild speculation
really. Is there anything else I can do to diagnose
the problem?
At the moment I'm thinking the best thing to do might
be to #define the code to use a single AppDomain on
Windows and just do the secondary AppDomain creation
and unloading on Linux, which is our production
environment anyway. I'm loathe to make the 2 versions
behave differently, but it would allow me to make some
forward progress while trying to sort this problem out
as a background task.
Thanks for all your help on this.
Cheers,
Jim.
--- Robert Jordan [EMAIL PROTECTED] wrote:
Hi Jim,
Hi Robert/Everyone,
You can and *should* invoke the managed
AppDomain methods to load and unload domains.
You don't need an intermediate managed assembly
to do that (untested):
MonoAppDomain*
createDomain (char *name)
void
unloadDomain (MonoAppDomain *domain) ...
That gets me a MonoAppDomain*, which I can
presumably
use to call AppDomain.Load(Byte[]) to load a
script's
assembly, which will be unloaded (along with JIT
output etc.) when I call unloadDomain?
MonoAppDomain is the unmanaged representation of
System.AppDomain.
You can call every System.AppDomain method using
mono_runtime_invoke,
like in my sample above.
How do I turn the MonoAppDomain in to a MonoDomain
required by mono_object_new, mono_string_new etc.?
Indeed, there is no accessor defined for it, but you
can
define this struct somewhere after you include
appdomain.h:
struct _MonoAppDomain {
MonoObject obj;
MonoObject *identity;
MonoDomain *data;
};
That is what my intermediate managed assemblies
were
doing: executing an assembly in the new domain
which
would call mono_domain_get() to get me a
MonoDomain*
for the new MonoAppDomain.
Do I even need a MonoDomain* for the new
MonoAppDomain? At the moment I try to allocate any
objects used by a script in the AppDomain that I
loaded the script's assembly in to, but I suppose
I
could allocate the other objects in the root
domain.
Would there be any problems doing this? My concern
would be that the root domain would end up loading
the
script's assembly which I then couldn't unload.
You have to use the proper MonoDomain.
Robert
___
Yahoo! Photos NEW, now offering a quality print service from just 8p a photo
http://uk.photos.yahoo.com
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Creating AppDomains From Embedded Mono
Thanks Zoltan,
I've got it working on Linux too and it used to work
on Windows until my hard drive died and I needed to
reinstall. I can't think how my old and new Windows
installations differ, so if you get it working I'll be
interested to know how your Windows machine is set up
and also how you're building mono on Windows. I build
mono from source in cygwin using then build the
mono.dll using:
gcc -mno-cygwin -O -g -O2 -fno-strict-aliasing -g
-Wall -Wunused -Wmissing-prototypes
-Wmissing-declarations -Wstrict-prototypes
-Wmissing-prototypes -Wnested-externs -Wpointer-arith
-Wno-cast-qual -Wcast-align -Wwrite-strings -shared -o
mono.dll main.o -Wl,--export-dynamic
-Wl,--export-dynamic ./.libs/libmono.a
-L/usr/local/lib -lgthread-2.0 -lgmodule-2.0
-lglib-2.0 -lintl -liconv -lws2_32 -lpsapi -lole32
I've looked in to building Mono in VS before, but at
the time I looked in to it there were problems with
stack walking, so not everything worked when you built
Mono with VS and also you needed VS 2005 which was
only in beta at the time and SL only built in VS 2003.
Cheers,
Jim.
--- Zoltan Varga [EMAIL PROTECTED] wrote:
Hi,
I tried the example code which creates appdomains
from C code and it
really does crash
because some things are only set up in the runtime
when mono_jit_exec
() is called. So
your workaround of calling CreateDomain () from
managed code seems to
be the way to
go, at least for now. I tried it and it seems to
work on linux, I will
try to look into the
windows problems shortly.
Zoltan
On 1/11/06, Jim Purbrick [EMAIL PROTECTED]
wrote:
Hi Robert/Lupus/Everyone,
I've tried Robert's approach (which cleans my code
up,
but is vulnerable to changes in _MonoAppDomain as
Paolo said), but I still get the same crash when
making the mono_runtime_invoke() call to
AppDomain.CreateDomain().
At this point I'm struggling with the limited
debugging I can do in VS 2003 when the code calls
in
to glib or the mono. The error I get is Unhandled
exception at 0x10059acc in debugsim.exe:
0xC005:
Access violation reading location 0x0007. And
the
call stack is:
mono.dll!10059acc()
libglib-2.0-0.dll!0032bb3e()
mono.dll!10059bad()
libglib-2.0-0.dll!0032baf9()
mono.dll!10059736()
mono.dll!1002b3ba()
mono.dll!1002b7a0()
mono.dll!10067527()
debugsim.exe!createDomain(const char *
name=0x0302edf8) Line 31 + 0x11C++
Which doesn't tell me a lot. I wonder whether the
problem is something to do with my new Windows
configuration as the code worked fine with mono
1.1.9.2 in my old Windows installation, but now I
get
this problem with mono 1.1.9.2, 1.1.12.1 and
1.1.13.
Could some differences in the environment be
causing
the problem? The value of MONO_PATH or some other
environment variable? This is just wild
speculation
really. Is there anything else I can do to
diagnose
the problem?
At the moment I'm thinking the best thing to do
might
be to #define the code to use a single AppDomain
on
Windows and just do the secondary AppDomain
creation
and unloading on Linux, which is our production
environment anyway. I'm loathe to make the 2
versions
behave differently, but it would allow me to make
some
forward progress while trying to sort this problem
out
as a background task.
Thanks for all your help on this.
Cheers,
Jim.
--- Robert Jordan [EMAIL PROTECTED] wrote:
Hi Jim,
Hi Robert/Everyone,
You can and *should* invoke the managed
AppDomain methods to load and unload domains.
You don't need an intermediate managed
assembly
to do that (untested):
MonoAppDomain*
createDomain (char *name)
void
unloadDomain (MonoAppDomain *domain) ...
That gets me a MonoAppDomain*, which I can
presumably
use to call AppDomain.Load(Byte[]) to load a
script's
assembly, which will be unloaded (along with
JIT
output etc.) when I call unloadDomain?
MonoAppDomain is the unmanaged representation of
System.AppDomain.
You can call every System.AppDomain method using
mono_runtime_invoke,
like in my sample above.
How do I turn the MonoAppDomain in to a
MonoDomain
required by mono_object_new, mono_string_new
etc.?
Indeed, there is no accessor defined for it, but
you
can
define this struct somewhere after you include
appdomain.h:
struct _MonoAppDomain {
MonoObject obj;
MonoObject *identity;
MonoDomain *data;
};
That is what my intermediate managed
assemblies
were
doing: executing an assembly in the new domain
which
would call mono_domain_get() to get me a
MonoDomain*
for the new MonoAppDomain.
Do I even need a MonoDomain* for the new
MonoAppDomain
Re: [Mono-dev] Creating AppDomains From Embedded Mono
Hmmm, I may be on to something... I just noticed a warning saying that System.dll, which is referenced by another loaded assembly, couldn't be found and that the Mono-INFO messages (output as I have MONO_LOG_LEVEL set to debug as a Windows environment variable) suggest that mono isn't looking for assemblied in the MONO_PATH, which is set to C:\Apps\Mono-1.1.13\lib. If I copy System.dll from C:\Apps\Mono-1.1.13\lib\mono\1.0 to the same directory as debugsim.exe then I no longer get the warning about System.dll _AND_ the error message on the call to AppDomain.CreateDomain() changes to Unhandled exception at 0x7c964ed1 in debugsim.exe: 0xC008: An invalid HANDLE was specified. With the following stack trace: ntdll.dll!7c964ed1() ntdll.dll!7c964ed1() ntdll.dll!7c9268ad() ntdll.dll!7c91056d() ntdll.dll!7c90e9c0() ntdll.dll!7c91901b() ntdll.dll!7c94243c() msvcrt.dll!77c2c2de() ntdll.dll!7c91056d() msvcrt.dll!77c2c2de() msvcrt.dll!77c2c2e3() ntdll.dll!7c90104b() mono.dll!1005a60e() mono.dll!10079c8a() mono.dll!10078dc9() mono.dll!1007a813() mono.dll!1007a327() mono.dll!1007a4e0() mono.dll!1007a5e0() debugsim.exe!load_class(_MonoDomain * domain=0x03839ae8) So, have I just set up MONO_PATH incorrectly so the embedded mono VM can't find the assemblies it needs when it makes the AppDomain.CreateDomain() and mono_assembly_open() calls? That would fit my theory that it's something to do with my new Windows set up that is causing the problem. Currently my MONO_PATH is set to C:\Apps\Mono-1.1.13\lib in the User variables section of the environment. Does that sound right? Should I see Mono-INFO messages about probing directories on the MONO_PATH if everything is working properly? I don't remember seeing warnings about unfound assemblies or having to copy assemblies to the debugsim.exe directory before. Thanks again for all your help, Cheers, Jim. --- Jim Purbrick [EMAIL PROTECTED] wrote: Thanks Zoltan, I've got it working on Linux too and it used to work on Windows until my hard drive died and I needed to reinstall. I can't think how my old and new Windows installations differ, so if you get it working I'll be interested to know how your Windows machine is set up and also how you're building mono on Windows. I build mono from source in cygwin using then build the mono.dll using: gcc -mno-cygwin -O -g -O2 -fno-strict-aliasing -g -Wall -Wunused -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wno-cast-qual -Wcast-align -Wwrite-strings -shared -o mono.dll main.o -Wl,--export-dynamic -Wl,--export-dynamic ./.libs/libmono.a -L/usr/local/lib -lgthread-2.0 -lgmodule-2.0 -lglib-2.0 -lintl -liconv -lws2_32 -lpsapi -lole32 I've looked in to building Mono in VS before, but at the time I looked in to it there were problems with stack walking, so not everything worked when you built Mono with VS and also you needed VS 2005 which was only in beta at the time and SL only built in VS 2003. Cheers, Jim. --- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, I tried the example code which creates appdomains from C code and it really does crash because some things are only set up in the runtime when mono_jit_exec () is called. So your workaround of calling CreateDomain () from managed code seems to be the way to go, at least for now. I tried it and it seems to work on linux, I will try to look into the windows problems shortly. Zoltan On 1/11/06, Jim Purbrick [EMAIL PROTECTED] wrote: Hi Robert/Lupus/Everyone, I've tried Robert's approach (which cleans my code up, but is vulnerable to changes in _MonoAppDomain as Paolo said), but I still get the same crash when making the mono_runtime_invoke() call to AppDomain.CreateDomain(). At this point I'm struggling with the limited debugging I can do in VS 2003 when the code calls in to glib or the mono. The error I get is Unhandled exception at 0x10059acc in debugsim.exe: 0xC005: Access violation reading location 0x0007. And the call stack is: mono.dll!10059acc() libglib-2.0-0.dll!0032bb3e() mono.dll!10059bad() libglib-2.0-0.dll!0032baf9() mono.dll!10059736() mono.dll!1002b3ba() mono.dll!1002b7a0() mono.dll!10067527() debugsim.exe!createDomain(const char * name=0x0302edf8) Line 31 + 0x11C++ Which doesn't tell me a lot. I wonder whether the problem is something to do with my new Windows configuration as the code worked fine with mono 1.1.9.2
RE: [Mono-dev] Problems Building Mono 1.1.12.1 on Windows
Hi Jonathan (all), Do you have any anti-virus software installed? I (had) Norton installed and I could only build if Norton was shutoff. Bingo! I did have Norton 2005 AV installed and now I've got rid of it I'm not seeing the build process hang anymore. I would have spent ages fiddling with cygwin and never have found the problem if you hadn't pointed me in the right direction! Thanks Jonathan! However, I'm still having problems building Mono on Windows. I've tried building the 1.1.12.1 and 1.1.13 source trees on the Monowin cygwin installations with both 1.1.12.1 and 1.1.13 windows mono installations mounted as /usr/local and here are the errors I get: Building mono 1.1.12.1 with mono 1.1.12.1 as /usr/local and building mono 1.1.13 with mono 1.1.12.1 as /usr/local: make[8]: Entering directory `/home/Jim/mono-1.1.13/mcs/class/Microsoft.VisualBas ic' MONO_PATH=../../class/lib/net_1_1_bootstrap;$MONO_PATH /home/Jim/mono-1.1.13/r untime/mono-wrapper ../../class/lib/net_1_1_bootstrap/ilasm.exe /debug /out:../ ../class/lib/default/Microsoft.VisualBasic.dll /dll fixup/default/Microsoft.Visu alBasic.il Assembling 'fixup/default/Microsoft.VisualBasic.il' , no listing file, to dll -- '../../class/lib/default/Microsoft.VisualBasic.dll' syntax error, got token `ID' Error at: line (1) column (5) Unhandled Exception: Mono.ILASM.yyParser.yyException: irrecoverable syntax error in 0x00f6b Mono.ILASM.ILParser:yyparse (yyInput yyLex) in 0x0008c Mono.ILASM.ILParser:yyparse (yyInput yyLex, System.Object yyd) in 0x001eb Mono.ILASM.Driver+DriverMain:ProcessFile (System.String file_path) make[8]: *** [../../class/lib/default/Microsoft.VisualBasic.dll] Error 1 Building mono 1.1.13 with mono 1.1.13 as /usr/local: make[8]: Entering directory `/home/Jim/mono-1.1.13/mcs/class/System.Security' Creating ../../build/deps/default_System.Security.dll.response ... MONO_PATH=../../class/lib/default;$MONO_PATH /home/Jim/mono-1.1.13/runtime/mon o-wrapper ../../class/lib/default/mcs.exe /codepage:28591 /nologo /optimize -d :NET_1_1 -d:ONLY_1_1 /debug+ /debug:full /noconfig -nowarn:618 -r:mscorlib.dll - r:System.dll -r:System.Xml.dll -debug+ -target:library -out:../../class/lib/defa ult/System.Security.dll @../../build/deps/default_System.Security.dll.response System.Security.Cryptography.Xml\Reference.cs(45,18): warning CS0169: The privat e field `System.Security.Cryptography.Xml.Reference.stream' is assigned but its value is never used System.Security.Cryptography.Xml\XmlDsigEnvelopedSignatureTransform.cs(44,16): w arning CS0169: The private field `System.Security.Cryptography.Xml.XmlDsigEnvelo pedSignatureTransform.comments' is assigned but its value is never used System.Security.Cryptography.Xml\XmlDsigXsltTransform.cs(47,16): warning CS0169: The private field `System.Security.Cryptography.Xml.XmlDsigXsltTransform.commen ts' is assigned but its value is never used Compilation succeeded - 3 warning(s) MONO_PATH=../../class/lib/net_1_1_bootstrap;$MONO_PATH /home/Jim/mono-1.1.13/r untime/mono-wrapper ../../class/lib/net_1_1_bootstrap/sn.exe -q -R ../../class/ lib/default/System.Security.dll ../../class/mono.snk /home/Jim/mono-1.1.13/libtool: line 1: #9787;: command not found Unhandled Exception: System.UnauthorizedAccessException: Access to the path '\' is denied. in 0x0015e System.IO.FileStream:.ctor (System.String name, FileMode mode, File Access access, FileShare share, Int32 bufferSize, Boolean isAsync, Boolean anony mous) in 0x0001f System.IO.FileStream:.ctor (System.String name, FileMode mode, File Access access, FileShare share) in (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,Syste m.IO.FileMode,System.IO.FileAccess,System.IO.FileShare) in 0x00024 System.IO.File:Open (System.String path, FileMode mode, FileAccess access, FileShare share) in 0x00028 Mono.Tools.SN:ReadFromFile (System.String fileName) in 0x0088e Mono.Tools.SN:Main (System.String[] args) make[8]: *** [../../class/lib/default/System.Security.dll] Error 1 If anyone could shine some light on these problems I'd appreciate it! Cheers, Jim. ___ Yahoo! Photos NEW, now offering a quality print service from just 8p a photo http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Creating AppDomains From Embedded Mono
Hi Everyone,
I seem to have fixed my build problems: I ran make
again without changing anything and managed to build
the mono 1.1.13 tree in cygwin with the mono 1.1.13
windows installation. Slightly disconcerting as it
crashed the last time I tried it...
I can run the Second Life simulator again and have it
load the new 1.1.13 mono.dll, run ilasm.exe and my
bytecode transformer, but the evil hack I've been
using to create new AppDomains for the Mono scripts
seems to be unravelling.
To create a domain I execute an assembly in the root
domain which creates another domain and executes
another assembly:
public static void Main()
{
AppDomain new_domain =
AppDomain.CreateDomain(Secondary Domain +
sDomainIndex++);
new_domain.ExecuteAssembly(DomainRegister.exe);
}
DomainRegister.exe then just makes an internal call
which uses mono_domain_get to give me a C reference to
the new domain, which is set up properly by the C#
code. mono_domain_set can then be used from C code to
switch
between secondary domains without any problems.
public class DomainRegister
{
[MethodImplAttribute(MethodImplOptions.InternalCall)]
public extern static void register_domain();
public static void Main()
{
register_domain();
}
}
Everything I tried to do in C to create the domain
left it partially initialised and things would crash
later on.
To unload domains I just call
ves_icall_System_AppDomain_InternalUnload which I have
to declare as an extern as it's not in the public
headers. Again, it's a hack, but it seems to work.
Does anyone know what might have changed recently to
cause this to crash on Windows when the
AppDomain.CreateDomain call is made?
Even better, is it now possible to create properly
initialised AppDomains using the embedding API without
having to call in to managed code and then back in to
unmanaged code again?
Thanks for all your help over the last couple of days.
Hopefully we can get over this (hopefully) last hurdle
and I can get the Mono SL running on Windows again.
Cheers,
Jim.
___
Does your mail provider give you FREE antivirus protection?
Get Yahoo! Mail http://uk.mail.yahoo.com
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Accessing Loaded Assembly Data
Hi everyone, I'm currently using Assembly.Load(byte[]) to load assemblies from memory. Is there a way to get the byte array back later using either the .NET or Mono embedding APIs? I don't want to have to keep a copy hanging around in memory if I can ask for the array back again. Merry Christmas! Jim. ___ Yahoo! Exclusive Xmas Game, help Santa with his celebrity party - http://santas-christmas-party.yahoo.net/ ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Saving And Loading Assemblies To/From Memory
Hi, I'm currently using Assembly.Load(Byte[]) to load assemblies and wondered if there was a way to save assemblies back to byte arrays either using the framework or embedding APIs? Saving and loading assemblies to and from memory in this way would be much easier for me than having to use files (and there seems to be a problem loading more than 800ish assemblies from files on Windows?). Could I do the same thing with AOT assemblies? In other news, I now have Mono executing the current Second Life scripts faster than the current LSL interpreter: http://secondlife.blogs.com/babbage/2005/11/mono_lsl2.html :-) Thanks for all your help. Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Debugging Busted CIL
Did you verify the assembly? Mono: pedump --verify code assembly I'm now getting an interesting error from pedump: Error: Incompatible type Managed Pointer in store at 0x029a What I'm trying to do is store the result of a ldflda instruction. The StackBehaviourPush of the ldflda instruction returns pushi, so I try to store the result in an int32, but MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemreflectionemitopcodesclassldfldatopic.asp) and the ECMA specs (http://dotnet.di.unipi.it/EcmaSpec/PartitionIII/cont4.html#_Toc526908960) both state that ldflda pushes a Managed Pointer on to the stack (which is presumably why pedump is complaining when I try to store the result in an Int32). Is is possible to store and load Managed Pointers in CIL? How? Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Debugging Busted CIL
Is is possible to store and load Managed Pointers in CIL? How? Sorry, should have googled a bit more before my last post. Looks like I can store a Managed Pointer in a local, but not a field :-( http://dotnet.di.unipi.it/EcmaSpec/PartitionIII/cont1.html#_Toc526908860 Looks like I won't be able to yield whenever there is a Managed Pointer on the stack :-( Do any other ops push Managed Pointers (and then claim that their StackBehaviourPush is pushi?) Can anyone see a way around this? Cheers, Jim. ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Debugging Busted CIL
Looks like I can store a Managed Pointer in a local, but not a field :-( Right, storing in a field would have both security and performance effects in the GC, so it's not allowed. OK, to solve that problem it looks as though the best approach is to make sure the value of the field is saved when I'm saving the stack and then inject code which pushes the (new) address of the field when I'm restoring the stack, which is a faff, but not impossible. I think I've found the cause of my current problems though with peverify 2 (which seems much better than 1.1 BTW). It looks as though the normal code pushes an LLVector on to the stack while the restore code unboxes a boxed LLVector which leaves a Managed Pointer to an LLVector on the stack. Is there an easy way to go from the pointer to a value? Thanks for all your help, Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Debugging Busted CIL
I'm currently trying to work out what is wrong with some assemblies I've generated and thought that people on the list might have some cunning techniques that I haven't thought of. Currently my process consists of running my app which embeds Mono, waiting for it to spit out an **ERROR**:Invalid IL code at blah aborting... message and then staring at dissassembly trying to work out stack states on a piece of paper and hoping I see what's wrong. Are there any better ways? I briefly tried using MS ildasm to generate CIL from my DLL, tacking on a Main method which called the broken method then assembling it with debugging information which let me step through the CIL in DbgClr, but when I tried to step in to the busted method DbgClr just told me that the method was busted and stopped. Any other ideas? Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Debugging Busted CIL
Hi Robert, How are you generating the assemblies? Are this assemblies generated by MS tools? Mono doesn't support incremental assemblies generated by CSC. I'm compiling LSL to CIL, then using the mono ILasm compiler to generate assemblies. Did you verify the assembly? Mono: pedump --verify code assembly MS: PEVerify.exe assembly Cool, both give me lots of errors that I can look at, thanks! Mono has a disassembler, too: monodis assembly Yes, I've been able to assemble and disassemble the assemblies without errors, the errors I'm getting have only manifested at runtime, but the verification tools look promising. Thanks again! Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] What about the Windows setup of Mono?
Hi Wade, I'd really like mono.dll and mono.lib files packaged with the Windows install so I can embed mono on Windows. Currently I have to use cygwin to build on Windows from source just to generate a dll and lib to link against (there may be an easier way to do this, please let me know). Cheers, Jim. --- Daniel Morgan [EMAIL PROTECTED] wrote: Hopefully, you can continue allowing the Mono Win32 installers to allow parallel installs. I need the ability to have multiple versions of mono installed. Some application working in one version, such as, Mono 1.1.6 do not work in Mono 1.1.8. Wade Berrier wrote: Hi Kornél, I'm working on the windows installer and it should be finished shortly. I'm shooting for today but it may be tomorrow. The good news is that after I'm done automating it the installers will be released in parallel with the mono releases. Your suggestions are very much appreciated. I believe (Paco, correct me if I'm wrong) the reason Paco used batch files instead of setting up environment variables was so that multiple mono versions could be installed in parallel. I will look into packaging libgdiplus with the installer. I think that's a great idea. Wade On Wed, 2005-09-28 at 20:08 +0200, Kornél Pál wrote: Hi, Is anyone working on it? When will it be released? Some suggestions: I don't like to have bat files in bin directory while having executables in lib directory. I think environment variables should be set in Windows instead of using batch files. And I would like to have an option to add mono\bin directory to PATH. Some others from the archive: http://lists.ximian.com/pipermail/mono-devel-list/2005-September/014803.html http://lists.ximian.com/pipermail/mono-devel-list/2005-September/014817.html Kornél ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] What about the Windows setup of Mono?
Hi Kornél, These files could be included in the installer. Great! But how do you compile Mono to have .lib files? I get no .lib files when compiling Mono. Or do you use Visual C++ .NET? I build the DLL using GCC then use a batch file called genlib (which uses the VC tool dumpbin) to generate the lib from the DLL. http://www.google.com/search?q=cache:KjuOidK5RrYJ:www.beginthread.com/Article/Ehsan/0030+genlib+dumpbin (the original URL seems busted) I'm happy to generate the lib for the release once you have the DLL if you like. Cheers, Jim. ___ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Re: [Mono-devel-list] AOT + Embedding = ?
Hi All, This is due to a problem in out embedding interface. It is now tracked as: http://bugzilla.ximian.com/show_bug.cgi?id=75194 I've now updated to 1.1.9 on Debian sarge and AOT assemblies seem to be looked for and loaded properly, thanks. However, I'm now oprofiling the server embedding mono and JIT compilation still seems to be my high nail (if I'm reading it right). Any idea what's going on here? Here's my oprofile top 10 for libmono.so.0.0.0: samples % symbol name 2687487.7870 mono_jit_compile_method_with_opt 2251476.5237 mono_class_vtable 2127856.1655 mono_jit_runtime_invoke 1735725.0293 mono_get_inflated_method 1628784.7194 EnterCriticalSection 1611654.6698 mono_marshal_get_runtime_invoke 1571514.5535 mono_gchandle_get_target 1420824.1169 anonymous symbol from section .plt 1411254.0891 do_mono_metadata_type_equal 1364193.9528 mono_runtime_class_init MONO_LOG_LEVEL=debug reports AOT loaded AOT Module for each of my assemblies. I double checked by hiding the AOT output so it wasn't found and got almost identical profile results, so it doesn't look like AOT compilation is saving any work. Also, assuming that AOT loading can be made to work and is worthwhile is there any way to load AOT modules from memory? Eventually I'll need to keep them in a remote server and ship them to the local server when needed. Otherwise, the job of moving Second Life to mono is going very well. More details here: http://secondlife.blogs.com/babbage/2005/08/second_life_in_.html http://secondlife.blogs.com/babbage/2005/08/flying_a_mono_b.html :-) Cheers, Jim. ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-list] Building Mono under Cygwin
I'm having a similar problem builing the mono 1.1.9 tarball on windows using cygwin with mcs and csc.exe symlinked to my mono 1.1.7 windows installation. Here are the errors: make[8]: Entering directory `/home/Jim/mono-1.1.9/mcs/class/System.XML' Creating ../../build/deps/basic_System.Xml.dll.response ... ../../jay/jay -ct ../../jay/skeleton.cs System.Xml.XPath/Parser.jay System.Xm l.XPath/Parser.cs C:\cygwin\home\Jim\mono-1.1.9\mcs\jay\jay.exe: 21 rules never reduced C:\cygwin\home\Jim\mono-1.1.9\mcs\jay\jay.exe: 1 shift/reduce conflict, 42 reduc e/reduce conflicts. echo #define XSLT_PATTERN Mono.Xml.Xsl/PatternParser.cs ../../jay/jay -ct Mono.Xml.Xsl/PatternParser.jay ../../jay/skeleton.cs Mono. Xml.Xsl/PatternParser.cs C:\cygwin\home\Jim\mono-1.1.9\mcs\jay\jay.exe: 3 rules never reduced C:\cygwin\home\Jim\mono-1.1.9\mcs\jay\jay.exe: 1 shift/reduce conflict, 46 reduc e/reduce conflicts. echo #define XSLT_PATTERN Mono.Xml.Xsl/PatternTokenizer.cs cat System.Xml.XPath/Tokenizer.cs Mono.Xml.Xsl/PatternTokenizer.cs MONO_PATH=../../class/lib/basic;$MONO_PATH /home/Jim/mono-1.1.9/runtime/mono-w rapper ../../class/lib/basic/mcs.exe /nologo /optimize -d:NET_1_1 -d:ONLY_1_1 -d:BOOTSTRAP_WITH_OLDLIB /debug+ /debug:full /noconfig -r:mscorlib.dll -r:System .dll -nowarn:0162 -nowarn:0618 -nowarn:0612 -target:library -out:System.Xml.dll `echo System.Xml.XPath/Parser.cs Mono.Xml.Xsl/PatternParser.cs Mono.Xml.Xsl/Patt ernTokenizer.cs | tr '/' ''` @../../build/deps/basic_System.Xml.dll.response make[8]: *** [../../class/lib/basic/System.Xml.dll] Error 255 make[8]: Leaving directory `/home/Jim/mono-1.1.9/mcs/class/System.XML' make[7]: *** [do-all] Error 2 make[7]: Leaving directory `/home/Jim/mono-1.1.9/mcs/class/System.XML' make[6]: *** [all-recursive] Error 1 make[6]: Leaving directory `/home/Jim/mono-1.1.9/mcs/class' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/home/Jim/mono-1.1.9/mcs' make[4]: *** [profile-do--basic--all] Error 2 make[4]: Leaving directory `/home/Jim/mono-1.1.9/mcs' make[3]: *** [profiles-do--all] Error 2 make[3]: Leaving directory `/home/Jim/mono-1.1.9/mcs' /home/Jim/mono-1.1.9/libtool: line 1: /home/Jim/mono-1.1.9/mono/handles/semdel: No such file or directory /home/Jim/mono-1.1.9/libtool: line 1: exec: /home/Jim/mono-1.1.9/mono/handles/se mdel: cannot execute: No such file or directory make[2]: *** [all-local] Error 1 make[2]: Leaving directory `/home/Jim/mono-1.1.9/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/Jim/mono-1.1.9' make: *** [all] Error 2 Any ideas? Cheers, Jim. --- Michael Thomsen [EMAIL PROTECTED] wrote: Yep. I have my directory structure like this: d:\mono ---\mono ---\mcs ---\libgdiplus I ran the autogen.sh script in d:\mono\mono and it didn't die. It went straight into configure and that ran to completion. I've built Mono on Linux before without incident from SVN, so I have a bit of an idea of what it looks like when something fails during the configuration stage. I guess I should try building from a nightly tarball to see if it is something wrong with cygwin or something like that. On 9/15/05, Kornél Pál [EMAIL PROTECTED] wrote: Consts.cs is generated by configure in mono tree. Are you sure that autogen.sh (in case of SVN) or configure (in case of source tarball) succeeds? Kornél - Original Message - From: Michael Thomsen [EMAIL PROTECTED] To: Kornél Pál [EMAIL PROTECTED] Cc: mono-list@lists.ximian.com Sent: Thursday, September 15, 2005 4:13 PM Subject: Re: [Mono-list] Building Mono under Cygwin I see this error just before it: Creating ../build/deps/mcs.exe.response ... make[2]: *** No rule to make target `../build/common/Consts.cs', needed by `../class/lib/default/mcs.exe'. Stop. make[2]: Leaving directory `/cygdrive/d/mono/mcs/mcs' On 9/14/05, Kornél Pál [EMAIL PROTECTED] wrote: This is not the error. I don't know why is semdel called because it does not exsist on Windows. The real error message that caused the build to fail is befor the semdel error. Kornél - Original Message - From: Michael Thomsen [EMAIL PROTECTED] To: Kornél Pál [EMAIL PROTECTED] Cc: mono-list@lists.ximian.com Sent: Wednesday, September 14, 2005 4:03 PM Subject: Re: [Mono-list] Building Mono under Cygwin Now I get all the way through the building of the C runtime, but when I get early on into the building of the C# runtime, I get an error that mono/mono/handlers/semdel: no such file or directory On 9/14/05, Kornél Pál [EMAIL PROTECTED] wrote: Simply unistall cygwing glib2-devel package. If you have all the other dependencies you should be able to run configure successfull. Kornél - Original Message - From: Michael Thomsen [EMAIL PROTECTED] To: mono-list@lists.ximian.com Sent: Tuesday,
Re: [Mono-devel-list] Preview of release notes.
Visual Studio solution to build Mono runtime natively on Windows. This is useful if you are embedding the Mono runtime into your application and want to debug and single step inside Mono. Thank you so much for this! It's going to make my life a lot easier! Keep up all the great work you're doing! Cheers, Jim. --- Miguel de Icaza [EMAIL PROTECTED] wrote: Here is a preview of the release notes, please send me updates before the release: http://www.go-mono.com/archive/1.1.8/ -- Miguel de Icaza [EMAIL PROTECTED] Novell, Inc. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] AOT + Embedding = ?
It doesn't look like the embedding runtime is looking for AOT files. When I run mono from the command line I get Mono-INFO: AOT loaded/failed to load AOT module messages, but I don't get these when I run the application embedding mono, just messages talking about loading the normal bytecode assemblies. Is there something I can do to get the embedded runtime to look for AOT files? Cheers, Jim. --- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, Try running mono with the MONO_LOG_LEVEL env variable set to 'debug'. That will show whenever the runtime is able to find the AOT compiled assemblies /methods. Zoltan On 6/8/05, Jim Purbrick [EMAIL PROTECTED] wrote: Hi all, Is there a way to use AOT compilation when embedding mono (or find out if AOT compiled assemblies are being used)? I'm getting a performance hiccup the first time I run methods, which I assume is down to JIT compilation, so want to try using AOT compiled assemblies to see if that solves the problem. I'm AOT compiling the assemblies and the AOT output is available in the same directory as the bytecode assemblies, but I'm still getting a performance hiccups. Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Re: Assembly Unloading
Hi Zoltan, Appdomains should be unloaded using mono_domain_unload (), not mono_domain_free (). mono_domain_unload() only seems to be declared and defined within appdomain.c, is there any reason for this, or could it be declared in appdomain.h so I can call it? When I removed all the calls to mono_domain_free the application ran fine, just as when I was running it with a single appdomain (but obviously not unloading anything). So, the calls to mono_domain_free seem to be the problem. The current domain of the current thread can be set using mono_domain_set (). When I try calling mono_domain_set () immediately after creating the new domains with mono_domain_create (), I get a crash the first time I try to save a thread to disk. Is this becuase I need to set the new domains up somehow? Is this because it's looking for types in the new domain when it serializes and can't find them? Am I right in thinking the current domain is used to load assemblues when deserializing objects? What else is the current domain used for? Cheers, Jim. The call stack I get when trying to serialize the thread after calling mono_domain_set is shown below. System.NullReferenceException: Object reference not set to an instance of an obj ect in 0x0023d System.Runtime.Serialization.Formatters.Binary.CodeGenerator:Genera teMetadataType (System.Type type, StreamingContext context) in 0x00037 System.Runtime.Serialization.Formatters.Binary.ObjectWriter:CreateM emberTypeMetadata (System.Type type) in 0x00587 System.Runtime.Serialization.Formatters.Binary.ObjectWriter:GetObje ctData (System.Object obj, System.Runtime.Serialization.Formatters.Binary.TypeMe tadata metadata, System.Object data) in 0x0003f System.Runtime.Serialization.Formatters.Binary.ObjectWriter:WriteOb ject (System.IO.BinaryWriter writer, Int64 id, System.Object obj) in 0x0014d System.Runtime.Serialization.Formatters.Binary.ObjectWriter:WriteOb jectInstance (System.IO.BinaryWriter writer, System.Object obj, Boolean isValueO bject) in 0x00039 System.Runtime.Serialization.Formatters.Binary.ObjectWriter:WriteQu euedObjects (System.IO.BinaryWriter writer) in 0x00049 System.Runtime.Serialization.Formatters.Binary.ObjectWriter:WriteOb jectGraph (System.IO.BinaryWriter writer, System.Object obj, System.Runtime.Remo ting.Messaging.Header[] headers) in 0x002ce System.Runtime.Serialization.Formatters.Binary.BinaryFormatter:Seri alize (System.IO.Stream serializationStream, System.Object graph, System.Runtime .Remoting.Messaging.Header[] headers) in 0x00015 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter:Seri alize (System.IO.Stream serializationStream, System.Object graph) in 0x000b5 UThread.UThread:Save (System.String filename) ___ Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] merging two assemblies and related problems
Hi Vladimir, --- Vladimir Vukicevic [EMAIL PROTECTED] wrote: 3) Use RAIL (http://rail.dei.uc.pt/). I'm currently using RAIL and Mono-1.1.7 on Windows to transform assemblies. I initially had some problems getting the test application running on Mono-1.1.4, but Zoltan fixed them. So, try running the RAIL examples on the latest Mono and see how you get on. Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-devel-list] Assembly Unloading
Hi Zoltan, Now that Sebastien has helped make the security situation clearer, I think unloading assemblies is (hopefully) the last big issue I need to resolve. I've had a look at mono_assembly_close and it looks like it might be OK to just call that to get rid of the assmembly for a script. Referenced assemblies won't be unloaded, but as scripts should only be referencing library assemblies, which can stay loaded, that seams OK. Looking at the other things that mono_domain_free does hightlights some issues which might be more problematic. It seems to go through trashing vtable, class info, static data and jit output hash tables. Presumably if I wanted to unload an assembly I'd need to selectively remove the data relating to the assembly from these hash tables. Is there a way to find out which entries relate to the assembly I want to unload? In the case of LSL assemblies there is no static data, which might make things easier, although there will presumably be class info and jit output. Could I use the mechanisms used by DynamicMethod to manage the collection of JIT output? Would that just leave me class info to clean up? Given that I know which class I want to get rid of would that be easy enough to find? Is this a reasonable approach to take? Is there anything else I need to think about? Should I just use separate domains for each of the (1000s) of scripts? Should I just implement Richard's suggestion of saving the scripts, destroying the domain and then loading the ones I need again to clear out the assemblies and JIT output? Thanks for all your help, Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
RE: [Mono-devel-list] Limiting Memory Allocation
Hi Sebastien, (Apologies for any confusion, I'm rationalising the private and public conversations I've ended up having with Sebastien and the list) So my big return question is what's kind of environment will you be supporting ?. The more accurate your answer is, the more accurate mine can be ;-) OK. It at _least_ needs to support user authored LSL [1] scripts which migrate between servers along with the virtual objects they are associated with [2][3], can make calls in to the C++ virtual world system and be called by the C++ virtual world system. [1] LSL (http://www.sluniverse.com/lsl.html) already compiles to CIL. [2] Planned via bytecode translation (with Cecil?) a la PicoThreads [3] I need a way to unload code and JIT output on migration. If we get to that point, we'll have the same functionality as the current implementation, but with 300 times the performance thanks to the Mono VM. Stage 2 is to open it up to any language that targets the CLR and allow access to (safe) framework classes. Is there anything that can be done in the meantime? Taking this as a yes|no question I can assure you the answer is: YES ;-) Good :-) Put extra checks in the bytecode translator? Yes, it's possible to pre-check assemblies to ensure they only use class/methods from within the script or from a white list. This sounds like a good approach for stage 1, as all of the IL opcodes will have to be scanned to add the thread serialization code, so checking that all method calls are within the script or from a single library class, shouldn't be too much more work. However this approach limit extensibility, existing code reuse... Remove dangerous bits of the framework so they can't be called? I'm unsure for your project but, in general, I don't believe this is a good approach. First the balance (functionality/security) is hard to keep and even when something gets completed you risk having a very crippled result (functionality-wise), a still insecure subset or (most probably) both. I can see that as desired functionality is increased either the white list or framework trimming approaches provide diminishing returns and require more and more effort. My (probably not very partial ;-) opinion is that both options requires a lot of efforts and time (not just to get there but to continue supporting it later). The same efforts could be invested in defining your security model requirements (well you would need that anyway) and helping to complete CAS to match them. It seems to me that the best approach would be to use a white list approach for stage 1, where we're only considering the very limited LSL language and library calls, then jump straight to CAS for stage 2 where we allow more complex languages and frameworks. Hopefully this would avoid the nasty middle ground where we're expending large amounts of effort trying to grow the white list approach which we could be spending helping to complete CAS. This is also nice in that we don't try to bite off everything in one go, but can get the minimal ammount working on Mono and then move on to the more ambitious goals. But at this stage it's hard to guess when the payoff would be... This is another reason to go for a simple implementation first. I'd rather have a subset working while we worked on completing CAS than have everything potentially delayed. (Now I just really need to find a way to unload assemblies and JIT output when I'm done with them) Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
Even with CAS on Microsoft's .NET, the system isn't really geared to running the way you need to on a game server. No, it's not, but with Mono you get a very good virtual machine that (hopefully) just needs some panel beating in places to work in a game server. In contrast with the Java JVM, the philosphy of .NET isn't one of a virtual machine within the machine. Instead .NET is meant to be a layer over OS functionality that just makes stuff easier for developers. Again, the JVM is a square peg too. The practical upshot of this is for example is if you need to schedule lots of code to run concurrently you have to use OS threads. That's OK for a web server that might have 10 or maybe 20 ASP.NET apps running on it, but if you gave every script on a SecondLife server a seperate thread the machine would grind to a halt. Which is why the plan is to inject bytecode to perform threading in a similar way to the .NET 2 iterators that Thomas was talking about. Likewise .NET doesn't have a mechanism that's explicity designed for handling memory allocation. Does Java? You can do some stuff using the profiler API as you've seen but it's clearly not intended for that use. No, but again if it's good enough, then that's OK. Also, if you need to inject code into a random .NET assembly, you can do that using the profiler API (with a lot of additional code of course!), but it's easier and more robust to create a modified assembly on disk ahead of time instead. Agreed. The solution will probably involve some bytecode translation of assemblies, some appropriation of runtime facilities like profiling and (hopefully not too much) hacking of the runtime and or libraries. A lot of stuff in .NET 1.x seems geared to support the ASP.NET server which figures as they are probably the most advanced 'customer' of the CLR in that version. I think you're right, but I also think there's enough good stuff in there that it's worth trying to bend it a bit to use elsewhere. Unless you're a big Microsoft product team though it's probably quite hard to get features added to the .NET CLR :) Yes, much easier to make use of the things everyday folk leave behind :-) Cheers, Jim. ___ Does your mail provider give you FREE antivirus protection? Get Yahoo! Mail http://uk.mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
--- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, Keep in mind that mono currently doesn't support code access security (CAS), so running assemblies containing untrusted CIL code is not a very safe thing to do. Zoltan Is there anything that can be done in the meantime? Put extra checks in the bytecode translator? Remove dangerous bits of the framework so they can't be called? Cheers, Jim. ___ Yahoo! Messenger - want a free and easy way to contact your friends online? http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
--- Jim Purbrick [EMAIL PROTECTED] wrote: Paolo was talking about DynamicMethod Lightweight Code Generation stuff as a potential solution to this problem, which I'm planing to look in to. Hmmm, I've had a look at this and I don't think DynamicMethod is going to be enough. Besides Richard's suggestion of saving all the scripts and restarting Mono, are there any other ways to unload assemblies and JIT output? I can work out when they aren't needed, so just need a way to remove them from Mono. Cheers, Jim. ___ Does your mail provider give you FREE antivirus protection? Get Yahoo! Mail http://uk.mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
--- Jim Purbrick [EMAIL PROTECTED] wrote: I've got the allocation tracking working with the profiling API, can work out how much memory a script has allocated and throw an exception from there when the limit is reached (admitedly after the allocation which breaks the limit). Is there a way to track how much of a scripts memory is freed so I can work out the current allocated total for each script? I imagine this might be complicated by the GC. It struck me that one way to trap memory freed by the GC would be to make a call from the finalizer of System.Object, which works fine. All I need to do now is mark each object with the Id of the currently running script when I see it allocated, so I know which script to credit with the object's memory when it's collected. The problem is, when I add a field to System.Object to store the Id of the allocated script, mcs crashes seemingly because I'm adding a field to a class with no base. Admitedly it's a pretty esoteric bug, but it would be nice to have it fixed so I can continue with my crazy plan, which looks like it might just work. Alternatively if anyone can see a less crazy way to make this work, I'd love to hear about that too! Cheers, Jim. Here's the stack trace: Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object in 0x0001f Mono.CSharp.TypeContainer:FindBaseMemberWithSameName (string,bool) in 0x0004f Mono.CSharp.FieldBase:CheckBase () in 0x000cf Mono.CSharp.FieldMember:Define () in 0x0001f Mono.CSharp.Field:Define () in 0x00073 MemberCoreArrayList:DefineContainerMembers () in 0x00014 Mono.CSharp.TypeContainer:DefineContainerMembers (Mono.CSharp.TypeC ontainer/MemberCoreArrayList) in 0x00196 Mono.CSharp.TypeContainer:DoDefineMembers () in 0x0002a Mono.CSharp.TypeContainer:DefineMembers (Mono.CSharp.TypeContainer) in 0x00042 Mono.CSharp.RootContext:PopulateCoreType (Mono.CSharp.TypeContainer ,string) in 0x00021 Mono.CSharp.RootContext:BootCorlib_PopulateCoreTypes () in 0x00ace Mono.CSharp.Driver:MainDriver (string[]) in 0xf Mono.CSharp.Driver:Main (string[]) Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
Hi Richard! Given that you are generating all the code from a source script at runtime, why don't you just inject code to monitor memory usage inline? Unless you are revising LSL (I'm a secondlife user) all data operations are value based, so it's pretty easy to see where memory is freed. Ultimately it would be nice if we weren't generating all the code and that this would work with any CIL from anywhere. Having said that, bytecode translation is currently part of the plan, so it could go in there. btw. I'm curious as to how you are dealing with the issue of assemblies (dynamically generated or otherwise) not being unloadable? Paolo was talking about DynamicMethod Lightweight Code Generation stuff as a potential solution to this problem, which I'm planing to look in to. As you need some way to serialize script state anyway, I guess you could serialize all the scripts periodically and unload the AppDomain and start again. At a pinch, this would do. Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Limiting Memory Allocation
--- Zoltan Varga [EMAIL PROTECTED] wrote: Hi, Keep in mind that mono currently doesn't support code access security (CAS), so running assemblies containing untrusted CIL code is not a very safe thing to do. You've both touched on the big issue here: should we be just running LSL on Mono as a new, faster virtual machine or look at LSL as just one language and aim towards running any language which compiles to CIL within SL? I've been keeping the former as a fallback position, but now I'm starting to work on implementing memory quotas and bytecode threading, which would both be much easier implemented in the LSL compiler than as general solutions for any language which targets the CLR, it would be good to make a decision one way or the other. Obviously it's a question we need to decide on, but I'd like to hear as many opinions from the mono community as possible. How long is it likely to be before CAS can be used in anger? Are there any other issues that would affect going down the multi-language route? Cheers, Jim. ___ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-devel-list] Passing Lists To Unmanaged Code
I'm currently embedding mono in a virtual world to support user scripting. The managed scripts need to call library functions implemented in unmanaged code that take or return lists. Is it possible to pass lists across the managed/unmanaged boundary? If so, what's the best way to do it? A lot of the documentation on interoperation suggests that this might be hard or impossible. Cheers, Jim. Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-devel-list] Saving And Loading Stacks (Embedding Mono in a Virtual World)
--- Jim Purbrick [EMAIL PROTECTED] wrote: --- Willibald Krenn [EMAIL PROTECTED] wrote: Is it possible to save and load mono stacks? I don't think this will work. You'd have to guarantee that all code (JIT compiled methods) is at the same place, the methods are actually compiled, all VMTs are correctly initialized, ... Hmm. I was hoping that I'd be able to store the stack in terms of CIL methods and offsets, then rebuild it at the other end. I've had a play with the stack walking code, which gets me the IP and methods on the stack, but looks like I effectively need to decompile the native code back in to CIL to recover the locals and arguments. Couldn't it be made to work in a similar way to a debugger? I still need to look at exactly what the debugger can and can't do, it may be extracting all the information I need from the native stacks and will hopefully give me some clues as to how to manipulate the stack in order to load it at the other end. If code locations and and ensuring compilation are a problem, could I compile ahead of time and ship the binary around between servers? This looks like it might be easier, just to deal with native code and native stacks, rather than try to transfer CIL stacks, but I'm not sure just tranferring the JIT output and native stack would be enough. Alternatively building continuations on the heap and doing saving and loading in bytecode like PicoThreads or JavaGoX might be the way to go after all... Cheers, Jim. Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Embedding Mono in a Virtual World
--- Willibald Krenn [EMAIL PROTECTED] wrote: Is it possible to save and load mono stacks? I don't think this will work. You'd have to guarantee that all code (JIT compiled methods) is at the same place, the methods are actually compiled, all VMTs are correctly initialized, ... Hmm. I was hoping that I'd be able to store the stack in terms of CIL methods and offsets, then rebuild it at the other end. Couldn't it be made to work in a similar way to a debugger? If code locations and and ensuring compilation are a problem, could I compile ahead of time and ship the binary around between servers? (What about all the objects on the heap?) I was planing to compile each script as a single object, so if I transfer the script object and it's stack I will have everything the script needs at the other end. Cheers, Jim. Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Embedding Mono in a Virtual World
--- Rodrigo B. de Oliveira [EMAIL PROTECTED] wrote: You might be able to cook up something with serializable objects and lightweight threads based on generators (yield) But this is just food for though... Yes, this is the other option. You end up with something that looks like PicoThreads for Java, or some of the other Java systems for serializing running threads. It boils down to builing continuations on the heap instead of using the stack. As the continuation objects are just objects you can serialize them like any other object. Then you just need to turn every method in to a state machine so you can jump to any point in the method. This is lots of work though and has a big impact on performance, so I'm hoping I don't have to go there. Trying to use debugging APIs to do this sounds like it might be an easier first option. Cheers, Jim. Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-devel-list] Embedding Mono in a Virtual World
It's been a while since I posted about this stuff, so
I thought I'd give you an update.
I've now got enough of the script language to CLR
compiler written to compile the script language
versions of the mono logic, fibonaci and loops
benchmarks. The current compiler generates code which
runs at about half the speed of the code generated by
mcs. This is still about 400 times faster than the
current interpreted implementation though, which is
good news.
I've also implemented a simple app which embeds mono
and uses NPTL to test the 1 thread per script approach
to concurrency. I could create NPTL threads with 32K
stacks which would run the logic benchmark and managed
to create 8000 threads which ran concurrently, which
is plenty. I think I may have found a bug while
building this application though: if garbage
collection occurs while a thread is being attached the
GC can abort complaining about an unknown thread.
There is code to avoid this happening when a thread is
created by mono, but not when one is attached.
Although using NPTL gives me the level of concurrency
I need, I still need to be able to transparently stop
running scripts, ship them over to another server and
restart them in order to deal with scripted objects
crossing server boundaries.
This brings me back to the issue of saving and loading
of stacks, which we established is hard. The 2
possible approaches seem to be to build continuations
on the heap as in PicoThreads or to write C functions
which are called from the CLR code which save and
restore the stack. Any idea how I'd go about
implementing the C function approach? Any idea which
approach is going to be easier?
In the future we'd also like to have more control over
scheduling so that we can allocate different fractions
of script CPU time to different scripts. Once we can
save and load stacks in order to migrate scripts we
might be able to use that mechanism as the basis of
user threading which would give us more control over
script scheduling.
Cheers,
Jim.
--- Paolo Molaro [EMAIL PROTECTED] wrote:
On 02/11/05 Jim Purbrick wrote:
I think this approach generalises to the
continuation
based approach used by PicoThreads in Java[1]: you
need to build chains of continuations on the heap
that
you can switch between to switch threads,
something
that Dan Sugalski suggests is really inefficient
and
one of the reasons you might want to use Parrot
instead of the JVM or Mono[2].
Except Parrot is still not suitable to develop any
of this stuff:-)
Anyway, as I said it will be slow: with mono 2.0
heap
allocations should become much faster, so, depending
on your timeframe it may not be so bad.
Fibers are cooperatively scheduled light weight
threads in Win32[3]. The core of the approach
boils
down to:
schedule()
{
saveCLRLogicalThreadState(currentThreadState);
switchOSFiber(nextFiber);
restoreCLRLogicalThreadState(nextThreadState);
}
Which is used to keep the logical managed thread
state
and fiber in sync. They had some problems with
exceptions using this approach and the logical
thread
save and restore methods maybe something that's
only
available in Rotor.
I don't see us implementing any of the support
needed for this,
though of corse we would accept good patches to do
it.
Personally, I don't think the complexity it
introduces is
worth it for the default mono behaviour (the good
thing about mono
is that you could write the support and use your own
build with
it enabled to run your app:-).
It should be pretty safe if you inject in the
user
code checks for a
global var that signals the event
Couldn't I just call Thread.Suspend from the main
thread after the timeout?
Yes, but that coiuld lead to deadlocks, since the
script might
hold a lock that another script or your engine needs
to acquire
or worse, it may have a lock in the mono runtime.
We'll be
putting some protection against the latter issue, in
the future, but
until then, it's better to make the script call
Suspend, because
you can ensure it's in a safe place wrt your engine
(and
calling Suspend from managed code on the current
thread is safe
wrt the mono runtime state).
The Suspend method is marked obsolete in 2.0,
because of the potential for deadlocks, but
using it
this way should
be safe.
Will I still be able to use this approach with
future
versions of Mono then?
Obsolete means the call will still work, but you'll
have
a warning when compiling (if you use a C# compiler
or the like).
Since most of the scripts should terminate
within
the timeout, if I understood correctly, you
should
just have a number
of threads created as many as the slow scripts
Yes, as all bar one of the threads would be
suspended
that presumably wouldn't cause any context
switching
problems. The only problem would be that each
suspended thread would still have a full OS thread
stack, so might
Re: [Mono-list] Mono/Cygwin CD - ISO File available for BitTorrent Download
--- Nik Derewianka [EMAIL PROTECTED] wrote: Is there anyone out there that can produce an OSX version?? It would also be good to have a .NET/MSVC version. I've got Mono 1.0.5 building on .NET 2003 using the MSVC6 project files which were mailed round a while ago, but cant seem to fix the mono_jit_runtime_invoke crash, even with the fix that was posted here. Having a clear tutorial for building mono 1.1.4 would be really good! Cheers, Jim. Send instant messages to your online friends http://uk.messenger.yahoo.com ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
