Re: [Mono-dev] Should we replace MemoryStream?
I like this idea a lot. The only gotcha is that the chunks would need to be zero'ed out before they go back to the pool to avoid leakage of sensitive information between streams. Since all the chunks are the same size, a simple lock-free stack can be used as pool of chunks. This would ensure pretty fast access to them. - Steve -- Steve G. Bjorg http://mindtouch.com http://twitter.com/bjorg irc.freenode.net #mindtouch On Nov 9, 2009, at 2:53 PM, pablosantosl...@terra.es wrote: > Hi, > > I think a very important place were this chunked mem streams HAVE TO > be > placed (in fact I'll be trying them tomorrow under heavy load on our > testing cluster) is in remoting: there's a huge number of mem streams > being created and destroyed (one for each call) and this can help... > only if the new ChunkedStream is based on a pool, which I don't > think is > the case but probably could be easily doable. > > What do you think about extending ChunkedMemoryStream to include an > option based on a chunk pool, so memory is actually reused instead of > freed and re-allocated? Wouldn't it be good to reduce fragmentation > (I'm > still thinking on remoting)?? > > > pablo > > > www.plasticscm.com > > > > Steve Bjorg wrote: >> The implementation could be adapted so that if the chunked memory >> stream is initialized with an existing byte array it behaves like it >> did in the past. It's possible that the best approached can be >> derived for the various MemoryStream constructors. >> >> The question is what is the most common usage pattern? If it's >> GetBuffer(), then there will be a performance and overhead hit. >> However, if it's using Write()/Read() as we do, then there are some >> significant gains to be had. GetBytes() would also benefit, though >> not as much. >> >> - Steve >> >> -- >> Steve G. Bjorg >> http://mindtouch.com >> http://twitter.com/bjorg >> irc.freenode.net #mindtouch >> >> On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote: >> >>> Are you talking about System.IO.MemoryStream? >>> Then imho this would be a problematic move. >>> Most people are used to new MemoryStream (someByteArray) being O(0) >>> time, >>> but with ChunkedStream would be O(n). In fact in those cases >>> ChunkedStream >>> would need twice the memory because it would still need to retain >>> the >>> original byte array (e.g. for GetBuffer). >>> >>> Happy hacking >>> Andreas >>> >>> -Ursprüngliche Nachricht- >>> Von: mono-devel-list-boun...@lists.ximian.com >>> [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von >>> Miguel de >>> Icaza >>> Gesendet: Montag, 9. November 2009 19:10 >>> An: mono-devel-list >>> Betreff: [Mono-dev] Should we replace MemoryStream? >>> >>> Hello folks, >>> >>> I just blogged about a memory fragmentation issue here: >>> >>> http://tirania.org/blog/archive/2009/Nov-09.html >>> >>> And I am wondering: since MemoryStream is one of these sources of >>> problems, we could replace this implementation with MindTouch's >>> ChunkedStream. >>> >>> Miguel. >>> >>> ___ >>> Mono-devel-list mailing list >>> Mono-devel-list@lists.ximian.com >>> http://lists.ximian.com/mailman/listinfo/mono-devel-list >>> >>> ___ >>> Mono-devel-list mailing list >>> Mono-devel-list@lists.ximian.com >>> http://lists.ximian.com/mailman/listinfo/mono-devel-list >> >> ___ >> Mono-devel-list mailing list >> Mono-devel-list@lists.ximian.com >> http://lists.ximian.com/mailman/listinfo/mono-devel-list >> > ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Soft debugger HOWTO
On 9-Nov-09, at 6:33 PM, pablosantosl...@terra.es wrote: >>> - Is there any other available interface to use the soft debugger >>> besides MD? >> You can create any interface you like on top of the >> Mono.Debugger.Soft.dll, MonoDevelop is the interface we support. > > Will it make sense to have some sort of command line interface? Maybe > not, MD will always be better... > We have no plans to make a command line interface, someone from the community is free to do so. Our suggested and supported interface is going to be MonoDevelop -g ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] on-the-spot Input method editing in mono WinForms
Hi,
I need to implement in a custom control on-the-spot Input method
editing. The Control is driven by WinForm events - although all the
drawing is completely custom.
I have had a first stab at adding support to mono's X11 classes to allow
a custom control to get at this information. I've attached the (not
finished) initial patch for comments and suggestions.
I have also attached a small test program which I use for testing this.
For which I run it with:
XMODIFIERS="@im=SCIM"
MONO_WINFORMS_XIM_STYLE=on-the-spot mono TestApp.exe
I have been testing with Scim using 'Amharic' and 'Chinese (Simplified)
- Wubi' (note: mono winforms seems to have problems rending Amharic -
but it's good enough for testing).
What I would like to know is what are the changes of accepting a patch
which allows custom controls to hook into mono's XIM use?
I have currently put externally access-able classes in a
System.Windows.Forms.X11 namespace, it this the right thing to do?
Another thing which I think may be necessary is the ability to have a
control specific xim or a least to be able to switch xim in application
code ( currently it seems to be one per applications) This would allow
some controls to do on-the-spot and some to do an alternative method
like off-the-spot.
Thanks
Tom
Index: class/Managed.Windows.Forms/System.Windows.Forms/X11Keyboard.cs
===
--- class/Managed.Windows.Forms/System.Windows.Forms/X11Keyboard.cs (revision 144586)
+++ class/Managed.Windows.Forms/System.Windows.Forms/X11Keyboard.cs (working copy)
@@ -32,11 +32,91 @@
//
using System;
using System.Collections;
+using System.Diagnostics;
using System.Drawing;
using System.Text;
using System.Globalization;
using System.Runtime.InteropServices;
+namespace System.Windows.Forms.X11
+{
+ public class XIM
+ {
+ ///
+ /// Allows user to request preedit notifications.
+ ///
+ public interface IPreedit
+ {
+ int PreeditStart(IntPtr xic, IntPtr clientData, IntPtr callData);
+ int PreeditDone(IntPtr xic, IntPtr clientData, IntPtr callData);
+ int PreeditDraw (IntPtr xic, IntPtr clientData, IntPtr callData);
+ int PreeditCaret (IntPtr xic, IntPtr clientData, IntPtr callData);
+ }
+
+ ///
+ /// Allow Applications to register interest in X Input methods preedit callbacks
+ /// TODO: Generalize this to allow specifiying window/control
+ ///
+ public static void RegisterPreeditNotification(IPreedit preedit)
+ {
+ System.Windows.Forms.XplatUIX11.Keyboard.RegisterPreeditNotification(preedit);
+ }
+
+ ///
+ /// Converts IntPtr if XIMPreeditDrawCallbackStruct into an easier to use form
+ ///
+ public class PreeditDrawInfo
+ {
+ public PreeditDrawInfo(IntPtr ximPreeditDrawPtr)
+ {
+if (ximPreeditDrawPtr != IntPtr.Zero)
+{
+ XIMPreeditDrawCallbackStruct preeditStruct = (XIMPreeditDrawCallbackStruct) Marshal.PtrToStructure (ximPreeditDrawPtr, typeof (XIMPreeditDrawCallbackStruct));
+ this.Caret = preeditStruct.Caret;
+ this.ChangeFirst = preeditStruct.ChangeFirst;
+ this.ChangeLength = preeditStruct.ChangeLength;
+ if (preeditStruct.Text != IntPtr.Zero)
+ {
+ XIMText text = (XIMText) Marshal.PtrToStructure(preeditStruct.Text, typeof(XIMText));
+ {
+ this.Length = text.Length;
+ if (text.Feedback != IntPtr.Zero)
+ {
+XIMFeedbackStruct Feedback = (XIMFeedbackStruct) Marshal.PtrToStructure(text.Feedback, typeof(XIMFeedbackStruct));
+this.Feedback = Feedback.FeedbackMask;
+ }
+ this.EncodingIsWChar = text.EncodingIsWChar;
+ if (text.String != IntPtr.Zero)
+ {
+if (text.EncodingIsWChar)
+ this.String = Marshal.PtrToStringUni(text.String);
+else
+ this.String = Marshal.PtrToStringAuto(text.String);
+ }
+ }
+ }
+}
+ }
+
+ public override string ToString()
+ {
+return String.Format("PreeditDrawInfo: Carret = {0} ChangeFirst = {1} ChangeLength = {2} Length = {3} Feedback = {4} EncodingIsWChar = {5} String = {6}",
+ Caret, ChangeFirst, ChangeLength, Length, Feedback, EncodingIsWChar, String);
+ }
+
+ public int Caret = 0;
+ public int ChangeFirst = 0;
+ public int ChangeLength = 0;
+ public ushort Length = 0;
+ public ulong Feedback = 0;
+ public bool EncodingIsWChar = false;
+ public String String = string.Empty;
+ }
+
+ }
+
+}
+
namespace System.Windows.Forms {
internal class X11Keyboard : IDisposable {
@@ -63,6 +143,8 @@
private int NumLockMask;
private int AltGrMask;
+
+ private System.Windows.Forms.X11.XIM.IPreedit ximPreedit;
public X11Keyboard (IntPtr display, IntPtr clientWindow)
{
@@ -1001,9 +1083,14 @@
}
}
+ public void RegisterPreeditNotification(System.Windows.Forms.X11.XIM.IPreedit preedit)
+ {
+ ximPreedit = preedit;
+ }
+
private IntPtr CreateOnTheSpotXic (IntPtr window, I
Re: [Mono-dev] Soft debugger HOWTO
Hi Geoff, >> - I understand MD is the primary interface *right now* for the soft >> debugger, is it possible to debug a Linux app too? > > You mean a native app? No. You mean a mono app on linux? Yes, on x86 > and amd64 and arm currently. I'm only concerned 99.99% of the time about managed code. So that's what I need. >> - Is there any other available interface to use the soft debugger >> besides MD? > You can create any interface you like on top of the > Mono.Debugger.Soft.dll, MonoDevelop is the interface we support. Will it make sense to have some sort of command line interface? Maybe not, MD will always be better... >> - As Miguel pointed to me last week, the new softdeb will allow debuggin >> gon any platform, I don't know if anyone experienced it yet (I'll >> definitely give it a try asap), if so, any feedback? >> > > We currently have it ported to: > > - linux (x86, amd64, arm, arm-full-aot) > - darwin (x86, arm, arm-full-aot) > > The point miguel was making is that porting to new architectures is a > minor (<1 day) task in most cases. Excellent! ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Soft debugger HOWTO
On 9-Nov-09, at 6:11 PM, pablosantosl...@terra.es wrote: > Hi, > > After the announcement last week of the new soft debugger, I have > several questions: > > - I understand MD is the primary interface *right now* for the soft > debugger, is it possible to debug a Linux app too? You mean a native app? No. You mean a mono app on linux? Yes, on x86 and amd64 and arm currently. > > - Is there any other available interface to use the soft debugger > besides MD? > You can create any interface you like on top of the Mono.Debugger.Soft.dll, MonoDevelop is the interface we support. > - As Miguel pointed to me last week, the new softdeb will allow > debuggin > gon any platform, I don't know if anyone experienced it yet (I'll > definitely give it a try asap), if so, any feedback? > We currently have it ported to: - linux (x86, amd64, arm, arm-full-aot) - darwin (x86, arm, arm-full-aot) The point miguel was making is that porting to new architectures is a minor (<1 day) task in most cases. For example, Jonathan Chambers has it mostly ported to win32 already -g ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Hello, > What do you think about extending ChunkedMemoryStream to include an > option based on a chunk pool, so memory is actually reused instead of > freed and re-allocated? Wouldn't it be good to reduce fragmentation (I'm > still thinking on remoting)?? This would work; Additionally, it might make sense to use the suggestion from Steve to use plain memory allocation up to a certain point (below some threshold) and then switch to chunk after that to avoid allocating 16k even for memory streams that might only have 100 bytes for example. > > > pablo > > > www.plasticscm.com > > > > Steve Bjorg wrote: > > The implementation could be adapted so that if the chunked memory > > stream is initialized with an existing byte array it behaves like it > > did in the past. It's possible that the best approached can be > > derived for the various MemoryStream constructors. > > > > The question is what is the most common usage pattern? If it's > > GetBuffer(), then there will be a performance and overhead hit. > > However, if it's using Write()/Read() as we do, then there are some > > significant gains to be had. GetBytes() would also benefit, though > > not as much. > > > > - Steve > > > > -- > > Steve G. Bjorg > > http://mindtouch.com > > http://twitter.com/bjorg > > irc.freenode.net #mindtouch > > > > On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote: > > > >> Are you talking about System.IO.MemoryStream? > >> Then imho this would be a problematic move. > >> Most people are used to new MemoryStream (someByteArray) being O(0) > >> time, > >> but with ChunkedStream would be O(n). In fact in those cases > >> ChunkedStream > >> would need twice the memory because it would still need to retain the > >> original byte array (e.g. for GetBuffer). > >> > >> Happy hacking > >> Andreas > >> > >> -Ursprüngliche Nachricht- > >> Von: mono-devel-list-boun...@lists.ximian.com > >> [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von > >> Miguel de > >> Icaza > >> Gesendet: Montag, 9. November 2009 19:10 > >> An: mono-devel-list > >> Betreff: [Mono-dev] Should we replace MemoryStream? > >> > >> Hello folks, > >> > >>I just blogged about a memory fragmentation issue here: > >> > >>http://tirania.org/blog/archive/2009/Nov-09.html > >> > >>And I am wondering: since MemoryStream is one of these sources of > >> problems, we could replace this implementation with MindTouch's > >> ChunkedStream. > >> > >> Miguel. > >> > >> ___ > >> Mono-devel-list mailing list > >> Mono-devel-list@lists.ximian.com > >> http://lists.ximian.com/mailman/listinfo/mono-devel-list > >> > >> ___ > >> Mono-devel-list mailing list > >> Mono-devel-list@lists.ximian.com > >> http://lists.ximian.com/mailman/listinfo/mono-devel-list > > > > ___ > > Mono-devel-list mailing list > > Mono-devel-list@lists.ximian.com > > http://lists.ximian.com/mailman/listinfo/mono-devel-list > > > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Soft debugger HOWTO
Hi, After the announcement last week of the new soft debugger, I have several questions: - I understand MD is the primary interface *right now* for the soft debugger, is it possible to debug a Linux app too? - Is there any other available interface to use the soft debugger besides MD? - As Miguel pointed to me last week, the new softdeb will allow debuggin gon any platform, I don't know if anyone experienced it yet (I'll definitely give it a try asap), if so, any feedback? I know it's still a little bit too soon, but once the softdeb gets some good visibility, I honestly trust we're about to see a huge change in the way Mono is used. pablo ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Hi, I think a very important place were this chunked mem streams HAVE TO be placed (in fact I'll be trying them tomorrow under heavy load on our testing cluster) is in remoting: there's a huge number of mem streams being created and destroyed (one for each call) and this can help... only if the new ChunkedStream is based on a pool, which I don't think is the case but probably could be easily doable. What do you think about extending ChunkedMemoryStream to include an option based on a chunk pool, so memory is actually reused instead of freed and re-allocated? Wouldn't it be good to reduce fragmentation (I'm still thinking on remoting)?? pablo www.plasticscm.com Steve Bjorg wrote: > The implementation could be adapted so that if the chunked memory > stream is initialized with an existing byte array it behaves like it > did in the past. It's possible that the best approached can be > derived for the various MemoryStream constructors. > > The question is what is the most common usage pattern? If it's > GetBuffer(), then there will be a performance and overhead hit. > However, if it's using Write()/Read() as we do, then there are some > significant gains to be had. GetBytes() would also benefit, though > not as much. > > - Steve > > -- > Steve G. Bjorg > http://mindtouch.com > http://twitter.com/bjorg > irc.freenode.net #mindtouch > > On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote: > >> Are you talking about System.IO.MemoryStream? >> Then imho this would be a problematic move. >> Most people are used to new MemoryStream (someByteArray) being O(0) >> time, >> but with ChunkedStream would be O(n). In fact in those cases >> ChunkedStream >> would need twice the memory because it would still need to retain the >> original byte array (e.g. for GetBuffer). >> >> Happy hacking >> Andreas >> >> -Ursprüngliche Nachricht- >> Von: mono-devel-list-boun...@lists.ximian.com >> [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von >> Miguel de >> Icaza >> Gesendet: Montag, 9. November 2009 19:10 >> An: mono-devel-list >> Betreff: [Mono-dev] Should we replace MemoryStream? >> >> Hello folks, >> >>I just blogged about a memory fragmentation issue here: >> >> http://tirania.org/blog/archive/2009/Nov-09.html >> >>And I am wondering: since MemoryStream is one of these sources of >> problems, we could replace this implementation with MindTouch's >> ChunkedStream. >> >> Miguel. >> >> ___ >> Mono-devel-list mailing list >> Mono-devel-list@lists.ximian.com >> http://lists.ximian.com/mailman/listinfo/mono-devel-list >> >> ___ >> Mono-devel-list mailing list >> Mono-devel-list@lists.ximian.com >> http://lists.ximian.com/mailman/listinfo/mono-devel-list > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list > ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Hello, > Are you talking about System.IO.MemoryStream? > Then imho this would be a problematic move. > Most people are used to new MemoryStream (someByteArray) being O(0) time, > but with ChunkedStream would be O(n). In fact in those cases ChunkedStream > would need twice the memory because it would still need to retain the > original byte array (e.g. for GetBuffer). I dont think it would be O(n), there is a dereference only. > > Happy hacking > Andreas > > -Ursprüngliche Nachricht- > Von: mono-devel-list-boun...@lists.ximian.com > [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von Miguel de > Icaza > Gesendet: Montag, 9. November 2009 19:10 > An: mono-devel-list > Betreff: [Mono-dev] Should we replace MemoryStream? > > Hello folks, > > I just blogged about a memory fragmentation issue here: > > http://tirania.org/blog/archive/2009/Nov-09.html > > And I am wondering: since MemoryStream is one of these sources of > problems, we could replace this implementation with MindTouch's > ChunkedStream. > > Miguel. > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list > ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
The implementation could be adapted so that if the chunked memory stream is initialized with an existing byte array it behaves like it did in the past. It's possible that the best approached can be derived for the various MemoryStream constructors. The question is what is the most common usage pattern? If it's GetBuffer(), then there will be a performance and overhead hit. However, if it's using Write()/Read() as we do, then there are some significant gains to be had. GetBytes() would also benefit, though not as much. - Steve -- Steve G. Bjorg http://mindtouch.com http://twitter.com/bjorg irc.freenode.net #mindtouch On Nov 9, 2009, at 11:41 AM, Andreas Nahr wrote: > Are you talking about System.IO.MemoryStream? > Then imho this would be a problematic move. > Most people are used to new MemoryStream (someByteArray) being O(0) > time, > but with ChunkedStream would be O(n). In fact in those cases > ChunkedStream > would need twice the memory because it would still need to retain the > original byte array (e.g. for GetBuffer). > > Happy hacking > Andreas > > -Ursprüngliche Nachricht- > Von: mono-devel-list-boun...@lists.ximian.com > [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von > Miguel de > Icaza > Gesendet: Montag, 9. November 2009 19:10 > An: mono-devel-list > Betreff: [Mono-dev] Should we replace MemoryStream? > > Hello folks, > >I just blogged about a memory fragmentation issue here: > > http://tirania.org/blog/archive/2009/Nov-09.html > >And I am wondering: since MemoryStream is one of these sources of > problems, we could replace this implementation with MindTouch's > ChunkedStream. > > Miguel. > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list > > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
On Mon, Nov 9, 2009 at 2:48 PM, Miguel de Icaza wrote: > Avery wrote: >> Probably stupid question: why is a compacting garbage collector >> actually needed? C programs have survived for a *long* time without >> any ability whatsoever to compact memory, simply by carefully >> optimizing their allocation algorithms to avoid fragmentation. Is the >> mono allocator very non-optimal in this respect? > > Long running applications tend to have this problem. > > Either people ignore the problem, or they come up with some creative > solutions, and the solutions are all over the map. > > In some cases, for long running processes, people split work into > separate processes and recycle the processes (for example Apache) in > other cases, they use custom memory allocation, pre-allocated pools, or > mark/release pools. Thanks for the response. That makes sense to me. Short answer: everybody has this problem, but it's only solvable in the general case when you have GC and compaction. Thanks :) Avery ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Hello, > Probably stupid question: why is a compacting garbage collector > actually needed? C programs have survived for a *long* time without > any ability whatsoever to compact memory, simply by carefully > optimizing their allocation algorithms to avoid fragmentation. Is the > mono allocator very non-optimal in this respect? Long running applications tend to have this problem. Either people ignore the problem, or they come up with some creative solutions, and the solutions are all over the map. In some cases, for long running processes, people split work into separate processes and recycle the processes (for example Apache) in other cases, they use custom memory allocation, pre-allocated pools, or mark/release pools. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Are you talking about System.IO.MemoryStream? Then imho this would be a problematic move. Most people are used to new MemoryStream (someByteArray) being O(0) time, but with ChunkedStream would be O(n). In fact in those cases ChunkedStream would need twice the memory because it would still need to retain the original byte array (e.g. for GetBuffer). Happy hacking Andreas -Ursprüngliche Nachricht- Von: mono-devel-list-boun...@lists.ximian.com [mailto:mono-devel-list-boun...@lists.ximian.com] Im Auftrag von Miguel de Icaza Gesendet: Montag, 9. November 2009 19:10 An: mono-devel-list Betreff: [Mono-dev] Should we replace MemoryStream? Hello folks, I just blogged about a memory fragmentation issue here: http://tirania.org/blog/archive/2009/Nov-09.html And I am wondering: since MemoryStream is one of these sources of problems, we could replace this implementation with MindTouch's ChunkedStream. Miguel. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Hey, On Mon, Nov 9, 2009 at 6:29 PM, Avery Pennarun wrote: > On Mon, Nov 9, 2009 at 1:10 PM, Miguel de Icaza wrote: > >I just blogged about a memory fragmentation issue here: > > > >http://tirania.org/blog/archive/2009/Nov-09.html > > > >And I am wondering: since MemoryStream is one of these sources of > > problems, we could replace this implementation with MindTouch's > > ChunkedStream. > > Probably stupid question: why is a compacting garbage collector > actually needed? C programs have survived for a *long* time without > any ability whatsoever to compact memory, simply by carefully > optimizing their allocation algorithms to avoid fragmentation. Is the > mono allocator very non-optimal in this respect? > One of the causes is that in a garbage collected language you allocate when you need something and discard it when you're done. If you do this with large buffers which are pinned in memory and have just the wrong allocation pattern, you can bloat your memory usage. There's nothing mono can do in this case as essentially it's the user causing the bloat. One thing you can do is keep a cache of buffers yourself and re-use them. For example if your application allocates 10 chunked memory streams a second and Dispose () them when you're done, you could add/remove the 'chunks' from a cache. This way you'd only ever allocate 10xsizeof (chunked stream) bytes of memory and you'd constantly re-use them. Alan. > Like I said, I feel like this is a stupid question. But I'm curious > about the answer, and neither your blog post nor the linked page on > the sgen collector addresses it. > > Thanks, > > Avery > ___ > Mono-devel-list mailing list > Mono-devel-list@lists.ximian.com > http://lists.ximian.com/mailman/listinfo/mono-devel-list > ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
Miguel de Icaza wrote: > Hello folks, > > I just blogged about a memory fragmentation issue here: > > http://tirania.org/blog/archive/2009/Nov-09.html > > And I am wondering: since MemoryStream is one of these sources of > problems, we could replace this implementation with MindTouch's > ChunkedStream. > I'm really liking the idea of a chunked stream like this. Even once sgen is complete and deployed, a chunked stream will still be more efficient - not just in avoiding fragmentation, but also because reallocating a new, larger buffer, has overhead associated with copying the content whereas allocating a new chunk is a cheaper operation. Jeff ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Should we replace MemoryStream?
On Mon, Nov 9, 2009 at 1:10 PM, Miguel de Icaza wrote: > I just blogged about a memory fragmentation issue here: > > http://tirania.org/blog/archive/2009/Nov-09.html > > And I am wondering: since MemoryStream is one of these sources of > problems, we could replace this implementation with MindTouch's > ChunkedStream. Probably stupid question: why is a compacting garbage collector actually needed? C programs have survived for a *long* time without any ability whatsoever to compact memory, simply by carefully optimizing their allocation algorithms to avoid fragmentation. Is the mono allocator very non-optimal in this respect? Like I said, I feel like this is a stupid question. But I'm curious about the answer, and neither your blog post nor the linked page on the sgen collector addresses it. Thanks, Avery ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Should we replace MemoryStream?
Hello folks, I just blogged about a memory fragmentation issue here: http://tirania.org/blog/archive/2009/Nov-09.html And I am wondering: since MemoryStream is one of these sources of problems, we could replace this implementation with MindTouch's ChunkedStream. Miguel. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Deadlock in System.Web.Caching.Cache class
Issue was in bad patch applied on mono/metadata/filewatcher.c - so no errors on mono side :-) I will try to create test case for this deadlock, since it is not OS dependent, but anyway I don't think it is very serious Best regards, Ivan Marek Habersack napisa: > Ivan Radovanovic wrote: >> Although this dead lock problem continues to potentially exists it >> seems that problem is after all OS specific - there is some weird >> behavior of fam/gamin reporting that bin/*.dll files are changed, >> causing ASP.Net runtime trying to restart application, while at the >> same time trying to compile *.aspx, *.ascx etc. > The issue might be in FAM FileSystemWatcher backend then. > >> Maybe deadlock can occur in normal conditions too (servicing some >> request that would need compiling of some control/page that is not >> compiled yet and replacing something in bin directory at the same >> time), but that should be rare enough :-) > The BuildManager in 2.4 has some bugs indeed, which in certain cases can > lead to deadlocks (it's, among others, related to recursive dependencies > between application files). For that reason I rewrote BuildManager for > 2.6+ - if the problem you're seeing turns out to be an issue with more > than few environments, I can just copy the new code to 2.4 branch for a > future release. If you can come up with a test case that triggers the > issue on your system, then please file a bug report and attach the > testcase including OS/environment details. > > marek >> >> Regards >> >> Ivan Radovanovic napisa: >>> Hello, I am experiencing weird deadlock in .net applications running >>> latest release version of mono (2.4.2.3) on FreeBSD (I don't think it >>> is OS specific, and it doesn't show all the times, but still often >>> enough so I can trace it) >>> >>> Stack from thread 1: >>>at >>> System.Web.Compilation.BuildManager.RemoveVirtualPathFromCaches(System.Web.VirtualPath >>> >>> virtualPath) >>>at >>> System.Web.Compilation.BuildManager.OnVirtualPathChanged(System.String >>> key, System.Object value, CacheItemRemovedReason removedReason) >>>at System.Web.Caching.Cache.InvokePrivateCallbacks() >>>at System.Web.HttpRuntime.ShutdownAppDomain(System.Object args) >>> >>> Stack from thread 2: >>>at System.Web.Caching.Cache.Add(System.String key, System.Object >>> value, System.Web.Caching.CacheDependency dependencies, DateTime >>> absoluteExpiration, TimeSpan slidingExpiration, CacheItemPriority >>> priority, System.Web.Caching.CacheItemRemovedCallback onRemoveCallback) >>>at System.Web.Compilation.BuildManager.AddToCache(System.String >>> virtualPath, System.Web.Compilation.BuildProvider bp) >>>at >>> System.Web.Compilation.BuildManager.GenerateAssembly(System.Web.Compilation.AssemblyBuilder >>> >>> abuilder, System.Collections.Generic.List`1 buildItems, >>> System.Web.VirtualPath virtualPath, BuildKind buildKind) >>>at >>> System.Web.Compilation.BuildManager.BuildAssembly(System.Web.VirtualPath >>> virtualPath) >>>at >>> System.Web.Compilation.BuildManager.GetCompiledType(System.String >>> virtualPath) >>>at >>> System.Web.Compilation.AspComponentFoundry+TagNameFoundry.LoadType() >>>at >>> System.Web.Compilation.AspComponentFoundry+TagNameFoundry.GetType(System.String >>> >>> componentName, System.String ByRef source, System.String ByRef ns) >>>at >>> System.Web.Compilation.AspComponentFoundry.CreateComponent(System.Web.Compilation.Foundry >>> >>> foundry, System.String tagName, System.String prefix, System.String tag) >>>at >>> System.Web.Compilation.AspComponentFoundry.GetComponent(System.String >>> tagName) >>>at System.Web.UI.RootBuilder.GetChildControlType(System.String >>> tagName, IDictionary attribs) >>>at System.Web.UI.ControlBuilder.CreateSubBuilder(System.String >>> tagid, System.Collections.Hashtable atts, System.Type childType, >>> System.Web.UI.TemplateParser parser, ILocation location) >>>at System.Web.Compilation.AspGenerator.ProcessTag(ILocation >>> location, System.String tagid, System.Web.Compilation.TagAttributes >>> atts, TagType tagtype, Boolean ByRef ignored) >>>at System.Web.Compilation.AspGenerator.TagParsed(ILocation >>> location, TagType tagtype, System.String tagid, >>> System.Web.Compilation.TagAttributes attributes) >>>at System.Web.Compilation.AspParser.OnTagParsed(TagType tagtype, >>> System.String id, System.Web.Compilation.TagAttributes attributes) >>>at System.Web.Compilation.AspParser.Parse() >>>at System.Web.Compilation.AspGenerator.Parse(System.IO.TextReader >>> reader, System.String filename, Boolean doInitParser) >>>at System.Web.Compilation.GenericBuildProvider`1.Parse() >>>at >>> System.Web.Compilation.GenericBuildProvider`1.get_CodeCompilerType() >>>at >>> System.Web.Compilation.BuildManager.GetCodeDomProviderType(System.Web.Compilation.B
Re: [Mono-dev] Unstable ThreadPool performance under massive QueueUserWorkItem requests.
What Mono version are you using? If you observe this behaviour with a recent version (2.4.2.3 is current), could you please post your test case? On Sat, Nov 7, 2009 at 11:55 AM, Laser Lu wrote: > I'm writing a program that utilizes ThreadPool heavily. More than thousands > of QueueUserWorkItem request may be generated during a second. And this > program works fine on Windows/.NET. > When I come to test on Linux/Mono, it becomes unstable. Large CPU > consumption often occurs. The same program usually takes 1-5% CPU on Windows > and similar hardware. However, on CentOS(2 core), the CPU usage may suddenly > jump to a very high (nearly 200%) level and then fall back to normal, and > then jump high again.. It seems very unstable. > > Below is the output of command 'top |grep mono' (run on CentOS, 2 core CPU): > 12926 root 15 0 43644 11m 4980 S 16.0 0.3 0:00.48 mono > > 12926 root 15 0 48068 15m 5052 S 18.6 0.5 0:01.04 mono > > 12926 root 15 0 80640 20m 5060 S 168.0 0.6 0:06.09 mono > > 12926 root 15 0 87372 22m 5064 S 189.3 0.7 0:11.78 mono > > 12926 root 15 0 85816 24m 5064 S 52.9 0.7 0:13.37 mono > > 12926 root 19 0 86408 24m 5064 S 13.3 0.7 0:13.77 mono > > 12926 root 19 0 86408 24m 5064 S 195.6 0.7 0:19.65 mono > > 12926 root 15 0 86408 24m 5064 S 195.9 0.7 0:25.54 mono > > 12926 root 18 0 91740 26m 5064 S 199.6 0.8 0:31.98 mono > > 12926 root 18 0 91740 26m 5064 S 191.2 0.8 0:37.96 mono > > 12926 root 15 0 91740 26m 5064 S 193.6 0.8 0:43.78 mono > > 12926 root 15 0 91740 26m 5064 S 199.3 0.8 0:49.77 mono > > 12926 root 15 0 91740 26m 5064 S 198.9 0.8 0:55.75 mono > > 12926 root 15 0 91740 26m 5064 S 199.9 0.8 1:01.76 mono > > 12926 root 15 0 91740 26m 5064 S 199.5 0.8 1:07.76 mono > > 12926 root 15 0 91740 26m 5064 S 199.9 0.8 1:13.77 mono > > 12926 root 15 0 91740 26m 5064 S 198.9 0.8 1:19.75 mono > > 12926 root 15 0 91740 26m 5064 S 199.3 0.8 1:25.74 mono > > 12926 root 15 0 91740 26m 5064 S 199.6 0.8 1:31.74 mono > > 12926 root 15 0 91740 26m 5064 S 199.9 0.8 1:37.75 mono > > 12926 root 23 0 91788 28m 5064 S 87.5 0.8 1:40.38 mono > > 12926 root 23 0 91196 28m 5064 S 0.3 0.8 1:40.39 mono > > 12926 root 15 0 91788 28m 5064 S 20.3 0.8 1:41.00 mono > > 12926 root 15 0 91196 28m 5064 S 4.3 0.8 1:41.13 mono > > 12926 root 15 0 91772 28m 5064 S 11.3 0.8 1:41.47 mono > > 12926 root 15 0 91772 27m 5064 S 22.0 0.8 1:42.13 mono > > 12926 root 15 0 91772 26m 5064 S 28.9 0.8 1:43.00 mono > > 12926 root 24 0 91196 26m 5064 S 1.3 0.8 1:43.04 mono > > 12926 root 15 0 91804 26m 5064 S 34.3 0.8 1:44.07 mono > > 12926 root 15 0 91196 26m 5064 S 4.3 0.8 1:44.20 mono > > 12926 root 17 0 91772 25m 5064 S 13.6 0.7 1:44.61 mono > > 12926 root 15 0 91196 25m 5064 S 16.0 0.7 1:45.09 mono > > 12926 root 15 0 91292 25m 5064 S 1.3 0.7 1:45.13 mono > > 12926 root 15 0 91804 24m 5064 S 134.6 0.7 1:49.40 mono > > 12926 root 15 0 91196 24m 5064 S 13.4 0.7 1:49.81 mono > > 12926 root 15 0 91196 24m 5064 S 0.7 0.7 1:49.83 mono > > 12926 root 15 0 91772 24m 5064 S 21.6 0.7 1:50.48 mono > > 12926 root 15 0 91804 23m 5064 S 3.3 0.7 1:50.58 mono > > 12926 root 15 0 91804 23m 5064 S 22.3 0.7 1:51.25 mono > > 12926 root 15 0 91804 23m 5064 S 3.3 0.7 1:51.35 mono > > 12926 root 24 0 91804 23m 5064 S 14.0 0.7 1:51.77 mono > > 12926 root 15 0 91196 23m 5064 S 14.6 0.7 1:52.21 mono > > 12926 root 15 0 91196 23m 5064 S 2.7 0.7 1:52.29 mono > > 12926 root 15 0 91804 23m 5064 S 178.9 0.7 1:57.67 mono > > 12926 root 15 0 91804 23m 5064 S 184.2 0.7 2:03.38 mono > > 12926 root 15 0 91804 23m 5064 S 196.5 0.7 2:09.29 mono > > 12926 root 15 0 91804 23m 5064 S 198.2 0.7 2:15.25 mono > > 12926 root 15 0 91804 23m 5064 S 198.1 0.7 2:21.21 mono > > 12926 root 15 0 91804 23m 5064 S 123.7 0.7 2:24.93 mono > > 12926 root 15 0 91196 24m 5064 S 1.7 0.7 2:24.98 mono > > 12926 root 15 0 91804 24m 5064 S 70.5 0.7 2:27.10 mono > > 12926 root 15 0 91804 24m 5064 S 198.7 0.7 2:33.27 mono > > 12926 root 15 0 91196 24m 5064 S 23.0 0.7 2:33.98 mono > > 12926 root 20 0 91212 24m 5064 S 1.0 0.7 2:34.01 mono > > 12926 root 15 0 91788 24m 5064 S 25.6 0.7 2:34.78 mono > > 12926 root 15 0 91196 24m 5064 S 1.0 0.7 2:34.81 mono > > 12926 root 17 0 91772 24m 5064 S 15.0 0.7 2:35.26 mono > > 12926 root 15 0 91196 24m 5064 S 12.0 0.7 2:35.62 mono > > 12926 root 15 0 91500 23m 5064 S 3.7 0.7 2:35.73 mono > > 12926 root 15 0 9
