Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-20 Thread Nick Piggin
On Friday 17 April 2009 17:08:07 Jared Hulbert wrote: > > As everyone knows, my favourite thing is to say nasty things about any > > new feature that adds complexity to common code. I feel like crying to > > hear about how many more instances of MS Office we can all run, if only > > we apply this p

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-17 Thread Jared Hulbert
> As everyone knows, my favourite thing is to say nasty things about any > new feature that adds complexity to common code. I feel like crying to > hear about how many more instances of MS Office we can all run, if only > we apply this patch. And the poorly written HPC app just sounds like > scrapi

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-16 Thread Izik Eidus
Nick Piggin wrote: On Wednesday 15 April 2009 08:09:03 Andrew Morton wrote: On Thu, 9 Apr 2009 06:58:37 +0300 Izik Eidus wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. Generally looks OK to me. But that doesn'

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-16 Thread Nick Piggin
On Wednesday 15 April 2009 08:09:03 Andrew Morton wrote: > On Thu, 9 Apr 2009 06:58:37 +0300 > Izik Eidus wrote: > > > KSM is a linux driver that allows dynamicly sharing identical memory > > pages between one or more processes. > > Generally looks OK to me. But that doesn't mean much. We sho

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v3

2009-04-14 Thread Andrew Morton
On Thu, 9 Apr 2009 06:58:37 +0300 Izik Eidus wrote: > KSM is a linux driver that allows dynamicly sharing identical memory > pages between one or more processes. Generally looks OK to me. But that doesn't mean much. We should rub bottles with words like "hugh" and "nick" on them to be sure.

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-07 Thread Andrea Arcangeli
On Sat, Apr 04, 2009 at 05:35:18PM +0300, Izik Eidus wrote: > From v1 to v2: > > 1)Fixed security issue found by Chris Wright: > Ksm was checking if page is a shared page by running !PageAnon. > Beacuse that Ksm scan only anonymous memory, all !PageAnons > inside ksm data strctures are

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Andrea Arcangeli
On Mon, Apr 06, 2009 at 05:04:49PM +1000, Nick Piggin wrote: > They should use a shared memory segment, or MAP_ANONYMOUS|MAP_SHARED etc. > Presumably they will probably want to control it to interleave it over > all numa nodes and use hugepages for it. It would be very little work. I thought it's

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Izik Eidus
Nikola Ciprich wrote: Hi Izik, Is there some user documentation available? (apart from RTFS?:)) I've compiled kernel with v2 of Your patches, loaded ksm module, did echo 1 > /proc/sys/kernel/mm/ksm/run, but I think it didn't do anything, at least no pages were collected.. Could You advise me a bi

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Izik Eidus
Nick Piggin wrote: On Sunday 05 April 2009 01:35:18 Izik Eidus wrote: This driver is very useful for KVM as in cases of runing multiple guests operation system of the same type. (For desktop work loads we have achived more than x2 memory overcommit (more like x3)) Interesting that it

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Nikola Ciprich
Hi Izik, Is there some user documentation available? (apart from RTFS?:)) I've compiled kernel with v2 of Your patches, loaded ksm module, did echo 1 > /proc/sys/kernel/mm/ksm/run, but I think it didn't do anything, at least no pages were collected.. Could You advise me a bit? thanks a lot in advan

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Avi Kivity
Nick Piggin wrote: On Sunday 05 April 2009 01:35:18 Izik Eidus wrote: This driver is very useful for KVM as in cases of runing multiple guests operation system of the same type. (For desktop work loads we have achived more than x2 memory overcommit (more like x3)) Interesting that it

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2009-04-06 Thread Nick Piggin
On Sunday 05 April 2009 01:35:18 Izik Eidus wrote: > This driver is very useful for KVM as in cases of runing multiple guests > operation system of the same type. > (For desktop work loads we have achived more than x2 memory overcommit > (more like x3)) Interesting that it is a desirable workload

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-04-02 Thread Jesper Juhl
On Thu, 2 Apr 2009, Chris Wright wrote: > * Jesper Juhl (j...@chaosbits.net) wrote: > > Do you rely only on the checksum or do you actually compare pages to check > > they are 100% identical before sharing? > > Checksum has absolutely nothing to do w/ finding if two pages match. > It's only used

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-04-02 Thread Chris Wright
* Jesper Juhl (j...@chaosbits.net) wrote: > Do you rely only on the checksum or do you actually compare pages to check > they are 100% identical before sharing? Checksum has absolutely nothing to do w/ finding if two pages match. It's only used as a heuristic to suggest whether a single page has

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-04-02 Thread Izik Eidus
Jesper Juhl wrote: Hi, On Tue, 31 Mar 2009, Izik Eidus wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. Unlike tradtional page sharing that is made at the allocation of the memory, ksm do it dynamicly after the memory was crea

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-04-02 Thread Jesper Juhl
Hi, On Tue, 31 Mar 2009, Izik Eidus wrote: > KSM is a linux driver that allows dynamicly sharing identical memory > pages between one or more processes. > > Unlike tradtional page sharing that is made at the allocation of the > memory, ksm do it dynamicly after the memory was created. > Memory i

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-03-31 Thread Izik Eidus
Anthony Liguori wrote: Izik Eidus wrote: I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Any reason to not take these through upstream QEMU instead of kvm-userspace? In prin

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2009-03-30 Thread Anthony Liguori
Izik Eidus wrote: I am sending another seires of patchs for kvm kernel and kvm-userspace that would allow users of kvm to test ksm with it. The kvm patchs would apply to Avi git tree. Any reason to not take these through upstream QEMU instead of kvm-userspace? In principle, I don't see anyth

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-28 Thread Alan Cox
> You have implemented second one, but seems it already was patented > http://www.google.com/patents?vid=USPAT6789156 > I'm not a lawyer but IMHO we have direct conflict here. > >From other point of view they have patented the WEEL, but at least we > have to know about this. Its an old idea and ap

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-28 Thread Dmitri Monakhov
Izik Eidus <[EMAIL PROTECTED]> writes: > (From v1 to v2 the main change is much more documentation) > > KSM is a linux driver that allows dynamicly sharing identical memory > pages between one or more processes. > > Unlike tradtional page sharing that is made at the allocation of the > memory, ksm

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-20 Thread Ryota OZAKI
2008/11/20 Izik Eidus <[EMAIL PROTECTED]>: > ציטוט Izik Eidus: >> >> ציטוט Ryota OZAKI: >>> >>> Hi Izik, >>> >>> I've tried your patch set, but ksm doesn't work in my machine. >>> >>> I compiled linux patched with the four patches and configured with KSM >>> and KVM enabled. After boot with the lin

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-20 Thread Izik Eidus
ציטוט Izik Eidus: ציטוט Ryota OZAKI: Hi Izik, I've tried your patch set, but ksm doesn't work in my machine. I compiled linux patched with the four patches and configured with KSM and KVM enabled. After boot with the linux, I run two VMs running linux using QEMU with a patch in your mail and s

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux v2

2008-11-20 Thread Izik Eidus
ציטוט Ryota OZAKI: Hi Izik, I've tried your patch set, but ksm doesn't work in my machine. I compiled linux patched with the four patches and configured with KSM and KVM enabled. After boot with the linux, I run two VMs running linux using QEMU with a patch in your mail and started KSM scanner

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Izik Eidus wrote: Andrew Morton wrote: On Tue, 11 Nov 2008 21:18:23 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: hm. There has been the occasional discussion about idenfifying all-zeroes pages and scavenging them, repointing them at the zero page. Could this infrastructure be used for that?

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Andrea Arcangeli
Hi Andrew, thanks for looking into this. On Tue, Nov 11, 2008 at 11:11:10AM -0800, Andrew Morton wrote: > What userspace-only changes could fix this? Identify the common data, > write it to a flat file and mmap it, something like that? The whole idea is to do something that works transparently

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 21:18:23 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: hm. There has been the occasional discussion about idenfifying all-zeroes pages and scavenging them, repointing them at the zero page. Could this infrastructure be used for that? (And how much wo

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 21:18:23 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: > > hm. > > > > There has been the occasional discussion about idenfifying all-zeroes > > pages and scavenging them, repointing them at the zero page. Could > > this infrastructure be used for that? (And how much would we g

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Avi Kivity
Andrew Morton wrote: For kvm, the kernel never knew those pages were shared. They are loaded from independent (possibly compressed and encrypted) disk images. These images are different; but some pages happen to be the same because they came from the same installation media. What users

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 21:07:10 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: > we have used KSM in production for about half year and the numbers that > came from our QA is: > using KSM for desktop (KSM was tested just for windows desktop workload) > you can run as many as > 52 windows xp with 1 gig

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 20:48:16 +0200 Avi Kivity <[EMAIL PROTECTED]> wrote: Andrew Morton wrote: The whole approach seems wrong to me. The kernel lost track of these pages and then we run around post-facto trying to fix that up again. Please explain (for the changel

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 20:48:16 +0200 Avi Kivity <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > > The whole approach seems wrong to me. The kernel lost track of these > > pages and then we run around post-facto trying to fix that up again. > > Please explain (for the changelog) why the kernel

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Avi Kivity wrote: Andrew Morton wrote: The whole approach seems wrong to me. The kernel lost track of these pages and then we run around post-facto trying to fix that up again. Please explain (for the changelog) why the kernel cannot get this right via the usual sharing, refcounting and COWin

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Izik Eidus
Andrew Morton wrote: On Tue, 11 Nov 2008 15:21:37 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: KSM is a linux driver that allows dynamicly sharing identical memory pages between one or more processes. unlike tradtional page sharing that is made at the allocation of the memory, ksm do it dyna

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Avi Kivity
Andrew Morton wrote: The whole approach seems wrong to me. The kernel lost track of these pages and then we run around post-facto trying to fix that up again. Please explain (for the changelog) why the kernel cannot get this right via the usual sharing, refcounting and COWing approaches.

Re: [PATCH 0/4] ksm - dynamic page sharing driver for linux

2008-11-11 Thread Andrew Morton
On Tue, 11 Nov 2008 15:21:37 +0200 Izik Eidus <[EMAIL PROTECTED]> wrote: > KSM is a linux driver that allows dynamicly sharing identical memory pages > between one or more processes. > > unlike tradtional page sharing that is made at the allocation of the > memory, ksm do it dynamicly after the m