Re: Review Request: Fixing massive memory consumption in KWin using the raster graphicssystem.
--- This is an automatically generated e-mail. To reply, visit: http://git.reviewboard.kde.org/r/101385/#review3402 --- those are some impressive results. my guess is that we likely have a very similar issue in plasma. in both cases, we are dealing with lots of small bits of memory in the form of pixmaps, strings, etc. that are fairly constantly coming and going. switching between compositing and no compositing is probably the 'worst case' real world scenario for this. if this does test out well, i'd suggest putting this code in libkworkspace so that all of our apps with this usage pattern (plasma-desktop, plasma-netbook, krunner, kwin, ..) can take advantage of a common implementation. a set of static methods in the KWorkspace namespace should suffice. if you could repeat your experiment with plasma-desktop with and without such a patch, that would be great and let us know whether we should adopt such a strategy across the plasma shell components. as for portability ... i'm not overly concerned about the portability of this to non-glibc systems as those systems are our primary targets. however, instead of checking for GLIBC the code could simply check for M_TRIM_THRESHOLD. for comparison, here is how busybox does this same trick for both the trim size and mmap request size threshold: #ifndef PAGE_SIZE # define PAGE_SIZE (4*1024) /* guess */ #endif #ifdef M_TRIM_THRESHOLD /* M_TRIM_THRESHOLD is the maximum amount of freed top-most memory * to keep before releasing to the OS * Default is way too big: 256k */ mallopt(M_TRIM_THRESHOLD, 2 * PAGE_SIZE); #endif #ifdef M_MMAP_THRESHOLD /* M_MMAP_THRESHOLD is the request size threshold for using mmap() * Default is too big: 256k */ mallopt(M_MMAP_THRESHOLD, 8 * PAGE_SIZE - 256); #endif i doubt we need to tweak the mmap strategy, but i like the checks for the definitions of mallopt there. for portability to non-unix-y systems, we could wrap this whole thing in a #ifndef Q_WS_WIN block which should keep the KDE Windows team from pulling their hair out over the use of such UNIX-y API ;) - Aaron J. On May 18, 2011, 8:53 p.m., Philipp Knechtges wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > http://git.reviewboard.kde.org/r/101385/ > --- > > (Updated May 18, 2011, 8:53 p.m.) > > > Review request for kwin and Plasma. > > > Summary > --- > > The intention of this patch is to lower the heap fragmentation in KWin when > using the raster backend. > > One can illustrate the issue with the following testcase: If one currently > uses the raster backend and > switches back to non-compositing mode one gets easily a similar memory usage > like the following: > > Situation: 14 windows, QtCurve > KWin started with compositing: 40 MB > KWin switched to non-compositing : more than 70 MB > > The first guess might be, that this is due to a memleak because of not > properly released pixmaps. > But calling malloc_stats() sheds some more light on the subject > (non-compositing mode): > > Arena 0: > system bytes = 72232960 > in use bytes =8180512 > Arena 1: > system bytes = 135168 > in use bytes = 4784 > Total (incl. mmap): > system bytes = 73080832 > in use bytes =8898000 > max mmap regions = 13 > max mmap bytes = 36343808 > > In other words, the glibc has allocated more than 70 MB memory although KWin > uses only less than 10 MB. > The problem is that glibc only resizes the heap if the heap has more than 128 > KB of free space at the end, but > many decoration pixmaps are smaller. Using mallopt to tune the threshold to > 20 KB (i'm open for other suggestions?) > fixes the issue. After the patch: > > KWin in compositing mode: 19 MB > KWin switched to non-compositing: 13 MB > > > Arena 0: > system bytes = 12374016 > in use bytes =6894544 > Arena 1: > system bytes = 135168 > in use bytes = 4784 > Total (incl. mmap): > system bytes = 13750272 > in use bytes =8140416 > max mmap regions = 67 > max mmap bytes = 99463168 > > Are there some negative effects? > The only negative effect i am aware of is, that Glibc free() is calling the > sbrk syscall more often. But this should > not be a bottleneck. > > > Diffs > - > > kwin/composite.cpp 9edb99d > kwin/main.cpp f767f54 > > Diff: http://git.reviewboard.kde.org/r/101385/diff > > > Testing > --- > > Tested using a standard Fedora 14. > Would be nice to know, whether other OSes have similar issues? > Martin Gräßlin had some concerns about the portability? > > > Thanks, > > Philipp > > ___ Plasma-devel mailing list Plasma-devel@kde
Re: questions about "remote web services"
On Wednesday, May 18, 2011 16:20:02 Aaron J. Seigo wrote: > * one source per service, e.g. youtube NO! :D For this kind of stuff, we should completely (== as much as possible) ignore the service itself. So instead of a youtube engine, we create a videoengine (which itself uses youtube as a backend). The backends are interchangeable, so if today youtube is hot, and tomorrow blip, we can easily change it in the client plasmoids. Same goes for other webservices. Overall your advice is good, of course, just for the case of "generic webservices", we've started to go into that direction. Cheers, -- sebas http://www.kde.org | http://vizZzion.org | GPG Key ID: 9119 0EF9 ___ Plasma-devel mailing list Plasma-devel@kde.org https://mail.kde.org/mailman/listinfo/plasma-devel
Re: questions about "remote web services"
On Thursday, May 19, 2011 10:40:11 Sebastian Kügler wrote:
> On Wednesday, May 18, 2011 16:20:02 Aaron J. Seigo wrote:
> > * one source per service, e.g. youtube
>
> NO! :D
>
> For this kind of stuff, we should completely (== as much as possible) ignore
> the service itself. So instead of a youtube engine, we create a videoengine
> (which itself uses youtube as a backend). The backends are interchangeable,
yes, that's essentially what i'm suggesting. something like:
Media DataEngine:
Youtube
Blip
Foobar
Baz
and the plasmoid can request a service object for each backend:
foreach (const QString &service, engine->sources()) {
services << engine->serviceForSource(service);
}
so one engine with multiple backends.
an altertnative is to forgoe the DataEngine entirely and just provide a
Service plugin which hides the idea that there are multiple providers
entirely. the implementation on the Service side would be nearly identical to
the above: one front end, multiple back ends.
however, with a "single Service that hides the backends" approach we lose the
ability for provider-specific features (e.g. one service might provide the
ability to search by "friends", another might not) and the ability to easily /
conveniently display to the user where it was coming from. with a DataEngine
approach that information would be contained in each source (where one source
represents one provider). with a Service, that information would need to be
encoded in each individual result returned, which is probably not great.
--
Aaron J. Seigo
humru othro a kohnu se
GPG Fingerprint: 8B8B 2209 0C6F 7C47 B1EA EE75 D6B7 2EB1 A7F1 DB43
KDE core developer sponsored by Qt Development Frameworks
signature.asc
Description: This is a digitally signed message part.
___
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
Python Plasmoids - 2 Questions
Hi there,
well, I am not exactly shure that I am on the right list here, yet
somewhere Aron's words about heading over to plasma-devel for any
qestions still ring in my head. If I am wrong, where should I ask instead?
So, currently I am working on a python plasmoid (first time plasmoid
development for me) and am currently struggling with two issues, namely:
1) dbus calls:
How do I write the following C++ call in python?
QDBusInterface("org.kde.krunner", "/App",
"org.kde.krunner.App").call(QDBus.NoBlock, "showTaskManager");
QDBusInterface is not wrapped by PyQT so that I should propably use the
dbus module. And then? How?
2) Implement toolTipAboutToShow():
In my plasmoid and to save cpu cycles I want to do some actions only
before / while showing a tooltip. For this purpose, the
toolTipAboutToShow() function can be used, I think. Unfortunatly, the
following snippet shows a working tooltip, yet the toolTipAboutToShow()
function of my class is never called but the following error / warning
is shown:
QMetaObject::invokeMethod: No such method
Plasma::Applet::toolTipAboutToShow()
So, how do I override toolTipAboutToShow with my own implementation?
class SimpleCpuMeter(plasmascript.Applet):
def __init__(self,parent,args=None):
plasmascript.Applet.__init__(self,parent)
def init(self):
...
self.setHasConfigurationInterface(False)
self.initToolTip()
def initToolTip(self):
self.tooltipdata = Plasma.ToolTipContent()
self.tooltipdata.setMainText("System Information")
self.tooltipdata.setImage(KIcon('view-statistics'))
Plasma.ToolTipManager.self().registerWidget(self.applet)
self.updateToolTip()
def toolTipAboutToShow(self):
print("toolTipAboutToShow")
Thanks, Michael
___
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
Re: Python Plasmoids - 2 Questions
In data giovedì 19 maggio 2011 13:38:49, Michael Zimmermann ha scritto:
> QDBusInterface is not wrapped by PyQT so that I should propably use the
> dbus module. And then? How?
Yes, you should use the dbus interface. Something like this:
import dbus
from dbus.mainloop.qt import DBusQtMainLoop
# This makes calls async
DBusQtMainLoop(set_as_default=True)
session_bus = dbus.SessionBus()
proxy = session_bus.get_object("org.kde.krunner", "/App")
proxy.showTaskManager(dbus_interface="org.kde.krunner.App")
Further details in this tutorial: http://dbus.freedesktop.org/doc/dbus-
python/doc/tutorial.html (ignore the fact that it says that the only event
loop supported is GLib).
> QMetaObject::invokeMethod: No such method
> Plasma::Applet::toolTipAboutToShow()
I think someone else raised this issue, but I'm not sure how and if it was
solved.
signature.asc
Description: This is a digitally signed message part.
___
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
Re: Python Plasmoids - 2 Questions
Luca, On 19/05/11 14:06, Luca Beltrame wrote: > Yes, you should use the dbus interface. Something like this: > this works. Thanks for the quick response :-) Michael ___ Plasma-devel mailing list Plasma-devel@kde.org https://mail.kde.org/mailman/listinfo/plasma-devel
Re: questions about "remote web services"
On Thursday, May 19, 2011 11:26:36 Aaron J. Seigo wrote:
> On Thursday, May 19, 2011 10:40:11 Sebastian Kügler wrote:
> > On Wednesday, May 18, 2011 16:20:02 Aaron J. Seigo wrote:
> > > * one source per service, e.g. youtube
> >
> >
> >
> > NO! :D
> >
> >
> >
> > For this kind of stuff, we should completely (== as much as possible)
> > ignore the service itself. So instead of a youtube engine, we create a
> > videoengine (which itself uses youtube as a backend). The backends are
> > interchangeable,
>
> yes, that's essentially what i'm suggesting. something like:
>
> Media DataEngine:
> Youtube
> Blip
> Foobar
> Baz
>
> and the plasmoid can request a service object for each backend:
>
> foreach (const QString &service, engine->sources()) {
> services << engine->serviceForSource(service);
> }
>
> so one engine with multiple backends.
>
> an altertnative is to forgoe the DataEngine entirely and just provide a
> Service plugin which hides the idea that there are multiple providers
> entirely. the implementation on the Service side would be nearly identical
> to the above: one front end, multiple back ends.
>
> however, with a "single Service that hides the backends" approach we lose
> the ability for provider-specific features (e.g. one service might
> provide the ability to search by "friends", another might not) and the
> ability to easily / conveniently display to the user where it was coming
> from. with a DataEngine approach that information would be contained in
> each source (where one source represents one provider). with a Service,
> that information would need to be encoded in each individual result
> returned, which is probably not great.
Yep, agree. It's not only service-specific features, but also users (/apps)
who want just one service, like Jérôme who wants to create a youtube player.
As far as I know, the video engine right now supports queries such as
"youtube:my little pony" (not sure about the exact syntax).
In the metadata engine, we ended up putting the results under queries (those
can have one or more types, and the added benefit that the results that come
from the same queries are shared across visulizations. (The latter being quite
useful since you then can just create DataSources anywhere in your QML code,
and end up having more self-contained components. If you need to share the
DataSource object across QML components, code will be messier and more
brittle.
Cheers,
--
sebas
http://www.kde.org | http://vizZzion.org | GPG Key ID: 9119 0EF9
___
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
Re: Review Request: Fixing massive memory consumption in KWin using the raster graphicssystem.
> On May 19, 2011, 7:34 a.m., Aaron J. Seigo wrote: > > those are some impressive results. my guess is that we likely have a very > > similar issue in plasma. in both cases, we are dealing with lots of small > > bits of memory in the form of pixmaps, strings, etc. that are fairly > > constantly coming and going. switching between compositing and no > > compositing is probably the 'worst case' real world scenario for this. > > > > if this does test out well, i'd suggest putting this code in libkworkspace > > so that all of our apps with this usage pattern (plasma-desktop, > > plasma-netbook, krunner, kwin, ..) can take advantage of a common > > implementation. a set of static methods in the KWorkspace namespace should > > suffice. > > > > if you could repeat your experiment with plasma-desktop with and without > > such a patch, that would be great and let us know whether we should adopt > > such a strategy across the plasma shell components. > > > > as for portability ... i'm not overly concerned about the portability of > > this to non-glibc systems as those systems are our primary targets. > > however, instead of checking for GLIBC the code could simply check for > > M_TRIM_THRESHOLD. for comparison, here is how busybox does this same trick > > for both the trim size and mmap request size threshold: > > > > #ifndef PAGE_SIZE > > # define PAGE_SIZE (4*1024) /* guess */ > > #endif > > #ifdef M_TRIM_THRESHOLD > > /* M_TRIM_THRESHOLD is the maximum amount of freed top-most memory > > * to keep before releasing to the OS > > * Default is way too big: 256k > > */ > >mallopt(M_TRIM_THRESHOLD, 2 * PAGE_SIZE); > > #endif > > #ifdef M_MMAP_THRESHOLD > >/* M_MMAP_THRESHOLD is the request size threshold for using mmap() > > * Default is too big: 256k > > */ > >mallopt(M_MMAP_THRESHOLD, 8 * PAGE_SIZE - 256); > > #endif > > > > i doubt we need to tweak the mmap strategy, but i like the checks for the > > definitions of mallopt there. for portability to non-unix-y systems, we > > could wrap this whole thing in a #ifndef Q_WS_WIN block which should keep > > the KDE Windows team from pulling their hair out over the use of such > > UNIX-y API ;) sysconf(_SC_PAGESIZE) returns the page size for mmap, no need for (4*1024) - Ivan --- This is an automatically generated e-mail. To reply, visit: http://git.reviewboard.kde.org/r/101385/#review3402 --- On May 18, 2011, 8:53 p.m., Philipp Knechtges wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > http://git.reviewboard.kde.org/r/101385/ > --- > > (Updated May 18, 2011, 8:53 p.m.) > > > Review request for kwin and Plasma. > > > Summary > --- > > The intention of this patch is to lower the heap fragmentation in KWin when > using the raster backend. > > One can illustrate the issue with the following testcase: If one currently > uses the raster backend and > switches back to non-compositing mode one gets easily a similar memory usage > like the following: > > Situation: 14 windows, QtCurve > KWin started with compositing: 40 MB > KWin switched to non-compositing : more than 70 MB > > The first guess might be, that this is due to a memleak because of not > properly released pixmaps. > But calling malloc_stats() sheds some more light on the subject > (non-compositing mode): > > Arena 0: > system bytes = 72232960 > in use bytes =8180512 > Arena 1: > system bytes = 135168 > in use bytes = 4784 > Total (incl. mmap): > system bytes = 73080832 > in use bytes =8898000 > max mmap regions = 13 > max mmap bytes = 36343808 > > In other words, the glibc has allocated more than 70 MB memory although KWin > uses only less than 10 MB. > The problem is that glibc only resizes the heap if the heap has more than 128 > KB of free space at the end, but > many decoration pixmaps are smaller. Using mallopt to tune the threshold to > 20 KB (i'm open for other suggestions?) > fixes the issue. After the patch: > > KWin in compositing mode: 19 MB > KWin switched to non-compositing: 13 MB > > > Arena 0: > system bytes = 12374016 > in use bytes =6894544 > Arena 1: > system bytes = 135168 > in use bytes = 4784 > Total (incl. mmap): > system bytes = 13750272 > in use bytes =8140416 > max mmap regions = 67 > max mmap bytes = 99463168 > > Are there some negative effects? > The only negative effect i am aware of is, that Glibc free() is calling the > sbrk syscall more often. But this should > not be a bottleneck. > > > Diffs > - > > kwin/composite.cpp 9edb99d > kwin/main.cpp f767f54 > > Diff: http://git.reviewboard.kde.org/r/101385/diff > > > Te
Python Plasmoids - 2 Questions
Hi there,
well, I am not exactly shure that I am on the right list here, yet
somewhere Aron's words about heading over to plasma-devel for any
qestions still ring in my head. If I am wrong, where should I ask instead?
So, currently I am working on a python plasmoid (first time plasmoid
development for me) and am currently struggling with two issues, namely:
1) dbus calls:
How do I write the following C++ call in python?
QDBusInterface("org.kde.krunner", "/App",
"org.kde.krunner.App").call(QDBus.NoBlock, "showTaskManager");
QDBusInterface is not wrapped by PyQT so that I should propably use the
dbus module. And then? How?
2) Implement toolTipAboutToShow():
In my plasmoid and to save cpu cycles I want to do some actions only
before / while showing a tooltip. For this purpose, the
toolTipAboutToShow() function can be used, I think. Unfortunatly, the
following snippet shows a working tooltip, yet the toolTipAboutToShow()
function of my class is never called but the following error / warning
is shown:
QMetaObject::invokeMethod: No such method
Plasma::Applet::toolTipAboutToShow()
So, how do I override toolTipAboutToShow with my own implementation?
class SimpleCpuMeter(plasmascript.Applet):
def __init__(self,parent,args=None):
plasmascript.Applet.__init__(self,parent)
def init(self):
...
self.setHasConfigurationInterface(False)
self.initToolTip()
def initToolTip(self):
self.tooltipdata = Plasma.ToolTipContent()
self.tooltipdata.setMainText("System Information")
self.tooltipdata.setImage(KIcon('view-statistics'))
Plasma.ToolTipManager.self().registerWidget(self.applet)
self.updateToolTip()
def toolTipAboutToShow(self):
print("toolTipAboutToShow")
Thanks, Michael
___
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
