Normal. When you change the parent relationship of a tasks, this task is updated, sending the updated signal and saving itself in the backend. The modified signal is also sent to the children and the parents of that particular task. All those modified-signal are catched by che tagstore to update itself.
The problem was that the "add_parent" method was not checking the previous situation. Thus, tasks were doing all that work even if the add_parent was adding a parent that previously existed and thus were not changing anything. This is particularly evident during the load time because, when reconstructing a tree, we use the "add_parent" method (even if the task already knows about its parent). What I simply did was to add a check to the add_parent method so it does nothing if the parent was already there. Results ? On my 300 tasks, during load, 100 were saved to the backend (yes, 100 file-write operation) and 3000 modified signals were sent. After adding this check, we are at 0-0, which is perfect :-) There are still some rooms for improvement : the tagstore, the fact that the task.sync is called a bit too much (3 times when opening a task) but, at least, we are in a shape which is 1000 times better than the trunk. (once I've solved the tag issue). On Thu, 25 Feb 2010 11:00:52 +0100, Bertrand Rousseau <[email protected]> wrote: > On Thu, Feb 25, 2010 at 10:14 AM, Lionel Dricot <[email protected]> wrote: >> >> Only for performances, could you please take the latest revision (658) of >> my refactoring branch and try it with a lot of tasks and confirm me that >> I'm not dreaming ? > > It seems quite fast, yes, but it's not using update_tags_for_task either: > > 109098 function calls (105962 primitive calls) in 3.730 CPU > seconds > > Ordered by: cumulative time > List reduced from 1042 to 15 due to restriction <15> > > ncalls tottime percall cumtime percall filename:lineno(function) > 1 0.000 0.000 3.730 3.730 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main) > 1 0.000 0.000 3.567 3.567 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser) > 1 0.000 0.000 3.428 3.428 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main) > 1 3.313 3.313 3.331 3.331 {gtk._gtk.main} > 1 0.000 0.000 0.139 0.139 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__) > 10 0.093 0.009 0.096 0.010 {method 'show' of > 'gtk.Widget' objects} > 1 0.000 0.000 0.069 0.069 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine) > 1 0.000 0.000 0.068 0.068 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins) > 46 0.001 0.000 0.068 0.001 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml) > 12 0.000 0.000 0.065 0.005 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__) > 12 0.000 0.000 0.062 0.005 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module) > 2 0.000 0.000 0.061 0.031 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore) > 12 0.001 0.000 0.060 0.005 {imp.load_module} > 1 0.000 0.000 0.049 0.049 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:66(check_instance) > 2 0.000 0.000 0.045 0.023 > /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:203(quit) > >> I just can't believe what I'm seeing but that computer is fast. >> >> >> Lionel >> >> PS: tags display is still broken. >> >> >> >> On Thu, 25 Feb 2010 01:27:16 +0100, Bertrand Rousseau >> <[email protected]> wrote: >>> On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <[email protected]> wrote: >>>> >>>> I've very good news for myself : I see the light. >>>> >>>> The FilteredTree is now working nearly perfectly. It is rock-solid, >> even >>>> when loading 300 tasks at the same time. I also implemented the Closed >>>> Tasks pane with it. The FiltersBank is also working great. >>> >>> Great news! >>> >>>> Bad news : with a lot of tasks, the start time is really really really >>>> slow. (like 10 times the current trunk launch time). On the other hand, >>>> it >>>> looks like other performance issues are mostly solved (like renaming a >>>> subtask in a task). >>> >>> Task renaming is maybe due to the fact that you disabled the refresh >>> methods from the browser. Originally, it can from the fact that >>> browser refresh is called at each letter you type by the editor. It >>> should be worth looking at this too or it will probably show again >>> sometime. >>> >>>> So now, I need *you*. I need testing and profiling. >>>> >>>> Do : bzr branch lp:~gtg/gtg/gtg-refactor >>>> and launch it first with debugging data then with actual data. Do some >>>> profiling ( http://live.gnome.org/gtg/development ) and send me the >>>> results. >>> >>> Here's mine (on my netbook, so slownesses are sensible here): >>> >>> Ordered by: cumulative time >>> List reduced from 947 to 15 due to restriction <15> >>> >>> ncalls tottime percall cumtime percall filename:lineno(function) >>> 1 0.000 0.000 10.889 10.889 >>> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main) >>> 1 0.000 0.000 10.127 10.127 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser) >>> 1 0.000 0.000 9.047 9.047 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main) >>> 1 4.523 4.523 8.653 8.653 {gtk._gtk.main} >>> 244 0.030 0.000 3.011 0.012 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified) >>> 244 0.027 0.000 2.595 0.011 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task) >>> 4032 0.263 0.000 2.457 0.001 {method 'row_changed' of >>> 'gtk.TreeModel' objects} >>> 7208 0.394 0.000 1.734 0.000 {method 'get_value' of >>> 'gtk.TreeModel' objects} >>> 7495 0.060 0.000 1.380 0.000 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value) >>> 732 0.023 0.000 1.347 0.002 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func) >>> 879 0.297 0.000 1.295 0.001 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr) >>> 12660 0.107 0.000 1.140 0.000 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task) >>> 1 0.000 0.000 1.074 1.074 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__) >>> 12660 0.231 0.000 1.034 0.000 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task) >>> 2709 0.175 0.000 0.825 0.000 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func) >>> >>> As you can see, a lot of time is spent in update_tags_for_task. If you >>> comment this out, get_value and on_task_modified don't even show up in >>> the profiling log: >>> >>> Ordered by: cumulative time >>> List reduced from 944 to 15 due to restriction <15> >>> >>> ncalls tottime percall cumtime percall filename:lineno(function) >>> 1 0.000 0.000 9.428 9.428 >>> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main) >>> 1 0.000 0.000 8.639 8.639 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser) >>> 1 0.000 0.000 7.443 7.443 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main) >>> 1 5.892 5.892 7.012 7.012 {gtk._gtk.main} >>> 1 0.001 0.001 1.173 1.173 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__) >>> 1 0.000 0.000 0.776 0.776 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine) >>> 1 0.002 0.002 0.775 0.775 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins) >>> 12 0.002 0.000 0.587 0.049 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__) >>> 12 0.001 0.000 0.544 0.045 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module) >>> 12 0.003 0.000 0.514 0.043 {imp.load_module} >>> 29 0.003 0.000 0.442 0.015 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml) >>> 2 0.000 0.000 0.430 0.215 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore) >>> 10 0.356 0.036 0.395 0.040 {method 'show' of >>> 'gtk.Widget' objects} >>> 975 0.064 0.000 0.383 0.000 >>> >> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value) >>> 29 0.001 0.000 0.334 0.012 >>> /usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml) >>> >>> It's probably not the task of the browser to refresh each tag related >>> to a task. Since Tags have references to their tasks, they are >>> modified as well when a Task is edited. So we should probably let the >>> tag emit a signal to warn everyone they changed (much like the Task >>> are doing now). Let the objets which care about it to register >>> themselve to this signal through the requester. >>> >>> That would imply having a "on_tag_modified" in the browser that would >>> trigger some job to perform on the browser (probably not much since, >>> if I understood well, most of the filtering work is now done outside >>> the browser) >>> >>>> If you want to contribute, there's a huge cleaning to do in the >>>> browser.py >>>> ! A lot of cruft should be removed (and I don't understand everything >>>> there). >>> >>> I agree with what you say, but it would be more efficient if you could >>> be more specific and point out some of these cruft. >>> >>>> >>>> What does not work : >>>> - tag filtering. They are implemented. We just need to do >>>> "self.req.apply_filter(tagname)" when a particular tag is selected but >> I >>>> don't know where it is in browser.py and I'm a bit tired. (any help is >>>> welcome here) >>>> - drag-n-drop : code is commented out in tasktree.py. Maybe it will >> work >>>> without any trouble. >>>> - multiparent (not tested yet) >>>> - tag count in tagtree >>>> >>>> What is broken : >>>> - plugins and plugin_api >>>> - dbus interface >>>> >>>> >>>> I was expecting better performances but FilteredTree appeared to be a >> lot >>>> more complex than expected. So now, it's time to optimize it a bit and >>>> I'm >>>> wondering where is the bottleneck. >>> >>> Like I wrote above, it seems to be situated around tags update >>> mechanisms and TagTreeModel updates. >>> >>>> >>>> Thanks for listenning, >>>> >>>> >>>> Lionel >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~gtg-contributors >>>> Post to : [email protected] >>>> Unsubscribe : https://launchpad.net/~gtg-contributors >>>> More help : https://help.launchpad.net/ListHelp >>>> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~gtg-contributors >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~gtg-contributors >> More help : https://help.launchpad.net/ListHelp >> _______________________________________________ Mailing list: https://launchpad.net/~gtg-contributors Post to : [email protected] Unsubscribe : https://launchpad.net/~gtg-contributors More help : https://help.launchpad.net/ListHelp
