[Zeitgeist] [Bug 844877] Re: Query execution slow
It is not *very* slow, it just seems to be at bit slower than it should be.
From reading the query plan the reason seems to be that two temp b-trees are
created, which is *slow*
sele order from deta
-
1 0 0 SCAN TABLE event USING INDEX event_subj_id (~125000
rows)
0 0 1 SCAN SUBQUERY 1 (~100 rows)
0 1 0 SEARCH TABLE event USING INDEX event_subj_id
(subj_id=?) (~2 rows)
0 0 0 USE TEMP B-TREE FOR GROUP BY
0 0 0 USE TEMP B-TREE FOR ORDER BY
It seems like all other queries are also using this temp b-trees (at least the
ones I tried)
Right now I don't see a way to optimize it.
--
You received this bug notification because you are a member of Zeitgeist
Framework Team, which is subscribed to Zeitgeist Framework.
https://bugs.launchpad.net/bugs/844877
Title:
Query execution slow
Status in Zeitgeist Framework:
New
Bug description:
Synapse is using various not-so-complex find_events queries to display
recent activities, and it seems that especially a query for All
category is taking much longer than any other query (~750ms with a DB
with 200k events).
Here's the output from zg with the debug_sql extension:
[DEBUG - root] # parameters: {'event_templates':
[Event([dbus.Array([dbus.String(u''), '1315492708863', dbus.String(u''),
dbus.String(u''), dbus.String(u''), ''], signature=dbus.Signature('s')),
[Subject([dbus.String(u''), dbus.String(u''), dbus.String(u''),
dbus.String(u''), dbus.String(u''), dbus.String(u''), dbus.String(u''),
dbus.String(u'')]), Subject([dbus.String(u''),
dbus.String(u'!http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Folder'),
dbus.String(u''), dbus.String(u''), dbus.String(u''), dbus.String(u''),
dbus.String(u''), dbus.String(u'')]), Subject([dbus.String(u''),
dbus.String(u'!http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#Software'),
dbus.String(u''), dbus.String(u''), dbus.String(u''), dbus.String(u''),
dbus.String(u''), dbus.String(u'')])], dbus.Array([],
signature=dbus.Signature('y'))])],
'max_events': dbus.UInt32(96L),
'order': dbus.UInt32(2L),
'return_mode': 1,
'self': _zeitgeist.engine.main.ZeitgeistEngine instance at 0x1b66248,
'sender': ':1.3604',
'storage_state': dbus.UInt32(2L),
'time_range': [1300977508860, 9223372036854775807]}
[DEBUG - root] # database size: {'actor': 129,
'event': 192746,
'interpretation': 68,
'manifestation': 27,
'mimetype': 353,
'uri': 60265}
[DEBUG - root] # BEGIN SQL QUERY #
no pretty sql: SELECT id FROM event_view
NATURAL JOIN (
SELECT subj_id,
max(timestamp) AS timestamp
FROM event_view WHERE (+timestamp = ?
AND ((NOT (subj_interpretation = ?) AND NOT (subj_interpretation = ? OR
subj_interpretation = ? OR subj_interpretation = ?
GROUP BY subj_id)
GROUP BY subj_id
ORDER BY timestamp DESC
LIMIT 96 : [1300977508860, 45, 46, 47, 43]
took 0.712120s
# END SQL QUERY
[DEBUG - zeitgeist.engine] Found 96 events IDs in 0.723284s
To manage notifications about this bug go to:
https://bugs.launchpad.net/zeitgeist/+bug/844877/+subscriptions
___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zeitgeist
More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 807076] Re: raptor2 not supported
The problem is not in zeitgeist itself, but in the recent version of raptor2-utils in ubuntu oneiric. See http://bugs.librdf.org/mantis/view.php?id=451 for the upstream bugreport. ** Bug watch added: librdf Mantis #451 http://bugs.librdf.org/mantis/view.php?id=451 -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/807076 Title: raptor2 not supported Status in Zeitgeist Framework: Triaged Status in “zeitgeist” package in Ubuntu: New Bug description: Ubuntu seems to have switched to some new Raptor which seems to fail really badly: make[3]: Entering directory `/tmp/buildd/zeitgeist-0.8.1/extra/ontology' echo -e #\n# Auto-generated from zg.trig. Do not edit\n# zg.py rapper -i trig -o rdfxml zg.trig | ../../extra/rdfxml2py zg.py rapper: Parsing URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/zg.trig with parser trig rapper: Serializing with serializer rdfxml rapper: Error - - The namespace prefix in zg was not declared. rapper: Error - URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/zg.trig:17 - turtle_qname_to_uri failed rapper: Parsing returned 0 triples Parsing from stdin echo -e #\n# Auto-generated from nie.trig. Do not edit\n# nie.py rapper -i trig -o rdfxml nie.trig | ../../extra/rdfxml2py nie.py rapper: Parsing URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/nie.trig with parser trig rapper: Serializing with serializer rdfxml rapper: Error - - The namespace prefix in nie was not declared. rapper: Error - URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/nie.trig:56 - turtle_qname_to_uri failed rapper: Parsing returned 0 triples Parsing from stdin To manage notifications about this bug go to: https://bugs.launchpad.net/zeitgeist/+bug/807076/+subscriptions ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 807076] Re: raptor2 not supported
Affected version is raptor2-utils = 2.0.2 -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/807076 Title: raptor2 not supported Status in Zeitgeist Framework: Triaged Status in “zeitgeist” package in Ubuntu: New Bug description: Ubuntu seems to have switched to some new Raptor which seems to fail really badly: make[3]: Entering directory `/tmp/buildd/zeitgeist-0.8.1/extra/ontology' echo -e #\n# Auto-generated from zg.trig. Do not edit\n# zg.py rapper -i trig -o rdfxml zg.trig | ../../extra/rdfxml2py zg.py rapper: Parsing URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/zg.trig with parser trig rapper: Serializing with serializer rdfxml rapper: Error - - The namespace prefix in zg was not declared. rapper: Error - URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/zg.trig:17 - turtle_qname_to_uri failed rapper: Parsing returned 0 triples Parsing from stdin echo -e #\n# Auto-generated from nie.trig. Do not edit\n# nie.py rapper -i trig -o rdfxml nie.trig | ../../extra/rdfxml2py nie.py rapper: Parsing URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/nie.trig with parser trig rapper: Serializing with serializer rdfxml rapper: Error - - The namespace prefix in nie was not declared. rapper: Error - URI file:///tmp/buildd/zeitgeist-0.8.1/extra/ontology/nie.trig:56 - turtle_qname_to_uri failed rapper: Parsing returned 0 triples Parsing from stdin To manage notifications about this bug go to: https://bugs.launchpad.net/zeitgeist/+bug/807076/+subscriptions ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 787868] Re: Encryption of database
I'm with Siegfried when it comes to add an extra encryption layer on top of the db, basically I fail to understand why putting the db in an encrypted filesystem is not good enough. But what I find interesting is the idea of limiting the ability to access the activity log to system-wide installed clients. And fortunately this is not hard to implement, as we already get the bus_name of the sender as 'sender' argument to all our public methods, and dbus has the GetConnectionUnixProcessID() which returns the process id of the client. All we now have to do is parsing the cmdline entry in proc (let's ignore possible attack vectors for now) -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/787868 Title: Encryption of database Status in Zeitgeist Framework: New Bug description: I think that Zeitgeist should encrypt databases in ~/.local/share/zeitgeist/* for anti-forensics reasons. While someone may happen to use an encrypted disk, Zeitgeist may serve as the ultimate accidental spyware to an unsuspecting user. One possible mitigation is to randomly generate a reasonable key, tie it into the login keychain and then use that key with something like http://sqlcipher.net/ rather than straight sqlite. In theory, a user will never know that this encryption/decryption is happening - no underlying assumptions about the disk need to be made to maintain any security guarantees. This should prevent anyone from learning the contents of the database without also learning the login password. Modern Ubuntu machines disallow non-root ptracing ( https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace ) and if the gnome keyring is locked, an attacker would have a much harder time grabbing meaningful Zeitgeist data without interacting with the user or bruteforcing the login keychain. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 772041] Re: Timestamp isn't considered when choosing events by popularity
10:17 thekorn RainCT: so the bug is: COUNT(boo) is working, but the additional sorting by timestamp is not? 10:18 RainCT thekorn: Right. So the question is if we want it to work (and if so I'll fix it together with a related bug) 10:19 thekorn RainCT: yes, we want it to work. picking them by timestamp makes sense, and returning *random* results does not 10:19 thekorn it's also an issue in our tests that we don't see this bug 10:20 RainCT thekorn: OK, agreed. 10:20 RainCT So that this in the bug and I'll try to fix it tonight. 10:20 thekorn it's because we are inserting events presorted by timestamp 10:20 RainCT The tests are just looking at the envet ids, they should be looking at the timestamps instead. 10:21 RainCT (I already swapped the order of 2 events, so checking the timestamps will expose the problem) -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/772041 Title: Timestamp isn't considered when choosing events by popularity Status in Zeitgeist Framework: New Bug description: Currently the MostPopular* and LeastPopular* methods are choosing a random (*) event from within all events matching the criteria. Is this desired? I believe it'd make more sense to return that one with the highest timestamp for MostPopular* and the one with the oldest timestamp for LeastPopular*, or something along that lines (there are currently such timestamp ASC/DESC statements in the code, but they aren't working, so if we agree on this property they need to be fixed). (* Actually, it's the last one which was inserted into the database, but that doesn't mean anything.) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 772265] Re: The datahub starts zeitgeist-daemon on startup
I'm not a big friend of the idea to allow someone else than the daemon itself modify config files like datasources.(pickle|json), because this would require the daemon to watch-out for changes to all config files it and its extensions are using. libzeitgeist and the datahub should instead defer any communication to the daemon until before the first event is inserted, e.g. the datahub should register itself as a datasource only once it inserts the first event. This doesn't necessarily solve the initial issue of this bugreport (zg slowing down login time) but it's a start. In a next step we have to make sure that the datahub inserts events as late as possible, *not during session login*. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/772265 Title: The datahub starts zeitgeist-daemon on startup Status in Zeitgeist Client Library: New Status in Zeitgeist Framework: New Status in Zeitgeist Datahub: New Bug description: We have been advising distros put zeitgeist-datahub into the autostart section for the sessions. This is better in theory because this way we only start a native daemon on login and can defer the launching of the heavier Python process zeitgeist-daemon. Unfortunately zeitgeist-datahub does DBus activation of the zeitgeist- daemon on startup so we really don't get the benefits we wanted. Bootcharts from Ubuntu 11.04 indicates that ZG eats about 1s on the login time and I don't think that is acceptable for ZG in the longer run. That is - no catastrophe for 11.04, but we should fix this. (a related issue seems to be that the dbus activated zeitgeist-daemon seems to launch another datahub instance that then immediately becomes a zombie... ? :-/) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 739780] Re: Session starting with a zeitgeist-datahub zombie process
revno: 1714 fixes bug(s): https://launchpad.net/bugs/739780 committer: Markus Korn thek...@gmx.de branch nick: trunk timestamp: Thu 2011-04-14 09:29:21 +0200 message: Use glib.spawn_async() to launch the datahub instead of subprocess.Popen to avoid zombie processes (LP: #739780) ** Changed in: zeitgeist Status: In Progress = Fix Committed -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/739780 Title: Session starting with a zeitgeist-datahub zombie process Status in Zeitgeist Framework: Fix Committed Status in Zeitgeist Datahub: Invalid Status in “zeitgeist” package in Ubuntu: Triaged Bug description: Binary package hint: zeitgeist To reproduce, either a.) Just start a desktop session b.) Kill the zeitgeist-daemon and manually run zeitgeist-datahub In both cases 'ps' reveals a zombie process: http://paste.ubuntu.com/583545/ ProblemType: Bug DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic i686 NonfreeKernelModules: nvidia Architecture: i386 Date: Mon Mar 21 23:07:05 2011 ProcEnviron: LANGUAGE=en_US:en LANG=en_US.utf8 SHELL=/bin/bash SourcePackage: zeitgeist UpgradeStatus: Upgraded to natty on 2010-12-08 (103 days ago) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 738555] Re: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle'
I was investigating on this bug a bit further. All Tracebacks are looking like Traceback (most recent call last): File /usr/bin/zeitgeist-daemon, line 180, in module handle_exit() File /usr/bin/zeitgeist-daemon, line 126, in handle_exit interface.Quit() File /usr/share/zeitgeist/_zeitgeist/engine/remote.py, line 349, in Quit self._engine.close() File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 133, in close self.extensions.unload() File /usr/share/zeitgeist/_zeitgeist/engine/extension.py, line 287, in unload self.unload(ext) File /usr/share/zeitgeist/_zeitgeist/engine/extension.py, line 292, in unload obj.unload() File /usr/share/zeitgeist/_zeitgeist/engine/extensions/datasource_registry.py, line 115, in unload self._write_to_disk() File /usr/share/zeitgeist/_zeitgeist/engine/extensions/datasource_registry.py, line 99, in _write_to_disk with open(DATA_FILE, w) as data_file: IOError: [Errno 2] No such file or directory: '$XDG_HOME_DATA/zeitgeist/datasources.pickle' This is telling us a few things: 1.) the crash happens when handle_exit() is called, this function is called when the daemon recieves a SIGHUB or SIGTERM signal, means if the systems wants zg to shut down. (apport will try to report this crash on the next session start, which makes people think this crash happen *while booting*) 2.) If you try to write to a certain location in python, and the parent directory of this location does not exist at this point, an IOError with [Errno 2] No such file or directory: [...] is raised. This indicates that $XDG_HOME_DATA or $XDG_HOME_DATA/zeitgeist does not exist for the bugreporters when the systems wants zg to shut-down. 3.) All bugreporters are having their $HOME (or parts of it) encrypted (EcryptfsInUse: Yes). This *could* be the reason for this bug, *maybe* the encrypted dir (and thus $XDG_HOME_DATA) get un-mounted before the system wants zg to stop. So this leads to a simple question: How do we want zg to behave if it is impossible to write the data to its DATA_PATH (which is in most cases $XDG_DATA_HOME/zeitgeist)? We are talking about these files atm: 1.) activity.sqlite 2.) datasources.pickle 3.) blacklist.(pickle|json) [4.) extension specific files, like the fts index] IMHO three of them are easy to handle. We should not care about 4.) at the first place, it's up to the (3rd party) extensions to do the right thing, but we have to tell extensions authors what we think is the best they could do. If the datasource registry fails to write its config file on shutdown it does not really matter, as its data is written on every change to this file anyway. We should just ignore this error on shutdown and log a warning. If config can not be written when AddTemplate or RemoveTemplate dbus method gets called we should log the error, and return the error over dbus, but not let the daemon crash. The requested config changes should not have any effect. Some goes for blacklists. I'm not sure how to handle the activity log here. Data gets written to this file (sqlite3.connection.commit()) as a result of dbus calls, for which we should log this error, revert in memory changes and send an error over dbus. But we are also writing to this file on start/shutdown of the daemon, which is not an explicit user action. We should log the write error in this cases and do the best to tell our users which kind of actions were not logged properly. As a future step we could also try to shutdown the daemon before the DATA_PATH becomes unavaiable. @Mikkel, Siegfried, Seif, et all. Any comments on this topic? -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/738555 Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle' Status in Zeitgeist Framework: Triaged Status in “zeitgeist” package in Ubuntu: Incomplete Bug description: Binary package hint: zeitgeist lsb_release -rd Description: Ubuntu Natty (development branch) Release: 11.04 ProblemType: Crash DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic x86_64 Architecture: amd64 Date: Sun Mar 20 07:40:48 2011 EcryptfsInUse: Yes ExecutablePath: /usr/bin/zeitgeist-daemon InstallationMedia: Ubuntu 11.04 Natty Narwhal - Alpha amd64 (20110318) InterpreterPath: /usr/bin/python2.7 ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon ProcEnviron: SHELL=/bin/bash LANGUAGE=en_AU:en LANG=en_AU.UTF-8 PythonArgs: ['/usr/bin/zeitgeist-daemon'] SourcePackage: zeitgeist Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory:
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist
Markus Korn has proposed merging
lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist.
Requested reviews:
Zeitgeist Framework Team (zeitgeist)
Related bugs:
Bug #598666 in Zeitgeist Framework: Invalid cache access (was: Error when
trying to fetch items)
https://bugs.launchpad.net/zeitgeist/+bug/598666
For more details, see:
https://code.launchpad.net/~thekorn/zeitgeist/fix-598666-remove_cache_entry/+merge/56710
This branch adds a fix/workaround for bug 598666, where the caches are not
updated properly if one cached item gets deleted in the database.
https://bugs.launchpad.net/zeitgeist/+bug/598666/comments/16 describes how this
works.
This branch also factors in all review comments by Mikkel and Siegfried made in
the bug.
--
https://code.launchpad.net/~thekorn/zeitgeist/fix-598666-remove_cache_entry/+merge/56710
Your team Zeitgeist Framework Team is requested to review the proposed merge of
lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist.
=== modified file '_zeitgeist/engine/main.py'
--- _zeitgeist/engine/main.py 2011-04-07 07:23:41 +
+++ _zeitgeist/engine/main.py 2011-04-07 07:56:30 +
@@ -5,7 +5,7 @@
# Copyright © 2009 Mikkel Kamstrup Erlandsen mikkel.kamst...@gmail.com
# Copyright © 2009-2010 Siegfried-Angel Gevatter Pujals rai...@ubuntu.com
# Copyright © 2009-2011 Seif Lotfy s...@lotfy.com
-# Copyright © 2009 Markus Korn thek...@gmx.net
+# Copyright © 2009-2011 Markus Korn thek...@gmx.net
# Copyright © 2011 Collabora Ltd.
# By Seif Lotfy s...@lotfy.com
#
@@ -133,6 +133,11 @@
def close(self):
log.debug(Closing down the zeitgeist engine...)
self.extensions.unload()
+ # make sure all symbol table are in good shape
+ # this Exception should never be raised, it would indicate a bug
+ # in the workaround for (LP: #598666)
+ if self._cursor.execute(SELECT * FROM _fix_cache).fetchone():
+ raise RuntimeError(Symbol cache is in bad state)
self._cursor.connection.close()
self._cursor = None
unset_cursor()
@@ -732,7 +737,16 @@
% ,.join([?] * len(ids)), ids)
self._cursor.connection.commit()
log.debug(Deleted %s % map(int, ids))
-
+ clean_fix_cache = False
+ for cached_table, cached_id in self._cursor.execute(SELECT * FROM _fix_cache):
+if cached_table not in (interpretation, manifestation, mimetype, actor):
+ raise ValueError(Unable to fix cache for '%s' %cached_table)
+clean_fix_cache = True
+getattr(self, _%s %cached_table).remove_id(cached_id)
+ if clean_fix_cache:
+# delete all rows from _fix_cache, as all caches are fixed by now
+self._cursor.execute(DELETE FROM _fix_cache)
+self._cursor.connection.commit()
self.extensions.apply_post_delete(ids, sender)
return timestamps
else:
=== modified file '_zeitgeist/engine/sql.py'
--- _zeitgeist/engine/sql.py 2011-03-25 18:30:46 +
+++ _zeitgeist/engine/sql.py 2011-04-07 07:56:30 +
@@ -4,7 +4,7 @@
#
# Copyright © 2009-2010 Siegfried-Angel Gevatter Pujals rai...@ubuntu.com
# Copyright © 2009 Mikkel Kamstrup Erlandsen mikkel.kamst...@gmail.com
-# Copyright © 2009 Markus Korn thek...@gmx.net
+# Copyright © 2009-2011 Markus Korn thek...@gmx.net
# Copyright © 2009 Seif Lotfy s...@lotfy.com
#
# This program is free software: you can redistribute it and/or modify
@@ -166,6 +166,11 @@
# one connection to the database is allowed to revert this setting set locking_mode to NORMAL.
cursor.execute(PRAGMA locking_mode = EXCLUSIVE)
+ # as part of the workaround for (LP: #598666) we need to
+ # create the '_fix_cache' TEMP table on every start,
+ # this table gets purged once the engine gets closed.
+ cursor.execute(CREATE TEMP TABLE _fix_cache (t VARCHAR, id INTEGER))
+
# Always assume that temporary memory backed DBs have good schemas
if constants.DATABASE_FILE != :memory::
if not new_database and _check_core_schema_upgrade(cursor):
@@ -453,6 +458,14 @@
self[row[value]] = row[id]
self._inv_dict = dict((value, key) for key, value in self.iteritems())
+
+ cursor.execute(
+ CREATE TEMP TRIGGER update_cache_%(table)s
+ BEFORE DELETE ON %(table)s
+ BEGIN
+INSERT INTO _fix_cache VALUES (%(table)s, OLD.id);
+ END;
+ % {table: table})
def __getitem__(self, name):
# Use this for inserting new properties into the database
@@ -483,6 +496,11 @@
# database already. Eg., in find_eventids.
return super(TableLookup, self).__getitem__(name)
+ def remove_id(self, id):
+ value = self.value(id)
+ del self._inv_dict[id]
+ del self[value]
+
def get_right_boundary(text):
returns the smallest string which is greater than `text`
if not text:
___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zeitgeist
More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist
The proposal to merge lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist has been updated. Description changed to: This branch adds a fix/workaround for bug 598666, where the caches are not updated properly if one cached item gets deleted in the database. https://bugs.launchpad.net/zeitgeist/+bug/598666/comments/16 describes how this works. This branch also factors in all review comments by Mikkel and Siegfried made in the bug. NOTE: I'm sure this changes do not require a db schema version dump, as only TEMP TABLE/TRIGGERS are added to the schema. And these are added on every engine start-up, no matter which version is used. Please correct me if I'm wrong with this assumtion. For more details, see: https://code.launchpad.net/~thekorn/zeitgeist/fix-598666-remove_cache_entry/+merge/56710 -- https://code.launchpad.net/~thekorn/zeitgeist/fix-598666-remove_cache_entry/+merge/56710 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/fix-598666-remove_cache_entry into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 738555] Re: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle'
Hi @all, we discussed this issue in the #zeitgeist IRC channel, our theory is: it's related to encrypted homedirs, and that this dirs get unmounted before the zeitgeist daemon stops. So the daemon is unable to write to $XDG_DATA_HOME. We'll investigate further, and keep you updated. Thanks for reporting this bug, Markus ** Changed in: zeitgeist Status: New = Triaged ** Changed in: zeitgeist Importance: Undecided = High -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/738555 Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle' Status in Zeitgeist Framework: Triaged Status in “zeitgeist” package in Ubuntu: Triaged Bug description: Binary package hint: zeitgeist lsb_release -rd Description: Ubuntu Natty (development branch) Release: 11.04 ProblemType: Crash DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic x86_64 Architecture: amd64 Date: Sun Mar 20 07:40:48 2011 EcryptfsInUse: Yes ExecutablePath: /usr/bin/zeitgeist-daemon InstallationMedia: Ubuntu 11.04 Natty Narwhal - Alpha amd64 (20110318) InterpreterPath: /usr/bin/python2.7 ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon ProcEnviron: SHELL=/bin/bash LANGUAGE=en_AU:en LANG=en_AU.UTF-8 PythonArgs: ['/usr/bin/zeitgeist-daemon'] SourcePackage: zeitgeist Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle' UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 744818] [NEW] Don't crash on daemon start if a running zeitgeist daemon instance is found
Public bug reported: Instead of exposing the RuntimeError to our users we should simply exit with a status !=0 and log the traceback. ** Affects: zeitgeist Importance: Undecided Status: New -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/744818 Title: Don't crash on daemon start if a running zeitgeist daemon instance is found Status in Zeitgeist Framework: New Bug description: Instead of exposing the RuntimeError to our users we should simply exit with a status !=0 and log the traceback. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 739780] Re: Session starting with a zeitgeist-datahub zombie process
** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist Status: Triaged = In Progress ** Changed in: zeitgeist-datahub Status: Confirmed = Invalid -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/739780 Title: Session starting with a zeitgeist-datahub zombie process Status in Zeitgeist Framework: In Progress Status in Zeitgeist Datahub: Invalid Status in “zeitgeist” package in Ubuntu: Triaged Bug description: Binary package hint: zeitgeist To reproduce, either a.) Just start a desktop session b.) Kill the zeitgeist-daemon and manually run zeitgeist-datahub In both cases 'ps' reveals a zombie process: http://paste.ubuntu.com/583545/ ProblemType: Bug DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic i686 NonfreeKernelModules: nvidia Architecture: i386 Date: Mon Mar 21 23:07:05 2011 ProcEnviron: LANGUAGE=en_US:en LANG=en_US.utf8 SHELL=/bin/bash SourcePackage: zeitgeist UpgradeStatus: Upgraded to natty on 2010-12-08 (103 days ago) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 739780] Re: Session starting with a zeitgeist-datahub zombie process
I can reproduce this issue in my natty installation. @Michal do you have an idea what's going on, maybe sth. wrong in the datahub? ** Also affects: zeitgeist-datahub Importance: Undecided Status: New ** Changed in: zeitgeist (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/739780 Title: Session starting with a zeitgeist-datahub zombie process Status in Zeitgeist Framework: New Status in Zeitgeist Datahub: New Status in “zeitgeist” package in Ubuntu: Confirmed Bug description: Binary package hint: zeitgeist To reproduce, either a.) Just start a desktop session b.) Kill the zeitgeist-daemon and manually run zeitgeist-datahub In both cases 'ps' reveals a zombie process: http://paste.ubuntu.com/583545/ ProblemType: Bug DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic i686 NonfreeKernelModules: nvidia Architecture: i386 Date: Mon Mar 21 23:07:05 2011 ProcEnviron: LANGUAGE=en_US:en LANG=en_US.utf8 SHELL=/bin/bash SourcePackage: zeitgeist UpgradeStatus: Upgraded to natty on 2010-12-08 (103 days ago) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 739780] Re: Session starting with a zeitgeist-datahub zombie process
Ok, now I know what's going on here http://paste.ubuntu.com/586129/ We just have to manage the case in the daemon that the datahub exists immediately on daemon startup, because an already running datahub was found. We can fix this in a few steps, first ist that the datahub should return an exit code !=0 if it fails to start because of this case. ** Changed in: zeitgeist (Ubuntu) Status: Confirmed = Triaged ** Changed in: zeitgeist Status: New = Triaged ** Changed in: zeitgeist-datahub Status: New = Confirmed ** Changed in: zeitgeist Importance: Undecided = Medium -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/739780 Title: Session starting with a zeitgeist-datahub zombie process Status in Zeitgeist Framework: Triaged Status in Zeitgeist Datahub: Confirmed Status in “zeitgeist” package in Ubuntu: Triaged Bug description: Binary package hint: zeitgeist To reproduce, either a.) Just start a desktop session b.) Kill the zeitgeist-daemon and manually run zeitgeist-datahub In both cases 'ps' reveals a zombie process: http://paste.ubuntu.com/583545/ ProblemType: Bug DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic i686 NonfreeKernelModules: nvidia Architecture: i386 Date: Mon Mar 21 23:07:05 2011 ProcEnviron: LANGUAGE=en_US:en LANG=en_US.utf8 SHELL=/bin/bash SourcePackage: zeitgeist UpgradeStatus: Upgraded to natty on 2010-12-08 (103 days ago) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 740911] [NEW] Timed-out testcases should be marked as SKIPPED, or FAILED
Public bug reported: We do not know if the testcases which time out (I currently have towo of them) really pass, so we should not mark them as such. In newer versions of python we should use SKIP, but I guess FAILED is fine too. Alternativly we should invest some time to find out why they time out and fix them. ** Affects: zeitgeist Importance: Undecided Status: New -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/740911 Title: Timed-out testcases should be marked as SKIPPED, or FAILED Status in Zeitgeist Framework: New Bug description: We do not know if the testcases which time out (I currently have towo of them) really pass, so we should not mark them as such. In newer versions of python we should use SKIP, but I guess FAILED is fine too. Alternativly we should invest some time to find out why they time out and fix them. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 598666] Re: Invalid cache access (was: Error when trying to fetch items)
Thanks for the reviews, I addressed them in my last commits to this branch, I still need to test if the TEMP triggers are working properly. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/598666 Title: Invalid cache access (was: Error when trying to fetch items) Status in Zeitgeist Framework: Triaged Bug description: When i try to fetch all items in one query i get Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/local/share/zeitgeist/_zeitgeist/engine/remote.py, line 254, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 378, in find_events return self._find_events(1, *args) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 366, in _find_events return self.get_events(rows=result, sender=sender) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 188, in get_events events[event.id].append_subject(self._get_subject_from_row(row)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 160, in _get_subject_from_row getattr(self, _ + field).value(row[subj_ + field])) File /usr/local/share/zeitgeist/_zeitgeist/engine/sql.py, line 422, in value return self._inv_dict[id] KeyError: 138 This looks serious ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 740912] Re: failing doctest: test-engine-extension.rst
** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist Status: New = Fix Committed -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/740912 Title: failing doctest: test-engine-extension.rst Status in Zeitgeist Framework: Fix Committed Bug description: As of today, I have (not sure what's going wrong here): == FAIL: /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst Doctest: test-engine-extension.rst -- Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 2166, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for test-engine-extension.rst File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 0 -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 14, in test-engine-extension.rst Failed example: engine = ZeitgeistEngine() Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[1], line 1, in module engine = ZeitgeistEngine() File /media/sda7/zeitgeist/fix_del_cache/test/../_zeitgeist/engine/main.py, line 101, in __init__ self._cursor = cursor = get_default_cursor() File /media/sda7/zeitgeist/fix_del_cache/test/../_zeitgeist/engine/sql.py, line 419, in get_default_cursor _cursor = create_db(dbfile) File /media/sda7/zeitgeist/fix_del_cache/test/../_zeitgeist/engine/sql.py, line 162, in create_db cursor.execute(PRAGMA journal_mode = DELETE) File /media/sda7/zeitgeist/fix_del_cache/test/../_zeitgeist/engine/sql.py, line 70, in execute return super(UnicodeCursor, self).execute(statement, parameters) OperationalError: database is locked -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 15, in test-engine-extension.rst Failed example: len(engine.extensions) Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[2], line 1, in module len(engine.extensions) NameError: name 'engine' is not defined -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 47, in test-engine-extension.rst Failed example: engine.extensions.load(SampleExtension) Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[5], line 1, in module engine.extensions.load(SampleExtension) NameError: name 'engine' is not defined -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 48, in test-engine-extension.rst Failed example: len(engine.extensions) Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[6], line 1, in module len(engine.extensions) NameError: name 'engine' is not defined -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 50, in test-engine-extension.rst Failed example: print engine.extensions Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[7], line 1, in module print engine.extensions NameError: name 'engine' is not defined -- File /media/sda7/zeitgeist/fix_del_cache/test/test-engine-extension.rst, line 52, in test-engine-extension.rst Failed example: sorted(engine.extensions.methods) Exception raised: Traceback (most recent call last): File /usr/lib/python2.7/doctest.py, line 1254, in __run compileflags, 1) in test.globs File doctest test-engine-extension.rst[8], line 1, in module
[Zeitgeist] [Bug 714676] Re: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range
Hi David, thanks for using zeitgeist, and reporting this bug. Can you please open a terminal tell us what the output of the following command gives you? $ python -c import time; print time.gmtime(0) this is finally a reproduction of bug 512166 which we closed because noone was able to reproduce this issue. ** Changed in: zeitgeist (Ubuntu) Status: New = Incomplete ** Visibility changed to: Public ** Also affects: zeitgeist Importance: Undecided Status: New ** Changed in: zeitgeist Status: New = Confirmed ** Changed in: zeitgeist Importance: Undecided = Medium -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/714676 Title: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range Status in Zeitgeist Framework: Confirmed Status in “zeitgeist” package in Ubuntu: Incomplete Bug description: Binary package hint: zeitgeist I do not what is the problem with the package, because i can see anything error in the system ProblemType: Crash DistroRelease: Ubuntu 11.04 Package: zeitgeist-datahub 0.5.2-0ubuntu1 ProcVersionSignature: Ubuntu 2.6.38-2.29-generic 2.6.38-rc3 Uname: Linux 2.6.38-2-generic x86_64 Architecture: amd64 Date: Mon Feb 7 08:58:11 2011 ExecutablePath: /usr/bin/zeitgeist-datahub InstallationMedia: Ubuntu 10.10 Maverick Meerkat - Release amd64 (20101007) InterpreterPath: /usr/bin/python2.7 PackageArchitecture: all ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-datahub ProcEnviron: SHELL=/bin/bash LC_MESSAGES=es_AR.utf8 LANGUAGE=es_MX.UTF-8:es:en_GB:en LANG=es_MX.UTF-8 PythonArgs: ['/usr/bin/zeitgeist-datahub'] SourcePackage: zeitgeist Title: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 738555] Re: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle'
This looks *very* weird to me, I'll investigate later this week. ** Visibility changed to: Public ** Changed in: zeitgeist (Ubuntu) Status: New = Triaged ** Also affects: zeitgeist Importance: Undecided Status: New ** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/738555 Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle' Status in Zeitgeist Framework: New Status in “zeitgeist” package in Ubuntu: Triaged Bug description: Binary package hint: zeitgeist lsb_release -rd Description: Ubuntu Natty (development branch) Release: 11.04 ProblemType: Crash DistroRelease: Ubuntu 11.04 Package: zeitgeist-core 0.7-1 ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38 Uname: Linux 2.6.38-7-generic x86_64 Architecture: amd64 Date: Sun Mar 20 07:40:48 2011 EcryptfsInUse: Yes ExecutablePath: /usr/bin/zeitgeist-daemon InstallationMedia: Ubuntu 11.04 Natty Narwhal - Alpha amd64 (20110318) InterpreterPath: /usr/bin/python2.7 ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon ProcEnviron: SHELL=/bin/bash LANGUAGE=en_AU:en LANG=en_AU.UTF-8 PythonArgs: ['/usr/bin/zeitgeist-daemon'] SourcePackage: zeitgeist Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle' UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 714676] Re: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range
@Everyone who is affected by this bug, could you please update your (natty) system and check if you have at least zeitgeist-datahub 0.7.0-0ubuntu1 installed, all these python and timezone related issues should be fixed in the new implementation of the datahub. Please open a new bug if you have this version (or newer) and see any similar crashes, or if after an update zeitgeist-datahub 0.7.0 is not available for you. I'm closing this bug now. Thanks, Markus ** Changed in: zeitgeist Status: Confirmed = Invalid ** Changed in: zeitgeist (Ubuntu) Status: Incomplete = Fix Released -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/714676 Title: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range Status in Zeitgeist Framework: Invalid Status in “zeitgeist” package in Ubuntu: Fix Released Bug description: Binary package hint: zeitgeist I do not what is the problem with the package, because i can see anything error in the system ProblemType: Crash DistroRelease: Ubuntu 11.04 Package: zeitgeist-datahub 0.5.2-0ubuntu1 ProcVersionSignature: Ubuntu 2.6.38-2.29-generic 2.6.38-rc3 Uname: Linux 2.6.38-2-generic x86_64 Architecture: amd64 Date: Mon Feb 7 08:58:11 2011 ExecutablePath: /usr/bin/zeitgeist-datahub InstallationMedia: Ubuntu 10.10 Maverick Meerkat - Release amd64 (20101007) InterpreterPath: /usr/bin/python2.7 PackageArchitecture: all ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-datahub ProcEnviron: SHELL=/bin/bash LC_MESSAGES=es_AR.utf8 LANGUAGE=es_MX.UTF-8:es:en_GB:en LANG=es_MX.UTF-8 PythonArgs: ['/usr/bin/zeitgeist-datahub'] SourcePackage: zeitgeist Title: zeitgeist-datahub crashed with OverflowError in /usr/share/zeitgeist/_zeitgeist/loggers/datasources/_recentmanager.py: mktime argument out of range UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 598666] Re: Invalid cache access (was: Error when trying to fetch items)
I've added a prototype of my idea as described in my last comment, if we want to do it this way, we have to bump the sql-schema-version, I guess. I'm not completely sure if this is the way to go, but given the fact that the number of entries in the _fix_cache table should be *very* low over time this should not be a big performance issue, and works well. Comments welcome. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/598666 Title: Invalid cache access (was: Error when trying to fetch items) Status in Zeitgeist Framework: Triaged Bug description: When i try to fetch all items in one query i get Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/local/share/zeitgeist/_zeitgeist/engine/remote.py, line 254, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 378, in find_events return self._find_events(1, *args) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 366, in _find_events return self.get_events(rows=result, sender=sender) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 188, in get_events events[event.id].append_subject(self._get_subject_from_row(row)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 160, in _get_subject_from_row getattr(self, _ + field).value(row[subj_ + field])) File /usr/local/share/zeitgeist/_zeitgeist/engine/sql.py, line 422, in value return self._inv_dict[id] KeyError: 138 This looks serious ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 598666] Re: Invalid cache access (was: Error when trying to fetch items)
Oh yeah, of course. Indeed, good work Siegfried. While reading the last few comments one possible fix came into my mind: Let's maintain a (temporary) helper table called '_fix_cache' with (table_name VARCHAR, id INTEGER) and create a 'BEFORE DELETE' trigger on each cached table (interpretation, manifestation, mimetype, ...) which inserts the table name and the deleted id in _fix_cache once a row in the related table got deleted. All we now have to do is run a 'SELECT * FROM _fix_cache' after deleting the events. If some rows in manifestation/interpretation/... got deleted while deleting events we get back the the ids of the cached values which we then can delete from the caches. I know that's not optimal, but looks like the only solution to me. This should not be a *big* performance issue, since the number of rows in the helper table should be very low. I'll hack on a prototype tomorrow, and see how it works. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/598666 Title: Invalid cache access (was: Error when trying to fetch items) Status in Zeitgeist Framework: Triaged Bug description: When i try to fetch all items in one query i get Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/local/share/zeitgeist/_zeitgeist/engine/remote.py, line 254, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 378, in find_events return self._find_events(1, *args) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 366, in _find_events return self.get_events(rows=result, sender=sender) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 188, in get_events events[event.id].append_subject(self._get_subject_from_row(row)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 160, in _get_subject_from_row getattr(self, _ + field).value(row[subj_ + field])) File /usr/local/share/zeitgeist/_zeitgeist/engine/sql.py, line 422, in value return self._inv_dict[id] KeyError: 138 This looks serious ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 704931] Re: Publish property names, accesslevel and type in introspection data
I *feel* that our solution could look much simpler than the one from aptdaemon, Let me just try and see how this can be implemented, we can mark this bug as won't fix laterone if it turns out to be too complex, slow or hackish compared to the gain. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/704931 Title: Publish property names, accesslevel and type in introspection data Status in Zeitgeist Framework: New Bug description: Our .Log dbus interface has two properties, version and extensions. It would be great if we could publish their existence in the introspect xml data. [0] is one example of how this can be achieved in python. [0] http://bazaar.launchpad.net/~aptdaemon- developers/aptdaemon/main/view/head:/aptdaemon/core.py#L150 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/dbus-inspect-properties into lp:zeitgeist
Markus Korn has proposed merging lp:~thekorn/zeitgeist/dbus-inspect-properties
into lp:zeitgeist.
Requested reviews:
Zeitgeist Framework Team (zeitgeist)
For more details, see:
https://code.launchpad.net/~thekorn/zeitgeist/dbus-inspect-properties/+merge/46890
* Changed dbus introspection output to show information about properties (LP:
#704931).
* raise a ValueError if the property methods are called on an unknown
interface.
--
https://code.launchpad.net/~thekorn/zeitgeist/dbus-inspect-properties/+merge/46890
Your team Zeitgeist Framework Team is requested to review the proposed merge of
lp:~thekorn/zeitgeist/dbus-inspect-properties into lp:zeitgeist.
=== modified file '_zeitgeist/engine/remote.py'
--- _zeitgeist/engine/remote.py 2011-01-18 10:49:04 +
+++ _zeitgeist/engine/remote.py 2011-01-20 11:21:12 +
@@ -5,6 +5,7 @@
# Copyright © 2009-2010 Siegfried-Angel Gevatter Pujals rai...@ubuntu.com
# Copyright © 2009 Mikkel Kamstrup Erlandsen mikkel.kamst...@gmail.com
# Copyright © 2010 Seif Lotfy s...@lotfy.com
+# Copyright © 2011 Markus Korn thek...@gmx.de
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
@@ -23,6 +24,8 @@
import dbus.service
import logging
+from xml.etree import ElementTree
+
from zeitgeist.datamodel import TimeRange, StorageState, ResultType, NULL_EVENT
from _zeitgeist.engine.datamodel import Event, Subject
from _zeitgeist.engine import get_engine
@@ -30,6 +33,18 @@
from _zeitgeist.engine import constants
from _zeitgeist.singleton import SingletonApplication
+class DBUSProperty(property):
+
+ def __init__(self, fget=None, fset=None, in_signature=None, out_signature=None):
+ assert not (fget and not out_signature), fget needs a dbus signature
+ assert not (fset and not in_signature), fset needs a dbus signature
+ assert (fget and not fset) or (fset and fget), \
+ dbus properties needs to be either readonly or readwritable
+ self.in_signature = in_signature
+ self.out_signature = out_signature
+ super(DBUSProperty, self).__init__(fget, fset)
+
+
class RemoteInterface(SingletonApplication):
Primary interface to the Zeitgeist engine. Used to update and query
@@ -42,8 +57,8 @@
:const:`org.gnome.zeitgeist.Engine`.
_dbus_properties = {
- version: property(lambda self: (0, 7, 0)),
- extensions: property(lambda self: list(self._engine.extensions.iter_names())),
+ version: DBUSProperty(lambda self: (0, 7, 0), out_signature=iii),
+ extensions: DBUSProperty(lambda self: list(self._engine.extensions.iter_names()), out_signature=as),
}
# Initialization
@@ -357,6 +372,11 @@
@dbus.service.method(dbus_interface=dbus.PROPERTIES_IFACE,
in_signature=ss, out_signature=v)
def Get(self, interface_name, property_name):
+ if interface_name != constants.DBUS_INTERFACE:
+ raise ValueError(
+'%s' doesn't know anything about the '%s' interface \
+%(constants.DBUS_INTERFACE, interface_name)
+ )
try:
return self._dbus_properties[property_name].fget(self)
except KeyError, e:
@@ -365,6 +385,11 @@
@dbus.service.method(dbus_interface=dbus.PROPERTIES_IFACE,
in_signature=ssv, out_signature=)
def Set(self, interface_name, property_name, value):
+ if interface_name != constants.DBUS_INTERFACE:
+ raise ValueError(
+'%s' doesn't know anything about the '%s' interface \
+%(constants.DBUS_INTERFACE, interface_name)
+ )
try:
prop = self._dbus_properties[property_name].fset(self, value)
except (KeyError, TypeError), e:
@@ -373,7 +398,32 @@
@dbus.service.method(dbus_interface=dbus.PROPERTIES_IFACE,
in_signature=s, out_signature=a{sv})
def GetAll(self, interface_name):
+ if interface_name != constants.DBUS_INTERFACE:
+ raise ValueError(
+'%s' doesn't know anything about the '%s' interface \
+%(constants.DBUS_INTERFACE, interface_name)
+ )
return dict((k, v.fget(self)) for (k,v) in self._dbus_properties.items())
+
+ # Instrospection Interface
+
+ @dbus.service.method(dbus.INTROSPECTABLE_IFACE, in_signature=, out_signature=s,
+ path_keyword=object_path, connection_keyword=connection)
+ def Introspect(self, object_path, connection):
+ data = dbus.service.Object.Introspect(self, object_path, connection)
+ xml = ElementTree.fromstring(data)
+ for iface in xml.findall(interface):
+ if iface.attrib[name] != constants.DBUS_INTERFACE:
+continue
+ for prop_name, prop_func in self._dbus_properties.iteritems():
+prop = {name: prop_name}
+if prop_func.fset is not None:
+ prop[access] = readwrite
+else:
+ prop[access] = read
+prop[type] = prop_func.out_signature
+iface.append(ElementTree.Element(property, prop))
+ return ElementTree.tostring(xml, encoding=UTF-8)
# Notifications interface
___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
[Zeitgeist] [Bug 704931] Re: Publish property names, accesslevel and type in introspection data
** Branch linked: lp:~thekorn/zeitgeist/dbus-inspect-properties ** Changed in: zeitgeist Status: New = In Progress -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/704931 Title: Publish property names, accesslevel and type in introspection data Status in Zeitgeist Framework: In Progress Bug description: Our .Log dbus interface has two properties, version and extensions. It would be great if we could publish their existence in the introspect xml data. [0] is one example of how this can be achieved in python. [0] http://bazaar.launchpad.net/~aptdaemon- developers/aptdaemon/main/view/head:/aptdaemon/core.py#L150 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~thekorn/zeitgeist/dbus-inspect-properties into lp:zeitgeist
I'm going to change this branch such that we only use one xml parser, minidom or etree, thanks Mikkel for pointing this out. -- https://code.launchpad.net/~thekorn/zeitgeist/dbus-inspect-properties/+merge/46890 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/dbus-inspect-properties into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 704931] [NEW] Publish property names, accesslevel and type in introspection data
Public bug reported: Our .Log dbus interface has two properties, version and extensions. It would be great if we could publish their existence in the introspect xml data. [0] is one example of how this can be achieved in python. [0] http://bazaar.launchpad.net/~aptdaemon- developers/aptdaemon/main/view/head:/aptdaemon/core.py#L150 ** Affects: zeitgeist Importance: Undecided Assignee: Markus Korn (thekorn) Status: New ** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/704931 Title: Publish property names, accesslevel and type in introspection data Status in Zeitgeist Framework: New Bug description: Our .Log dbus interface has two properties, version and extensions. It would be great if we could publish their existence in the introspect xml data. [0] is one example of how this can be achieved in python. [0] http://bazaar.launchpad.net/~aptdaemon- developers/aptdaemon/main/view/head:/aptdaemon/core.py#L150 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 598666] Re: Invalid cache access (was: Error when trying to fetch items)
Seif sent me an activity.log which he managed to create by [...] randomly clicking in GAJ and synapse[...]. This log is broken and results in the above mentioned KeyError. The db has a few broken entries: http://paste.ubuntu.com/553188/ -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/598666 Title: Invalid cache access (was: Error when trying to fetch items) Status in Zeitgeist Framework: Triaged Bug description: When i try to fetch all items in one query i get Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/local/share/zeitgeist/_zeitgeist/engine/remote.py, line 254, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 378, in find_events return self._find_events(1, *args) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 366, in _find_events return self.get_events(rows=result, sender=sender) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 188, in get_events events[event.id].append_subject(self._get_subject_from_row(row)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 160, in _get_subject_from_row getattr(self, _ + field).value(row[subj_ + field])) File /usr/local/share/zeitgeist/_zeitgeist/engine/sql.py, line 422, in value return self._inv_dict[id] KeyError: 138 This looks serious ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~manishsinha/zeitgeist/fix-691167 into lp:zeitgeist
While reading the python dbus docs [0] again I found out about the byte_array=True, this will solve all our problems immediately. By using this switch in the @dbus.service.method and @dbus.service.signal decorators, ay is not exposed as dbus.Array of dbus.Bytes anymore, but as dbus.ByteArray. Which gives us the simple mapping dbus.ByteArray - str [0] http://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html#return-values-and-the-byte-arrays-and-utf8-strings-options -- https://code.launchpad.net/~manishsinha/zeitgeist/fix-691167/+merge/43948 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~manishsinha/zeitgeist/fix-691167 into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 612344] Re: Blacklist API sucks
After re-reading this bugreport, the discussion it includes and the merge proposal, I'm still not confident that the API which is proposed here is good enough. This is why I started working on a blacklist API spec. It is still work in progress, and I'm not sure if this points in the right direction (comments are welcome) but I at least have a good feeling. current version is here: http://paste.ubuntu.com/551001/ -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/612344 Title: Blacklist API sucks Status in NULL Project: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: In Progress Bug description: Guys, GetBlacklist and SetBlacklist (without any signals) for an asynchrounous-by-nature API? Come on! How about changing it to Get, Add, Remove and a changed signal? That way it'd be actually usable... ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 612344] Re: Blacklist API sucks
My first idea related to RegEx based matches was that I thought we need something very powerful there. But maybe I'm wrong, So I'm happy to change my proposal there and only allow prefix-search and negation. And if we need something more powerful later on we can always change it. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/612344 Title: Blacklist API sucks Status in NULL Project: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: In Progress Bug description: Guys, GetBlacklist and SetBlacklist (without any signals) for an asynchrounous-by-nature API? Come on! How about changing it to Get, Add, Remove and a changed signal? That way it'd be actually usable... ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 612344] Re: Blacklist API sucks
@Siegfried, how do you want to identify a template in the collection of all templates, for example, what do you propose as an argument for the RemoveTemplate() method? Do you want automatically generated identifier (e.g. auto-integer-id, maybe in combination with sender string)? or do you have a system in mind which does not use identifiers at all? -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/612344 Title: Blacklist API sucks Status in NULL Project: Triaged Status in Unity Files Place: Triaged Status in Zeitgeist Framework: In Progress Bug description: Guys, GetBlacklist and SetBlacklist (without any signals) for an asynchrounous-by-nature API? Come on! How about changing it to Get, Add, Remove and a changed signal? That way it'd be actually usable... ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~zeitgeist/zeitgeist/bug695311 into lp:zeitgeist
Review: Needs Information Siegfried, you added an additional handler for SIGTERM. Can you please elaborate when we receive such kind of signals. I mean we clearly documented SIGHUB as a tool for distributors, and I would like to see the same documentation for SIGTERM too. -- https://code.launchpad.net/~zeitgeist/zeitgeist/bug695311/+merge/44850 Your team Zeitgeist Framework Team is subscribed to branch lp:~zeitgeist/zeitgeist/bug695311. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 695087] Re: Invalid cache access
*** This bug is a duplicate of bug 598666 *** https://bugs.launchpad.net/bugs/598666 Ok, my concern is: this script only uses one manifestation and one interpretation, so dict lookup and sql query (including sqlite's internal caching) might have the same speed. What I'm curious to know is: is there if difference if we have many different of these values, at which point are sql queries slower than our cache lookups? -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/695087 Title: Invalid cache access Status in Zeitgeist Framework: Triaged Bug description: ala...@alagos-desktop:~$ gnome-activity-journal ** Message: pygobject_register_sinkfunc is deprecated (GstObject) /usr/share/gnome-activity-journal/src/common.py:747: DeprecationWarning: object.__new__() takes no parameters GIO_FILES[subj] = object.__new__(classtype, *args, **kwargs) /usr/share/gnome-activity-journal/src/activity_widgets.py:303: GtkWarning: gtk_box_pack: assertion `child-parent == NULL' failed self.pack_end(hbox) Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/share/zeitgeist/_zeitgeist/engine/remote.py, line 253, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 395, in find_events return self._find_events(1, *args) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 382, in _find_events result = self.get_events(ids=[row[0] for row in result], sender=sender) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 187, in get_events event = self._get_event_from_row(row) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 152, in _get_event_from_row setattr(event, field, getattr(self, _ + field).value(row[field])) File /usr/share/zeitgeist/_zeitgeist/engine/sql.py, line 430, in value return self._inv_dict[id] KeyError: 1 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 695297] [NEW] unused helper ZeitgeistRemoteDataSourceRegistryTest._register_data_source_python_api() is broken
Public bug reported: I'm talking about def _register_data_source_python_api(self, *args): mainloop = gobject.MainLoop() self.client.register_data_source(*args, reply_handler=lambda: mainloop.quit(), error_handler=lambda: fail(Error registering data-source)) mainloop.run() * In error_handler it should be self.fail(...) * register_data_source() has no reply_handler/error_handler arguments On the other hand, this method isn't used anywhere @Siegfried, I think you added this code recently, can you please fix it. Thanks so much ;) ** Affects: zeitgeist Importance: Undecided Status: New -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/695297 Title: unused helper ZeitgeistRemoteDataSourceRegistryTest._register_data_source_python_api() is broken Status in Zeitgeist Framework: New Bug description: I'm talking about def _register_data_source_python_api(self, *args): mainloop = gobject.MainLoop() self.client.register_data_source(*args, reply_handler=lambda: mainloop.quit(), error_handler=lambda: fail(Error registering data-source)) mainloop.run() * In error_handler it should be self.fail(...) * register_data_source() has no reply_handler/error_handler arguments On the other hand, this method isn't used anywhere @Siegfried, I think you added this code recently, can you please fix it. Thanks so much ;) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 695311] [NEW] DataSourceRegistry is a huge performance bottleneck
Public bug reported: I was wondering today why the timings for inserting events when running our testsuite (or my benchmark scripts) are looking much better than the timings in a 'real' daemon instance. Some Data: * inserting 500 events at once in my benchmarks: ~0.09 sec * inserting the same 500 events at once in a 'real' zeitgeist session: ~0.185 sec You can reproduce this tests by running the attached script like ZEITGEIST_DEFAULT_EXTENSIONS=_zeitgeist.engine.extensions.datasource_registry.DataSourceRegistry PYTHONPATH=. python /tmp/test_insert_events.py PYTHONPATH=. python /tmp/test_insert_events.py and compare outputs. To simulate the scenario of multiple running instances of the same application, uncomment the lines 26+ in the script, you will get even worse results. ** Affects: zeitgeist Importance: Undecided Status: New -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/695311 Title: DataSourceRegistry is a huge performance bottleneck Status in Zeitgeist Framework: New Bug description: I was wondering today why the timings for inserting events when running our testsuite (or my benchmark scripts) are looking much better than the timings in a 'real' daemon instance. Some Data: * inserting 500 events at once in my benchmarks: ~0.09 sec * inserting the same 500 events at once in a 'real' zeitgeist session: ~0.185 sec You can reproduce this tests by running the attached script like ZEITGEIST_DEFAULT_EXTENSIONS=_zeitgeist.engine.extensions.datasource_registry.DataSourceRegistry PYTHONPATH=. python /tmp/test_insert_events.py PYTHONPATH=. python /tmp/test_insert_events.py and compare outputs. To simulate the scenario of multiple running instances of the same application, uncomment the lines 26+ in the script, you will get even worse results. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 695311] Re: DataSourceRegistry is a huge performance bottleneck
** Attachment added: test_insert_events.py https://bugs.edge.launchpad.net/bugs/695311/+attachment/1778807/+files/test_insert_events.py -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/695311 Title: DataSourceRegistry is a huge performance bottleneck Status in Zeitgeist Framework: New Bug description: I was wondering today why the timings for inserting events when running our testsuite (or my benchmark scripts) are looking much better than the timings in a 'real' daemon instance. Some Data: * inserting 500 events at once in my benchmarks: ~0.09 sec * inserting the same 500 events at once in a 'real' zeitgeist session: ~0.185 sec You can reproduce this tests by running the attached script like ZEITGEIST_DEFAULT_EXTENSIONS=_zeitgeist.engine.extensions.datasource_registry.DataSourceRegistry PYTHONPATH=. python /tmp/test_insert_events.py PYTHONPATH=. python /tmp/test_insert_events.py and compare outputs. To simulate the scenario of multiple running instances of the same application, uncomment the lines 26+ in the script, you will get even worse results. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 695087] Re: Invalid cache access
*** This bug is a duplicate of bug 598666 *** https://bugs.launchpad.net/bugs/598666 @Siegfried, what exactly are you benchmarking here? Eg. how many different (relevant) values are you using? How do the graphs look if you have some hundred different actors and mimetypes and a wide range of used manifestations and interpretations? -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/695087 Title: Invalid cache access Status in Zeitgeist Framework: Triaged Bug description: ala...@alagos-desktop:~$ gnome-activity-journal ** Message: pygobject_register_sinkfunc is deprecated (GstObject) /usr/share/gnome-activity-journal/src/common.py:747: DeprecationWarning: object.__new__() takes no parameters GIO_FILES[subj] = object.__new__(classtype, *args, **kwargs) /usr/share/gnome-activity-journal/src/activity_widgets.py:303: GtkWarning: gtk_box_pack: assertion `child-parent == NULL' failed self.pack_end(hbox) Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/share/zeitgeist/_zeitgeist/engine/remote.py, line 253, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 395, in find_events return self._find_events(1, *args) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 382, in _find_events result = self.get_events(ids=[row[0] for row in result], sender=sender) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 187, in get_events event = self._get_event_from_row(row) File /usr/share/zeitgeist/_zeitgeist/engine/main.py, line 152, in _get_event_from_row setattr(event, field, getattr(self, _ + field).value(row[field])) File /usr/share/zeitgeist/_zeitgeist/engine/sql.py, line 430, in value return self._inv_dict[id] KeyError: 1 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 686732] Re: add cache to get events
this is the set of scripts which helped me to create the benchmarks mentioned in the attached merge proposal ** Attachment added: cache_benchmark.tar.gz https://bugs.launchpad.net/zeitgeist/+bug/686732/+attachment/1759472/+files/cache_benchmark.tar.gz -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/686732 Title: add cache to get events Status in Zeitgeist Framework: In Progress Bug description: Sometimes the same events get requested by several different apps. having a cache for the get_events cache can cut off some time, and enhance performance. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 686674] Re: zeitgeist crashes in case of non existing .cache folder (KDE)
Ohh? Is KDE (or the distro you are using) not using xdg, or why is ~/.cache not there? And if KDE uses something else for such kind of `local log files` we should use this location instead of creating XDG_CACHE with a big hammer. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/686674 Title: zeitgeist crashes in case of non existing .cache folder (KDE) Status in Zeitgeist Framework: In Progress Bug description: When trying to run trunk on a fresh KDE install I get s...@wumbo-vm:~$ zeitgeist-daemon --replace Traceback (most recent call last): File /usr/local/bin/zeitgeist-daemon, line 158, in module setup_logger(_config.options.log_level) File /usr/local/bin/zeitgeist-daemon, line 142, in setup_logger os.mkdir(os.path.dirname(log_file)) OSError: [Errno 2] No such file or directory: '/home/seif/.cache/zeitgeist' This is due to our latest change for logging outputs ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 680360] Re: Zeitgeist builds man page for zeitgeist-datahub
** Changed in: zeitgeist Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/680360 Title: Zeitgeist builds man page for zeitgeist-datahub Status in Zeitgeist Framework: Fix Committed Bug description: I haven't checked this against trunk, but it's the case for zeitgeist 0.6. It currently builds the man page for zeitgeist-datahub, which zeitgeist-datahub of course also does. Two packages claiming ownership to the same file is a very bad thing, so we should probably fix this. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 683847] Re: URI scheme for actors without a .desktop file
completely agree, 'officially' adding a 'dbus://'-scheme would also solve the issue of the telepathy dataprovider where we (until now) are not sure which value to use in the actor field. ** Changed in: zeitgeist Importance: Undecided = Wishlist ** Changed in: zeitgeist Status: New = Triaged ** Changed in: zeitgeist Milestone: None = 0.7.0 -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/683847 Title: URI scheme for actors without a .desktop file Status in Zeitgeist Framework: Triaged Bug description: What should actors that don't have a .desktop file use as their actor URI? Currently we use application://$desktop_file_id which is very nice to wrok with because one can do g_desktop_app_info_new($desktop_file_id). I propose we define a similar scheme in our API docs where services and daemons without .desktop files can use the basename of their DBus .service file instead and put that in a dbus:// URI scheme ala dbus://$dbus_service_id. Consider the example of the Ubuntu One syncdamon. The DBus service files are normally stored in /usr/share/dbus-1/services/ and it has indeed com.ubuntuone.SyncDaemon.service stored there. This daemon would then have actor URI dbus://com.ubuntuone.SyncDaemon.service. This is useful because it allows me to obtain both the executable name of the daemon process as well as the DBus name it claims. Together with the service file this gives me 3 pieces of crucial metadata on the actor. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 683280] Re: problem building from trunk in KDE
Ok, first of all, thanks Harald for the input and your offer, as much as I appreciate it I don't think we really want to move to some random other build system. Installing gnome-common shouldn't be a problem for anyone, because this package has no gnome specific dependencies. A downside of moving to cmake is that (afaik) noone in our core team has experience with cmake, which makes maintaining the build system hard for us. If we decide to switch the build system (someone has to come up with a very good reason in this bug, please) we should move to a system which is optimized to what we need - building python packages, distutils seems to be the only alternative for me. -- You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. https://bugs.launchpad.net/bugs/683280 Title: problem building from trunk in KDE Status in Zeitgeist Framework: New Bug description: Building zeitgeist from trunk requires gnome-common since we use gnome-autogen.sh While this makes our lives easy in GNOME, KDE people don't really want to install it. So there must be another way to build from trunk easily. Building from a source package works. But again some devs want trunk. Lets make this easier for them. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 683146] Re: fix for LP: #672965 breaks returns for AJ and Synapse
** Changed in: zeitgeist Status: New = Triaged ** Changed in: zeitgeist Importance: Undecided = Critical ** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) -- fix for LP: #672965 breaks returns for AJ and Synapse https://bugs.launchpad.net/bugs/683146 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Triaged Bug description: running revision 1639+ breaks returns for find_events when called from AJ and Synapse. I always get: DEBUG:zeitgeist.engine:Found 0 events IDs in 0.007377s This can be fixed by just making the queries to use the index. So just remove the + from infront of the timestamp --- if min_time != 0: where.add(+timestamp = ?, min_time) if max_time != sys.maxint: where.add(+timestamp = ?, max_time) --- ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 683146] Re: fix for LP: #672965 breaks returns for AJ and Synapse
** Changed in: zeitgeist Status: Triaged = Fix Committed -- fix for LP: #672965 breaks returns for AJ and Synapse https://bugs.launchpad.net/bugs/683146 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: running revision 1639+ breaks returns for find_events when called from AJ and Synapse. I always get: DEBUG:zeitgeist.engine:Found 0 events IDs in 0.007377s This can be fixed by just making the queries to use the index. So just remove the + from infront of the timestamp --- if min_time != 0: where.add(+timestamp = ?, min_time) if max_time != sys.maxint: where.add(+timestamp = ?, max_time) --- ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 681887] Re: Moving Zeitgeist dbus Bus name from gnome to Freedesktop
I don't agree with changing the namepsace for dbus names. There is simply no good reason why we should make this move. After all the gnome part is just a random string. And noone on KDE, Windows or whatever platform should have a problem using this string. Changing the name, maybe by even using *two* names at the same time for a period of time is adding extra complexity to our work, and I don't want this - unless someone comes up with a very good reason (like zg would be banned from certain systems if we keep on using 'gnome', etc.) -- Moving Zeitgeist dbus Bus name from gnome to Freedesktop https://bugs.launchpad.net/bugs/681887 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: Zeitgeist currently uses the org.gnome dbus domain. As we are no longer a GNOME-specific project, I think it would make more sense to move the project to the org.freedesktop domain, as org.freedesktop.Zeitgeist.engine or something similar. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 670358] Re: Add new aggregate API
I updated my proposal at http://wiki.zeitgeist- project.com/index.php?title=Aggregation_API -- Add new aggregate API https://bugs.launchpad.net/bugs/670358 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Confirmed Bug description: Zeitgeist API can give us vague information but not statistics over the API Use case: - Give me counts of every subject_text from actor = Unity Currently to do that one will need to either: Request all events with Unity as an Actor and count the subject_text This can be done much better IMHO I am thinking of an Aggregation Extension. But before we hack on that we need to agree if its necessary and how the API will look like. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 670358] Re: Add new aggregate API
@Siegfried: yes, but it is only redundant for FindEventsStats as FindEventIdsStats only returns ids and not events. It can't be avoided in this particular case, as the stats field has to return *someting*. @Michal: as we already clearified on irc, the mapping is done based on index, so the first element in the stats array points to the first element in the result one, etc. -- Add new aggregate API https://bugs.launchpad.net/bugs/670358 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Confirmed Bug description: Zeitgeist API can give us vague information but not statistics over the API Use case: - Give me counts of every subject_text from actor = Unity Currently to do that one will need to either: Request all events with Unity as an Actor and count the subject_text This can be done much better IMHO I am thinking of an Aggregation Extension. But before we hack on that we need to agree if its necessary and how the API will look like. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673452] Re: Using the subj_origin column of event_view is slower than it should be
** Changed in: zeitgeist Status: In Progress = Fix Committed -- Using the subj_origin column of event_view is slower than it should be https://bugs.launchpad.net/bugs/673452 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: Let me summarize what we have wrt subj_origin 1) we have a subj_origin column in the event table, which is an integer pointing to an entry in the uri table 2) we have an index on the subj_origin column of the event table 3) we have subj_origin in event view, which is not the id, but the resolved uri 4) we have subj_origin_id in event_view which is in fact the same as event.subj_uri, but we get this value by using a sub-query on the uri table. Points 3) and 4) are the reason why the index on event.subj_origin is never used when querying the event_view, example: DEBUG:zeitgeist.sql:Got query: QUERY: SELECT id FROM event_view WHERE subj_origin_id=1 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] By simply renaming event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin in combination with removing the sub-query we can make queries for the origin id on event_view *much* faster. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673394] Re: Queries for subj_uri and subj_origin are using no index
** Changed in: zeitgeist Status: In Progress = Fix Committed -- Queries for subj_uri and subj_origin are using no index https://bugs.launchpad.net/bugs/673394 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: We have a bunch of queries which are using no index and therefor are much slower than others, this is one example: Running: engine.find_eventids(TimeRange.always(), [], StorageState.Any, 6, ResultType.MostRecentSubjects) DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY subj_uri ORDER BY timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.571514s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.571749s All quries with subj_uri and subj_origin are affected. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/db_schema_3 into lp:zeitgeist
Markus Korn has proposed merging lp:~thekorn/zeitgeist/db_schema_3 into lp:zeitgeist. Requested reviews: Zeitgeist Framework Team (zeitgeist) Related bugs: #673394 Queries for subj_uri and subj_origin are using no index https://bugs.launchpad.net/bugs/673394 #673452 Using the subj_origin column of event_view is slower than it should be https://bugs.launchpad.net/bugs/673452 This branch introduces a new db schema (version 3) and bundles a few performance related fixes, [0] compares the performance of this branch lp:zeitgeist (each query is run ~10 times, the plot shows the average). Changes in detail: * renamed event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin, this makes subj_origin point to an id in the uri table, and subj_origin_uri the resolved value. Also we are not using the redundant SELECT id FROM uri WHERE uri.id=event.subj_origin query anymore. (LP: #673452) * removed event_view.subj_uri_id and added event.subj_id, both values are in fact the same, but we again sting on another SELECT query in the event_view. * Optimization in FindEvent queries: we are not ordering and grouping by resolved values anymore, we are doing it on integer ids, which is much faster (LP: #673394) [0] https://dl.dropbox.com/u/174479/compare_queries.svg -- https://code.launchpad.net/~thekorn/zeitgeist/db_schema_3/+merge/40877 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/db_schema_3 into lp:zeitgeist. === modified file '_zeitgeist/engine/__init__.py' --- _zeitgeist/engine/__init__.py 2010-10-19 13:54:12 + +++ _zeitgeist/engine/__init__.py 2010-11-15 15:11:50 + @@ -55,7 +55,7 @@ # Required version of DB schema CORE_SCHEMA=core - CORE_SCHEMA_VERSION = 2 + CORE_SCHEMA_VERSION = 3 USER_EXTENSION_PATH = os.path.join(DATA_PATH, extensions) === modified file '_zeitgeist/engine/main.py' --- _zeitgeist/engine/main.py 2010-11-12 17:39:28 + +++ _zeitgeist/engine/main.py 2010-11-15 15:11:50 + @@ -148,8 +148,9 @@ def _get_subject_from_row(self, row): subject = Subject() - for field in (uri, origin, text, storage): + for field in (uri, text, storage): setattr(subject, field, row[subj_ + field]) + setattr(subject, origin, row[subj_origin_uri]) for field in (interpretation, manifestation, mimetype): setattr(subject, field, getattr(self, _ + field).value(row[subj_ + field])) @@ -353,10 +354,13 @@ sql += ( ORDER BY timestamp DESC, ORDER BY timestamp ASC, - GROUP BY subj_uri ORDER BY timestamp DESC, - GROUP BY subj_uri ORDER BY timestamp ASC, - GROUP BY subj_uri ORDER BY COUNT(subj_uri) DESC, timestamp DESC, - GROUP BY subj_uri ORDER BY COUNT(subj_uri) ASC, timestamp ASC, + # thekorn: please note, event.subj_id == uri.id, as in + # the subj_id points directly to an entry in the uri table, + # so we are in fact grouping by subj_uris here + GROUP BY subj_id ORDER BY timestamp DESC, + GROUP BY subj_id ORDER BY timestamp ASC, + GROUP BY subj_id ORDER BY COUNT(subj_id) DESC, timestamp DESC, + GROUP BY subj_id ORDER BY COUNT(subj_id) ASC, timestamp ASC, GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC, GROUP BY actor ORDER BY COUNT(actor) ASC, timestamp ASC, GROUP BY actor ORDER BY timestamp DESC, === modified file '_zeitgeist/engine/sql.py' --- _zeitgeist/engine/sql.py 2010-10-25 20:26:03 + +++ _zeitgeist/engine/sql.py 2010-11-15 15:11:50 + @@ -369,14 +369,12 @@ AS payload, (SELECT value FROM uri WHERE uri.id=event.subj_id) AS subj_uri, -(SELECT id FROM uri WHERE uri.id=event.subj_id) - AS subj_uri_id, +event.subj_id, -- #this directly points to an uri event.subj_interpretation, event.subj_manifestation, +event.subj_origin, (SELECT value FROM uri WHERE uri.id=event.subj_origin) - AS subj_origin, -(SELECT id FROM uri WHERE uri.id=event.subj_origin) - AS subj_origin_id, + AS subj_origin_uri, event.subj_mimetype, (SELECT value FROM text WHERE text.id = event.subj_text) AS subj_text, @@ -528,14 +526,17 @@ def add_text_condition(self, column, value, like=False, negation=False, cache=None): if like: assert column in (subj_uri, subj_origin, actor, subj_mimetype), \ -prefix search on the %r column is not supported by zeitgeist - if column in (subj_uri, subj_origin): -view_column = %s_id %column +prefix search on the %r column is not supported by zeitgeist %column + if column == subj_uri: +# subj_id directly points to the id of an uri entry +view_column = subj_id else: view_column = column optimized_glob, value = self.optimize_glob(id, TABLE_MAP.get(column, column), value) sql = %s %sIN (%s) %(view_column, self.NOT if negation else , optimized_glob) else: + if column == subj_origin: +column = subj_origin_uri sql = %s %s= ? %(column
[Zeitgeist] [Bug 673916] Re: ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument
** Changed in: zeitgeist Status: In Progress = Fix Committed -- ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument https://bugs.launchpad.net/bugs/673916 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673916] Re: ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument
** Branch linked: lp:~thekorn/zeitgeist/db_schema_3 ** Branch unlinked: lp:~thekorn/zeitgeist/db_schema_3 -- ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument https://bugs.launchpad.net/bugs/673916 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673452] Re: Using the subj_origin column of event_view is slower than it should be
** Branch linked: lp:~thekorn/zeitgeist/db_schema_3 ** Branch unlinked: lp:~thekorn/zeitgeist/faster_origin_queries ** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist Status: Triaged = In Progress -- Using the subj_origin column of event_view is slower than it should be https://bugs.launchpad.net/bugs/673452 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: Let me summarize what we have wrt subj_origin 1) we have a subj_origin column in the event table, which is an integer pointing to an entry in the uri table 2) we have an index on the subj_origin column of the event table 3) we have subj_origin in event view, which is not the id, but the resolved uri 4) we have subj_origin_id in event_view which is in fact the same as event.subj_uri, but we get this value by using a sub-query on the uri table. Points 3) and 4) are the reason why the index on event.subj_origin is never used when querying the event_view, example: DEBUG:zeitgeist.sql:Got query: QUERY: SELECT id FROM event_view WHERE subj_origin_id=1 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] By simply renaming event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin in combination with removing the sub-query we can make queries for the origin id on event_view *much* faster. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~seif/zeitgeist/fix-673922 into lp:zeitgeist
Just for a reference, the last usage of the 'row' argument was removed with https://code.launchpad.net/~zeitgeist/zeitgeist/small-find-events-optimization/+merge/34065 -- https://code.launchpad.net/~seif/zeitgeist/fix-673922/+merge/40734 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~seif/zeitgeist/fix-673922 into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673922] Re: ZeitgeistEngine.get_events(rows=x) is broken
** Changed in: zeitgeist Status: Confirmed = Fix Committed -- ZeitgeistEngine.get_events(rows=x) is broken https://bugs.launchpad.net/bugs/673922 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: If one hands over a list of rows instead of ids to the method, get_events will fail with a TypeError. The reason is simple: ids will become a generator (line 174) and in line 183 there is a len(ids) call, which is undefined for generators. This can be solved by either fixing the code, or removing the rows argument, as a short grep through the zeitgeist source and testsuite shows that we are not using this argument at all. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist
I think we don't want to merge this branch into lp:zeitgeist until we understand the whole consequences of this change. -- https://code.launchpad.net/~thekorn/zeitgeist/fix-672965-opt_timerange_queries/+merge/40412 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673907] [NEW] engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist
Public bug reported: mar...@thekorn /tmp/zeitgeist % bzr revno 1630 == ERROR: testGetHook (engine-extension-test.TestExtensionHooks) -- Traceback (most recent call last): File /tmp/zeitgeist/test/engine-extension-test.py, line 139, in testGetHook events = self.engine.get_events(ids) File /tmp/zeitgeist/test/../_zeitgeist/engine/main.py, line 197, in get_events sorted_events[id_hash[event.id]] = event AttributeError: 'NoneType' object has no attribute 'id' -- ** Affects: zeitgeist Importance: High Status: New ** Changed in: zeitgeist Importance: Undecided = High ** Changed in: zeitgeist Milestone: None = 0.7.0 -- engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist https://bugs.launchpad.net/bugs/673907 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: mar...@thekorn /tmp/zeitgeist % bzr revno 1630 == ERROR: testGetHook (engine-extension-test.TestExtensionHooks) -- Traceback (most recent call last): File /tmp/zeitgeist/test/engine-extension-test.py, line 139, in testGetHook events = self.engine.get_events(ids) File /tmp/zeitgeist/test/../_zeitgeist/engine/main.py, line 197, in get_events sorted_events[id_hash[event.id]] = event AttributeError: 'NoneType' object has no attribute 'id' -- ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673907] Re: engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist
** Changed in: zeitgeist Assignee: (unassigned) = Seif Lotfy (seif) -- engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist https://bugs.launchpad.net/bugs/673907 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: mar...@thekorn /tmp/zeitgeist % bzr revno 1630 == ERROR: testGetHook (engine-extension-test.TestExtensionHooks) -- Traceback (most recent call last): File /tmp/zeitgeist/test/engine-extension-test.py, line 139, in testGetHook events = self.engine.get_events(ids) File /tmp/zeitgeist/test/../_zeitgeist/engine/main.py, line 197, in get_events sorted_events[id_hash[event.id]] = event AttributeError: 'NoneType' object has no attribute 'id' -- ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673907] Re: engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist
** Changed in: zeitgeist Status: New = Fix Committed -- engine-extension-test.TestExtensionHooks fails in today's lp:zeitgeist https://bugs.launchpad.net/bugs/673907 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: mar...@thekorn /tmp/zeitgeist % bzr revno 1630 == ERROR: testGetHook (engine-extension-test.TestExtensionHooks) -- Traceback (most recent call last): File /tmp/zeitgeist/test/engine-extension-test.py, line 139, in testGetHook events = self.engine.get_events(ids) File /tmp/zeitgeist/test/../_zeitgeist/engine/main.py, line 197, in get_events sorted_events[id_hash[event.id]] = event AttributeError: 'NoneType' object has no attribute 'id' -- ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673916] [NEW] ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument
Public bug reported: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ** Affects: zeitgeist Importance: High Status: New ** Branch linked: lp:~zeitgeist/zeitgeist/fix-673916-GetEvent-duplicate- id ** Changed in: zeitgeist Importance: Undecided = High ** Changed in: zeitgeist Milestone: None = 0.7.0 -- ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument https://bugs.launchpad.net/bugs/673916 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673916] Re: ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument
As a starting-point to fix this bug I attached a branch with a testcase -- ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument https://bugs.launchpad.net/bugs/673916 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673922] [NEW] ZeitgeistEngine.get_events(rows=x) is broken
Public bug reported: If one hands over a list of rows instead of ids to the method, get_events will fail with a TypeError. The reason is simple: ids will become a generator (line 174) and in line 183 there is a len(ids) call, which is undefined for generators. This can be solved by either fixing the code, or removing the rows argument, as a short grep through the zeitgeist source and testsuite shows that we are not using this argument at all. ** Affects: zeitgeist Importance: Undecided Status: New -- ZeitgeistEngine.get_events(rows=x) is broken https://bugs.launchpad.net/bugs/673922 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: If one hands over a list of rows instead of ids to the method, get_events will fail with a TypeError. The reason is simple: ids will become a generator (line 174) and in line 183 there is a len(ids) call, which is undefined for generators. This can be solved by either fixing the code, or removing the rows argument, as a short grep through the zeitgeist source and testsuite shows that we are not using this argument at all. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673916] Re: ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument
** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist Status: New = In Progress -- ZeitgeistEngine.get_events() returns wrong result if there are duplicates in the ids argument https://bugs.launchpad.net/bugs/673916 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: As of today, ZeitgeistEngine.get_events([1, 1]) returns [None, Event with id=1] instead of [Event with id=1, Event with id=1] This worked in rev1629 but is not working in rev 1631 anymore. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673386] [NEW] 'iso_strptime.py' is mentioned in COPYRIGHT files although we don't ship this file anymore
Public bug reported: mar...@thekorn ~/devel/zeitgeist/trunk % grep -ri iso_strptime.py . ./COPYRIGHT:26:File zeitgeist/loggers/iso_strptime.py is part of wadllib, which is ./extra/packaging/debian/copyright:20:Files: zeitgeist/loggers/iso_strptime.py From my understanding it is save to remove this two extra clauses. ** Affects: zeitgeist Importance: Undecided Status: New -- 'iso_strptime.py' is mentioned in COPYRIGHT files although we don't ship this file anymore https://bugs.launchpad.net/bugs/673386 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: mar...@thekorn ~/devel/zeitgeist/trunk % grep -ri iso_strptime.py . ./COPYRIGHT:26:File zeitgeist/loggers/iso_strptime.py is part of wadllib, which is ./extra/packaging/debian/copyright:20:Files: zeitgeist/loggers/iso_strptime.py From my understanding it is save to remove this two extra clauses. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673394] [NEW] Queries for subj_uri and subj_origin are using no index
Public bug reported: We have a bunch of queries which are using no index and therefor are much slower than others, this is one example: Running: engine.find_eventids(TimeRange.always(), [], StorageState.Any, 6, ResultType.MostRecentSubjects) DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY subj_uri ORDER BY timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.571514s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.571749s All quries with subj_uri and subj_origin are affected. ** Affects: zeitgeist Importance: Undecided Status: New -- Queries for subj_uri and subj_origin are using no index https://bugs.launchpad.net/bugs/673394 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: We have a bunch of queries which are using no index and therefor are much slower than others, this is one example: Running: engine.find_eventids(TimeRange.always(), [], StorageState.Any, 6, ResultType.MostRecentSubjects) DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY subj_uri ORDER BY timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.571514s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.571749s All quries with subj_uri and subj_origin are affected. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 632363] Re: Slow queries: SQL indexes not used
Bug 673394 describes anopther class of slow queries -- Slow queries: SQL indexes not used https://bugs.launchpad.net/bugs/632363 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Confirmed Bug description: THE PROBLEM: I am seeing query times around 200ms (and 150ms with my latest performance tweak in trunk), which surprised me as quite slow since my queries where quite simple. I had expected times around 1-2ms. Reading up on the sqlite documentation I see that the queries we generate are pretty far from optimized in an sqlite world [1]. The case is that when ever you use an OR sqlite will no longer use an index. Thus this query is NOT using the indexes: SELECT * FROM event WHERE interpretation=1 OR interpretation=2 But if we rewrite it using IN instead the indexes will be used: SELECT * FROM event WHERE interpretation IN (1, 2) This also explains the case where Michal where seeing tremendously slow query times when searching for a big range of mimetypes. Looking in _zeitgeist.engine.sql.WhereClause.add_text_condition() I am also pretty sure we are not using the indexes for prefix queries (eg file://home/*). THE SOLUTION I *definitely* don't think we should panic and feverishly start rewriting our query compilation. Here's what I propose: 1) Implement an envvar ZEITGEIST_DEBUG_QUERY_PLANS which will spit out all our SQL calls and the query plans for each of our calls. The query plan will tell us how the db is queried and which indexes are used if any. The query plan is obtained by prefixing the SQL statement with EXPLAIN QUERY PLAN. 2) Collect some useful intelligence with this new tool, and generally learn more about how we can optimize sqlite queries. A big question here is how the event_view VIEW impacts the query plan. 3) Write a new template - SQL compilation engine that generates SQL optimized for sqlite. We can actually be quite clever about grouping our OR statements into IN clauses - but it will be tricky to get right. NOTE: That this doesn't imply any change in the public API or event template system. That would be the wrong solution imho. Our current API is nice and simple by my standards. Let's keep it that way. [1]: See fx the section Using Indexes To Speed Searching in http://www.sqlite.org/vdbe.html ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 632363] Re: Slow queries: SQL indexes not used
Also, bug 673452 -- Slow queries: SQL indexes not used https://bugs.launchpad.net/bugs/632363 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Confirmed Bug description: THE PROBLEM: I am seeing query times around 200ms (and 150ms with my latest performance tweak in trunk), which surprised me as quite slow since my queries where quite simple. I had expected times around 1-2ms. Reading up on the sqlite documentation I see that the queries we generate are pretty far from optimized in an sqlite world [1]. The case is that when ever you use an OR sqlite will no longer use an index. Thus this query is NOT using the indexes: SELECT * FROM event WHERE interpretation=1 OR interpretation=2 But if we rewrite it using IN instead the indexes will be used: SELECT * FROM event WHERE interpretation IN (1, 2) This also explains the case where Michal where seeing tremendously slow query times when searching for a big range of mimetypes. Looking in _zeitgeist.engine.sql.WhereClause.add_text_condition() I am also pretty sure we are not using the indexes for prefix queries (eg file://home/*). THE SOLUTION I *definitely* don't think we should panic and feverishly start rewriting our query compilation. Here's what I propose: 1) Implement an envvar ZEITGEIST_DEBUG_QUERY_PLANS which will spit out all our SQL calls and the query plans for each of our calls. The query plan will tell us how the db is queried and which indexes are used if any. The query plan is obtained by prefixing the SQL statement with EXPLAIN QUERY PLAN. 2) Collect some useful intelligence with this new tool, and generally learn more about how we can optimize sqlite queries. A big question here is how the event_view VIEW impacts the query plan. 3) Write a new template - SQL compilation engine that generates SQL optimized for sqlite. We can actually be quite clever about grouping our OR statements into IN clauses - but it will be tricky to get right. NOTE: That this doesn't imply any change in the public API or event template system. That would be the wrong solution imho. Our current API is nice and simple by my standards. Let's keep it that way. [1]: See fx the section Using Indexes To Speed Searching in http://www.sqlite.org/vdbe.html ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 673452] [NEW] Using the subj_origin column of event_view is slower than it should be
Public bug reported: Let me summarize what we have wrt subj_origin 1) we have a subj_origin column in the event table, which is an integer pointing to an entry in the uri table 2) we have an index on the subj_origin column of the event table 3) we have subj_origin in event view, which is not the id, but the resolved uri 4) we have subj_origin_id in event_view which is in fact the same as event.subj_uri, but we get this value by using a sub-query on the uri table. Points 3) and 4) are the reason why the index on event.subj_origin is never used when querying the event_view, example: DEBUG:zeitgeist.sql:Got query: QUERY: SELECT id FROM event_view WHERE subj_origin_id=1 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] By simply renaming event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin in combination with removing the sub-query we can make queries for the origin id on event_view *much* faster. ** Affects: zeitgeist Importance: Undecided Status: New -- Using the subj_origin column of event_view is slower than it should be https://bugs.launchpad.net/bugs/673452 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: Let me summarize what we have wrt subj_origin 1) we have a subj_origin column in the event table, which is an integer pointing to an entry in the uri table 2) we have an index on the subj_origin column of the event table 3) we have subj_origin in event view, which is not the id, but the resolved uri 4) we have subj_origin_id in event_view which is in fact the same as event.subj_uri, but we get this value by using a sub-query on the uri table. Points 3) and 4) are the reason why the index on event.subj_origin is never used when querying the event_view, example: DEBUG:zeitgeist.sql:Got query: QUERY: SELECT id FROM event_view WHERE subj_origin_id=1 ([]) PLAN: [0, 0, u'TABLE event'] [0, 0, u'TABLE uri USING PRIMARY KEY'] By simply renaming event_view.subj_origin to event_view.subj_origin_uri and event_view.subj_origin_id to event_view.subj_origin in combination with removing the sub-query we can make queries for the origin id on event_view *much* faster. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 672965] [NEW] find_event() queries with timerange other than TimeRange.always() are slow
Public bug reported: When running the attached script you can see that when giving a timerange which does not start at 0 and end at maxint the queries get 400% slower. python sample_timerange_query.py DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event WITH INDEX event_actor ORDER BY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.080176s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.080375s DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view WHERE (timestamp = ? AND timestamp = ?) GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([u'1', u'5']) PLAN: [0, 0, u'TABLE event WITH INDEX event_timestamp'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.260648s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.260838s The reason is simply that the second query uses the 'wrong' index. ** Affects: zeitgeist Importance: Undecided Status: New -- find_event() queries with timerange other than TimeRange.always() are slow https://bugs.launchpad.net/bugs/672965 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: When running the attached script you can see that when giving a timerange which does not start at 0 and end at maxint the queries get 400% slower. python sample_timerange_query.py DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event WITH INDEX event_actor ORDER BY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.080176s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.080375s DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view WHERE (timestamp = ? AND timestamp = ?) GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([u'1', u'5']) PLAN: [0, 0, u'TABLE event WITH INDEX event_timestamp'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.260648s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.260838s The reason is simply that the second query uses the 'wrong' index. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 632363] Re: Slow queries: SQL indexes not used
I've just filed bug 672965 for one class of slow queries -- Slow queries: SQL indexes not used https://bugs.launchpad.net/bugs/632363 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Confirmed Bug description: THE PROBLEM: I am seeing query times around 200ms (and 150ms with my latest performance tweak in trunk), which surprised me as quite slow since my queries where quite simple. I had expected times around 1-2ms. Reading up on the sqlite documentation I see that the queries we generate are pretty far from optimized in an sqlite world [1]. The case is that when ever you use an OR sqlite will no longer use an index. Thus this query is NOT using the indexes: SELECT * FROM event WHERE interpretation=1 OR interpretation=2 But if we rewrite it using IN instead the indexes will be used: SELECT * FROM event WHERE interpretation IN (1, 2) This also explains the case where Michal where seeing tremendously slow query times when searching for a big range of mimetypes. Looking in _zeitgeist.engine.sql.WhereClause.add_text_condition() I am also pretty sure we are not using the indexes for prefix queries (eg file://home/*). THE SOLUTION I *definitely* don't think we should panic and feverishly start rewriting our query compilation. Here's what I propose: 1) Implement an envvar ZEITGEIST_DEBUG_QUERY_PLANS which will spit out all our SQL calls and the query plans for each of our calls. The query plan will tell us how the db is queried and which indexes are used if any. The query plan is obtained by prefixing the SQL statement with EXPLAIN QUERY PLAN. 2) Collect some useful intelligence with this new tool, and generally learn more about how we can optimize sqlite queries. A big question here is how the event_view VIEW impacts the query plan. 3) Write a new template - SQL compilation engine that generates SQL optimized for sqlite. We can actually be quite clever about grouping our OR statements into IN clauses - but it will be tricky to get right. NOTE: That this doesn't imply any change in the public API or event template system. That would be the wrong solution imho. Our current API is nice and simple by my standards. Let's keep it that way. [1]: See fx the section Using Indexes To Speed Searching in http://www.sqlite.org/vdbe.html ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 672965] Re: find_event() queries with timerange other than TimeRange.always() are slow
** Description changed: When running the attached script you can see that when giving a timerange which does not start at 0 and end at maxint the queries get - 400% slower. - + 300% slower. python sample_timerange_query.py DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event WITH INDEX event_actor ORDER BY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.080176s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.080375s DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view WHERE (timestamp = ? AND timestamp = ?) GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([u'1', u'5']) PLAN: [0, 0, u'TABLE event WITH INDEX event_timestamp'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.260648s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.260838s The reason is simply that the second query uses the 'wrong' index. -- find_event() queries with timerange other than TimeRange.always() are slow https://bugs.launchpad.net/bugs/672965 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: When running the attached script you can see that when giving a timerange which does not start at 0 and end at maxint the queries get 300% slower. python sample_timerange_query.py DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event WITH INDEX event_actor ORDER BY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.080176s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.080375s DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view WHERE (timestamp = ? AND timestamp = ?) GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([u'1', u'5']) PLAN: [0, 0, u'TABLE event WITH INDEX event_timestamp'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.260648s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.260838s The reason is simply that the second query uses the 'wrong' index. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist
Markus Korn has proposed merging lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist. Requested reviews: Zeitgeist Framework Team (zeitgeist) Related bugs: #672965 find_event() queries with timerange other than TimeRange.always() are slow https://bugs.launchpad.net/bugs/672965 Made FindEvent queries faster if a timerange other than TimeRange.always() is used (LP: #672965) This is achieved by forcing the query planner to not use the timestamp index for this kind of queries by using the unary '+' operator. (see http://www.sqlite.org/optoverview.html section 6.0) -- https://code.launchpad.net/~thekorn/zeitgeist/fix-672965-opt_timerange_queries/+merge/40412 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist. === modified file '_zeitgeist/engine/main.py' --- _zeitgeist/engine/main.py 2010-10-25 09:48:49 + +++ _zeitgeist/engine/main.py 2010-11-09 11:12:51 + @@ -302,9 +302,9 @@ where = WhereClause(WhereClause.AND) min_time, max_time = time_range if min_time != 0: - where.add(timestamp = ?, min_time) + where.add(+timestamp = ?, min_time) if max_time != sys.maxint: - where.add(timestamp = ?, max_time) + where.add(+timestamp = ?, max_time) where.extend(self._build_sql_from_event_templates(templates)) === modified file '_zeitgeist/engine/sql.py' --- _zeitgeist/engine/sql.py 2010-10-25 20:26:03 + +++ _zeitgeist/engine/sql.py 2010-11-09 11:12:51 + @@ -47,6 +47,8 @@ @staticmethod def fix_unicode(obj): + if isinstance(obj, int): + return obj if isinstance(obj, str): obj = obj.decode(UTF-8) return unicode(obj) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-672965-opt_timerange_queries into lp:zeitgeist
The question which came in my mind after changing the code was: does the performance of other queries suffer from this change, so I changed the script attached to the bug to test all ResultType Before the change: --- LeastPopularActor: get len(ids)=6 using .find_eventids() in 0.078450s --- LeastPopularActor: get len(ids)=6 using .find_eventids() in 0.261053s ++ LeastPopularActor 332.763498996 --- LeastPopularMimeType: get len(ids)=1 using .find_eventids() in 0.066008s --- LeastPopularMimeType: get len(ids)=1 using .find_eventids() in 0.206841s ++ LeastPopularMimeType 313.35702779 --- LeastPopularOrigin: get len(ids)=1 using .find_eventids() in 0.200927s --- LeastPopularOrigin: get len(ids)=1 using .find_eventids() in 0.257056s ++ LeastPopularOrigin 127.935007932 --- LeastPopularSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.065896s --- LeastPopularSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.207888s ++ LeastPopularSubjectInterpretation 315.479743982 --- LeastPopularSubjects: get len(ids)=6 using .find_eventids() in 0.721558s --- LeastPopularSubjects: get len(ids)=6 using .find_eventids() in 0.774241s ++ LeastPopularSubjects 107.301266111 --- LeastRecentActor: get len(ids)=6 using .find_eventids() in 0.071438s --- LeastRecentActor: get len(ids)=6 using .find_eventids() in 0.258033s ++ LeastRecentActor 361.19819913 --- LeastRecentEvents: get len(ids)=6 using .find_eventids() in 0.133322s --- LeastRecentEvents: get len(ids)=6 using .find_eventids() in 0.247588s ++ LeastRecentEvents 185.706902626 --- LeastRecentMimeType: get len(ids)=1 using .find_eventids() in 0.064972s --- LeastRecentMimeType: get len(ids)=1 using .find_eventids() in 0.211424s ++ LeastRecentMimeType 325.407228279 --- LeastRecentOrigin: get len(ids)=1 using .find_eventids() in 0.172430s --- LeastRecentOrigin: get len(ids)=1 using .find_eventids() in 0.234929s ++ LeastRecentOrigin 136.246031658 --- LeastRecentSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.055954s --- LeastRecentSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.185784s ++ LeastRecentSubjectInterpretation 332.029758658 --- LeastRecentSubjects: get len(ids)=6 using .find_eventids() in 0.488676s --- LeastRecentSubjects: get len(ids)=6 using .find_eventids() in 0.537375s ++ LeastRecentSubjects 109.965477134 --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.069163s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.224269s ++ MostPopularActor 324.261352472 --- MostPopularMimeType: get len(ids)=1 using .find_eventids() in 0.056728s --- MostPopularMimeType: get len(ids)=1 using .find_eventids() in 0.186340s ++ MostPopularMimeType 328.480587054 --- MostPopularOrigin: get len(ids)=1 using .find_eventids() in 0.176185s --- MostPopularOrigin: get len(ids)=1 using .find_eventids() in 0.221803s ++ MostPopularOrigin 125.892285645 --- MostPopularSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.057706s --- MostPopularSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.179496s ++ MostPopularSubjectInterpretation 311.053314383 --- MostPopularSubjects: get len(ids)=6 using .find_eventids() in 0.617611s --- MostPopularSubjects: get len(ids)=6 using .find_eventids() in 0.685492s ++ MostPopularSubjects 110.990917401 --- MostRecentActor: get len(ids)=6 using .find_eventids() in 0.062013s --- MostRecentActor: get len(ids)=6 using .find_eventids() in 0.218987s ++ MostRecentActor 353.129925952 --- MostRecentEvents: get len(ids)=6 using .find_eventids() in 0.128710s --- MostRecentEvents: get len(ids)=6 using .find_eventids() in 0.256136s ++ MostRecentEvents 199.00231361 --- MostRecentMimeType: get len(ids)=1 using .find_eventids() in 0.064722s --- MostRecentMimeType: get len(ids)=1 using .find_eventids() in 0.216302s ++ MostRecentMimeType 334.201220051 --- MostRecentOrigin: get len(ids)=1 using .find_eventids() in 0.172238s --- MostRecentOrigin: get len(ids)=1 using .find_eventids() in 0.229335s ++ MostRecentOrigin 133.149875626 --- MostRecentSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.064796s --- MostRecentSubjectInterpretation: get len(ids)=1 using .find_eventids() in 0.211639s ++ MostRecentSubjectInterpretation 326.623591661 --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.574415s --- MostRecentSubjects: get len(ids)=6 using .find_eventids() in 0.620583s ++ MostRecentSubjects 108.03736898 --- OldestActor: get len(ids)=6 using .find_eventids() in 0.187836s --- OldestActor: get len(ids)=6 using .find_eventids() in 0.363429s ++ OldestActor 193.482187396 After the change: --- LeastPopularActor: get len(ids)=6 using .find_eventids() in 0.067574s --- LeastPopularActor: get len(ids)=6 using .find_eventids() in 0.073416s ++ LeastPopularActor 108.645290129 --- LeastPopularMimeType: get len(ids)=1 using .find_eventids() in
[Zeitgeist] [Bug 672965] Re: find_event() queries with timerange other than TimeRange.always() are slow
** Changed in: zeitgeist Status: New = In Progress ** Changed in: zeitgeist Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist Importance: Undecided = Low ** Changed in: zeitgeist Milestone: None = 0.7.0 -- find_event() queries with timerange other than TimeRange.always() are slow https://bugs.launchpad.net/bugs/672965 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: When running the attached script you can see that when giving a timerange which does not start at 0 and end at maxint the queries get 300% slower. python sample_timerange_query.py DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([]) PLAN: [0, 0, u'TABLE event WITH INDEX event_actor ORDER BY'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.080176s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.080375s DEBUG:zeitgeist.sql:Got query: QUERY: SELECT DISTINCT id FROM event_view WHERE (timestamp = ? AND timestamp = ?) GROUP BY actor ORDER BY COUNT(actor) DESC, timestamp DESC LIMIT 6 ([u'1', u'5']) PLAN: [0, 0, u'TABLE event WITH INDEX event_timestamp'] DEBUG:zeitgeist.engine:Found 6 event IDs in 0.260648s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.260838s The reason is simply that the second query uses the 'wrong' index. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 670355] Re: dynamic loading of extensions
Restarting of zeitgeist after installation the GAJ package requires no user action, because GAJ does the right thing (tm), in its package install process a SIGHUB signal is send to zeitgeist, which shuts the daemon down. - If this is not the case right now, it's a bug in the packaging of GAJ. And to make one thing a bit clearer, there is not need to 'restart' the daemon, all we have to do is to *stop* the daemon, the daemon will then start again whenever needed. -- dynamic loading of extensions https://bugs.launchpad.net/bugs/670355 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: When wanting to use a new extension one has to restart zeitgeist. Problem with that is that all monitors are then gone. My suggestion would be to be able to dynamically load extension by monitoring the extensions directories and pulling in the new extension on runtime. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 670355] Re: dynamic loading of extensions
ups, pressed 'post comment' to early, from my POV this bug is 'won't fix' -- dynamic loading of extensions https://bugs.launchpad.net/bugs/670355 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: When wanting to use a new extension one has to restart zeitgeist. Problem with that is that all monitors are then gone. My suggestion would be to be able to dynamically load extension by monitoring the extensions directories and pulling in the new extension on runtime. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 670358] Re: Add new aggregate API
I suggest something like: def find_events_and_data(*find_event_arguments, datatype_const): ... return result result = [events, data] datatype_const: DATATYPE_COUNT - result of COUNT() from within the sql-query DATATYPE_RELATIVE_COUNT - result of COUNT() from within the sql-query relative to the overall result ... same with ids which is more flexible than seif's proposal, data is not necessarily a count... -- Add new aggregate API https://bugs.launchpad.net/bugs/670358 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: New Bug description: Zeitgeist API can give us vague information but not statistics over the API Use case: - Give me counts of every subject_text from actor = Unity Currently to do that one will need to either: Request all events with Unity as an Actor and count the subject_text This can be done much better IMHO I am thinking of an Aggregation Extension. But before we hack on that we need to agree if its necessary and how the API will look like. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 668771] Re: `make distcheck` fails badly
** Changed in: zeitgeist-dataproviders Assignee: (unassigned) = Markus Korn (thekorn) ** Changed in: zeitgeist-dataproviders Status: New = Fix Committed -- `make distcheck` fails badly https://bugs.launchpad.net/bugs/668771 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Data-Sources. Status in Zeitgeist Data-Sources: Fix Committed Bug description: Hey, I've finally looked into packaging the data-sources but I can't get make distcheck to work, I'm getting failures with the Firefox, Chromium and Totem plugins (plus some other I've already fixed). For example: make[2]: Entering directory `/tmp/zeitgeist-dataproviders/zeitgeist-dataproviders-0.1.0/_build/totem-libzg' CC libtotem_zeitgeist_plugin_la-totem-zeitgeist-plugin.lo ../../totem-libzg/totem-zeitgeist-plugin.c:9:26: error: totem-plugin.h: File or directory doesn't exist ../../totem-libzg/totem-zeitgeist-plugin.c:11:19: error: totem.h: File or directory doesn't exist ../../totem-libzg/totem-zeitgeist-plugin.c:12:32: error: bacon-video-widget.h: File or directory doesn't exist ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 665607] Re: DB upgrades not backwards compatible (even when their are)
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- DB upgrades not backwards compatible (even when their are) https://bugs.launchpad.net/bugs/665607 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: Not sure if we already fixed this, but just in case. INFO:zeitgeist.sql:Upgrading database 'core' from version 2 to 1. This may take a while CRITICAL:zeitgeist.sql:Failed to upgrade database 'core' from version 2 to 1: No module named core_2_1 ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 643303] Re: Upgrade of the db schema strategy for version jumps
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Upgrade of the db schema strategy for version jumps https://bugs.launchpad.net/bugs/643303 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: We have to discuss how to support versions jump in the upgrade path of our db schema, like from (core, 0) to (core, 2). This becomes even more important when we reach the next version. For me there are two solution: 1.) write 'dummy' upgrade scripts, like core_0_2.py, which do not more than running core_0_1.run() and core_1_2.run() one after another 2.) add some magic to sql._do_schema_upgrade() which automatically tries to find an upgrade path and run the 'shortest' possible one. There is an issue with both solutions: right now the upgrade mechanism is designed in a way that after each data upgrade the schema upgrade is done. In both solutions the schema upgrade will be done *after* the full upgrade path. This can be handled by solution 1.) in the sanest possible way, the downside is that 1.) does not scale well for big numbers of schema versions around. Any Ideas? ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 630593] Re: Replace old datahub with vala port
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Replace old datahub with vala port https://bugs.launchpad.net/bugs/630593 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Status in Zeitgeist Datahub: In Progress Bug description: Now with mhr3's vala datahub port, I think we should get rid of the python version ASAP. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 655164] Re: Support more ResultType
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Support more ResultType https://bugs.launchpad.net/bugs/655164 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: I am in discussion with the Software Center team and it seems like they are growing fond of Zeitgeist. There is a feature in their queue that they would like to implement and where Zeitgeist can make it happen: - Software Center wants to suggest applications to be installed based on the Most or Recently Used Subject Interpretations and/or mimetypes. Currently I plan to call all events within a specific timeperiod and sort internally but it would be more helpful to have this info provided by Zeitgeist. So long story short I suggest 2 new ResultTypes: 1) MostPopular/MostRecent/LeastPopular/LeastRecent Interpretation 2) MostPopular/MostRecent/LeastPopular/LeastRecent Mimetype ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 650930] Re: Make timestamp related queries faster
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Make timestamp related queries faster https://bugs.launchpad.net/bugs/650930 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: When grep'ing the source of `unity-place-applications` and `unity-place-files` you will find out that these two products are using the FindEvents method, and both products are using this method to query for the complete time interval [0, sys.maxint] (aka. TimeRange.always()). And I've seen similar queries in other clients too. So I think it is worth trying to optimize the query for this kind of queries. The attached script is timing the exactly same query which is used in `unity-place-applications` on a database which contains 50k events, with 100 randomly choosen different actors. On lp:zeitgeist the find_events query returns the 6 events with most popular actors in ~0.22s --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.224661s By making the sql queries on the timestamp column conditional we will be able to reduce this time to --- MostPopularActor: get len(ids)=6 using .find_eventids() in 0.069161s ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 634744] Re: test suite fails if zeitgeist is installed on the system
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- test suite fails if zeitgeist is installed on the system https://bugs.launchpad.net/bugs/634744 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: Today I tried to run our test suite on a system which has zeitgeist installed and running (maverick netbook edition). The test suite fails with a lot of errors because it was unable to kill the running daemon instance. We should make sure to run our tests on its own private dbus message bus. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 660440] Re: Using logging output for debugging purposes
** Changed in: zeitgeist Milestone: 0.6 = 0.7 -- Using logging output for debugging purposes https://bugs.launchpad.net/bugs/660440 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: Right now, whenever we get a bugreport from a user who is zeitgeist `indirectly` (via unity, gnome-activity-journal, ...) we have to ask the reporter to open a terminal, run zeitgeist-daemon, do your usual stuff, get us the logging output This is not optimal, as we a requesting some extra work from our users (who maybe don't care about zeitgeist, they just want there clients to work) and it might also take a long time to get the debugging output, we are depending on our users here. It would be much easier if zeitgeist would also log to a file (or even better to a rotating set of files). On ubuntu we can then make apport to add such files to a bugreport. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 642772] Re: event_view changes requires schema upgrade
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- event_view changes requires schema upgrade https://bugs.launchpad.net/bugs/642772 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: The recently merged changes to use journal_mode=WAL and adding some columns to event_view requires schema upgrades. 1) Setting WAL journalling *permanently* makes the DB unreadable with sqlite 3.6 which is still widely deployed. Notably if I boot back into Ubuntu Lucid Lynx Zeitgeist will no longer work as lucid only ships sqlite 3.6. 2) Contrary to what is stated in the merge request for the event_view changes the event_view VIEW is *not* regenerated on each startup. That is one of the whole points of doing schema versioning. This change makes Zeitgeist trunk unable to return any results if run on an older db. Strangely enough it doesn't raise any errors... Guys - if you are making low level changes to our DB structure like both of these changes are, please make sure you understand all the ramifications before merging. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 660415] Re: zeitgeist-daemon.py has a bad code structure
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- zeitgeist-daemon.py has a bad code structure https://bugs.launchpad.net/bugs/660415 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: The code of zeitgeist-daemon.py is very hard to read, let's restructure the code! (This is more like a reminder for myself to work on it, but I would like to fix it after bug #655164 has been fixed) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 642686] Re: Add subj_uri_id and subj_origin_id columns to event_view
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Add subj_uri_id and subj_origin_id columns to event_view https://bugs.launchpad.net/bugs/642686 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: If we add subj_uri_id and subj_origin_id columns to the event_view we could make the prefix searches about 10% faster. Combined with the upcoming optimization of the GLOB statement it will be possible to reduce the avg. time for this kind of queries from ~0.20s to ~0.12s. As always I've added a script where you can test the proposed change to event_view yourself. *** 551 entries starting with 'http://aa' SELECT DISTINCT id FROM event_view WHERE(timestamp = ? AND timestamp = ? AND ((subj_uri IN (SELECT value FROM uri WHERE value GLOB ? ORDER BY timestamp DESC [u'0', u'9223372036854775807', u'http://aa*'] --- get len(ids)=551 using pure sql in 0.206047s SELECT DISTINCT id FROM event_view_new WHERE(timestamp = ? AND timestamp = ? AND ((subj_uri_id IN (SELECT id FROM uri WHERE value GLOB ? ORDER BY timestamp DESC [u'0', u'9223372036854775807', u'http://aa*'] --- get len(ids)=551 using pure sql in 0.179763s ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 641198] Re: Prefix search is not using an index
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- Prefix search is not using an index https://bugs.launchpad.net/bugs/641198 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: In the prefix search we are using a 'GLOB' sql-statement, this statement is not using the index and thus slow, some timings from me running the attached script: * Database with ~50k events * ~1% of these event have a subject with uri beginning with 'http://aa' * searching for all uris beginning with 'http://aa' using the 'GLOB' expression (what we have now in lp:zeitgeist) takes ~0.03 sec * changing this query to some (x = S AND x T) statement reduces this time to ~0.002 sec This is easy to implement (and test) for ascii chars, but as we support unicode esp. testing seems to be a bit tricky ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 634740] Re: explicitly define on a per testcase basis which extension needs to be loaded
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- explicitly define on a per testcase basis which extension needs to be loaded https://bugs.launchpad.net/bugs/634740 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: Right now we load all available extensions in our test runs. So if someone as a random (experimental) extension in ~/.local/share/zeitgeist/extensions which fails all our remote and engine testcases fail too. As we have no influence on the installed extensions on the users system we should explicitly configure them on each test run. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 598666] Re: Error when trying to fetch items
** Changed in: zeitgeist Milestone: 0.6 = None -- Error when trying to fetch items https://bugs.launchpad.net/bugs/598666 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Incomplete Bug description: When i try to fetch all items in one query i get Error from Zeitgeist engine: org.freedesktop.DBus.Python.KeyError: Traceback (most recent call last): File /usr/lib/pymodules/python2.6/dbus/service.py, line 702, in _message_cb retval = candidate_method(self, *args, **keywords) File /usr/local/share/zeitgeist/_zeitgeist/engine/remote.py, line 254, in FindEvents event_templates, storage_state, num_events, result_type, sender)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 378, in find_events return self._find_events(1, *args) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 366, in _find_events return self.get_events(rows=result, sender=sender) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 188, in get_events events[event.id].append_subject(self._get_subject_from_row(row)) File /usr/local/share/zeitgeist/_zeitgeist/engine/main.py, line 160, in _get_subject_from_row getattr(self, _ + field).value(row[subj_ + field])) File /usr/local/share/zeitgeist/_zeitgeist/engine/sql.py, line 422, in value return self._inv_dict[id] KeyError: 138 This looks serious ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 660423] Re: add `make run` targets to rootlevel Makefile
** Changed in: zeitgeist Status: Fix Committed = Fix Released -- add `make run` targets to rootlevel Makefile https://bugs.launchpad.net/bugs/660423 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Released Bug description: While working on some changes in zeitgeist-daemon.py I realized that testing changes to the actual zeitgeist-daemon is hard, I always have to remember to run make ./zeitgeist-daemon The `make run` target should do the right thing and simply wrap the command above. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Bug 660415] Re: zeitgeist-daemon.py has a bad code structure
** Changed in: zeitgeist Status: In Progress = Fix Committed -- zeitgeist-daemon.py has a bad code structure https://bugs.launchpad.net/bugs/660415 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: Fix Committed Bug description: The code of zeitgeist-daemon.py is very hard to read, let's restructure the code! (This is more like a reminder for myself to work on it, but I would like to fix it after bug #655164 has been fixed) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~seif/zeitgeist/fix-643303 into lp:zeitgeist
okidoki, my take on this: I would like to have some kind of tests for the upgrade pathes, but please let's not do it like Seif tried in rev 1625. Therefor I suggest the following, let's merge this branch without rev 1625 into lp:zeitgeist, and then put some work into writing a testing framework for db schemes and upgrades. -- https://code.launchpad.net/~seif/zeitgeist/fix-643303/+merge/39201 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~seif/zeitgeist/fix-643303 into lp:zeitgeist. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-660415-improve-zeitgeist-daemon into lp:zeitgeist
Markus Korn has proposed merging
lp:~thekorn/zeitgeist/fix-660415-improve-zeitgeist-daemon into lp:zeitgeist.
Requested reviews:
Zeitgeist Framework Team (zeitgeist)
Related bugs:
#660415 zeitgeist-daemon.py has a bad code structure
https://bugs.launchpad.net/bugs/660415
zeitgeist-daemon.py has now a much more readable code structure (LP: #660415)
While I was on it I also fixed the `--log-level` option, by removing
logging.basicConfig() from all modules in _zeitgeist/ and zeitgeist/
--
https://code.launchpad.net/~thekorn/zeitgeist/fix-660415-improve-zeitgeist-daemon/+merge/38914
Your team Zeitgeist Framework Team is requested to review the proposed merge of
lp:~thekorn/zeitgeist/fix-660415-improve-zeitgeist-daemon into lp:zeitgeist.
=== modified file '_zeitgeist/engine/__init__.py'
--- _zeitgeist/engine/__init__.py 2010-09-19 14:15:21 +
+++ _zeitgeist/engine/__init__.py 2010-10-20 07:31:07 +
@@ -31,7 +31,6 @@
constants
]
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.engine)
_engine = None
=== modified file '_zeitgeist/engine/extension.py'
--- _zeitgeist/engine/extension.py 2010-09-18 14:30:45 +
+++ _zeitgeist/engine/extension.py 2010-10-20 07:31:07 +
@@ -22,7 +22,6 @@
import logging
import weakref # avoid circular references as they confuse garbage collection
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.extension)
import zeitgeist
=== modified file '_zeitgeist/engine/extensions/blacklist.py'
--- _zeitgeist/engine/extensions/blacklist.py 2010-07-31 13:13:22 +
+++ _zeitgeist/engine/extensions/blacklist.py 2010-10-20 07:31:07 +
@@ -28,7 +28,6 @@
from _zeitgeist.engine.extension import Extension
from _zeitgeist.engine import constants
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.blacklist)
CONFIG_FILE = os.path.join(constants.DATA_PATH, blacklist.pickle)
=== modified file '_zeitgeist/engine/extensions/datasource_registry.py'
--- _zeitgeist/engine/extensions/datasource_registry.py 2010-08-28 15:08:49 +
+++ _zeitgeist/engine/extensions/datasource_registry.py 2010-10-20 07:31:07 +
@@ -30,7 +30,6 @@
from _zeitgeist.engine.extension import Extension
from _zeitgeist.engine import constants
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.datasource_registry)
DATA_FILE = os.path.join(constants.DATA_PATH, datasources.pickle)
=== modified file '_zeitgeist/engine/main.py'
--- _zeitgeist/engine/main.py 2010-10-19 10:25:33 +
+++ _zeitgeist/engine/main.py 2010-10-20 07:31:07 +
@@ -35,7 +35,6 @@
from _zeitgeist.engine.sql import get_default_cursor, unset_cursor, \
TableLookup, WhereClause
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.engine)
class NegationNotSupported(ValueError):
=== modified file '_zeitgeist/engine/notify.py'
--- _zeitgeist/engine/notify.py 2010-09-25 13:19:51 +
+++ _zeitgeist/engine/notify.py 2010-10-20 07:31:07 +
@@ -22,7 +22,6 @@
from zeitgeist.datamodel import TimeRange
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.notify)
class _MonitorProxy (dbus.Interface):
=== modified file '_zeitgeist/engine/sql.py'
--- _zeitgeist/engine/sql.py 2010-09-21 16:15:14 +
+++ _zeitgeist/engine/sql.py 2010-10-20 07:31:07 +
@@ -27,7 +27,6 @@
from _zeitgeist.engine import constants
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.sql)
TABLE_MAP = {
=== modified file '_zeitgeist/engine/upgrades/core_0_1.py'
--- _zeitgeist/engine/upgrades/core_0_1.py 2010-06-09 06:29:47 +
+++ _zeitgeist/engine/upgrades/core_0_1.py 2010-10-20 07:31:07 +
@@ -3,7 +3,6 @@
import logging
import sqlite3
-logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger(zeitgeist.sql)
INTERPRETATION_RENAMES = \
=== modified file 'zeitgeist-daemon.py'
--- zeitgeist-daemon.py 2010-10-14 09:42:08 +
+++ zeitgeist-daemon.py 2010-10-20 07:31:07 +
@@ -4,6 +4,7 @@
# Zeitgeist
#
# Copyright © 2009 Siegfried-Angel Gevatter Pujals rai...@ubuntu.com
+# Copyright © 2010 Markus Korn thek...@gmx.de
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
@@ -26,7 +27,6 @@
import logging
import optparse
import signal
-from copy import copy
from subprocess import Popen, PIPE
# Make sure we can find the private _zeitgeist namespace
@@ -38,75 +38,68 @@
from _zeitgeist.engine import constants
sys.path.insert(0, constants.USER_EXTENSION_PATH)
-gettext.install(zeitgeist, _config.localedir, unicode=1)
-DATAHUB = zeitgeist-datahub
-
-def check_loglevel(option, opt, value):
- value = value.upper()
- if value in Options.log_levels:
- return value
- raise optparse.OptionValueError(
- option %s: invalid value: %s % (opt, value))
-
+gettext.install(zeitgeist, _config.localedir, unicode=True)
+
+class
[Zeitgeist] [Bug 660415] Re: zeitgeist-daemon.py has a bad code structure
** Branch linked: lp:~thekorn/zeitgeist/fix-660415-improve-zeitgeist- daemon ** Changed in: zeitgeist Status: Confirmed = In Progress -- zeitgeist-daemon.py has a bad code structure https://bugs.launchpad.net/bugs/660415 You received this bug notification because you are a member of Zeitgeist Framework Team, which is subscribed to Zeitgeist Framework. Status in Zeitgeist Framework: In Progress Bug description: The code of zeitgeist-daemon.py is very hard to read, let's restructure the code! (This is more like a reminder for myself to work on it, but I would like to fix it after bug #655164 has been fixed) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
Re: [Zeitgeist] [Merge] lp:~zeitgeist/zeitgeist/remove-datahub into lp:zeitgeist
Review: Approve All tests are running fine, zeitgeist-daemon is looking good, and it seems to me that you managed to remove everything related to the old datahub, Good work. -- https://code.launchpad.net/~zeitgeist/zeitgeist/remove-datahub/+merge/38339 Your team Zeitgeist Framework Team is subscribed to branch lp:~zeitgeist/zeitgeist/remove-datahub. ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
[Zeitgeist] [Merge] lp:~thekorn/zeitgeist/fix-660423-make-run into lp:zeitgeist
Markus Korn has proposed merging lp:~thekorn/zeitgeist/fix-660423-make-run into lp:zeitgeist. Requested reviews: Zeitgeist Framework Team (zeitgeist) Related bugs: #660423 add `make run` targets to rootlevel Makefile https://bugs.launchpad.net/bugs/660423 Added `make run` target to the rootlevel Makefile (LP: #660423) This command makes sure the ontology is updated, zeitgeist-daemon.py and zeitgeist-daemon are in sync and then runs zeitgeist-daemon. -- https://code.launchpad.net/~thekorn/zeitgeist/fix-660423-make-run/+merge/38542 Your team Zeitgeist Framework Team is requested to review the proposed merge of lp:~thekorn/zeitgeist/fix-660423-make-run into lp:zeitgeist. === modified file 'Makefile.am' --- Makefile.am 2010-10-13 15:33:17 + +++ Makefile.am 2010-10-15 14:43:23 + @@ -69,3 +69,14 @@ else \ echo Cannot run the testsuite, tests are not shipped; \ fi + +# do the right thing to start a daemon +# wrap commandline options +# * to run the daemon without any datahub, set ZG_NODATAHUB, +# * to use a certain log level, set ZG_LOGLEVEL=log-level +# example: +# make run ZG_LOGLEVEL=INFO ZG_NODATAHUB=1 +ZG_DAEMON_ARGS := $(if $(ZG_LOGLEVEL), --log-level $(ZG_LOGLEVEL),) +ZG_DAEMON_ARGS += $(if $(ZG_NODATAHUB), --no-datahub,) +run: all + ./zeitgeist-daemon $(ZG_DAEMON_ARGS) ___ Mailing list: https://launchpad.net/~zeitgeist Post to : zeitgeist@lists.launchpad.net Unsubscribe : https://launchpad.net/~zeitgeist More help : https://help.launchpad.net/ListHelp
