seth vidal wrote:
On Wed, 2007-08-22 at 16:08 +0200, Tim Lauridsen wrote:
Florian Festi wrote:
Tim Lauridsen wrote:
I did some basic testing of the current yum git HEAD vs. yum 3.2.2.
Yes, current git head is a bit slower than 3.2.2. There is a simple
reason for that: Linear searches on disk are even slower than linear
searches in memory.
In other words: We are still missing some indexes in the sqlite
database to really gain speed from the changes. For performance runs
you can temporary remove patch
3c2621ba8f8070f24ad3e979f6bd1699f6f6b394 or readd
42283902f929ac131cda7b3497ae047b497e02bc.
There are two possible permanent solutions:
Readd the patch mentions above and extend it that all indexes are
created if they are not present yet. I posted timings for creating
these indexes some weeks ago (2. Aug "Sqlite DB Indexes").
As alternative or in addition we could patch yum-metadata-parser to
just add the needed indexes.
I can provide patches for both as soon as there is a decision with
way(s) we want to go.
I added this
http://devel.linux.duke.edu/gitweb/?p=yum.git;a=commitdiff;h=42283902f929ac131cda7b3497ae047b497e02bc
It speed up things a lot, but makes yum load the metadata every time,
because the index creation changes the checksum i think.
I we want to create the indexes locally, how do we know when to download
the sqlite tarball from the repo ?
I don't think we want to make them locally. For verification purposes
it's a dicey proposition.
Think about this:
if we see that the sqlite file is changed - how do we know it was
changed due to index generation and not due to someone mucking with our
metadata?
While I belief it is possible to create the indexes in a sane way locally
this would require some more thought. As we are only a few days before F8
feature freeze modifying the metadata-parser is the best solution for now.
We can easily revisit that topic later on - or stay with that.
Patch is attached and compiles. Can someone please check if it really works
as I am heading home now?
Thanks
Florian
>From 86458223ee104756922fa5969b48aa8277954851 Mon Sep 17 00:00:00 2001
From: Florian Festi <[EMAIL PROTECTED]>
Date: Wed, 22 Aug 2007 19:57:31 +0200
Subject: [PATCH] Add more indexes: filenames, dirnames, (PRCO)name
---
db.c | 40 ++++++++++++++++++++++++++++++----------
1 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/db.c b/db.c
index 019faac..b31a79b 100644
--- a/db.c
+++ b/db.c
@@ -380,6 +380,15 @@ yum_db_create_primary_tables (sqlite3 *db, GError **err)
return;
}
+ sql = "CREATE INDEX filenames ON files (name)";
+ rc = sqlite3_exec (db, sql, NULL, NULL, NULL);
+ if (rc != SQLITE_OK) {
+ g_set_error (err, YUM_DB_ERROR, YUM_DB_ERROR,
+ "Can not create filenames index: %s",
+ sqlite3_errmsg (db));
+ return;
+ }
+
sql =
"CREATE TABLE %s ("
" name TEXT,"
@@ -392,7 +401,8 @@ yum_db_create_primary_tables (sqlite3 *db, GError **err)
const char *deps[] = { "requires", "provides", "conflicts", "obsoletes", NULL };
int i;
- const char *indexsql = "CREATE INDEX IF NOT EXISTS pkg%s on %s (pkgKey)";
+ const char *pkgindexsql = "CREATE INDEX pkg%s on %s (pkgKey)";
+ const char *nameindexsql = "CREATE INDEX %sname ON %s (name)";
for (i = 0; deps[i]; i++) {
const char *prereq;
@@ -414,7 +424,7 @@ yum_db_create_primary_tables (sqlite3 *db, GError **err)
return;
}
- query = g_strdup_printf(indexsql, deps[i], deps[i]);
+ query = g_strdup_printf(pkgindexsql, deps[i], deps[i]);
rc = sqlite3_exec (db, query, NULL, NULL, NULL);
g_free (query);
@@ -424,15 +434,16 @@ yum_db_create_primary_tables (sqlite3 *db, GError **err)
deps[i], sqlite3_errmsg (db));
return;
}
- }
- sql = "CREATE INDEX providesname ON provides (name)";
- rc = sqlite3_exec (db, sql, NULL, NULL, NULL);
- if (rc != SQLITE_OK) {
- g_set_error (err, YUM_DB_ERROR, YUM_DB_ERROR,
- "Can not create providesname index: %s",
- sqlite3_errmsg (db));
- return;
+ query = g_strdup_printf(nameindexsql, deps[i], deps[i]);
+ rc = sqlite3_exec (db, sql, NULL, NULL, NULL);
+ if (rc != SQLITE_OK) {
+ g_set_error (err, YUM_DB_ERROR, YUM_DB_ERROR,
+ "Can not create %sname index: %s",
+ deps[i], sqlite3_errmsg (db));
+ return;
+ }
+
}
sql =
@@ -682,6 +693,15 @@ yum_db_create_filelist_tables (sqlite3 *db, GError **err)
return;
}
+ sql = "CREATE INDEX dirnames ON filelist (dirname)";
+ rc = sqlite3_exec (db, sql, NULL, NULL, NULL);
+ if (rc != SQLITE_OK) {
+ g_set_error (err, YUM_DB_ERROR, YUM_DB_ERROR,
+ "Can not create dirnames index: %s",
+ sqlite3_errmsg (db));
+ return;
+ }
+
sql =
"CREATE TRIGGER remove_filelist AFTER DELETE ON packages"
" BEGIN"
--
1.5.2.2
_______________________________________________
Yum-devel mailing list
[email protected]
https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
