The branch, 2.5 has been updated
via f9fbccbbd8e98c514648ff5094e6b4c7697a1909 (commit)
via bf08af9a64a9046e2eeee8eb3b362459df65af72 (commit)
via d8a3afa25b103d79f608489961f14d6fd2425e5a (commit)
via a0996731866c53eeda95a48f3eeff2c8d0dbafac (commit)
via 4907196db2b72c9955cc313feecd561bc245fe04 (commit)
via 089bea7c052ddd3b09a2fdb8ed8ca457c6c6dca8 (commit)
via d5dbff4883770a74fd1faf4223e6bc6f140075dd (commit)
via 73a4b383a28a7c658f163ad2b28cb849b7c1c4eb (commit)
via d3bd7178b471c0bdc0ba91c39e88f0886bb29c06 (commit)
via 59733dc79cd20e0acdfabb6d824e31ee05e0b262 (commit)
via f29e77f08d9ee8cef8348d4d7bd42839cdeae4d0 (commit)
via afb37c2f39cf8f62fe6cccd6c93827d40fb930c8 (commit)
via 0fcf454e9459b8be637e633c25f029b64c989e2f (commit)
via c7925a731cd35fa3a9cb4597214b5db1d7c3acee (commit)
via 1f74d3555fcfd321d68dfdb902061d07f5643037 (commit)
via 509d75cdd42a5e6451f343232690c53da9c4ae20 (commit)
via 037247395e26d8a4d6e3ca721d0175f8bda9c150 (commit)
via d83842d614711154a5f3389b444df843dde1462c (commit)
via b2ce5b02d45e1bd3f26f3cefbbc1113f0d5cad66 (commit)
via 1ba830f049273c9916ff03fe51f9831623a783f0 (commit)
via 551d06837a4a004b5ffd7d3cce1e29c8126237cb (commit)
via c346d1e1faaf0d8ed853d9cc4b9a1d877fe45610 (commit)
via 061d9a00ea5f4a7202a89f12cbeb74b8c6f8db3f (commit)
via 92b4bd69c8b65441c64906ff293970c3663a5211 (commit)
via 8aa6055c7e63b96dcee2eeafef8343816a7f2371 (commit)
via ff9aeb06bbc68b6bc7110c9bcbe1b7652457c660 (commit)
via 682319e09c01d7b3e563235b9f17bb06c38f44e1 (commit)
via feff0e8ccff8b84115522c6cbd324cf6a5879655 (commit)
via cefaf11204b25d193bde1ee63323d67c59048a73 (commit)
via 5b9e418c90ebba044901f556d6242fd3f30c108f (commit)
via d925b5f9c614dcf45178e9db61778d975ced0ee3 (commit)
via fa6a40667fba56eb53101a43353af144eee938e2 (commit)
via f0b64e4ff84d43702b5348291c1d53e3d4757eaa (commit)
via f8630b7ef0df2f3342730af4d52460ef97011748 (commit)
via 8803e62ac7ed3f244993af1ff7171c94cb420a4b (commit)
from f33b1f208a668d881144ead0eb3cad8e918748ed (commit)
http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=2.5
- Log -----------------------------------------------------------------
commit f9fbccbbd8e98c514648ff5094e6b4c7697a1909
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 21 23:43:17 2014 +0100
vacuum: fix delete list counts in delete_marshall_traverse_first
when bumping skipped, decrement left, so the sum is correct
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
Autobuild-User(master): Amitay Isaacs <ami...@samba.org>
Autobuild-Date(master): Thu Mar 6 03:32:33 CET 2014 on sn-devel-104
(Imported from commit d8e110ed7dacda18860cce0c86e4e44f0b83dd42)
commit bf08af9a64a9046e2eeee8eb3b362459df65af72
Author: Michael Adam <ob...@samba.org>
Date: Thu Feb 20 00:58:17 2014 +0100
vacuum: fix possible cause for delelete_list processing counts left records
> 0
We need to have left records == 0 at the end of the delete list processing.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 5b81848e50b43b7ab7889f5217e05ca42e452c8f)
commit d8a3afa25b103d79f608489961f14d6fd2425e5a
Author: Michael Adam <ob...@samba.org>
Date: Thu Feb 20 00:32:08 2014 +0100
vacuum: systematize counters into various structs
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 551e9d791c146473b45f8a9fb1574e0ad7cca6b2)
commit a0996731866c53eeda95a48f3eeff2c8d0dbafac
Author: Michael Adam <ob...@samba.org>
Date: Thu Feb 20 00:29:47 2014 +0100
vacuum: remove unused counter vdata->total
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit ec3f33c314207f93529d7c9a2bfb82bf05e3a08b)
commit 4907196db2b72c9955cc313feecd561bc245fe04
Author: Michael Adam <ob...@samba.org>
Date: Sun Feb 16 01:08:18 2014 +0100
vacuum: make ctdb_process_delete_list() void
The overall return code was not really used anyways.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 49eb503c5d3133e1476a15f9d11ce4269407e6c6)
commit 089bea7c052ddd3b09a2fdb8ed8ca457c6c6dca8
Author: Michael Adam <ob...@samba.org>
Date: Sun Feb 16 00:37:43 2014 +0100
vacuum: make ctdb_process_vacuum_fetch_lists() void.
This constantly returns 0 anyways.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 776d4e88f2a6068016dedd37003cdde10f8090a5)
commit d5dbff4883770a74fd1faf4223e6bc6f140075dd
Author: Michael Adam <ob...@samba.org>
Date: Sun Feb 16 00:35:34 2014 +0100
vacuum: make ctdb_vacuum_traverse_db() void.
Failure in traversal of the DB should not
prevent further processing.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 19948702992c94553e1a611540ad398de9f9d8b9)
commit 73a4b383a28a7c658f163ad2b28cb849b7c1c4eb
Author: Michael Adam <ob...@samba.org>
Date: Sun Feb 16 00:26:00 2014 +0100
vacuum: don't stop in process_vacuum_fetch_lists when sending to one node
fails.
We should try to continue vacuuming as much as possible.
Failure to send records to one lmaster doesn't mean the
others will fail too.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 7983946680ac0de8f82dfee6f0f849a11653d042)
commit d3bd7178b471c0bdc0ba91c39e88f0886bb29c06
Author: Michael Adam <ob...@samba.org>
Date: Sat Feb 15 18:06:09 2014 +0100
vacuum: catch and log errors to traverse the delete list in
ctdb_process_delete_list()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit f3483de240987f05cc20f747ac467c8ed81bb03e)
commit 59733dc79cd20e0acdfabb6d824e31ee05e0b262
Author: Michael Adam <ob...@samba.org>
Date: Sat Feb 15 17:59:22 2014 +0100
vacuum: catch and log error of traverse in ctdb_process_delete_queue()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 72ea9759930bd29b59c0f831d2cb2f4f1e2e643d)
commit f29e77f08d9ee8cef8348d4d7bd42839cdeae4d0
Author: Michael Adam <ob...@samba.org>
Date: Sat Feb 15 13:03:51 2014 +0100
vacuum: use tdb_parse_record instead of tdb_fetch in
delete_marshall_traverse_first()
Spare malloc and free.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 83fa09e78c6ca8e08cb2659f013a05b4b340f0aa)
commit afb37c2f39cf8f62fe6cccd6c93827d40fb930c8
Author: Michael Adam <ob...@samba.org>
Date: Sat Feb 15 13:01:33 2014 +0100
vacuum: simplify delete_marshall_traverse_first: use tdb_null
we know anyways the record to store is empty at this point.
So skip pointer calculations.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 974aa73160d50b7cf63b4a5e6dd7a7b1408ece42)
commit 0fcf454e9459b8be637e633c25f029b64c989e2f
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 22:05:21 2014 +0100
vacuum: remove VacuumLimit criterion for triggering a repack
With the new vacuuming, we consider it an error if there are
records left for deletion after processing the various lists.
All records that can be deleted should have been deleted by
tdb_delete calls.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 16837bc309aa9a86fc21d7f59a8fce0b947428a3)
commit c7925a731cd35fa3a9cb4597214b5db1d7c3acee
Author: Michael Adam <ob...@samba.org>
Date: Wed Feb 12 17:41:28 2014 +0100
vacuum: treat value 0 of tunable VacuumLimit as turning off repacking
I.e. no number of records found to delete will trigger the
repacking.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 6fdd25008f384408dfc103b90ab40b8e64ce18b0)
commit 1f74d3555fcfd321d68dfdb902061d07f5643037
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 22:02:41 2014 +0100
vacuum: add consistency check for counts at end of process_delete_list()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 1b2fc1f096f80db5974eab021f12f0ad9af24882)
commit 509d75cdd42a5e6451f343232690c53da9c4ae20
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 22:01:38 2014 +0100
vacuum: log error if records are left for deletion after
ctdb_process_delete_list()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit cf407d208afdc70b31ce5013591c869b36f588f1)
commit 037247395e26d8a4d6e3ca721d0175f8bda9c150
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 21:50:59 2014 +0100
vacuum: use tdb_parse_record instead of tdb_fetch in
delete_record_traverse()
Spare malloc and free.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 23be632449524e74d73fcb6fd3875a6d5a428d89)
commit d83842d614711154a5f3389b444df843dde1462c
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:48:02 2014 +0100
vacuum: update comment for ctdb_process_delete_queue
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit be2f1a0c790d08571dd757fb7b2941d367175008)
commit b2ce5b02d45e1bd3f26f3cefbbc1113f0d5cad66
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:47:25 2014 +0100
vacuum: rename ctdb_vacuum_db_fast --> ctdb_process_delete_queue
This describes more precisely what this does.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit cd877b6a21a5080b3d9ae8ee7ac8cf27c4fd9512)
commit 1ba830f049273c9916ff03fe51f9831623a783f0
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:46:49 2014 +0100
vacuum: update comment for ctdb_vacuum_traverse_db
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit c4478bd40daee22bbe5696bb17d0a1bff164c7f7)
commit 551d06837a4a004b5ffd7d3cce1e29c8126237cb
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:42:37 2014 +0100
vacuum: rename ctdb_vacuum_db_full --> ctdb_vacuum_traverse_db
This describes more precisely what it actually is, nowadays.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 0309d5eda4f28c7e99e93e0da6c02757ea0cae8f)
commit c346d1e1faaf0d8ed853d9cc4b9a1d877fe45610
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:38:31 2014 +0100
vacuum: change full db traverse vacuuming to fill delete queue
This lets the "fast vacuum" delete queue traverse do the actual work.
On the positive side, we note that this lets the "full vacuuming"
treat the records that have never been migrated with data correctly.
These had previously been added to the delete list for complicated
cross-node deletion instead of directly deleting them.
On the other hand side, there might be a slight overhead
since the records are read again in the delete queu traverse,
but this is OK because this change is in preparation of
untangling the db traverse altogether from the vacuum run,
making it independent.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit ba49deb2344c0a9a8f76c9fd0136bdeadad6af89)
commit 061d9a00ea5f4a7202a89f12cbeb74b8c6f8db3f
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:27:14 2014 +0100
vacuum: run the fast vacuum after the db traverse
This in preparation of modifying the db traverse to
fill the delete_queue that is processed by the fast
vacuum run, instead of filling the same lists as the
fast vacuum run for further processing.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit d0b7b3882511769b1bfc1d0d4fdc0dba288e6ccd)
commit 92b4bd69c8b65441c64906ff293970c3663a5211
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:08:20 2014 +0100
vacuum: rename private->private_data in repack_traverse
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 04c2115b606f2346fc7315a503b3dae0189e0737)
commit 8aa6055c7e63b96dcee2eeafef8343816a7f2371
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:07:55 2014 +0100
vacuum: rename private->private_data in vacuum_traverse
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 951efa1097a113910c06ce78d1c9fb70e3f4d75e)
commit ff9aeb06bbc68b6bc7110c9bcbe1b7652457c660
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 18:03:02 2014 +0100
vacuum: extract check for full vacuum run out of ctdb_vacuum_db_full()
This is more consistent.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 01f359cafccb5ae3bea312d628dad92746520527)
commit 682319e09c01d7b3e563235b9f17bb06c38f44e1
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 17:58:01 2014 +0100
vacuum: add consistency check for counts to ctdb_vacuum_db_fast()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit c88fd19714b98769887dbff59d8c1d077cf351d5)
commit feff0e8ccff8b84115522c6cbd324cf6a5879655
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 15:28:22 2014 +0100
vacuum: use tdb_parse_record instead of tdb_fetch in delete_queue_traverse()
this spares malloc and free
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 5d5907c7cf09567e73092578917624c8789c7471)
commit cefaf11204b25d193bde1ee63323d67c59048a73
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 15:35:01 2014 +0100
vacuum: simplify delete_record_traverse() - free treats NULL
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit fe68b3c4942a4660c9b35c6316856644c32f5631)
commit 5b9e418c90ebba044901f556d6242fd3f30c108f
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 15:34:23 2014 +0100
vacuum: simplify delete_queue_traverse() - free treats NULL pointers.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 593bddf2e82fcb9666449c40625b972ff9c7961c)
commit d925b5f9c614dcf45178e9db61778d975ced0ee3
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 15:30:08 2014 +0100
vacuum: reduce indentation in delete_queue_traverse
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 24bec3d31b16c4c83b5ed76ecffccbfda53858fd)
commit fa6a40667fba56eb53101a43353af144eee938e2
Author: Michael Adam <ob...@samba.org>
Date: Wed Feb 12 17:40:31 2014 +0100
vacuum: treat value 0 of tunable RepackLimit as turned off.
I.e. when RepackLimit is set to 0, no size of the freelist
should trigger a repack in vacuuming.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 48f2d1158820bfb063ba0a0bbfb6f496a8e7522d)
commit f0b64e4ff84d43702b5348291c1d53e3d4757eaa
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 01:55:39 2014 +0100
vacuum: fix treatment of remaining records and statistics in
delete_record_traverse()
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit af5568b26761dadbb652d92f8c8ced477b38c7cc)
commit f8630b7ef0df2f3342730af4d52460ef97011748
Author: Michael Adam <ob...@samba.org>
Date: Wed Feb 12 17:38:56 2014 +0100
vacuum: cast freelist_size in comparison.
At this point, it is >= 0 anyways.
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit b4e0b01a8c8415bec9c7dbbe4494813917dddfe5)
commit 8803e62ac7ed3f244993af1ff7171c94cb420a4b
Author: Michael Adam <ob...@samba.org>
Date: Fri Feb 14 00:53:23 2014 +0100
vacuum: improve output of delete list statistics
Signed-off-by: Michael Adam <ob...@samba.org>
Reviewed-by: Amitay Isaacs <ami...@gmail.com>
(Imported from commit 6a46a255307a070c887525ee1d79810ba12442bb)
-----------------------------------------------------------------------
Summary of changes:
server/ctdb_vacuum.c | 579 +++++++++++++++++++++++++-------------------------
1 files changed, 293 insertions(+), 286 deletions(-)
Changeset truncated at 500 lines:
diff --git a/server/ctdb_vacuum.c b/server/ctdb_vacuum.c
index 22603b8..574ad87 100644
--- a/server/ctdb_vacuum.c
+++ b/server/ctdb_vacuum.c
@@ -53,36 +53,43 @@ struct ctdb_vacuum_handle {
/* a list of records to possibly delete */
struct vacuum_data {
- uint32_t vacuum_limit;
uint32_t repack_limit;
struct ctdb_context *ctdb;
struct ctdb_db_context *ctdb_db;
struct tdb_context *dest_db;
trbt_tree_t *delete_list;
- uint32_t delete_count;
struct ctdb_marshall_buffer **vacuum_fetch_list;
struct timeval start;
bool traverse_error;
bool vacuum;
- uint32_t total;
- uint32_t vacuumed;
- uint32_t copied;
- uint32_t fast_added_to_vacuum_fetch_list;
- uint32_t fast_added_to_delete_list;
- uint32_t fast_deleted;
- uint32_t fast_skipped;
- uint32_t fast_error;
- uint32_t fast_total;
- uint32_t full_added_to_vacuum_fetch_list;
- uint32_t full_added_to_delete_list;
- uint32_t full_skipped;
- uint32_t full_error;
- uint32_t full_total;
- uint32_t delete_left;
- uint32_t delete_remote_error;
- uint32_t delete_local_error;
- uint32_t delete_deleted;
- uint32_t delete_skipped;
+ struct {
+ struct {
+ uint32_t added_to_vacuum_fetch_list;
+ uint32_t added_to_delete_list;
+ uint32_t deleted;
+ uint32_t skipped;
+ uint32_t error;
+ uint32_t total;
+ } delete_queue;
+ struct {
+ uint32_t scheduled;
+ uint32_t skipped;
+ uint32_t error;
+ uint32_t total;
+ } db_traverse;
+ struct {
+ uint32_t total;
+ uint32_t remote_error;
+ uint32_t local_error;
+ uint32_t deleted;
+ uint32_t skipped;
+ uint32_t left;
+ } delete_list;
+ struct {
+ uint32_t vacuumed;
+ uint32_t copied;
+ } repack;
+ } count;
};
/* this structure contains the information for one record to be deleted */
@@ -99,6 +106,10 @@ struct delete_records_list {
struct vacuum_data *vdata;
};
+static int insert_record_into_delete_queue(struct ctdb_db_context *ctdb_db,
+ const struct ctdb_ltdb_header *hdr,
+ TDB_DATA key);
+
/**
* Store key and header in a tree, indexed by the key hash.
*/
@@ -158,7 +169,7 @@ static int add_record_to_delete_list(struct vacuum_data
*vdata, TDB_DATA key,
return -1;
}
- vdata->delete_count++;
+ vdata->count.delete_list.total++;
return 0;
}
@@ -200,8 +211,6 @@ static int add_record_to_vacuum_fetch_list(struct
vacuum_data *vdata,
memcpy(old_size+(uint8_t *)vfl, rec, rec->length);
talloc_free(rec);
- vdata->total++;
-
return 0;
}
@@ -209,23 +218,39 @@ static int add_record_to_vacuum_fetch_list(struct
vacuum_data *vdata,
static void ctdb_vacuum_event(struct event_context *ev, struct timed_event *te,
struct timeval t, void *private_data);
+static int vacuum_record_parser(TDB_DATA key, TDB_DATA data, void
*private_data)
+{
+ struct ctdb_ltdb_header *header =
+ (struct ctdb_ltdb_header *)private_data;
+
+ if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
+ return -1;
+ }
+
+ *header = *(struct ctdb_ltdb_header *)data.dptr;
+
+ return 0;
+}
/*
* traverse function for gathering the records that can be deleted
*/
-static int vacuum_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA
data, void *private)
+static int vacuum_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA
data,
+ void *private_data)
{
- struct vacuum_data *vdata = talloc_get_type(private, struct
vacuum_data);
+ struct vacuum_data *vdata = talloc_get_type(private_data,
+ struct vacuum_data);
struct ctdb_context *ctdb = vdata->ctdb;
+ struct ctdb_db_context *ctdb_db = vdata->ctdb_db;
uint32_t lmaster;
struct ctdb_ltdb_header *hdr;
int res = 0;
- vdata->full_total++;
+ vdata->count.db_traverse.total++;
lmaster = ctdb_lmaster(ctdb, &key);
if (lmaster >= ctdb->num_nodes) {
- vdata->full_error++;
+ vdata->count.db_traverse.error++;
DEBUG(DEBUG_CRIT, (__location__
" lmaster[%u] >= ctdb->num_nodes[%u] for key"
" with hash[%u]!\n",
@@ -237,42 +262,29 @@ static int vacuum_traverse(struct tdb_context *tdb,
TDB_DATA key, TDB_DATA data,
if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
/* it is not a deleted record */
- vdata->full_skipped++;
+ vdata->count.db_traverse.skipped++;
return 0;
}
hdr = (struct ctdb_ltdb_header *)data.dptr;
if (hdr->dmaster != ctdb->pnn) {
- vdata->full_skipped++;
+ vdata->count.db_traverse.skipped++;
return 0;
}
- if (lmaster == ctdb->pnn) {
- /*
- * We are both lmaster and dmaster, and the record is empty.
- * So we should be able to delete it.
- */
- res = add_record_to_delete_list(vdata, key, hdr);
- if (res != 0) {
- vdata->full_error++;
- } else {
- vdata->full_added_to_delete_list++;
- }
+ /*
+ * Add the record to this process's delete_queue for processing
+ * in the subsequent traverse in the fast vacuum run.
+ */
+ res = insert_record_into_delete_queue(ctdb_db, hdr, key);
+ if (res != 0) {
+ vdata->count.db_traverse.error++;
} else {
- /*
- * We are not lmaster.
- * Add the record to the blob ready to send to the nodes.
- */
- res = add_record_to_vacuum_fetch_list(vdata, key);
- if (res != 0) {
- vdata->full_error++;
- } else {
- vdata->full_added_to_vacuum_fetch_list++;
- }
+ vdata->count.db_traverse.scheduled++;
}
- return res;
+ return 0;
}
/*
@@ -319,8 +331,7 @@ static int delete_marshall_traverse_first(void *param, void
*data)
struct delete_records_list *recs = talloc_get_type(param, struct
delete_records_list);
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data, ctdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
uint32_t hash = ctdb_hash(&(dd->key));
int res;
@@ -331,7 +342,8 @@ static int delete_marshall_traverse_first(void *param, void
*data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- recs->vdata->delete_skipped++;
+ recs->vdata->count.delete_list.skipped++;
+ recs->vdata->count.delete_list.left--;
talloc_free(dd);
return 0;
}
@@ -341,26 +353,13 @@ static int delete_marshall_traverse_first(void *param,
void *data)
* changed and that we are still its lmaster and dmaster.
*/
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] does not exist or is not"
- " a ctdb-record. skipping.\n",
- hash, ctdb_db->db_name));
- goto skip;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] has been recycled. "
- "skipping.\n",
- hash, ctdb_db->db_name));
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
goto skip;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->flags & CTDB_REC_RO_FLAGS) {
+ if (header.flags & CTDB_REC_RO_FLAGS) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has read-only flags. "
"skipping.\n",
@@ -368,7 +367,7 @@ static int delete_marshall_traverse_first(void *param, void
*data)
goto skip;
}
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has been migrated away. "
"skipping.\n",
@@ -376,7 +375,7 @@ static int delete_marshall_traverse_first(void *param, void
*data)
goto skip;
}
- if (header->rsn != dd->hdr.rsn) {
+ if (header.rsn != dd->hdr.rsn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] seems to have been "
"migrated away and back again (with empty "
@@ -404,10 +403,7 @@ static int delete_marshall_traverse_first(void *param,
void *data)
* on the record's dmaster.
*/
- ctdb_data.dptr = tdb_data.dptr + sizeof(struct ctdb_ltdb_header);
- ctdb_data.dsize = tdb_data.dsize - sizeof(struct ctdb_ltdb_header);
-
- res = ctdb_ltdb_store(ctdb_db, dd->key, header, ctdb_data);
+ res = ctdb_ltdb_store(ctdb_db, dd->key, &header, tdb_null);
if (res != 0) {
DEBUG(DEBUG_ERR, (__location__ ": Failed to store record with "
"key hash [0x%08x] on database db[%s].\n",
@@ -422,15 +418,12 @@ static int delete_marshall_traverse_first(void *param,
void *data)
skip:
tdb_chainunlock(ctdb_db->ltdb->tdb, dd->key);
- recs->vdata->delete_skipped++;
+ recs->vdata->count.delete_list.skipped++;
+ recs->vdata->count.delete_list.left--;
talloc_free(dd);
dd = NULL;
done:
- if (tdb_data.dptr != NULL) {
- free(tdb_data.dptr);
- }
-
if (dd == NULL) {
return 0;
}
@@ -466,12 +459,11 @@ static int delete_queue_traverse(void *param, void *data)
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb; /* or dd->ctdb ??? */
int res;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
uint32_t hash = ctdb_hash(&(dd->key));
- vdata->fast_total++;
+ vdata->count.delete_queue.total++;
res = tdb_chainlock(ctdb_db->ltdb->tdb, dd->key);
if (res != 0) {
@@ -479,29 +471,22 @@ static int delete_queue_traverse(void *param, void *data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
return 0;
}
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- /* Does not exist or not a ctdb record. Skip. */
- goto skipped;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- /* The record has been recycled (filled with data). Skip. */
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
goto skipped;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
/* The record has been migrated off the node. Skip. */
goto skipped;
}
- if (header->rsn != dd->hdr.rsn) {
+ if (header.rsn != dd->hdr.rsn) {
/*
* The record has been migrated off the node and back again.
* But not requeued for deletion. Skip it.
@@ -527,9 +512,9 @@ static int delete_queue_traverse(void *param, void *data)
DEBUG(DEBUG_ERR,
(__location__ " Error adding record to list "
"of records to send to lmaster.\n"));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
} else {
- vdata->fast_added_to_vacuum_fetch_list++;
+ vdata->count.delete_queue.added_to_vacuum_fetch_list++;
}
goto done;
}
@@ -542,9 +527,9 @@ static int delete_queue_traverse(void *param, void *data)
DEBUG(DEBUG_ERR,
(__location__ " Error adding record to list "
"of records for deletion on lmaster.\n"));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
} else {
- vdata->fast_added_to_delete_list++;
+ vdata->count.delete_queue.added_to_delete_list++;
}
} else {
res = tdb_delete(ctdb_db->ltdb->tdb, dd->key);
@@ -554,25 +539,23 @@ static int delete_queue_traverse(void *param, void *data)
(__location__ " Error deleting record with key "
"hash [0x%08x] from local data base db[%s].\n",
hash, ctdb_db->db_name));
- vdata->fast_error++;
- } else {
- DEBUG(DEBUG_DEBUG,
- (__location__ " Deleted record with key hash "
- "[0x%08x] from local data base db[%s].\n",
- hash, ctdb_db->db_name));
- vdata->fast_deleted++;
+ vdata->count.delete_queue.error++;
+ goto done;
}
+
+ DEBUG(DEBUG_DEBUG,
+ (__location__ " Deleted record with key hash "
+ "[0x%08x] from local data base db[%s].\n",
+ hash, ctdb_db->db_name));
+ vdata->count.delete_queue.deleted++;
}
goto done;
skipped:
- vdata->fast_skipped++;
+ vdata->count.delete_queue.skipped++;
done:
- if (tdb_data.dptr != NULL) {
- free(tdb_data.dptr);
- }
tdb_chainunlock(ctdb_db->ltdb->tdb, dd->key);
return 0;
@@ -591,10 +574,8 @@ static int delete_record_traverse(void *param, void *data)
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb;
int res;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
- bool deleted = false;
uint32_t hash = ctdb_hash(&(dd->key));
res = tdb_chainlock(ctdb_db->ltdb->tdb, dd->key);
@@ -603,7 +584,9 @@ static int delete_record_traverse(void *param, void *data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- vdata->delete_local_error++;
+ vdata->count.delete_list.local_error++;
+ vdata->count.delete_list.left--;
+ talloc_free(dd);
return 0;
}
@@ -612,46 +595,29 @@ static int delete_record_traverse(void *param, void *data)
* changed and that we are still its lmaster and dmaster.
*/
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] does not exist or is not"
- " a ctdb-record. skipping.\n",
- hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] has been recycled. "
- "skipping.\n",
- hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
+ goto skip;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->flags & CTDB_REC_RO_FLAGS) {
+ if (header.flags & CTDB_REC_RO_FLAGS) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has read-only flags. "
"skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has been migrated away. "
"skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
- if (header->rsn != dd->hdr.rsn + 1) {
+ if (header.rsn != dd->hdr.rsn + 1) {
/*
* The record has been migrated off the node and back again.
* But not requeued for deletion. Skip it.
@@ -663,8 +629,7 @@ static int delete_record_traverse(void *param, void *data)
"migrated away and back again (with empty "
"data). skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
lmaster = ctdb_lmaster(ctdb_db->ctdb, &dd->key);
@@ -674,8 +639,7 @@ static int delete_record_traverse(void *param, void *data)
"delete list (key hash [0x%08x], db[%s]). "
"Strange! skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
--
CTDB repository
