Thanks Han and Ben for suggestions. Sent v6 https://patchwork.ozlabs.org/patch/1155510/ to address comments from both of you. PTAL.
Ali From: Han Zhou <zhou...@gmail.com> Date: Thursday, August 29, 2019 at 11:33 AM To: Ali Gin <amgin...@gmail.com> Cc: ovs dev <d...@openvswitch.org>, "Ginwala, Aliasgar" <aginw...@ebay.com> Subject: Re: [ovs-dev] [PATCH v5] ovsdb-tool: Convert clustered db to standalone db. Thanks for the update. Please see some minor comments below, mostly naming and documentation. On Wed, Aug 28, 2019 at 6:32 PM <amgin...@gmail.com<mailto:amgin...@gmail.com>> wrote: > > From: Aliasgar Ginwala <aginw...@ebay.com<mailto:aginw...@ebay.com>> > > Add support in ovsdb-tool for migrating clustered dbs to standalone dbs. > E.g. usage to migrate nb/sb db to standalone db from raft: > ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db > > Signed-off-by: Aliasgar Ginwala <aginw...@ebay.com<mailto:aginw...@ebay.com>> > --- > Documentation/ref/ovsdb.7.rst | 3 + > > ovsdb/ovsdb-tool.1.in<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fovsdb-tool.1.in&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972538452&sdata=2hQS%2Fs8BLQbo%2FiFN0w247r9GUy25uG1ZnbKuyjEa2ZY%3D&reserved=0> > | 8 +++ > ovsdb/ovsdb-tool.c | 110 +++++++++++++++++++++++++++++++++- > 3 files changed, 120 insertions(+), 1 deletion(-) > > diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst > index cd1c63d64..b12d8066c 100644 > --- a/Documentation/ref/ovsdb.7.rst > +++ b/Documentation/ref/ovsdb.7.rst > @@ -514,6 +514,9 @@ standalone database from the contents of a running > clustered database. > When the cluster is down and cannot be revived, ``ovsdb-client backup`` will > not work. > > +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to > +standalone database when the cluster is down and cannot be revived. > + > Upgrading or Downgrading a Database > ----------------------------------- > > diff --git > a/ovsdb/ovsdb-tool.1.in<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fovsdb-tool.1.in&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972538452&sdata=2hQS%2Fs8BLQbo%2FiFN0w247r9GUy25uG1ZnbKuyjEa2ZY%3D&reserved=0> > > b/ovsdb/ovsdb-tool.1.in<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fovsdb-tool.1.in&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972548440&sdata=9tGlwoHlI8GuGgmf6z4Q5FMMhVMvNr3Paip89PiHD3w%3D&reserved=0> > index ec85e14c4..8c7962ab3 100644 > --- > a/ovsdb/ovsdb-tool.1.in<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fovsdb-tool.1.in&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972558436&sdata=1vbNfTp5J78v0D7Qhe8PNpenliVv86westyfSDWjw1A%3D&reserved=0> > +++ > b/ovsdb/ovsdb-tool.1.in<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fovsdb-tool.1.in&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972558436&sdata=1vbNfTp5J78v0D7Qhe8PNpenliVv86westyfSDWjw1A%3D&reserved=0> > @@ -147,6 +147,14 @@ avoid this possibility, specify \fB\-\-cid=\fIuuid\fR, > where > \fIuuid\fR is the cluster ID of the cluster to join, as printed by > \fBovsdb\-tool get\-cid\fR. > . > +.SS "Database Migration Commands" > +This commands will convert cluster database to standalone database. s/This/These > +. > +.IP "\fBcluster-to-standalone \fR[\fIdb\fR [\fIdb\fR]]" The 2 args are both required (not optional), so please don't add "[ ]". Same comments to the help messages in ovsdb-tool.c > +Use this command to convert to standalone database from clustered database > +when the cluster is down and cannot be revived. It creates new standalone > +\fIdb\fR file from the given cluster \fIdb\fR file. > +. > .SS "Version Management Commands" > .so ovsdb/ovsdb-schemas.man > .PP > diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c > index 438f97590..8039078a3 100644 > --- a/ovsdb/ovsdb-tool.c > +++ b/ovsdb/ovsdb-tool.c > @@ -173,6 +173,9 @@ usage(void) > " compare-versions A OP B compare OVSDB schema version > numbers\n" > " query [DB] TRNS execute read-only transaction on DB\n" > " transact [DB] TRNS execute read/write transaction on DB\n" > + " cluster-to-standalone [DB [DB]] Convert clustered DB to\n" > + "standalone DB when cluster is down and cannot be\n" > + "revived\n" > " [-m]... show-log [DB] print DB's log entries\n" > "The default DB is %s.\n" > "The default SCHEMA is %s.\n", > @@ -942,6 +945,64 @@ print_raft_record(const struct raft_record *r, > } > } > > +static void > +write_raft_header_to_file(const struct json *data, > + struct ovsdb_log *db_log_data) Suggest to rename the functions starting with "write_raft_" to something more clearly tell it is converting from raft to standalone file format, instead of writing a raft header/record to a clustered DB format. Such as: raft_header_to_standalone_log(). > +{ > + if (!data || json_array(data)->n != 2) { > + ovs_fatal(0, "***Invalid data***\n"); > + } > + > + struct json *schema_json = json_array(data)->elems[0]; > + if (schema_json->type != JSON_NULL) { > + struct ovsdb_schema *schema; > + check_ovsdb_error(ovsdb_schema_from_json(schema_json, &schema)); > + ovsdb_schema_destroy(schema); > + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data, > schema_json)); > + } > + > + struct json *data_json = json_array(data)->elems[1]; > + if (!data_json || data_json->type != JSON_OBJECT) { > + ovs_fatal(0, "***invalid data***\n"); > + } > + if (data_json->type != JSON_NULL) { > + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data, data_json)); > + } > +} > + > +static void > +write_raft_header(const struct raft_header *h, struct ovsdb_log *db_log_data) The function name write_raft_header() and write_raft_header_to_file() seems doesn't tell the real difference between these two funcitons. Maybe they can be combined as just one function. Same comment for write_raft_record() and write_raft_record_to_file(). > +{ > + if (h->snap_index) { > + write_raft_header_to_file(h->snap.data, db_log_data); > + } > +} > + > +static void > +write_raft_record_to_file(const struct json *data, > + struct ovsdb_log *db_log_data) > +{ > + if (json_array(data)->n != 2) { > + ovs_fatal(0, "***invalid data***\n"); > + } > + > + struct json *data_json = json_array(data)->elems[1]; > + if (data_json->type != JSON_NULL) { > + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data, data_json)); > + } > +} > + > +static void > +write_raft_record(const struct raft_record *r, struct ovsdb_log *db_log_data) > +{ > + if (r->type == RAFT_REC_ENTRY) { > + if (!r->entry.data) { > + return; > + } > + write_raft_record_to_file(r->entry.data, db_log_data); > + } > +} > + > static void > do_show_log_cluster(struct ovsdb_log *log) > { > @@ -1511,6 +1572,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx) > exit(result ? 0 : 2); > } > > +static void > +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log > *db_log_data) > +{ > + for (unsigned int i = 0; ; i++) { > + struct json *json; > + check_ovsdb_error(ovsdb_log_read(log, &json)); > + if (!json) { > + break; > + } > + > + if (i == 0) { > + struct raft_header h; > + check_ovsdb_error(raft_header_from_json(&h, json)); > + write_raft_header(&h, db_log_data); > + raft_header_uninit(&h); > + } else { > + struct raft_record r; > + check_ovsdb_error(raft_record_from_json(&r, json)); > + write_raft_record(&r, db_log_data); > + raft_record_uninit(&r); > + } > + } > +} > + > +static void > +do_cluster_standalone(struct ovs_cmdl_context *ctx) > +{ > + const char *db_file_name = ctx->argv[1]; > + const char *cluster_db_file_name = ctx->argv[2]; > + struct ovsdb_log *log; > + struct ovsdb_log *db_log_data; > + > + check_ovsdb_error(ovsdb_log_open(cluster_db_file_name, > + OVSDB_MAGIC"|"RAFT_MAGIC, > + OVSDB_LOG_READ_ONLY, -1, &log)); > + check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC, > + OVSDB_LOG_CREATE_EXCL, -1, > &db_log_data)); > + if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) { > + ovs_fatal(0, "Database is not clustered db.\n"); > + } > + do_convert_to_standalone(log, db_log_data); > + check_ovsdb_error(ovsdb_log_commit_block(db_log_data)); > + ovsdb_log_close(db_log_data); > + ovsdb_log_close(log); > +} > > static void > do_help(struct ovs_cmdl_context *ctx OVS_UNUSED) > @@ -1550,7 +1656,9 @@ static const struct ovs_cmdl_command all_commands[] = { > { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO }, > { "help", NULL, 0, INT_MAX, do_help, OVS_RO }, > { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO }, > - { NULL, NULL, 0, 0, NULL, OVS_RO }, > + { "cluster-to-standalone", "[db [clusterdb]]", 2, 2, > + do_cluster_standalone, OVS_RW }, > + { NULL, NULL, 2, 2, NULL, OVS_RO }, > }; > > static const struct ovs_cmdl_command *get_all_commands(void) > -- > 2.20.1 (Apple Git-117) > > _______________________________________________ > dev mailing list > d...@openvswitch.org<mailto:d...@openvswitch.org> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-dev&data=02%7C01%7Caginwala%40ebay.com%7Ce71785eafb58484924b908d72caf5b4b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637027003972568434&sdata=Sm4AgxV3q%2FGE7nsghbdzclHwphMQ85qj1Td5%2B8KG4z8%3D&reserved=0> _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev