Change in osmo-hlr[master]: add db_upgrade test
neels has submitted this change. ( https://gerrit.osmocom.org/c/osmo-hlr/+/15913 ) Change subject: add db_upgrade test .. add db_upgrade test We have a database schema upgrade path, but so far nothing that verifies that we don't break it. It almost seems like the user data weren't important to us!? Add a db upgrade test: - Create a db with an .sql dump taken from a db created with an old osmo-hlr, producing DB schema version 0. - Run osmo-hlr --db-upgrade --db-check - Verify that the upgrade exited successfully. - Verify that restarting with the upgraded DB works. If python tests are enabled, also: - create a new database using the new osmo-hlr (just built). - replay a VTY transcript to create subscribers as in the old snapshot. - replay some sql modifications as done in the old snapshot. - Get a list of sorted table names, - a list of their sorted columns with all their properties, - and dump the table contents in a column- and value-sorted way. - Compare the resulting dumps and error if there are any diffs. (This is how I found the difference in the imei column that was fixed in I68a00014a3d603fcba8781470bc5285f78b538d0) Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 --- M configure.ac M tests/Makefile.am A tests/db_upgrade/Makefile.am A tests/db_upgrade/create_subscribers.vty A tests/db_upgrade/create_subscribers_step2.sql A tests/db_upgrade/db_upgrade_test.err A tests/db_upgrade/db_upgrade_test.ok A tests/db_upgrade/db_upgrade_test.sh A tests/db_upgrade/hlr_db_v0.sql A tests/db_upgrade/osmo-hlr.cfg M tests/testsuite.at 11 files changed, 417 insertions(+), 0 deletions(-) Approvals: Jenkins Builder: Verified laforge: Looks good to me, but someone else must approve osmith: Looks good to me, approved diff --git a/configure.ac b/configure.ac index 993d4d5..ca78f38 100644 --- a/configure.ac +++ b/configure.ac @@ -186,4 +186,5 @@ tests/gsup_server/Makefile tests/gsup/Makefile tests/db/Makefile + tests/db_upgrade/Makefile ) diff --git a/tests/Makefile.am b/tests/Makefile.am index 4da8ab1..357fbac 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -3,6 +3,7 @@ gsup_server \ db \ gsup \ + db_upgrade \ $(NULL) # The `:;' works around a Bash 3.2 bug when the output is not writeable. @@ -44,6 +45,7 @@ # don't run vty and ctrl tests concurrently so that the ports don't conflict $(MAKE) vty-test $(MAKE) ctrl-test + $(MAKE) db-upgrade-equivalence-test else python-tests: echo "Not running python-based external tests (determined at configure-time)" @@ -81,6 +83,9 @@ -rm -f $(CTRL_TEST_DB) -rm $(CTRL_TEST_DB)-* +db-upgrade-equivalence-test: + $(MAKE) -C db_upgrade upgrade-equivalence-test + check-local: atconfig $(TESTSUITE) $(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS) $(MAKE) $(AM_MAKEFLAGS) python-tests diff --git a/tests/db_upgrade/Makefile.am b/tests/db_upgrade/Makefile.am new file mode 100644 index 000..79136c9 --- /dev/null +++ b/tests/db_upgrade/Makefile.am @@ -0,0 +1,14 @@ +EXTRA_DIST = \ + db_upgrade_test.sh \ + db_upgrade_test.err \ + db_upgrade_test.ok \ + hlr_db_v0.sql \ + osmo-hlr.cfg \ + create_subscribers.vty \ + $(NULL) + +update_exp: + $(srcdir)/db_upgrade_test.sh $(srcdir) $(builddir) >"$(srcdir)/db_upgrade_test.ok" 2>"$(srcdir)/db_upgrade_test.err" + +upgrade-equivalence-test: + $(srcdir)/db_upgrade_test.sh $(srcdir) $(builddir) do-equivalence-test diff --git a/tests/db_upgrade/create_subscribers.vty b/tests/db_upgrade/create_subscribers.vty new file mode 100644 index 000..30eeba6 --- /dev/null +++ b/tests/db_upgrade/create_subscribers.vty @@ -0,0 +1,47 @@ +OsmoHLR> enable +OsmoHLR# subscriber imsi 123456789012345 create +% Created subscriber 123456789012345 +ID: 1 +IMSI: 123456789012345 +MSISDN: none +OsmoHLR# subscriber imsi 123456789012345 update msisdn 098765432109876 +% Updated subscriber IMSI='123456789012345' to MSISDN='098765432109876' +OsmoHLR# subscriber imsi 123456789012345 update aud2g comp128v1 ki BeefedCafeFaceAcedAddedDecadeFee +OsmoHLR# subscriber imsi 123456789012345 update aud3g milenage k C01ffedC1cadaeAc1d1f1edAcac1aB0a opc CededEffacedAceFacedBadFadedBeef +OsmoHLR# subscriber imsi 1 create +% Created subscriber 1 +ID: 2 +IMSI: 1 +MSISDN: none +OsmoHLR# subscriber imsi 2 create +% Created subscriber 2 +ID: 3 +IMSI: 2 +MSISDN: none +OsmoHLR# subscriber imsi 2 update msisdn 2 +% Updated subscriber IMSI='2' to MSISDN='2' +OsmoHLR# subscriber imsi 33 create +% Created subscriber 33 +ID: 4 +IMSI: 33 +MSISDN: none +OsmoHLR# subscriber imsi 33 update msisdn 3 +% Updated subscriber IMSI='33' to MSISDN='3' +OsmoHLR# subscriber imsi 33 update
Change in osmo-hlr[master]: add db_upgrade test
osmith has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-hlr/+/15913 ) Change subject: add db_upgrade test .. Patch Set 5: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 5 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge Gerrit-Reviewer: neels Gerrit-Reviewer: osmith Gerrit-CC: pespin Gerrit-Comment-Date: Mon, 04 Nov 2019 09:58:27 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in osmo-hlr[master]: add db_upgrade test
laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-hlr/+/15913 ) Change subject: add db_upgrade test .. Patch Set 5: Code-Review+1 -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 5 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: laforge Gerrit-Reviewer: neels Gerrit-CC: pespin Gerrit-Comment-Date: Fri, 01 Nov 2019 14:57:35 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in osmo-hlr[master]: add db_upgrade test
Hello Jenkins Builder, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 to look at the new patch set (#5). Change subject: add db_upgrade test .. add db_upgrade test We have a database schema upgrade path, but so far nothing that verifies that we don't break it. It almost seems like the user data weren't important to us!? Add a db upgrade test: - Create a db with an .sql dump taken from a db created with an old osmo-hlr, producing DB schema version 0. - Run osmo-hlr --db-upgrade --db-check - Verify that the upgrade exited successfully. - Verify that restarting with the upgraded DB works. If python tests are enabled, also: - create a new database using the new osmo-hlr (just built). - replay a VTY transcript to create subscribers as in the old snapshot. - replay some sql modifications as done in the old snapshot. - Get a list of sorted table names, - a list of their sorted columns with all their properties, - and dump the table contents in a column- and value-sorted way. - Compare the resulting dumps and error if there are any diffs. (This is how I found the difference in the imei column that was fixed in I68a00014a3d603fcba8781470bc5285f78b538d0) Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 --- M configure.ac M tests/Makefile.am A tests/db_upgrade/Makefile.am A tests/db_upgrade/create_subscribers.vty A tests/db_upgrade/create_subscribers_step2.sql A tests/db_upgrade/db_upgrade_test.err A tests/db_upgrade/db_upgrade_test.ok A tests/db_upgrade/db_upgrade_test.sh A tests/db_upgrade/hlr_db_v0.sql A tests/db_upgrade/osmo-hlr.cfg M tests/testsuite.at 11 files changed, 417 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/13/15913/5 -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 5 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: neels Gerrit-CC: pespin Gerrit-MessageType: newpatchset
Change in osmo-hlr[master]: add db_upgrade test
Hello Jenkins Builder, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 to look at the new patch set (#4). Change subject: add db_upgrade test .. add db_upgrade test We have a database schema upgrade path, but so far nothing that verifies that we don't break it. It almost seems like the user data weren't important to us!? Add a db upgrade test: - Create a db with an .sql dump taken from a db created with an old osmo-hlr, producing DB schema version 0. - Run osmo-hlr --db-upgrade --db-check - Verify that the upgrade exited successfully. - Verify that restarting with the upgraded DB works. If python tests are enabled, also: - create a new database using the new osmo-hlr (just built). - replay a VTY transcript to create subscribers as in the old snapshot. - replay some sql modifications as done in the old snapshot. - Get a list of sorted table names, - a list of their sorted columns with all their properties, - and dump the table contents in a column- and value-sorted way. - Compare the resulting dumps and error if there are any diffs. (This is how I found the difference in the imei column that was fixed in I68a00014a3d603fcba8781470bc5285f78b538d0) Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 --- M configure.ac M tests/Makefile.am A tests/db_upgrade/Makefile.am A tests/db_upgrade/create_subscribers.vty A tests/db_upgrade/create_subscribers_step2.sql A tests/db_upgrade/db_upgrade_test.err A tests/db_upgrade/db_upgrade_test.ok A tests/db_upgrade/db_upgrade_test.sh A tests/db_upgrade/hlr_db_v0.sql A tests/db_upgrade/osmo-hlr.cfg M tests/testsuite.at 11 files changed, 416 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/13/15913/4 -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 4 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: neels Gerrit-CC: pespin Gerrit-MessageType: newpatchset
Change in osmo-hlr[master]: add db_upgrade test
neels has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-hlr/+/15913 ) Change subject: add db_upgrade test .. Patch Set 2: hrm the changes in sqlite3 clients makes it hard to get this test to act reproducably... -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 2 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: neels Gerrit-CC: pespin Gerrit-Comment-Date: Thu, 31 Oct 2019 16:57:01 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in osmo-hlr[master]: add db_upgrade test
Hello Jenkins Builder, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 to look at the new patch set (#2). Change subject: add db_upgrade test .. add db_upgrade test We have a database schema upgrade path, but so far nothing that verifies that we don't break it. It almost seems like the user data weren't important to us!? Add a db upgrade test: - Create a db with an .sql dump taken from a db created with an old osmo-hlr, producing DB schema version 0. - Run osmo-hlr --db-upgrade --db-check - Verify that the upgrade exited successfully. - Verify that restarting with the upgraded DB works. If python tests are enabled, also: - create a new database using the new osmo-hlr (just built). - replay a VTY transcript to create subscribers as in the old snapshot. - replay some sql modifications as done in the old snapshot. - Get a list of sorted table names, - a list of their sorted columns with all their properties, - and dump the table contents in a column- and value-sorted way. - Compare the resulting dumps and error if there are any diffs. (This is how I found the difference in the imei column that was fixed in I68a00014a3d603fcba8781470bc5285f78b538d0) Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 --- M configure.ac M tests/Makefile.am A tests/db_upgrade/Makefile.am A tests/db_upgrade/create_subscribers.vty A tests/db_upgrade/create_subscribers_step2.sql A tests/db_upgrade/db_upgrade_test.err A tests/db_upgrade/db_upgrade_test.ok A tests/db_upgrade/db_upgrade_test.sh A tests/db_upgrade/hlr_db_v0.sql A tests/db_upgrade/osmo-hlr.cfg M tests/testsuite.at 11 files changed, 415 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/13/15913/2 -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 2 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-CC: pespin Gerrit-MessageType: newpatchset
Change in osmo-hlr[master]: add db_upgrade test
pespin has posted comments on this change. ( https://gerrit.osmocom.org/c/osmo-hlr/+/15913 ) Change subject: add db_upgrade test .. Patch Set 1: Test fails. -- To view, visit https://gerrit.osmocom.org/c/osmo-hlr/+/15913 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-hlr Gerrit-Branch: master Gerrit-Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4 Gerrit-Change-Number: 15913 Gerrit-PatchSet: 1 Gerrit-Owner: neels Gerrit-Reviewer: Jenkins Builder Gerrit-CC: pespin Gerrit-Comment-Date: Thu, 31 Oct 2019 11:32:40 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in osmo-hlr[master]: add db_upgrade test
neels has uploaded this change for review. (
https://gerrit.osmocom.org/c/osmo-hlr/+/15913 )
Change subject: add db_upgrade test
..
add db_upgrade test
We have a database schema upgrade path, but so far nothing that verifies that
we don't break it. It almost seems like the user data weren't important to us!?
Add a db upgrade test:
- Create a db with an .sql dump taken from a db created with an old osmo-hlr,
producing DB schema version 0.
- Run osmo-hlr --db-upgrade --db-check
- Verify that the upgrade exited successfully.
- Verify that restarting with the upgraded DB works.
If python tests are enabled, also:
- create a new database using the new osmo-hlr (just built).
- replay a VTY transcript to create subscribers as in the old snapshot.
- replay some sql modifications as done in the old snapshot.
- Get a list of sorted table names,
- a list of their sorted columns with all their properties,
- and dump the table contents in a column- and value-sorted way.
- Compare the resulting dumps and error if there are any diffs.
(This is how I found the difference in the imei column that was fixed in
I68a00014a3d603fcba8781470bc5285f78b538d0)
Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
---
M configure.ac
M src/hlr.c
M tests/Makefile.am
A tests/db_upgrade/Makefile.am
A tests/db_upgrade/create_subscribers.vty
A tests/db_upgrade/create_subscribers_step2.sql
A tests/db_upgrade/db_upgrade_test.err
A tests/db_upgrade/db_upgrade_test.ok
A tests/db_upgrade/db_upgrade_test.sh
A tests/db_upgrade/hlr_db_v0.sql
A tests/db_upgrade/osmo-hlr.cfg
M tests/testsuite.at
12 files changed, 434 insertions(+), 6 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-hlr refs/changes/13/15913/1
diff --git a/configure.ac b/configure.ac
index 993d4d5..ca78f38 100644
--- a/configure.ac
+++ b/configure.ac
@@ -186,4 +186,5 @@
tests/gsup_server/Makefile
tests/gsup/Makefile
tests/db/Makefile
+ tests/db_upgrade/Makefile
)
diff --git a/src/hlr.c b/src/hlr.c
index f9cc2f5..6bfc141 100644
--- a/src/hlr.c
+++ b/src/hlr.c
@@ -862,12 +862,6 @@
return rc;
}
- /* start telnet after reading config for vty_get_bind_addr() */
- rc = telnet_init_dynif(hlr_ctx, NULL, vty_get_bind_addr(),
- OSMO_VTY_PORT_HLR);
- if (rc < 0)
- return rc;
-
LOGP(DMAIN, LOGL_NOTICE, "hlr starting\n");
rc = rand_init();
@@ -895,6 +889,13 @@
exit(0);
}
+ /* start telnet after reading config for vty_get_bind_addr() */
+ rc = telnet_init_dynif(hlr_ctx, NULL, vty_get_bind_addr(),
+ OSMO_VTY_PORT_HLR);
+ if (rc < 0)
+ return rc;
+
+
g_hlr->gs = osmo_gsup_server_create(hlr_ctx, g_hlr->gsup_bind_addr,
OSMO_GSUP_PORT,
read_cb, &g_lu_ops, g_hlr);
if (!g_hlr->gs) {
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4da8ab1..357fbac 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -3,6 +3,7 @@
gsup_server \
db \
gsup \
+ db_upgrade \
$(NULL)
# The `:;' works around a Bash 3.2 bug when the output is not writeable.
@@ -44,6 +45,7 @@
# don't run vty and ctrl tests concurrently so that the ports don't conflict
$(MAKE) vty-test
$(MAKE) ctrl-test
+ $(MAKE) db-upgrade-equivalence-test
else
python-tests:
echo "Not running python-based external tests (determined at
configure-time)"
@@ -81,6 +83,9 @@
-rm -f $(CTRL_TEST_DB)
-rm $(CTRL_TEST_DB)-*
+db-upgrade-equivalence-test:
+ $(MAKE) -C db_upgrade upgrade-equivalence-test
+
check-local: atconfig $(TESTSUITE)
$(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS)
$(MAKE) $(AM_MAKEFLAGS) python-tests
diff --git a/tests/db_upgrade/Makefile.am b/tests/db_upgrade/Makefile.am
new file mode 100644
index 000..79136c9
--- /dev/null
+++ b/tests/db_upgrade/Makefile.am
@@ -0,0 +1,14 @@
+EXTRA_DIST = \
+ db_upgrade_test.sh \
+ db_upgrade_test.err \
+ db_upgrade_test.ok \
+ hlr_db_v0.sql \
+ osmo-hlr.cfg \
+ create_subscribers.vty \
+ $(NULL)
+
+update_exp:
+ $(srcdir)/db_upgrade_test.sh $(srcdir) $(builddir)
>"$(srcdir)/db_upgrade_test.ok" 2>"$(srcdir)/db_upgrade_test.err"
+
+upgrade-equivalence-test:
+ $(srcdir)/db_upgrade_test.sh $(srcdir) $(builddir) do-equivalence-test
diff --git a/tests/db_upgrade/create_subscribers.vty
b/tests/db_upgrade/create_subscribers.vty
new file mode 100644
index 000..30eeba6
--- /dev/null
+++ b/tests/db_upgrade/create_subscribers.vty
@@ -0,0 +1,47 @@
+OsmoHLR> enable
+OsmoHLR# subscriber imsi 123456789012345 create
+% Created subscriber 123456789012345
+ID: 1
+IMSI: 123456789012345
+MSISDN: none
+OsmoHLR# subscriber imsi 123456789012345 update msisdn 098765432109876
+%
