Change in osmo-ttcn3-hacks[master]: add ttcn3 test suites slides from OsmoDevCon 2018
Harald Welte has submitted this change and it was merged. (
https://gerrit.osmocom.org/10709 )
Change subject: add ttcn3 test suites slides from OsmoDevCon 2018
..
add ttcn3 test suites slides from OsmoDevCon 2018
Those slides are copied from
http://git.gnumonks.org/laforge-slides/plain/2018/osmodevcon2018-ttcn3_test_suites/
to make sure they are more easily found. It would be nice to convert
them into an actual asciidoc book with one chapter per test suite which
would be built from within this repository. This way we can keep
documentation and code better in sync.
Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79
---
A doc/osmodevcon2018-ttcn3_test_suites.adoc
1 file changed, 296 insertions(+), 0 deletions(-)
Approvals:
Jenkins Builder: Verified
Harald Welte: Looks good to me, approved
diff --git a/doc/osmodevcon2018-ttcn3_test_suites.adoc
b/doc/osmodevcon2018-ttcn3_test_suites.adoc
new file mode 100644
index 000..6eaee82
--- /dev/null
+++ b/doc/osmodevcon2018-ttcn3_test_suites.adoc
@@ -0,0 +1,296 @@
+Osmocom TTCN-3 Test Suites
+==
+:author: Harald Welte
+:copyright:2018 by Harald Welte (License: CC-BY-SA)
+:backend: slidy
+:max-width:45em
+
+
+== Osmocom TTCN-3 Test Suites
+
+* developed in 2017+2018
+* compiled using Eclipse TITAN
+** uses just a command-line compiler + Makefiles
+** no IDE needed at all, don't let _Eclipse_ fool you
+* containerized in Docker
+* executed by Jenkins CI
+
+== Terminology
+
+ATS:: Abstract Test Suite
+MTC:: Main Test Component
+PTC:: Parallel Test Component
+IUT:: Implementation Under Test
+
+== Test Suite Philosophy
+
+* test one network element (our IUT)
+* test external behavior (3GPP and non-3GPP)
+* emulate entire environment from TTCN-3
+* don't reuse Osmocom C-code protocol implementations in the tests
+* test against independent TTCN-3 implementations!
+
+== What to test?
+
+* successful cases
+* erroneous cases (no answer, NACK, ...)
+** many difficult to reproduce with real phones/devices
+* load / resource exhaustion
+* spec compliance
+* focus on functionality actually relevant to IUT
+
+== Why TTCN-3 + TITAN
+
+* TTCN-3 specifically designed for telecom protocol testing
+* TITAN team released many telecom protocols in TTCN-3, such as
+** BSSAP, L3 (RR/MM/CC), SMS (CP/RP/TP), SS, M3UA, SCCP, GTP, NS, BSSGP, ...
+** shortens our test development cycle
+** permits us to test against known working industry implementations
+
+== Test suites for Osmocom CNI components
+
+* `osmo-bts`
+* `osmo-bsc`
+* `osmo-msc`
+* `osmo-mgw`
+* `osmo-hlr`
+* `osmo-sip-connector`
+* `osmo-sgsn`
+* `osmo-ggsn`
+
+== Test suites in progress
+
+* `osmo-pcu`
+* `osmo-bsc_nat`
+
+
+
+
+== BTS_Tests.ttcn
+
+* external interfaces
+** A-bis side: RSL (emulates BSC-side server)
+** Um side: L1CTL to control MS
+** PCU side: pcu_socket
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ { rank=same; BTS, BSC};
+ BTS [label="IUT\nosmo-bts-trx",shape="box"];
+ ATS [label="ATS\nBTS_Tests.ttcn"];
+ BSC [label="osmo-bsc\nOML only"];
+ BTS -> fake_trx [label="bursts"];
+ fake_trx -> trxcon [label="bursts"];
+ trxcon -> ATS [label="bursts"];
+
+ BTS -> BSC [label="A-bis OML"];
+ BTS -> ATS [label="A-bis RSL"];
+
+ ATS -> BTS [label="pcu_sock"];
+ ATS -> BSC [label="VTY"];
+ ATS -> BTS [label="CTRL"];
+}
+
+
+
+
+== BSC_Tests.ttcn
+
+* external interfaces
+** A-bis side: RSL (emulates BTS-side client)
+** A-side: BSSAP/SCCP/M3UA (emulates MSC-side)
+** MGW side: MGCP (emulates MGW side)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ { rank=same; BTS; STP; };
+ BSC [label="IUT\nosmo-bsc",shape="box"];
+ ATS [label="ATS\nBSC_Tests.ttcn"];
+ BTS [label="osmo-bts-omldummy\nOML only"];
+
+ BTS -> BSC [label="A-bis OML"];
+ ATS -> BSC [label="A-bis RSL"];
+ ATS -> BSC [label="CTRL"];
+ ATS -> BSC [label="VTY"];
+ ATS -> STP [label="A BSSAP\nSCCP/M3UA"];
+ BSC -> STP [label="A BSSAP\nSCCP/M3UA"];
+}
+
+
+== MSC_Tests.ttcn
+
+* external interfaces
+** A: BSSAP/SCCP/M3UA (emulates BSC-side)
+** MNCC: MNCC/unix-domain (emulates ext. MNCC side)
+** MGW: MGCP (emulates MGW side)
+** GSUP (emulates HLR side)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ MSC [label="IUT\nosmo-msc",shape="box"];
+ ATS [label="ATS\nMSC_Tests.ttcn"];
+
+ ATS -> MSC [label="MNCC"];
+ ATS -> MSC [label="SMPP",style="dashed"];
+ ATS -> MSC [label="CTRL"];
+ ATS -> MSC [label="VTY"];
+ MSC -> ATS [label="GSUP"];
+ ATS -> STP [label="A BSSAP\nSCCP/M3UA"];
+ MSC -> STP [label="A BSSAP\nSCCP/M3UA"];
+}
+
+
+
+== MGCP_Test.ttcn
+
+* external interfaces
+** MGCP (emulates call agent)
+** RTP (stream source/sink)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ MGW [label="IUT\nosmo-mgw",shape="box"];
+ ATS [label="ATS\nMGCP_Test.ttcn"];
+
+ ATS -> MGW [label="RTP"];
+ ATS -> MGW [label="MGCP"];
+ MGW -> ATS [label="RTP"];
+}
+
+
+== HLR_Tests.ttcn
+
+
Change in osmo-ttcn3-hacks[master]: add ttcn3 test suites slides from OsmoDevCon 2018
Harald Welte has posted comments on this change. ( https://gerrit.osmocom.org/10709 ) Change subject: add ttcn3 test suites slides from OsmoDevCon 2018 .. Patch Set 2: Code-Review+2 -- To view, visit https://gerrit.osmocom.org/10709 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79 Gerrit-Change-Number: 10709 Gerrit-PatchSet: 2 Gerrit-Owner: Harald Welte Gerrit-Reviewer: Harald Welte Gerrit-Reviewer: Jenkins Builder (102) Gerrit-CC: Vadim Yanitskiy Gerrit-Comment-Date: Sun, 16 Sep 2018 10:42:30 + Gerrit-HasComments: No Gerrit-HasLabels: Yes
Change in osmo-ttcn3-hacks[master]: add ttcn3 test suites slides from OsmoDevCon 2018
Hello Jenkins Builder, I'd like you to reexamine a change. Please visit https://gerrit.osmocom.org/10709 to look at the new patch set (#2). Change subject: add ttcn3 test suites slides from OsmoDevCon 2018 .. add ttcn3 test suites slides from OsmoDevCon 2018 Those slides are copied from http://git.gnumonks.org/laforge-slides/plain/2018/osmodevcon2018-ttcn3_test_suites/ to make sure they are more easily found. It would be nice to convert them into an actual asciidoc book with one chapter per test suite which would be built from within this repository. This way we can keep documentation and code better in sync. Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79 --- A doc/osmodevcon2018-ttcn3_test_suites.adoc 1 file changed, 296 insertions(+), 0 deletions(-) git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/09/10709/2 -- To view, visit https://gerrit.osmocom.org/10709 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79 Gerrit-Change-Number: 10709 Gerrit-PatchSet: 2 Gerrit-Owner: Harald Welte Gerrit-Reviewer: Jenkins Builder (102) Gerrit-CC: Vadim Yanitskiy
Change in osmo-ttcn3-hacks[master]: add ttcn3 test suites slides from OsmoDevCon 2018
Vadim Yanitskiy has posted comments on this change. ( https://gerrit.osmocom.org/10709 ) Change subject: add ttcn3 test suites slides from OsmoDevCon 2018 .. Patch Set 1: (1 comment) https://gerrit.osmocom.org/#/c/10709/1/doc/osmodevcon2018-ttcn3_test_suites.adoc File doc/osmodevcon2018-ttcn3_test_suites.adoc: https://gerrit.osmocom.org/#/c/10709/1/doc/osmodevcon2018-ttcn3_test_suites.adoc@130 PS1, Line 130: impllements typo: implements suggestion: s/implements/emulates/? -- To view, visit https://gerrit.osmocom.org/10709 To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-ttcn3-hacks Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79 Gerrit-Change-Number: 10709 Gerrit-PatchSet: 1 Gerrit-Owner: Harald Welte Gerrit-Reviewer: Jenkins Builder (102) Gerrit-CC: Vadim Yanitskiy Gerrit-Comment-Date: Thu, 30 Aug 2018 19:07:29 + Gerrit-HasComments: Yes Gerrit-HasLabels: No
Change in osmo-ttcn3-hacks[master]: add ttcn3 test suites slides from OsmoDevCon 2018
Harald Welte has uploaded this change for review. (
https://gerrit.osmocom.org/10709
Change subject: add ttcn3 test suites slides from OsmoDevCon 2018
..
add ttcn3 test suites slides from OsmoDevCon 2018
Those slides are copied from
http://git.gnumonks.org/laforge-slides/plain/2018/osmodevcon2018-ttcn3_test_suites/
to make sure they are more easily found. It would be nice to convert
them into an actual asciidoc book with one chapter per test suite which
would be built from within this repository. This way we can keep
documentation and code better in sync.
Change-Id: I8a0d66babc165678ec53a1ee1c673e93708b8b79
---
A doc/osmodevcon2018-ttcn3_test_suites.adoc
1 file changed, 296 insertions(+), 0 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks
refs/changes/09/10709/1
diff --git a/doc/osmodevcon2018-ttcn3_test_suites.adoc
b/doc/osmodevcon2018-ttcn3_test_suites.adoc
new file mode 100644
index 000..3273314
--- /dev/null
+++ b/doc/osmodevcon2018-ttcn3_test_suites.adoc
@@ -0,0 +1,296 @@
+Osmocom TTCN-3 Test Suites
+==
+:author: Harald Welte
+:copyright:2018 by Harald Welte (License: CC-BY-SA)
+:backend: slidy
+:max-width:45em
+
+
+== Osmocom TTCN-3 Test Suites
+
+* developed in 2017+2018
+* compiled using Eclipse TITAN
+** uses just a command-line compiler + Makefiles
+** no IDE needed at all, don't let _Eclipse_ fool you
+* containerized in Docker
+* executed by Jenkins CI
+
+== Terminology
+
+ATS:: Abstract Test Suite
+MTC:: Main Test Component
+PTC:: Parallel Test Component
+IUT:: Implementation Under Test
+
+== Test Suite Philosophy
+
+* test one network element (our IUT)
+* test external behavior (3GPP and non-3GPP)
+* emulate entire environment from TTCN-3
+* don't reuse Osmocom C-code protocol implementations in the tests
+* test against independent TTCN-3 implementations!
+
+== What to test?
+
+* successful cases
+* erroneous cases (no answer, NACK, ...)
+** many difficult to reproduce with real phones/devices
+* load / resource exhaustion
+* spec compliance
+* focus on functionality actually relevant to IUT
+
+== Why TTCN-3 + TITAN
+
+* TTCN-3 specifically designed for telecom protocol testing
+* TITAN team released many telecom protocols in TTCN-3, such as
+** BSSAP, L3 (RR/MM/CC), SMS (CP/RP/TP), SS, M3UA, SCCP, GTP, NS, BSSGP, ...
+** shortens our test development cycle
+** permits us to test against known working industry implementations
+
+== Test suites for Osmocom CNI components
+
+* `osmo-bts`
+* `osmo-bsc`
+* `osmo-msc`
+* `osmo-mgw`
+* `osmo-hlr`
+* `osmo-sip-connector`
+* `osmo-sgsn`
+* `osmo-ggsn`
+
+== Test suites in progress
+
+* `osmo-pcu`
+* `osmo-bsc_nat`
+
+
+
+
+== BTS_Tests.ttcn
+
+* external interfaces
+** A-bis side: RSL (emulates BSC-side server)
+** Um side: L1CTL to control MS
+** PCU side: pcu_socket
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ { rank=same; BTS, BSC};
+ BTS [label="IUT\nosmo-bts-trx",shape="box"];
+ ATS [label="ATS\nBTS_Tests.ttcn"];
+ BSC [label="osmo-bsc\nOML only"];
+ BTS -> fake_trx [label="bursts"];
+ fake_trx -> trxcon [label="bursts"];
+ trxcon -> ATS [label="bursts"];
+
+ BTS -> BSC [label="A-bis OML"];
+ BTS -> ATS [label="A-bis RSL"];
+
+ ATS -> BTS [label="pcu_sock"];
+ ATS -> BSC [label="VTY"];
+ ATS -> BTS [label="CTRL"];
+}
+
+
+
+
+== BSC_Tests.ttcn
+
+* external interfaces
+** A-bis side: RSL (emulates BTS-side client)
+** A-side: BSSAP/SCCP/M3UA (emulates MSC-side)
+** MGW side: MGCP (emulates MGW side)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ { rank=same; BTS; STP; };
+ BSC [label="IUT\nosmo-bsc",shape="box"];
+ ATS [label="ATS\nBSC_Tests.ttcn"];
+ BTS [label="osmo-bts-omldummy\nOML only"];
+
+ BTS -> BSC [label="A-bis OML"];
+ ATS -> BSC [label="A-bis RSL"];
+ ATS -> BSC [label="CTRL"];
+ ATS -> BSC [label="VTY"];
+ ATS -> STP [label="A BSSAP\nSCCP/M3UA"];
+ BSC -> STP [label="A BSSAP\nSCCP/M3UA"];
+}
+
+
+== MSC_Tests.ttcn
+
+* external interfaces
+** A: BSSAP/SCCP/M3UA (emulates BSC-side)
+** MNCC: MNCC/unix-domain (emulates ext. MNCC side)
+** MGW: MGCP (emulates MGW side)
+** GSUP (impllements HLR side)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ MSC [label="IUT\nosmo-msc",shape="box"];
+ ATS [label="ATS\nMSC_Tests.ttcn"];
+
+ ATS -> MSC [label="MNCC"];
+ ATS -> MSC [label="SMPP",style="dashed"];
+ ATS -> MSC [label="CTRL"];
+ ATS -> MSC [label="VTY"];
+ MSC -> ATS [label="GSUP"];
+ ATS -> STP [label="A BSSAP\nSCCP/M3UA"];
+ MSC -> STP [label="A BSSAP\nSCCP/M3UA"];
+}
+
+
+
+== MGCP_Test.ttcn
+
+* external interfaces
+** MGCP (emulates call agent)
+** RTP (stream source/sink)
+
+[graphviz]
+
+digraph G {
+ rankdir=LR;
+ MGW [label="IUT\nosmo-mgw",shape="box"];
+ ATS [label="ATS\nMGCP_Test.ttcn"];
+
+ ATS -> MGW [label="RTP"];
+ ATS -> MGW [label="MGCP"];
+ MGW -> ATS [label="RTP"];
+}
+
+
+== HLR_Tests.ttcn
+
+* ex
