Hi all,
I am running a scenario which contains ims registration and ims_uac with ims bench. Although the scenario is completed without any failure: all successful calls. In the produced report.html, some graphs are missing. It is normal that there are no graphs related deregistration and messaging, since I do not run these scenarios. But I expect to see all of the registration and call related graphs . Some of the call graphs are unfortunately missing. Any hint is more than welcome. Or are there any other way to produce reports from the outputs of the ims bench tests? Produced report.html is attached. Thanks... KaanTitle: Test Report (ubuntu-vm)
Test Report (ubuntu-vm) |
Summary
This report shows the result of a benchmark run performed by "IMS Bench SIPp", an implementation of the IMS/NGN Performance Benchmark suite, ETSI TS 186.008.
The test was started on 15-Jul-2014 13:25, and the total time for the test execution was 0h 3m 17s. The Design Objective Capacity (DOC) has not been reached. The DOC is >= 1 scenarios per second.
The following systems and parameters were used for the test.
|
|
The following table shows the average of the key measurements for each step of the test. Each steps is characterized by the requested load, the effective load, the global IHS (total of all Inadequately handled scenarios for this step divided by number of Session Attempts for this step) the scenario IHS (number of inadequately handled scenarios for this step divided by the number of scenario attempts for this step), the CPU utilization and the available Memory on the SUT. The available Memory is expressed in MegaBytes, and the requested and effective loads in Scenarios Attempts Per Seconds (SAPS).
Note that the IHS percentages represented in this table are the number of failures for a step divided by the number of scenario attempts for this step, and so is not the average of (IHS per seconds)
| Pre-registration | |
|---|---|
| Requested load | 1 |
| Effective Load | 0.90 |
| Ratio ims_reg % | 100.00 |
| SIPP CPU ubuntu | 8.46 |
| SIPP MEM ubuntu | 409.23 |
| global IHS % | 0.00 |
The following chapters show details on different measurement, like delay between two messages,
response time or number of messages per seconds.
Each measurement can be represented in one of the four following forms.
- Evolution in function of the time. On such graphs, the raw information is plotted, like number of messages per seconds, or response time of each scenario. This graph is useful in giving for instance a good idea on the distribution of response times, and it's evolution over the time.
- Evolution (mean) in function of the time. While previous graph gives a good indication, it may sometimes be easier to see the evolution of the mean of the measurement over a second in function of the time.
- Histogram. This graph shows the histogram of the measurement, so how many times each value of the measurement occured.
- Probability. This graph gives the probability of the measurement to be higher than a certain value. This graph can be used to determine percentile for instances.
For some graphs, a cubic Bezier curve is plotted as well.
- 1 Scenario Attempts Per Second
- 1.1 Scenario Attempts Per Second (Mean per second)
- 2 SUT CPU %
- 3 SUT Available Memory [MB]
- 4 ALL SIPP CPU %
- 4.1 ALL SIPP CPU % over time
- 5 ALL SIPP Free Memory [MB]
- 5.1 ALL SIPP Free Memory [MB] over time
- 6 Inadequately handled scenario Percentage
- 6.1 Inadequately handled scenario Percentage over time
- 7 Scenario retransmissions - all scenarios
- 7.1 Scenario retransmissions - all scenarios over time
- 7 Registration
- 7.1 PX_TRT-REG1: Time of the first register transaction
- 7.1.1 PX_TRT-REG1: Time of the first register transaction(Registration use case) (Mean per second)
- 7.2 PX_TRT-REG2: Time of the second register transaction
- 7.2.1 PX_TRT-REG2: Time of the second register transaction(Registration use case) (Mean per second)
- 7.3 ims_rereg : Time of the re-register transaction
- 7.4 ims_dereg : PX_TRT-REG1: Time of the first register transaction
- 7.5 ims_reg : Scenario retransmissions
- 7.5.1 Scenario retransmissions(ims_reg scenario) over time
- 7.6 ims_reg : PX_TRT-REG1: Time of the first register transaction
- 7.6.1 PX_TRT-REG1: Time of the first register transaction(ims_reg scenario) (Mean per second)
- 8 Calling
- 8.1 PX_TRT-SES1: Session Setup Time
- 8.2 PX_TRT-SES2: Session Initiation transversal time
- 8.3 PX_TRT-REL1: Delay Between BYE and 200 OK
- 8.3.1 PX_TRT-REL1: Delay Between BYE and 200 OK (Calling use case) (Mean per second)
- 8.4 PX_TRT-SES3: INVITE and re-INVITE cost
- 8.5 ims_uac : Scenario retransmissions
- 8.5.1 Scenario retransmissions(ims_uac scenario) over time
- 8.6 ims_uac : Inadequately handled scenario Percentage
- 8.6.1 Inadequately handled scenario Percentage(ims_uac scenario) over time
- 9 Messaging
- 9.1 PX_TRT-PMM1: Message Transmission time
- 9.2 PX_TRT-PMM2: Message Transmission time (error case)
- Appendix
1 Scenario Attempts Per Second ![Index]( "Index")
This graph represents the number of scenario per seconds generated by the test system. For each step, the generation was based on a Poisson.
| Effective Load | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Variance | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 0.90 | 0.09 | 0.30 | 0.00 | 1.00 | 1.0 | 1.0 | 1.0 | 1.0 |
1.1 Scenario Attempts Per Second (Mean per second) ![Index]( "Index")
2 SUT CPU % ![Index]( "Index")
This graph represents the CPU of the system under test (SUT).
3 SUT Available Memory [MB] ![Index]( "Index")
This graph represents the Available memory on the system under test, in MBytes (SUT).
4 ALL SIPP CPU % ![Index]( "Index")
This graph represents the CPU of SIPP on ALL Test Machines
| SIPP CPU ubuntu | |||||
|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum |
| Pre-reg | 1 | 8.46 | 2.99 | 4.02 | 14.50 |
4.1 ALL SIPP CPU % over time ![Index]( "Index")
5 ALL SIPP Free Memory [MB] ![Index]( "Index")
This graph represents the free memory of SIPP on ALL Test Machines, in MBytes
| SIPP MEM ubuntu | |||||
|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum |
| Pre-reg | 1 | 409.23 | 0.06 | 409.17 | 409.36 |
5.1 ALL SIPP Free Memory [MB] over time ![Index]( "Index")
6 Inadequately handled scenario Percentage ![Index]( "Index")
This graph represents the percentage of inadequately handled scenarios.
| IHS per use_case % | |||||
|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum |
| Pre-reg | 1 | 0.00 | 0.00 | 0.00 | 0.00 |
6.1 Inadequately handled scenario Percentage over time ![Index]( "Index")
7 Scenario retransmissions - all scenarios ![Index]( "Index")
This graph represents the number of retransmissions per seconds for all scenarios.
| RETRANSMIT | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | 0.0 | 0.0 |
7.1 Scenario retransmissions - all scenarios over time ![Index]( "Index")
7 Registration ![Index]( "Index")
7.1 PX_TRT-REG1: Time of the first register transaction ![Index]( "Index")
This graph represents the time of the first register transaction in the registration use_cases i.e. the time between the REGISTER and the 401 Unautorized for all scenarios in the Registration use_case.
| PX_TRT-REG1 (msec) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 55.02 | 28.81 | 24.85 | 106.49 | 37.2 | 106.4 | 106.4 | 106.4 |
7.1.1 PX_TRT-REG1: Time of the first register transaction(Registration use case) (Mean per second) ![Index]( "Index")
7.2 PX_TRT-REG2: Time of the second register transaction ![Index]( "Index")
This graph represents the time of the second register transaction in the registration use_cases, i.e. the delay between the second REGISTER and the 200 OK.
| PX_TRT-REG2 (msec) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 53.55 | 21.82 | 23.82 | 105.00 | 49.5 | 105.0 | 105.0 | 105.0 |
7.2.1 PX_TRT-REG2: Time of the second register transaction(Registration use case) (Mean per second) ![Index]( "Index")
7.3 ims_rereg : Time of the re-register transaction ![Index]( "Index")
This graph represents the time of re-register transaction i.e. the time between the REGISTER and the 200 OK.
7.4 ims_dereg : PX_TRT-REG1: Time of the first register transaction ![Index]( "Index")
This graph represents the time of the first register transaction i.e. the time between the REGISTER and the 401 Unautorized.
7.5 ims_reg : Scenario retransmissions ![Index]( "Index")
This graph represents the number of retransmissions per seconds for this scenario.
| RETRANSMIT | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | 0.0 | 0.0 |
7.5.1 Scenario retransmissions(ims_reg scenario) over time ![Index]( "Index")
7.6 ims_reg : PX_TRT-REG1: Time of the first register transaction ![Index]( "Index")
This graph represents the time of the first register transaction i.e. the time between the REGISTER and the 401 Unautorized.
| PX_TRT-REG1 (msec) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 55.02 | 28.81 | 24.85 | 106.49 | 37.2 | 106.4 | 106.4 | 106.4 |
7.6.1 PX_TRT-REG1: Time of the first register transaction(ims_reg scenario) (Mean per second) ![Index]( "Index")
8 Calling ![Index]( "Index")
8.1 PX_TRT-SES1: Session Setup Time ![Index]( "Index")
This graph represents the delay between the Caller sending INVITE and callee receiving ACK.
8.2 PX_TRT-SES2: Session Initiation transversal time ![Index]( "Index")
This graph represents the delay between the caller sending INVITE and the callee receiving INVITE.
8.3 PX_TRT-REL1: Delay Between BYE and 200 OK ![Index]( "Index")
This graph represents the delay between the first BYE and the corresponding 200 OK.
| PX_TRT-REL1 (msec) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
8.3.1 PX_TRT-REL1: Delay Between BYE and 200 OK (Calling use case) (Mean per second) ![Index]( "Index")
8.4 PX_TRT-SES3: INVITE and re-INVITE cost ![Index]( "Index")
This graph represents the caller sending first INVITE and callee receiving second ACK.
8.5 ims_uac : Scenario retransmissions ![Index]( "Index")
This graph represents the number of retransmissions per seconds for this scenario.
| RETRANSMIT | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum | Percentile 50 | Percentile 90 | Percentile 95 | Percentile 99 |
| Pre-reg | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.0 | 0.0 | 0.0 | 0.0 |
8.5.1 Scenario retransmissions(ims_uac scenario) over time ![Index]( "Index")
8.6 ims_uac : Inadequately handled scenario Percentage ![Index]( "Index")
This graph represents the percentage of Inadequately handled scenarios for the uac.
| IHS per scenario % | |||||
|---|---|---|---|---|---|
| Step | Requested Load | Mean | Standard Deviation | Minimum | Maximum |
| Pre-reg | 1 | 0.00 | 0.00 | 0.00 | 0.00 |
8.6.1 Inadequately handled scenario Percentage(ims_uac scenario) over time ![Index]( "Index")
9 Messaging ![Index]( "Index")
9.1 PX_TRT-PMM1: Message Transmission time ![Index]( "Index")
This graph represents the delay between the message and the 200 OK.
9.2 PX_TRT-PMM2: Message Transmission time (error case) ![Index]( "Index")
This graph represents the delay between the message and the 404 Not Found.
Appendix ![Index]( "Index")
The following information is also available for the test
| Parameter Name | Parameter Value | Parameter Info |
|---|---|---|
| rand_seed | 1405455923 | Value used to initialize the random number generators |
| prep_offset | 2000 | Time (ms) for scenario preparation (user reservation, etc.) prior to actual execution |
| highest_measured_time_offset | 17 | Highest time offset observed at startup between any test system and the manager (microseconds) |
| System | Command Line |
|---|---|
| TS1 | /opt/ims_bench/sipp -id 1 -i 192.168.2.11 -user_inf ./ims_users_100.inf -rmctrl 192.168.2.11:5000 192.168.2.5:4060 -trace_err -trace_cpumem -trace_scen -trace_retrans -trace_stat |
| Manager | ./manager -f managersmall.xml |
| SUT 1 | ./cpum 192.168.2.11:5000 |
------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho
_______________________________________________ Sipp-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sipp-users
