Re: [sfc-dev] CSIT verify job timing analysis

[Sam Hague]

On Wed, Jan 3, 2018 at 3:36 PM, Brady Johnson 
wrote:

>
> Hello,
>
> Recently, we had problems with the ODL SFC CSIT verify jobs timing out
> after 90 minutes. That was due mainly to problems with the testing
> infrastructure when ODL migrated to a new cloud provider. In light of those
> problems, I decided to start analyzing how long the SFC CSIT verify jobs
> take. Now the jobs are down to about 49 minutes as can be seen here [0],
> which still seems like a lot to me.
>
> I wrote a simple python script that analyzes CSIT verify log files. The
> script has a configurable input file to specify the "events" to collect
> timing data for. Basically, you just specify a start-tag and end-tag.
> Here's the config I used to generate these results [1]
>
> {
> "timing-analysis" : {
> "events" : [
> { "name"   : "Java Build",
>   "start-tag"  : "Building ODL :: sfc :: sfc",
>   "end-tag": "T E S T S"
> },
> { "name"   : "Features Build",
>   "start-tag"  : "Building ODL :: sfc :: odl",
>   "end-tag": "T E S T S"
> },
> { "name"   : "Karaf Build",
>   "start-tag"  : "Building ODL :: sfc :: sfc-karaf",
>   "end-tag": "Reactor Summary"
> },
> { "name"   : "Unit-Test",
>   "start-tag"  : "Surefire report directory:
> /w/workspace/sfc-maven-verify-oxygen-mvn33-openjdk8/sfc",
>   "end-tag": "Results :"
> },
> { "name"   : "SingleFeatureTest",
>   "start-tag"  : "Surefire report directory:
> /w/workspace/sfc-maven-verify-oxygen-mvn33-openjdk8/features",
>   "end-tag": "in org.opendaylight.odlparent.
> featuretest.SingleFeatureTest"
> }
> ]
> }
> }
>
> Here are the interesting parts from the output:
>
> Total accumulated time [0:44:26]
> Timing data for [Java Build] total-time [0:08:25]
> Timing data for [Feature Build] total-time [0:02:46]
> Timing data for [Karaf Build] total-time [0:00:40]
> Timing data for [Unit-Test] total-time [0:05:57]
> Timing data for [SingleFeatureTest] total-time [0:26:38]
>
> In the detailed output here [1] you can see that there are 3
> SingleFeatureTest runs that take [odl-sfc-genius
> 0:05:33, odl-sfc-provider-rest 0:05:42, odl-sfc-ovs 0:05:38] (about 17
> minutes for just those 3) which is much longer than the rest. I'll look
> into reducing those to see if we can get back some 10-15 minutes from the
> time it takes the verify jobs to run.
>
Yeah, the SFT is always the long pole. Would be great if we could get SFT
to run quicker across the board or all projects, but until then I think the
only thing to do is reduce dependencies in a  feature or collapse/remove
features.

>
> I'll post the code to github if anyone want to use it.
>
> [0] https://git.opendaylight.org/gerrit/#/c/65007/
> [1] https://pastebin.com/YGm2BfQU
>
>
>
> ___
> sfc-dev mailing list
> sfc-dev@lists.opendaylight.org
> https://lists.opendaylight.org/mailman/listinfo/sfc-dev
>
>
___
sfc-dev mailing list
sfc-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/sfc-dev

Re: [sfc-dev] CSIT verify job timing analysis

[Brady Johnson]

Here is the code in github:

https://github.com/bradyallenjohnson/odlUtils

Regards,

Brady


On Wed, Jan 3, 2018 at 2:36 PM Brady Johnson 
wrote:

>
> Hello,
>
> Recently, we had problems with the ODL SFC CSIT verify jobs timing out
> after 90 minutes. That was due mainly to problems with the testing
> infrastructure when ODL migrated to a new cloud provider. In light of those
> problems, I decided to start analyzing how long the SFC CSIT verify jobs
> take. Now the jobs are down to about 49 minutes as can be seen here [0],
> which still seems like a lot to me.
>
> I wrote a simple python script that analyzes CSIT verify log files. The
> script has a configurable input file to specify the "events" to collect
> timing data for. Basically, you just specify a start-tag and end-tag.
> Here's the config I used to generate these results [1]
>
> {
> "timing-analysis" : {
> "events" : [
> { "name"   : "Java Build",
>   "start-tag"  : "Building ODL :: sfc :: sfc",
>   "end-tag": "T E S T S"
> },
> { "name"   : "Features Build",
>   "start-tag"  : "Building ODL :: sfc :: odl",
>   "end-tag": "T E S T S"
> },
> { "name"   : "Karaf Build",
>   "start-tag"  : "Building ODL :: sfc :: sfc-karaf",
>   "end-tag": "Reactor Summary"
> },
> { "name"   : "Unit-Test",
>   "start-tag"  : "Surefire report directory:
> /w/workspace/sfc-maven-verify-oxygen-mvn33-openjdk8/sfc",
>   "end-tag": "Results :"
> },
> { "name"   : "SingleFeatureTest",
>   "start-tag"  : "Surefire report directory:
> /w/workspace/sfc-maven-verify-oxygen-mvn33-openjdk8/features",
>   "end-tag": "in
> org.opendaylight.odlparent.featuretest.SingleFeatureTest"
> }
> ]
> }
> }
>
> Here are the interesting parts from the output:
>
> Total accumulated time [0:44:26]
> Timing data for [Java Build] total-time [0:08:25]
> Timing data for [Feature Build] total-time [0:02:46]
> Timing data for [Karaf Build] total-time [0:00:40]
> Timing data for [Unit-Test] total-time [0:05:57]
> Timing data for [SingleFeatureTest] total-time [0:26:38]
>
> In the detailed output here [1] you can see that there are 3
> SingleFeatureTest runs that take [odl-sfc-genius
> 0:05:33, odl-sfc-provider-rest 0:05:42, odl-sfc-ovs 0:05:38] (about 17
> minutes for just those 3) which is much longer than the rest. I'll look
> into reducing those to see if we can get back some 10-15 minutes from the
> time it takes the verify jobs to run.
>
> I'll post the code to github if anyone want to use it.
>
> [0] https://git.opendaylight.org/gerrit/#/c/65007/
> [1] https://pastebin.com/YGm2BfQU
>
>
>
___
sfc-dev mailing list
sfc-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/sfc-dev