Author: jleroux
Date: Tue Jan 16 16:37:18 2018
New Revision: 1821264
URL: http://svn.apache.org/viewvc?rev=1821264&view=rev
Log:
Updates after recent Infra Jiras
Modified:
ofbiz/tools/documentation/BuildBot/BuildBot.md
ofbiz/tools/wiki-files/BuildBot/BuildBot.md.html
Modified: ofbiz/tools/documentation/BuildBot/BuildBot.md
URL:
http://svn.apache.org/viewvc/ofbiz/tools/documentation/BuildBot/BuildBot.md?rev=1821264&r1=1821263&r2=1821264&view=diff
==============================================================================
--- ofbiz/tools/documentation/BuildBot/BuildBot.md (original)
+++ ofbiz/tools/documentation/BuildBot/BuildBot.md Tue Jan 16 16:37:18 2018
@@ -21,23 +21,35 @@ OFBiz uses [BuilbBot](https://en.wikiped
## OFBiz and BuildBot
-[Here is the OFBiz BuildBot
script](https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf)
+[Here is the OFBiz BuildBot
script](https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf).
If you are interested on modifying it, please understand its content and read
the BuildBot documentation. Note that the Infra team is always there to help
you :smile:
-You can see the OFBiz builders and their states in the [common Buildbot
waterfall view](https://ci.apache.org/waterfall) (look for "ofb").
-Here are the [test results](ci.apache.org/projects/ofbiz/logs/)
-The [ofbizTrunkFramework
builder](https://ci.apache.org/builders/ofbizTrunkFramework) is triggered on
commits in the trunk framework branch. This also triggers the
[ofbizTrunkFrameworkPlugins](https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins)
to check that a commit in the framework puts no regressions in plugins.
-The same apply to the
[ofbizBranch17Framework](https://ci.apache.org/builders/ofbizBranch17Framework)
and the
[ofbizBranch17FrameworkPlugins](https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins)
+### Schedulers and Builders
+BuildBot uses schedulers and builders. Schedulers decide which builders to
trigger on svn commits.
+
+You can see the OFBiz builders and their states in the [common Buildbot
waterfall view](https://ci.apache.org/waterfall) (look for "ofb").
+
+The [ofbizTrunkFramework](https://ci.apache.org/builders/ofbizTrunkFramework)
builder is triggered on commits in the trunk framework branch. Then only the
framework integration tests are run. This also triggers the
[ofbizTrunkFrameworkPlugins](https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins)
and then the whole integration tests are run. To check that a commit in the
framework puts no regressions in plugins and especially because the
dependencies of ofbiz-framework without ofbiz-plugins is different from the
dependencies of ofbiz-framework + ofbiz-plugins (plugins components are Gradle
sub-projects). So testing needs to happen in both scenarios because you might
face library version bugs.
+Of course a commit into the trunk plugins also triggers the
ofbizTrunkFrameworkPlugins.
+
+The same apply to the
[ofbizBranch17Framework](https://ci.apache.org/builders/ofbizBranch17Framework)
and the
[ofbizBranch17FrameworkPlugins](https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins)
and next releases branches, they follows the same structure than the trunk now.
+
The current stable branch R16.12 has only [one
builder](https://ci.apache.org/builders/ofbizBranch16). This will be gone in
future. All new releases needs 2 builders.
-In an OFBiz builder page you can see [an history of the builds up to
100+](https://ci.apache.org/builders/ofbizTrunkFramework). OFBiz randomly
(except for RAT) uses 3 servers to build as you can see at the bottom of this
page. You may want to explore more to get acquainted with the tool...
+In an OFBiz builder page you can see [an history of the builds up to
100+](https://ci.apache.org/builders/ofbizTrunkFramework). OFBiz randomly
(except for RAT which always only uses lares_ubuntu) uses 3 servers to build as
you can see at the bottom of this page. You may want to explore more to get
acquainted with the tool...
There is also the [RAT
builder](https://ci.apache.org/builders/ofbizTrunkFrameworkRat) which check the
licenses in OFBiz. You can find the result
[here](https://ci.apache.org/projects/ofbiz/rat-output.html)
-## Technical information
+
+### tests results
+Here are the [test results](ci.apache.org/projects/ofbiz/logs/)
+The folders structure [will soon be
updated](https://issues.apache.org/jira/browse/INFRA-15842) to reflect the new
svn repo structure.
+
+### Technical information
When you create a new branch you need to let know BuildBot about it. This
because BuildBot uses svn hooks to triggers builds on commits. [The file to
change is
here](https://github.com/apache/infrastructure-puppet/blob/deployment/modules/subversion_server/files/hooks/buildbot_project_paths)
You can't make commit in infrastructure-puppet if you are not an infra
committer. So you need to clone the repo and make a Pull Request.
+
## Handling issues
Sometimes (rarely) you can get transient tests errors in BuildBot. This mean
tests don't all pass in BuildBot, though they pass in your local instance. In
such case, it's most certainly an issue with servers. Those are hard workers
and make errors from time to time, which shows that not only human make errors
:smirk:.
@@ -48,7 +60,6 @@ Some other errors may happen, like
* upload not working
* you name it...
-
In such case you can trigger a build from IRC to see if the problem resolves
by itself. Most of the time tests and svn issues are resolved this way. If it
does not then the best is to ask Infra help, either on [HipChat infra
room](https://apache.hipchat.com/chat/room/669587) or through the [service
desk](https://issues.apache.org/jira/servicedesk/customer/portal/1/create/3)
When something like that happens, I get to IRC (using
https://webchat.freenode.net). There using a recognisable username (I use
jleroux) I get to the ofbiz channel. I wait for ofbiz-bot to appear, click on
it, click on query and then I can make a request in the chat line to restart a
scheduler.
@@ -64,7 +75,7 @@ So the whole request is of the form
force build ofbizTrunkFramework forces manual build after weird error
-Note though that with our last config (see
[INFRA-15394](https://issues.apache.org/jira/browse/INFRA-15394)) the
ofbizTrunkFrameworkPlugins and ofbizBranch17FrameworkPlugins builders are
dependent and respectively automatically launched by the ofbizTrunkFramework
and ofbizBranch17Framework builders but only on commits. So if you use an IRC
command like `force build ofbizTrunkFramework` only this builder will be
launched not the dependent ofbizTrunkFrameworkPlugins.
+Note though that with our last config (see
[INFRA-15394](https://issues.apache.org/jira/browse/INFRA-15394)) the
ofbizTrunkFrameworkPlugins and ofbizBranch17FrameworkPlugins builders are
dependent and respectively automatically launched by the ofbizTrunkFramework
and ofbizBranch17Framework builders but only on commits. So if you use an IRC
command like `force build ofbizTrunkFramework` only this builder will be
launched not the dependent ofbizTrunkFrameworkPlugins. We can't call a sheduler
from IRC. It needs a svn commit.
### Randon conflicts on port 8080 during tests
One case which comes back from time to time is a conflit on port 8080 due to
the automatic startup of tomcat. It's due to security patches being applied on
Silvanus (one of 3 the servers BuildBot uses for OFBiz, hence the random
aspect, only Sylvanus is concerned). In such case we need to ask infra to
manually disable Tomcat on Silvanus. This happened 4th already, last case was
[INFRA-15829](https://issues.apache.org/jira/browse/INFRA-15829)) where things
are best explained.
\ No newline at end of file
Modified: ofbiz/tools/wiki-files/BuildBot/BuildBot.md.html
URL:
http://svn.apache.org/viewvc/ofbiz/tools/wiki-files/BuildBot/BuildBot.md.html?rev=1821264&r1=1821263&r2=1821264&view=diff
==============================================================================
--- ofbiz/tools/wiki-files/BuildBot/BuildBot.md.html (original)
+++ ofbiz/tools/wiki-files/BuildBot/BuildBot.md.html Tue Jan 16 16:37:18 2018
@@ -29,12 +29,18 @@ under the License.
<h2 id="what-is-builbbot-and-why-using-it">What is BuilbBot and why using
it?</h2>
<p>OFBiz uses <a href="https://en.wikipedia.org/wiki/Buildbot";>BuilbBot</a> as
its Continuous Integration (CI) tool. There are other CI tools <a
href="https://ci.apache.org/";>used at the ASF</a>.</p>
<h2 id="ofbiz-and-buildbot">OFBiz and BuildBot</h2>
-<p><a
href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf";>Here
is the OFBiz BuildBot script</a></p>
-<p>You can see the OFBiz builders and their states in the <a
href="https://ci.apache.org/waterfall";>common Buildbot waterfall view</a> (look
for "ofb"). Here are the <a
href="ci.apache.org/projects/ofbiz/logs/">test results</a></p>
-<p>The <a
href="https://ci.apache.org/builders/ofbizTrunkFramework";>ofbizTrunkFramework
builder</a> is triggered on commits in the trunk framework branch. This also
triggers the <a
href="https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins";>ofbizTrunkFrameworkPlugins</a>
to check that a commit in the framework puts no regressions in plugins. The
same apply to the <a
href="https://ci.apache.org/builders/ofbizBranch17Framework";>ofbizBranch17Framework</a>
and the <a
href="https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins";>ofbizBranch17FrameworkPlugins</a>
The current stable branch R16.12 has only <a
href="https://ci.apache.org/builders/ofbizBranch16";>one builder</a>. This will
be gone in future. All new releases needs 2 builders.</p>
-<p>In an OFBiz builder page you can see <a
href="https://ci.apache.org/builders/ofbizTrunkFramework";>an history of the
builds up to 100+</a>. OFBiz randomly (except for RAT) uses 3 servers to build
as you can see at the bottom of this page. You may want to explore more to get
acquainted with the tool...</p>
+<p><a
href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf";>Here
is the OFBiz BuildBot script</a>. If you are interested on modifying it,
please understand its content and read the BuildBot documentation. Note that
the Infra team is always there to help you :smile:</p>
+<h3 id="schedulers-and-builders">Schedulers and Builders</h3>
+<p>BuildBot uses schedulers and builders. Schedulers decide which builders to
trigger on svn commits.</p>
+<p>You can see the OFBiz builders and their states in the <a
href="https://ci.apache.org/waterfall";>common Buildbot waterfall view</a> (look
for "ofb").</p>
+<p>The <a
href="https://ci.apache.org/builders/ofbizTrunkFramework";>ofbizTrunkFramework</a>
builder is triggered on commits in the trunk framework branch. Then only the
framework integration tests are run. This also triggers the <a
href="https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins";>ofbizTrunkFrameworkPlugins</a>
and then the whole integration tests are run. To check that a commit in the
framework puts no regressions in plugins and especially because the
dependencies of ofbiz-framework without ofbiz-plugins is different from the
dependencies of ofbiz-framework + ofbiz-plugins (plugins components are Gradle
sub-projects). So testing needs to happen in both scenarios because you might
face library version bugs. Of course a commit into the trunk plugins also
triggers the ofbizTrunkFrameworkPlugins.</p>
+<p>The same apply to the <a
href="https://ci.apache.org/builders/ofbizBranch17Framework";>ofbizBranch17Framework</a>
and the <a
href="https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins";>ofbizBranch17FrameworkPlugins</a>
and next releases branches, they follows the same structure than the trunk
now.</p>
+<p>The current stable branch R16.12 has only <a
href="https://ci.apache.org/builders/ofbizBranch16";>one builder</a>. This will
be gone in future. All new releases needs 2 builders.</p>
+<p>In an OFBiz builder page you can see <a
href="https://ci.apache.org/builders/ofbizTrunkFramework";>an history of the
builds up to 100+</a>. OFBiz randomly (except for RAT which always only uses
lares_ubuntu) uses 3 servers to build as you can see at the bottom of this
page. You may want to explore more to get acquainted with the tool...</p>
<p>There is also the <a
href="https://ci.apache.org/builders/ofbizTrunkFrameworkRat";>RAT builder</a>
which check the licenses in OFBiz. You can find the result <a
href="https://ci.apache.org/projects/ofbiz/rat-output.html";>here</a></p>
-<h2 id="technical-information">Technical information</h2>
+<h3 id="tests-results">tests results</h3>
+<p>Here are the <a href="ci.apache.org/projects/ofbiz/logs/">test results</a>
The folders structure <a
href="https://issues.apache.org/jira/browse/INFRA-15842";>will soon be
updated</a> to reflect the new svn repo structure.</p>
+<h3 id="technical-information">Technical information</h3>
<p>When you create a new branch you need to let know BuildBot about it. This
because BuildBot uses svn hooks to triggers builds on commits. <a
href="https://github.com/apache/infrastructure-puppet/blob/deployment/modules/subversion_server/files/hooks/buildbot_project_paths";>The
file to change is here</a> You can't make commit in infrastructure-puppet if
you are not an infra committer. So you need to clone the repo and make a Pull
Request.</p>
<h2 id="handling-issues">Handling issues</h2>
<p>Sometimes (rarely) you can get transient tests errors in BuildBot. This
mean tests don't all pass in BuildBot, though they pass in your local instance.
In such case, it's most certainly an issue with servers. Those are hard workers
and make errors from time to time, which shows that not only human make errors
:smirk:.</p>
@@ -47,7 +53,7 @@ under the License.
<pre><code>forces manual build after weird error</code></pre>
<p>So the whole request is of the form</p>
<pre><code>force build ofbizTrunkFramework forces manual build after weird
error</code></pre>
-<p>Note though that with our last config (see <a
href="https://issues.apache.org/jira/browse/INFRA-15394";>INFRA-15394</a>) the
ofbizTrunkFrameworkPlugins and ofbizBranch17FrameworkPlugins builders are
dependent and respectively automatically launched by the ofbizTrunkFramework
and ofbizBranch17Framework builders but only on commits. So if you use an IRC
command like <code>force build ofbizTrunkFramework</code> only this builder
will be launched not the dependent ofbizTrunkFrameworkPlugins.</p>
+<p>Note though that with our last config (see <a
href="https://issues.apache.org/jira/browse/INFRA-15394";>INFRA-15394</a>) the
ofbizTrunkFrameworkPlugins and ofbizBranch17FrameworkPlugins builders are
dependent and respectively automatically launched by the ofbizTrunkFramework
and ofbizBranch17Framework builders but only on commits. So if you use an IRC
command like <code>force build ofbizTrunkFramework</code> only this builder
will be launched not the dependent ofbizTrunkFrameworkPlugins. We can't call a
sheduler from IRC. It needs a svn commit.</p>
<h3 id="randon-conflicts-on-port-8080-during-tests">Randon conflicts on port
8080 during tests</h3>
<p>One case which comes back from time to time is a conflit on port 8080 due
to the automatic startup of tomcat. It's due to security patches being applied
on Silvanus (one of 3 the servers BuildBot uses for OFBiz, hence the random
aspect, only Sylvanus is concerned). In such case we need to ask infra to
manually disable Tomcat on Silvanus. This happened 4th already, last case was
<a href="https://issues.apache.org/jira/browse/INFRA-15829";>INFRA-15829</a>)
where things are best explained.</p>
</body>
