Okay great! Now the next question. Does buildbot fail on simultaneous commits to the same branch or for different branches only? Meaning if we commit at the same time to trunk it's okay but if we commit to trunk and 16.11 then it fails?. I am asking because I assume only one build for one branch happens at a time. Is this correct?
Cheers, Taher Alkhateeb On Mar 19, 2017 3:28 PM, "Jacques Le Roux" <jacques.le.r...@les7arts.com> wrote: > Yes, that should do it > > Jacques > > > Le 19/03/2017 à 13:01, Taher Alkhateeb a écrit : > >> Okay, so, to be absolutely exact, the repeat behavior is the following: >> >> - checkout ofbiz r1744041 >> - ant clean-all load-demo >> - ant start >> - open a new terminal >> - ant run-tests >> - observe tests run successfully >> >> And then >> >> - checkout ofbiz r1744107 >> - ant clean-all load-demo >> - ant start >> - open a new terminal >> - ant run-tests >> - observe tests fail with the message Couldn't create server socket(/ >> 127.0.0.1:10523) (Address already in use: JVM_Bind)Start.java >> >> Is this correct? >> >> On Sun, Mar 19, 2017 at 2:47 PM, Jacques Le Roux < >> jacques.le.r...@les7arts.com> wrote: >> >> yep, you got it. But you also need an OFBiz instance running (previously) >>> to reproduce, of course! >>> >>> Jacques >>> >>> >>> Le 19/03/2017 à 12:37, Taher Alkhateeb a écrit : >>> >>> Sorry, I'm still lost, trying to explain it again: >>>> >>>> Before the commit: "ant clean-all load-demo run-tests" works >>>> after the commit: "ant clean-all load-demo run-tests" does not work with >>>> the socket error message? >>>> >>>> Is this now what you mean? >>>> >>>> >>>> On Sun, Mar 19, 2017 at 2:32 PM, Jacques Le Roux < >>>> jacques.le.r...@les7arts.com> wrote: >>>> >>>> It's quite simple, I repeat. >>>> >>>>> It's not specifically related with Buildbot despite the title. It was >>>>> 1st >>>>> identified there. >>>>> >>>>> If you run an OFBiz instance (I ran one at the "beforeGradle" tag, but >>>>> we >>>>> know any commit version fits since it's currently reproducible on >>>>> Buildbot) >>>>> >>>>> Before your commit (r1744107 in disappeared trunk) below (ie using the >>>>> last previous commit, to be even more clear r1744041) "ant clean-all >>>>> load-demo run-tests" works >>>>> >>>>> After (ie using r1744107) it fails with the error reported below. I >>>>> repeat >>>>> it here in case >>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------------------------------------------------ >>>>> ------------------------------ >>>>> >>>>> run-tests: >>>>> [java] org.ofbiz.base.start.StartupException: Couldn't create >>>>> server >>>>> socket(/127.0.0.1:10523) (Address already in use: JVM_Bind)Start.java >>>>> using configuration file org/ofbiz/base/start/test.properties >>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------------------------------------------------ >>>>> ------------------------------ >>>>> >>>>> I hope it's now clear. And yes it's in the commit and we need to find >>>>> where! >>>>> >>>>> You asked me to answer on HipChat, sorry I'll not, I must move now... >>>>> >>>>> HTH >>>>> >>>>> Jacques >>>>> >>>>> >>>>> Le 19/03/2017 à 12:00, Taher Alkhateeb a écrit : >>>>> >>>>> The commit in question is not important because there are too many >>>>> >>>>>> changes. >>>>>> >>>>>> What we need is a "repeat" behavior. So my questions from this thread >>>>>> remain to be answered before we can investigate. >>>>>> >>>>>> On Sun, Mar 19, 2017 at 1:53 PM, Jacques Le Roux < >>>>>> jacques.le.r...@les7arts.com> wrote: >>>>>> >>>>>> Here it is >>>>>> >>>>>> ------------------------------------------------------------ >>>>>>> ------------------------------------------------------------ >>>>>>> ------------------------------ >>>>>>> >>>>>>> git.exe bisect good >>>>>>> >>>>>>> 298c831a3aa1cea45b4aab66acebd9617f6615e0 is the first bad commit >>>>>>> commit 298c831a3aa1cea45b4aab66acebd9617f6615e0 >>>>>>> Author: Taher A. Alkhateeb <ta...@apache.org> >>>>>>> Date: Mon May 16 18:40:49 2016 +0000 >>>>>>> >>>>>>> major change in the start component related to OFBIZ-6783 >>>>>>> >>>>>>> this is a big commit that achieves the following new features in >>>>>>> ofbiz: >>>>>>> >>>>>>> - add the commons-cli library >>>>>>> - fix build.xml in start menu to include the commons-cli. It is done >>>>>>> in a way to ensure that the classpath continues to work when >>>>>>> ofbiz.jar >>>>>>> is copied to the root folder >>>>>>> - set a default portoffset value of 0 when not selected in build.xml >>>>>>> - fully redefine the commands passed in java -jar ofbiz.jar using the >>>>>>> commons-cli >>>>>>> api. The commands are now much more consistent and clean >>>>>>> - remove ofbiz dependency on String[] args across the different >>>>>>> components >>>>>>> and isolate >>>>>>> them in a new entity class called StartupCommand. This decouples >>>>>>> ofbiz >>>>>>> from both >>>>>>> String[] args and commons-cli at the same time >>>>>>> - fix all the calls in the master build.xml to comply with the new >>>>>>> commands >>>>>>> - fix the Config.java to remove dependecies on the args array >>>>>>> - create a utility class StartupCommandUtil that takes care of all >>>>>>> commons-cli >>>>>>> operations and abstracts away the implementation in private methods >>>>>>> - substantially reduce the size of main and init in Start.java by >>>>>>> refactoring >>>>>>> them in different places >>>>>>> - create an intermediate function called populateLoaderArgs. This is >>>>>>> the >>>>>>> first >>>>>>> step in removing the dependecies on args by "adapting" them using >>>>>>> this >>>>>>> method >>>>>>> - unify the exception model everywhere to StartupException. This >>>>>>> makes >>>>>>> exception >>>>>>> propagation cleaner and easier >>>>>>> - lots of cleanup of the code related in all these areas >>>>>>> >>>>>>> Big thanks to Jacques for the substantial help in testing. >>>>>>> >>>>>>> >>>>>>> git-svn-id: https://svn.apache.org/repos/asf/ofbiz/trunk@1744107 >>>>>>> 13f79535-47bb-0310-9956-ffa450edef68 >>>>>>> >>>>>>> :100644 100644 e7ffd6978ec0b072e73fa128ec9e2e83ba81524a >>>>>>> 74787e5942a4d0bae826bb06c15a19e24066fc30 M .classpath >>>>>>> :100644 100644 042fda025e68f8f86c359abe70b87b7138c31133 >>>>>>> 5053165b4a75e71d9f63768a84ff5351afe783a5 M build.xml >>>>>>> :040000 040000 31fa9e03d92cc7629cba86898c34da77a7a88ace >>>>>>> bb1541c40aa7f92ef5b4f125dc415eb394e1f93f M framework >>>>>>> >>>>>>> Success (422 ms @ 19/03/2017 11:51:29) >>>>>>> >>>>>>> ------------------------------------------------------------ >>>>>>> ------------------------------------------------------------ >>>>>>> ------------------------------ >>>>>>> >>>>>>> Now we need to find where it is exactly... >>>>>>> >>>>>>> Jacques >>>>>>> >>>>>>> >>>>>>> >>>>>>> Le 19/03/2017 à 10:53, Taher Alkhateeb a écrit : >>>>>>> >>>>>>> Hi Jacques, >>>>>>> >>>>>>> Sorry I still don't understand, let me try to capture this. >>>>>>>> >>>>>>>> Scenario 1 - Using ant >>>>>>>> - Start OFBiz using: ant start >>>>>>>> - Run tests in another OFBiz using: ant run-tests >>>>>>>> - OFBiz fails with an error message >>>>>>>> >>>>>>>> Scenario 2 - Using gradle >>>>>>>> - Start OFBiz using: ./gradlew ofbiz >>>>>>>> - Run tests in another OFBiz using: ./gradlew "ofbiz --test" >>>>>>>> - OFBiz freezes? >>>>>>>> >>>>>>>> Is this correct? Is this what you are witnessing? >>>>>>>> >>>>>>>> On Sun, Mar 19, 2017 at 12:06 PM, Jacques Le Roux < >>>>>>>> jacques.le.r...@les7arts.com> wrote: >>>>>>>> >>>>>>>> Hi Taher, >>>>>>>> >>>>>>>> Le 19/03/2017 à 04:18, Taher Alkhateeb a écrit : >>>>>>>> >>>>>>>>> Hi Jacques, >>>>>>>>> >>>>>>>>> Okay just to try and understand your point, what are the >>>>>>>>> >>>>>>>>>> circumstances >>>>>>>>>> in >>>>>>>>>> which you are witnessing failure in post gradle but not >>>>>>>>>> pre-gradle? >>>>>>>>>> >>>>>>>>>> Fortunately it's now simple. >>>>>>>>>> >>>>>>>>>> You run an OFBiz instance with "ant start" >>>>>>>>>> >>>>>>>>> You run tests with the same (or another) OFBiz instance with "ant >>>>>>>>> run-tests", bingo the port 10523 is already used tests stop with >>>>>>>>> the >>>>>>>>> error >>>>>>>>> below >>>>>>>>> >>>>>>>>> In other words what is the repeat process, the expected outcome and >>>>>>>>> actual >>>>>>>>> >>>>>>>>> outcome. I ask because I'm a bit confused not sure if you're >>>>>>>>> referring >>>>>>>>> >>>>>>>>> to >>>>>>>>>> web ports or admin ports pr something else? >>>>>>>>>> >>>>>>>>>> Any port used by OFBiz I guess. But the tests fails on the the 1st >>>>>>>>>> >>>>>>>>>> called >>>>>>>>>> >>>>>>>>> port: 10523 by default. >>>>>>>>> We got issues with the port 8080 on Buildbot so it was confusing. >>>>>>>>> Those >>>>>>>>> were due to Buildbot global config (not specific to OFBiz Buildbot >>>>>>>>> config) >>>>>>>>> ans has been fixed by Infra. >>>>>>>>> >>>>>>>>> Jacques >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Taher Alkhateeb >>>>>>>>> >>>>>>>>>> On Mar 18, 2017 10:24 PM, "Jacques Le Roux" < >>>>>>>>>> jacques.le.r...@les7arts.com >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> OK, I locally kept a version of the trunk just prior the Gradle >>>>>>>>>> switch. >>>>>>>>>> >>>>>>>>>> When running an OFBiz instance (any one fits, I used the >>>>>>>>>> >>>>>>>>>> ofbiz-framework+plugin HEAD) you can run tests with R15, but not >>>>>>>>>>> with >>>>>>>>>>> this >>>>>>>>>>> preGradle version. You get >>>>>>>>>>> >>>>>>>>>>> run-tests: >>>>>>>>>>> [java] org.ofbiz.base.start.StartupException: Couldn't >>>>>>>>>>> create >>>>>>>>>>> server >>>>>>>>>>> socket(/127.0.0.1:10523) (Address already in use: >>>>>>>>>>> JVM_Bind)Start.java >>>>>>>>>>> using configuration file org/ofbiz/base/start/test.properties >>>>>>>>>>> >>>>>>>>>>> So the problem is anterior the Gradle switch and after the R15 >>>>>>>>>>> freezing. >>>>>>>>>>> >>>>>>>>>>> But unfortunately we no longer have a trunk to get back into >>>>>>>>>>> commits. >>>>>>>>>>> Fortunately we have https://github.com/apache/ofbiz >>>>>>>>>>> >>>>>>>>>>> I'll try that :) >>>>>>>>>>> >>>>>>>>>>> Jacques >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Le 18/03/2017 à 17:43, Jacques Le Roux a écrit : >>>>>>>>>>> >>>>>>>>>>> Le 18/03/2017 à 17:07, Taher Alkhateeb a écrit : >>>>>>>>>>> >>>>>>>>>>> I think we discussed this in the past, but I will ask again for >>>>>>>>>>> >>>>>>>>>>> clarity >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> what does gradle have to do with port blocking? >>>>>>>>>>>>> >>>>>>>>>>>>> For now I have no clear evidences. Only that before the Gradle >>>>>>>>>>>>> move >>>>>>>>>>>>> we >>>>>>>>>>>>> >>>>>>>>>>>>> had not the ports issues (locally and on Buildbot) I describe >>>>>>>>>>>>> in >>>>>>>>>>>>> >>>>>>>>>>>>> this >>>>>>>>>>>> Jira. >>>>>>>>>>>> We need to dig deeper in this and localise the commit which >>>>>>>>>>>> introduced >>>>>>>>>>>> this >>>>>>>>>>>> error. >>>>>>>>>>>> In the meantime better not committing in parallel in the trunk >>>>>>>>>>>> and >>>>>>>>>>>> R16 >>>>>>>>>>>> if >>>>>>>>>>>> we want to avoid the ports conflicts on Buildbot >>>>>>>>>>>> >>>>>>>>>>>> How can gradle or any build tool be responsible for blocking >>>>>>>>>>>> ports >>>>>>>>>>>> that >>>>>>>>>>>> >>>>>>>>>>>> are >>>>>>>>>>>> >>>>>>>>>>>> not related to the build system but rather used from OFBiz >>>>>>>>>>>>> directly? >>>>>>>>>>>>> >>>>>>>>>>>>> It could be code changes while refactoring, still a >>>>>>>>>>>>> supposition. >>>>>>>>>>>>> You >>>>>>>>>>>>> >>>>>>>>>>>>> spoke about Config.java, at 1st glance I saw nothing there. >>>>>>>>>>>>> >>>>>>>>>>>>> Jacques >>>>>>>>>>>> >>>>>>>>>>>> On Mar 18, 2017 7:00 PM, "Jacques Le Roux" < >>>>>>>>>>>> >>>>>>>>>>>> jacques.le.r...@les7arts.com >>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> It's clearly related with OFBIZ-9196 "Regression: the >>>>>>>>>>>>> testIntegration >>>>>>>>>>>>> >>>>>>>>>>>>> Gradle taks shoud not use/block the ports" >>>>>>>>>>>>> >>>>>>>>>>>>> The build in trunk and R16 overlap in time: >>>>>>>>>>>>> >>>>>>>>>>>>>> https://ci.apache.org/builders/ofbiz-trunk-framework/builds/ >>>>>>>>>>>>>> 44/ >>>>>>>>>>>>>> Start Sat Mar 18 09:25:12 2017 >>>>>>>>>>>>>> End Sat Mar 18 09:26:56 2017 >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://ci.apache.org/builders/ofbiz-branch16/builds/17 >>>>>>>>>>>>>> Start Sat Mar 18 09:25:30 2017 >>>>>>>>>>>>>> End Sat Mar 18 09:30:44 2017 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Jacques >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Le 18/03/2017 à 15:01, Taher Alkhateeb a écrit : >>>>>>>>>>>>>> >>>>>>>>>>>>>> The problem after a second review of the logs is pretty clear. >>>>>>>>>>>>>> You >>>>>>>>>>>>>> have >>>>>>>>>>>>>> >>>>>>>>>>>>>> two >>>>>>>>>>>>>> >>>>>>>>>>>>>> instances of OFBiz running and conflicting on ports. Either >>>>>>>>>>>>>> the >>>>>>>>>>>>>> >>>>>>>>>>>>>>> first >>>>>>>>>>>>>>> build >>>>>>>>>>>>>>> did not terminate while the second starts or shutdown is not >>>>>>>>>>>>>>> occurring >>>>>>>>>>>>>>> cleanly. In other words, if you start ./gradlew "ofbiz >>>>>>>>>>>>>>> load-data" >>>>>>>>>>>>>>> && >>>>>>>>>>>>>>> ./gradlew "ofbiz load-data" then it would work correctly >>>>>>>>>>>>>>> because >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> first >>>>>>>>>>>>>>> scripts terminates with exit code 0 before the second one >>>>>>>>>>>>>>> begins. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Most likely a buildbot script issue. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Mar 18, 2017 at 4:24 PM, Taher Alkhateeb < >>>>>>>>>>>>>>> slidingfilame...@gmail.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ctrl-C does not kill gradle and gradle does not need time to >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> release >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> resources >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mar 18, 2017 4:21 PM, "Jacques Le Roux" < >>>>>>>>>>>>>>>> jacques.le.r...@les7arts.com >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks James, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yes indeed, it takes a bit for Gradle, and especially its >>>>>>>>>>>>>>>> wrapper, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> disconnect resources. I noticed that too on Windows, thought >>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>> specific to Windows . >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This could be the reason, though we don't use the wrapper >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> Buildbot. >>>>>>>>>>>>>>>>> We >>>>>>>>>>>>>>>>> could try to add a small pause (say 30 seconds) before >>>>>>>>>>>>>>>>> launching >>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>> build. >>>>>>>>>>>>>>>>> Because we (especially me) sometimes launch Buildbot builds >>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> burst >>>>>>>>>>>>>>>>> when >>>>>>>>>>>>>>>>> backporting. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> But it also happens for isolated build so I fear it's not >>>>>>>>>>>>>>>>> enough >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Jacques >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Le 18/03/2017 à 13:41, james yong a écrit : >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I got this issue too when I pressed CTL-C to exit OFBiz and >>>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>> ran >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> "./gradew ofbiz" immediately. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Jacques Le Roux wrote >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Mmm, this is pretty bad >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://ci.apache.org/builders >>>>>>>>>>>>>>>>>> /ofbiz-trunk-framework/builds/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 44/steps/shell_1/logs/stdio >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Building 84% > :ofbiz --load-dataSet OFBIZ_HOME to - >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> /home/buildslave32/slave32/ofbiz-trunk-framework/build >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> org.apache.ofbiz.base.start.StartupException: Couldn't >>>>>>>>>>>>>>>>>>> create >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> server >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> socket(/127.0.0.1:10523) (Address already in use (Bind >>>>>>>>>>>>>>>>>>>> failed)) >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> org.apache.ofbiz.base.start.AdminServer. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> <init> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> (AdminServer.java:56) Something is >>>>>>>>>>>>>>>>>>> wrong with Buildbot. I already asked infra about several >>>>>>>>>>>>>>>>>>> instances >>>>>>>>>>>>>>>>>>> running >>>>>>>>>>>>>>>>>>> at the same time, I don't know much yet... Jacques >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Le 18/03/2017 à 10:26, >>>>>>>>>>>>>>>>>>> buildbot@ >>>>>>>>>>>>>>>>>>> a écrit : >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The Buildbot has detected a build exception on builder >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ofbiz-trunk-framework while building . Full details are >>>>>>>>>>>>>>>>>>> available >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> at: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> https://ci.apache.org/builders/ofbiz-trunk-framework/ >>>>>>>>>>>>>>>>>>>> builds/44 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Buildbot URL: https://ci.apache.org/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Buildslave for this Build: silvanus_ubuntu >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Build Reason: The AnyBranchScheduler scheduler named >>>>>>>>>>>>>>>>>>>> 'on-ofbiz-framework-commit' triggered this build >>>>>>>>>>>>>>>>>>>> Build Source Stamp: [branch ofbiz/ofbiz-framework/trunk] >>>>>>>>>>>>>>>>>>>> 1787535 >>>>>>>>>>>>>>>>>>>> Blamelist: jleroux >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> BUILD FAILED: exception shell_1 upload_1 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Sincerely, >>>>>>>>>>>>>>>>>>>> -The Buildbot >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> View this message in context: >>>>>>>>>>>>>>>>>>>> http://ofbiz.135035.n4.nabble >>>>>>>>>>>>>>>>>>>> . >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> com/buildbot-exception-in-on-o >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> fbiz-trunk-framework-tp4703602 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> p4703620.html >>>>>>>>>>>>>>>>>> Sent from the OFBiz - Dev mailing list archive at >>>>>>>>>>>>>>>>>> Nabble.com. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >
