----- Original Message ----- > From: "Alon Bar-Lev" <alo...@redhat.com> > To: "Sandro Bonazzola" <sbona...@redhat.com> > Cc: "Vojtech Szocs" <vsz...@redhat.com>, "Juan Hernandez" > <jhern...@redhat.com>, "Kiril Nesenko" > <knese...@redhat.com>, "arch" <arch@ovirt.org> > Sent: Wednesday, February 26, 2014 12:03:08 PM > Subject: Re: [urgent] Re: ovirt-engine build failure > > > > ----- Original Message ----- > > From: "Sandro Bonazzola" <sbona...@redhat.com> > > To: "Vojtech Szocs" <vsz...@redhat.com> > > Cc: "Juan Hernandez" <jhern...@redhat.com>, "Kiril Nesenko" > > <knese...@redhat.com>, "arch" <arch@ovirt.org> > > Sent: Wednesday, February 26, 2014 12:58:29 PM > > Subject: Re: [urgent] Re: ovirt-engine build failure > > > > Il 26/02/2014 11:51, Vojtech Szocs ha scritto: > > > > > > > > > ----- Original Message ----- > > >> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >> To: "Tomas Jelinek" <tjeli...@redhat.com> > > >> Cc: "Vojtech Szocs" <vsz...@redhat.com>, "Juan Hernandez" > > >> <jhern...@redhat.com>, "Kiril Nesenko" > > >> <knese...@redhat.com>, "arch" <arch@ovirt.org> > > >> Sent: Wednesday, February 26, 2014 10:22:22 AM > > >> Subject: Re: [urgent] Re: ovirt-engine build failure > > >> > > >> Il 26/02/2014 09:06, Tomas Jelinek ha scritto: > > >>> Hey, > > >>> > > >>> from this logs it seems that the root cause is: > > >>> [ERROR] java.io.FileNotFoundException: > > >>> /home/jenkins/workspace/manual-build-tarball/label/centos6-host/rpmbuild/BUILD/ovirt-engine-3.5.0/frontend/webadmin/modules/webadmin/target/tmp/ImageResourceGenerator4201639982561998434.png > > >>> (Too many open files) > > >>> > > >>> e.g. the OS does not allow to open all the files the gwt compilation > > >>> needs. > > >>> Could you please try to increase the limit of max opened files on the > > >>> host > > >>> and try to rebuild? > > >> > > >> David is increasing max open files. We'll give it another run. > > >> I really think that having to reduce localWorkers to 1 for having decent > > >> build requirements on RAM and still hit too many opened files is really > > >> bad. > > >> Any chance of lowering build requirements in terms of RAM and opened > > >> files? > > > > > > In general, GWT compiler generates/reads lots of intermediate files > > > during > > > its runtime. IIRC, this file amount is proportional to number of > > > permutations > > > being compiled, i.e. num_of_browsers x num_of_locales. This is why we > > > don't > > > hit "Too many open files" in small-scale builds (i.e. development) but we > > > do > > > hit it on large-scale builds (i.e. all_browsers x all_locales nightly > > > build). > > > > > > But is increasing max open files for GWT compiler really that big of a > > > deal? > > > (i.e. we know that compiler needs to process lots of files per each > > > permutation) > > > > No, it's not that big of a deal *for our packaging* but since we've people > > on > > Gentoo building engine > > on their own systems, it may impact all of them... > > We can have clear note that limits should be modified to support locale > build.
+1 Maybe this (or similar): "When building Engine for multiple/all locales, please check max open files limit to avoid errors during GWT compilation." > > We can run limit -n at Makefile and fail. I'm not sure we can get good-enough/accurate "n" value since this depends on many different factors (including number of small images used in GWT application), so I'd just vote for clear note as suggested above. > > > > > > > > > Alternatively, we could implement "remote GWT worker grid" -> offload > > > compilation > > > for each specific (browser x locale) permutation to a remote machine, > > > this > > > would > > > require some extra Java implementation though. > > > > > > I agree on excessive RAM usage, I'm currently experimenting with some > > > compiler > > > options, I'll share results when I have something. > > > > > > Another approach we could take (something I'll discuss with my peers) is > > > the > > > idea of eliminating locale vector entirely -> re-implement i18n support > > > in > > > UI > > > in a way that for X browsers, there will be only X total permutations > > > (i.e. > > > not X x num_of_locales). This would reduce RAM usage && compilation time > > > && > > > perhaps max files opened too. > > > > +1 on whatever can reduce build time and build requirements. > > > > > > > > > >> > > >> > > >>> > > >>> ----- Original Message ----- > > >>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>> Cc: "Juan Hernandez" <jhern...@redhat.com>, "Kiril Nesenko" > > >>>> <knese...@redhat.com>, "arch" <arch@ovirt.org> > > >>>> Sent: Wednesday, February 26, 2014 8:37:07 AM > > >>>> Subject: Re: [urgent] Re: ovirt-engine build failure > > >>>> > > >>>> Il 25/02/2014 16:52, Vojtech Szocs ha scritto: > > >>>>> > > >>>>> > > >>>>> ----- Original Message ----- > > >>>>>> From: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>> To: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>> Cc: "Juan Hernandez" <jhern...@redhat.com>, "Kiril Nesenko" > > >>>>>> <knese...@redhat.com>, "arch" <arch@ovirt.org> > > >>>>>> Sent: Tuesday, February 25, 2014 1:05:35 PM > > >>>>>> Subject: Re: [urgent] Re: ovirt-engine build failure > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> ----- Original Message ----- > > >>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>> Cc: "Juan Hernandez" <jhern...@redhat.com>, "arch" > > >>>>>>> <arch@ovirt.org>, > > >>>>>>> "Kiril > > >>>>>>> Nesenko" <knese...@redhat.com> > > >>>>>>> Sent: Tuesday, February 25, 2014 9:02:11 AM > > >>>>>>> Subject: Re: [urgent] Re: ovirt-engine build failure > > >>>>>>> > > >>>>>>> Il 25/02/2014 08:47, Sandro Bonazzola ha scritto: > > >>>>>>>> Il 24/02/2014 18:28, Vojtech Szocs ha scritto: > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ----- Original Message ----- > > >>>>>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > > >>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>> Sent: Monday, February 24, 2014 9:08:09 AM > > >>>>>>>>>> Subject: [urgent] Re: ovirt-engine build failure > > >>>>>>>>>> > > >>>>>>>>>> Top posting, > > >>>>>>>>>> > > >>>>>>>>>> Please give this task really urgent priority, as you can see: > > >>>>>>>>>> > > >>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-engine_3.4_create_rpms/ > > >>>>>>>>>> and > > >>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-engine_master_create_rpms/ > > >>>>>>>>>> > > >>>>>>>>>> are failing consistently so we don't have nightly builds. > > >>>>>>>>> > > >>>>>>>>> (CC'ing Einav) > > >>>>>>>>> > > >>>>>>>>> Attached TEST patch, please apply & rebuild ovirt-engine as > > >>>>>>>>> usual. > > >>>>>>>>> > > >>>>>>>>> This patch should cause following output during GWT compilation: > > >>>>>>>>> > > >>>>>>>>> [ERROR] ######### Util.readURLAsBytes > > >>>>>>>>> file:/home/vszocs/work/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt/org/ovirt/engine/ui/webadmin/images/search_button.png > > >>>>>>>>> > > >>>>>>>>> If IOException occurs, exception stacktrace will be printed right > > >>>>>>>>> below > > >>>>>>>>> that line. > > >>>>>>>>> > > >>>>>>>>> Note: please ignore the [ERROR] prefix, it's a technical issue, > > >>>>>>>>> more > > >>>>>>>>> details at > > >>>>>>>>> https://www.mail-archive.com/engine-devel@ovirt.org/msg05735.html > > >>>>>>>>> > > >>>>>>>>> I just did a local upstream build with TEST patch applied, GWT > > >>>>>>>>> compilation > > >>>>>>>>> passed OK. > > >>>>>>>> > > >>>>>>>> build job running with your patch here: > > >>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/235/ > > >>>>>>>> waiting for a free executor right now. > > >>>>>>>> Let's see how it goes. > > >>>>>>> > > >>>>>>> Failed on CentOS: > > >>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/235/label=centos6-host/consoleFull > > >>>>>> > > >>>>>> Hm, in that ^^ build, following exceptions occured: > > >>>>>> > > >>>>>> 1. > > >>>>>> [INFO] java.lang.NullPointerException > > >>>>>> [INFO] at > > >>>>>> com.google.gwt.dev.util.Util.computeStrongName(Util.java:172) > > >>>>>> > > >>>>>> 2. > > >>>>>> [INFO] java.lang.NullPointerException: No ImageRect ever computed > > >>>>>> for > > >>>>>> downDisabledImage > > >>>>>> [INFO] at > > >>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475 > > >>>>>> > > >>>>>> 1. is caused by Util#readURLAsBytes but I don't see any IOException > > >>>>>> in > > >>>>>> the > > >>>>>> logs > > >>>>>> 2. is just follow-up consequence of 1. (i.e. image metadata missing > > >>>>>> in > > >>>>>> GWT > > >>>>>> image cache) > > >>>>>> > > >>>>>> Now I noticed I missed one possible IOException inside > > >>>>>> Util#readURLConnectionAsBytes ... > > >>>>>> > > >>>>>> Long story short, I'll send updated TEST patch soon, apologies for > > >>>>>> the > > >>>>>> inconvenience. > > >>>>> > > >>>>> TEST-v2 patch attached. Please apply & rebuild ovirt-engine as usual. > > >>>> > > >>>> Done, here is the console output: > > >>>> http://jenkins.ovirt.org/job/manual-build-tarball/label=centos6-host/239/consoleFull > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>> > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Il 20/02/2014 13:34, Vojtech Szocs ha scritto: > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> ----- Original Message ----- > > >>>>>>>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > > >>>>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>>>> Sent: Thursday, February 20, 2014 10:02:00 AM > > >>>>>>>>>>>> Subject: Re: ovirt-engine build failure > > >>>>>>>>>>>> > > >>>>>>>>>>>> Il 19/02/2014 16:51, Vojtech Szocs ha scritto: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> ----- Original Message ----- > > >>>>>>>>>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" > > >>>>>>>>>>>>>> <knese...@redhat.com>, > > >>>>>>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>>>>>> Sent: Wednesday, February 19, 2014 2:43:44 PM > > >>>>>>>>>>>>>> Subject: Re: ovirt-engine build failure > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Il 19/02/2014 14:41, Vojtech Szocs ha scritto: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> ----- Original Message ----- > > >>>>>>>>>>>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>>>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" > > >>>>>>>>>>>>>>>> <knese...@redhat.com>, > > >>>>>>>>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>>>>>>>> Sent: Wednesday, February 19, 2014 2:23:16 PM > > >>>>>>>>>>>>>>>> Subject: Re: ovirt-engine build failure > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Il 19/02/2014 14:19, Vojtech Szocs ha scritto: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> ----- Original Message ----- > > >>>>>>>>>>>>>>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>>>>>>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" > > >>>>>>>>>>>>>>>>>> <knese...@redhat.com>, > > >>>>>>>>>>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>>>>>>>>>> Sent: Wednesday, February 19, 2014 1:18:28 PM > > >>>>>>>>>>>>>>>>>> Subject: Re: ovirt-engine build failure > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Il 19/02/2014 12:45, Vojtech Szocs ha scritto: > > >>>>>>>>>>>>>>>>>>> Hi, I checked GWT sources: > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect > > >>>>>>>>>>>>>>>>>>>> ever > > >>>>>>>>>>>>>>>>>>>> computed > > >>>>>>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>> removeBrickCommitRequired > > >>>>>>>>>>>>>>>>>>>> [INFO] at > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> This happened because internal image cache for bundled > > >>>>>>>>>>>>>>>>>>> (composite > > >>>>>>>>>>>>>>>>>>> PNG) > > >>>>>>>>>>>>>>>>>>> images was corrput. > > >>>>>>>>>>>>>>>>>>> It seems to be caused by disk I/O error while reading > > >>>>>>>>>>>>>>>>>>> file > > >>>>>>>>>>>>>>>>>>> images/glusterAsyncTasks/remove_brick_commit_required.png > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> [INFO] java.lang.NullPointerException > > >>>>>>>>>>>>>>>>>>>> [INFO] at > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> This happened because image data was null, again > > >>>>>>>>>>>>>>>>>>> possible > > >>>>>>>>>>>>>>>>>>> disk > > >>>>>>>>>>>>>>>>>>> I/O > > >>>>>>>>>>>>>>>>>>> error > > >>>>>>>>>>>>>>>>>>> while reading image file. > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Can we enable "TRACE" logging during GWT compilation? > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> -Dgwt.logLevel=TRACE > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Building master right now with your suggestions here: > > >>>>>>>>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/ > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Looking at > > >>>>>>>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=fedora20-host/console > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> So far I see no errors.. There should be total 5 browsers > > >>>>>>>>>>>>>>>>> x > > >>>>>>>>>>>>>>>>> 8 > > >>>>>>>>>>>>>>>>> locales > > >>>>>>>>>>>>>>>>> = > > >>>>>>>>>>>>>>>>> 40 > > >>>>>>>>>>>>>>>>> GWT permutations. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> So no changes with and without > > >>>>>>>>>>>>>>>> -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera. > > >>>>>>>>>>>>>>>> We've still 40 permutations. Maybe you defined > > >>>>>>>>>>>>>>>> userAgent=ie8,ie9,gecko1_8,safari,opera somewhere in the > > >>>>>>>>>>>>>>>> POM > > >>>>>>>>>>>>>>>> files? > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> You are right. I checked frontend/webadmin/modules/pom.xml > > >>>>>>>>>>>>>>> -> > > >>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>> "gwt.userAgent" > > >>>>>>>>>>>>>>> property is copied into "locale" GWT property in > > >>>>>>>>>>>>>>> {WebAdmin,UserPortal}.gwt.xml > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Ok, so I'll remove > > >>>>>>>>>>>>>> -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera > > >>>>>>>>>>>>>> from > > >>>>>>>>>>>>>> build > > >>>>>>>>>>>>>> command. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Agreed. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> This: > > >>>>>>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=centos6-host/console > > >>>>>>>>>>>>>>>> failed, you can look at it, it has -Dgwt.logLevel=TRACE > > >>>>>>>>>>>>>>>> -X. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Hm, it contains: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> [DEBUG] (f) logLevel = INFO > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Somehow the TRACE log level for GWT wasn't applied.. not > > >>>>>>>>>>>>>>> sure > > >>>>>>>>>>>>>>> why. > > >>>>>>>>>>>>>>> Let > > >>>>>>>>>>>>>>> me > > >>>>>>>>>>>>>>> check > > >>>>>>>>>>>>>>> this on local build. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Ok, let me know if I've to change command line. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I wrote upstream patch: http://gerrit.ovirt.org/#/c/24789/ > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> When it's merged, setting "gwt.logLevel" will work (as you > > >>>>>>>>>>>>> did > > >>>>>>>>>>>>> before) > > >>>>>>>>>>>>> and we'll be able to see more details in the build log. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Here is the new build with your patch merged: > > >>>>>>>>>>>> > > >>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/228/label=centos6-host/consoleFull > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks. I did some more investigation on this. > > >>>>>>>>>>> > > >>>>>>>>>>> Again it's NullPointerException when rebinding > > >>>>>>>>>>> ApplicationResources, > > >>>>>>>>>>> but > > >>>>>>>>>>> this time for different image file: > > >>>>>>>>>>> ApplicationResources.wrenchImage > > >>>>>>>>>>> ("images/wrench.png"). > > >>>>>>>>>>> > > >>>>>>>>>>> Looking at stacktrace & GWT sources, the problematic code seems > > >>>>>>>>>>> to > > >>>>>>>>>>> be > > >>>>>>>>>>> (AbstractResourceContext:74): > > >>>>>>>>>>> > > >>>>>>>>>>> byte[] bytes = Util.readURLAsBytes(resource); > > >>>>>>>>>>> > > >>>>>>>>>>> where "resource" is (ImageResourceGenerator:311): > > >>>>>>>>>>> > > >>>>>>>>>>> localized.getUrl(); // LocalizedImage localized > > >>>>>>>>>>> > > >>>>>>>>>>> Note: in oVirt UI we don't use localized images (i.e. > > >>>>>>>>>>> "wrench_fr_FR.png") > > >>>>>>>>>>> so "localized" in this case points to "wrench.png". > > >>>>>>>>>>> > > >>>>>>>>>>> So the problem seems to be reading this file: > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> frontend/webadmin/modules/webadmin/src/main/resources/org/ovirt/engine/ui/webadmin/images/wrench.png > > >>>>>>>>>>> > > >>>>>>>>>>> Since we're having many GWT permutations, this file ^^ (along > > >>>>>>>>>>> with > > >>>>>>>>>>> similar > > >>>>>>>>>>> ones) is attempted to be read multiple times *from the same > > >>>>>>>>>>> (single) > > >>>>>>>>>>> GWT > > >>>>>>>>>>> compiler Java process, before actual permutation worker Java > > >>>>>>>>>>> sub-processes > > >>>>>>>>>>> are spawned*. (More precisely, above NullPointerException > > >>>>>>>>>>> happens > > >>>>>>>>>>> at > > >>>>>>>>>>> deferred binding phase which precedes actual GWT compilation > > >>>>>>>>>>> per > > >>>>>>>>>>> each > > >>>>>>>>>>> permutation.) > > >>>>>>>>>>> > > >>>>>>>>>>> I think the root cause is Util.readURLAsBytes method used > > >>>>>>>>>>> internally > > >>>>>>>>>>> by > > >>>>>>>>>>> GWT > > >>>>>>>>>>> compiler: > > >>>>>>>>>>> > > >>>>>>>>>>> public static byte[] readURLAsBytes(URL url) { > > >>>>>>>>>>> try { > > >>>>>>>>>>> URLConnection conn = url.openConnection(); > > >>>>>>>>>>> conn.setUseCaches(false); > > >>>>>>>>>>> return readURLConnectionAsBytes(conn); > > >>>>>>>>>>> } catch (IOException e) { > > >>>>>>>>>>> return null; // This causes NullPointerException, but > > >>>>>>>>>>> this > > >>>>>>>>>>> exception > > >>>>>>>>>>> isn't logged (wtf, GWT!) > > >>>>>>>>>>> } > > >>>>>>>>>>> } > > >>>>>>>>>>> > > >>>>>>>>>>> To summarize, I'll need to write another patch which replaces > > >>>>>>>>>>> Util.readURLAsBytes implementation via AOP and prints > > >>>>>>>>>>> IOException > > >>>>>>>>>>> details. > > >>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> # > > >>>>>>>>>>>>>>>>>> CI_MAVEN_SETTINGS=$WORKSPACE/artifactory-ovirt-org-settings.xml > > >>>>>>>>>>>>>>>>>> # export MAVEN_OPTS="${MAVEN_OPTS} -Xmx2048M > > >>>>>>>>>>>>>>>>>> -XX:MaxPermSize=1024m > > >>>>>>>>>>>>>>>>>> -Dgwt.compiler.localWorkers=1" > > >>>>>>>>>>>>>>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -ts "${TARBALL}" > > >>>>>>>>>>>>>>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -D > > >>>>>>>>>>>>>>>>>> "ovirt_build_extra_flags > > >>>>>>>>>>>>>>>>>> -gs > > >>>>>>>>>>>>>>>>>> ${CI_MAVEN_SETTINGS} > > >>>>>>>>>>>>>>>>>> -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera > > >>>>>>>>>>>>>>>>>> -Dgwt.logLevel=TRACE -X" --rebuild > > >>>>>>>>>>>>>>>>>> rpmbuild/SRPMS/*.src.rpm > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Let me know if you need anything else. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Vojtech > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> ----- Original Message ----- > > >>>>>>>>>>>>>>>>>>>> From: "Vojtech Szocs" <vsz...@redhat.com> > > >>>>>>>>>>>>>>>>>>>> To: "Sandro Bonazzola" <sbona...@redhat.com> > > >>>>>>>>>>>>>>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > > >>>>>>>>>>>>>>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" > > >>>>>>>>>>>>>>>>>>>> <knese...@redhat.com>, > > >>>>>>>>>>>>>>>>>>>> "arch" <arch@ovirt.org> > > >>>>>>>>>>>>>>>>>>>> Sent: Tuesday, February 18, 2014 6:54:20 PM > > >>>>>>>>>>>>>>>>>>>> Subject: Re: ovirt-engine build failure > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> In your log I see following error: > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>>>>> [INFO] Computing all possible rebind results for > > >>>>>>>>>>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > > >>>>>>>>>>>>>>>>>>>> [INFO] Rebinding > > >>>>>>>>>>>>>>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources > > >>>>>>>>>>>>>>>>>>>> [INFO] Invoking generator > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator > > >>>>>>>>>>>>>>>>>>>> [INFO] [ERROR] Generator > > >>>>>>>>>>>>>>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' > > >>>>>>>>>>>>>>>>>>>> threw > > >>>>>>>>>>>>>>>>>>>> an exception while rebinding > > >>>>>>>>>>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > > >>>>>>>>>>>>>>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect > > >>>>>>>>>>>>>>>>>>>> ever > > >>>>>>>>>>>>>>>>>>>> computed > > >>>>>>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>> removeBrickCommitRequired > > >>>>>>>>>>>>>>>>>>>> [INFO] at > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) > > >>>>>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> I checked again logs at > > >>>>>>>>>>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/212/label=fedora19-host/consoleFull > > >>>>>>>>>>>>>>>>>>>> and indeed there is a similar error: > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>>>>> [INFO] Computing all possible rebind results for > > >>>>>>>>>>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > > >>>>>>>>>>>>>>>>>>>> [INFO] Rebinding > > >>>>>>>>>>>>>>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources > > >>>>>>>>>>>>>>>>>>>> [INFO] Invoking generator > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator > > >>>>>>>>>>>>>>>>>>>> [INFO] [ERROR] Generator > > >>>>>>>>>>>>>>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' > > >>>>>>>>>>>>>>>>>>>> threw > > >>>>>>>>>>>>>>>>>>>> an exception while rebinding > > >>>>>>>>>>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > > >>>>>>>>>>>>>>>>>>>> [INFO] java.lang.NullPointerException > > >>>>>>>>>>>>>>>>>>>> [INFO] at > > >>>>>>>>>>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) > > >>>>>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> NullPointerException when processing > > >>>>>>>>>>>>>>>>>>>> ApplicationResources > > >>>>>>>>>>>>>>>>>>>> but > > >>>>>>>>>>>>>>>>>>>> occuring > > >>>>>>>>>>>>>>>>>>>> from > > >>>>>>>>>>>>>>>>>>>> different places (ImageResourceGenerator vs. > > >>>>>>>>>>>>>>>>>>>> InlineResourceContext) > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Now I understand why Maven build fails without any > > >>>>>>>>>>>>>>>>>>>> details: > > >>>>>>>>>>>>>>>>>>>> - GWT deferred binding fails for ApplicationResources > > >>>>>>>>>>>>>>>>>>>> (i.e. > > >>>>>>>>>>>>>>>>>>>> uncaught > > >>>>>>>>>>>>>>>>>>>> NPE) > > >>>>>>>>>>>>>>>>>>>> - GWT compilation proceeds, fails to link with > > >>>>>>>>>>>>>>>>>>>> ApplicationResources > > >>>>>>>>>>>>>>>>>>>> - GWT compilation task fails with non-OK status code > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> So the issue here is GWT deferred binding generator > > >>>>>>>>>>>>>>>>>>>> (InlineClientBundleGenerator) failure during GWT > > >>>>>>>>>>>>>>>>>>>> compilation. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> GWT compiler writes extra Java files during deferred > > >>>>>>>>>>>>>>>>>>>> binding > > >>>>>>>>>>>>>>>>>>>> phase, so this could point to issue with local > > >>>>>>>>>>>>>>>>>>>> environment.. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> I will try to investigate this and look at GWT source > > >>>>>>>>>>>>>>>>>>>> code. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>>>>> Sandro Bonazzola > > >>>>>>>>>>>>>>>>>> Better technology. Faster innovation. Powered by > > >>>>>>>>>>>>>>>>>> community > > >>>>>>>>>>>>>>>>>> collaboration. > > >>>>>>>>>>>>>>>>>> See how it works at redhat.com > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>>> Sandro Bonazzola > > >>>>>>>>>>>>>>>> Better technology. Faster innovation. Powered by community > > >>>>>>>>>>>>>>>> collaboration. > > >>>>>>>>>>>>>>>> See how it works at redhat.com > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>> Sandro Bonazzola > > >>>>>>>>>>>>>> Better technology. Faster innovation. Powered by community > > >>>>>>>>>>>>>> collaboration. > > >>>>>>>>>>>>>> See how it works at redhat.com > > >>>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> -- > > >>>>>>>>>>>> Sandro Bonazzola > > >>>>>>>>>>>> Better technology. Faster innovation. Powered by community > > >>>>>>>>>>>> collaboration. > > >>>>>>>>>>>> See how it works at redhat.com > > >>>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> -- > > >>>>>>>>>> Sandro Bonazzola > > >>>>>>>>>> Better technology. Faster innovation. Powered by community > > >>>>>>>>>> collaboration. > > >>>>>>>>>> See how it works at redhat.com > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Sandro Bonazzola > > >>>>>>> Better technology. Faster innovation. Powered by community > > >>>>>>> collaboration. > > >>>>>>> See how it works at redhat.com > > >>>>>>> > > >>>>>> _______________________________________________ > > >>>>>> Arch mailing list > > >>>>>> Arch@ovirt.org > > >>>>>> http://lists.ovirt.org/mailman/listinfo/arch > > >>>>>> > > >>>> > > >>>> > > >>>> -- > > >>>> Sandro Bonazzola > > >>>> Better technology. Faster innovation. Powered by community > > >>>> collaboration. > > >>>> See how it works at redhat.com > > >>>> _______________________________________________ > > >>>> Arch mailing list > > >>>> Arch@ovirt.org > > >>>> http://lists.ovirt.org/mailman/listinfo/arch > > >>>> > > >> > > >> > > >> -- > > >> Sandro Bonazzola > > >> Better technology. Faster innovation. Powered by community > > >> collaboration. > > >> See how it works at redhat.com > > >> > > > > > > -- > > Sandro Bonazzola > > Better technology. Faster innovation. Powered by community collaboration. > > See how it works at redhat.com > > _______________________________________________ > > Arch mailing list > > Arch@ovirt.org > > http://lists.ovirt.org/mailman/listinfo/arch > > > _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
