----- Original Message ----- > From: "Sandro Bonazzola" <sbona...@redhat.com> > To: "Vojtech Szocs" <vsz...@redhat.com> > Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" <jhern...@redhat.com>, > "Kiril Nesenko" <knese...@redhat.com>, > "arch" <arch@ovirt.org> > Sent: Thursday, February 20, 2014 10:02:00 AM > Subject: Re: ovirt-engine build failure > > Il 19/02/2014 16:51, Vojtech Szocs ha scritto: > > > > > > ----- Original Message ----- > >> From: "Sandro Bonazzola" <sbona...@redhat.com> > >> To: "Vojtech Szocs" <vsz...@redhat.com> > >> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > >> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > >> "arch" <arch@ovirt.org> > >> Sent: Wednesday, February 19, 2014 2:43:44 PM > >> Subject: Re: ovirt-engine build failure > >> > >> Il 19/02/2014 14:41, Vojtech Szocs ha scritto: > >>> > >>> > >>> ----- Original Message ----- > >>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > >>>> To: "Vojtech Szocs" <vsz...@redhat.com> > >>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > >>>> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > >>>> "arch" <arch@ovirt.org> > >>>> Sent: Wednesday, February 19, 2014 2:23:16 PM > >>>> Subject: Re: ovirt-engine build failure > >>>> > >>>> Il 19/02/2014 14:19, Vojtech Szocs ha scritto: > >>>>> > >>>>> > >>>>> ----- Original Message ----- > >>>>>> From: "Sandro Bonazzola" <sbona...@redhat.com> > >>>>>> To: "Vojtech Szocs" <vsz...@redhat.com> > >>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > >>>>>> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > >>>>>> "arch" <arch@ovirt.org> > >>>>>> Sent: Wednesday, February 19, 2014 1:18:28 PM > >>>>>> Subject: Re: ovirt-engine build failure > >>>>>> > >>>>>> Il 19/02/2014 12:45, Vojtech Szocs ha scritto: > >>>>>>> Hi, I checked GWT sources: > >>>>>>> > >>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect ever computed > >>>>>>>> for > >>>>>>>> removeBrickCommitRequired > >>>>>>>> [INFO] at > >>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) > >>>>>>> > >>>>>>> This happened because internal image cache for bundled (composite > >>>>>>> PNG) > >>>>>>> images was corrput. > >>>>>>> It seems to be caused by disk I/O error while reading file > >>>>>>> images/glusterAsyncTasks/remove_brick_commit_required.png > >>>>>>> > >>>>>>>> [INFO] java.lang.NullPointerException > >>>>>>>> [INFO] at > >>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) > >>>>>>> > >>>>>>> This happened because image data was null, again possible disk I/O > >>>>>>> error > >>>>>>> while reading image file. > >>>>>>> > >>>>>>> Can we enable "TRACE" logging during GWT compilation? > >>>>>>> > >>>>>>> -Dgwt.logLevel=TRACE > >>>>>> > >>>>>> > >>>>>> Building master right now with your suggestions here: > >>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/ > >>>>> > >>>>> Looking at > >>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=fedora20-host/console > >>>>> > >>>>> So far I see no errors.. There should be total 5 browsers x 8 locales = > >>>>> 40 > >>>>> GWT permutations. > >>>> > >>>> So no changes with and without > >>>> -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera. > >>>> We've still 40 permutations. Maybe you defined > >>>> userAgent=ie8,ie9,gecko1_8,safari,opera somewhere in the POM files? > >>> > >>> You are right. I checked frontend/webadmin/modules/pom.xml -> the > >>> "gwt.userAgent" > >>> property is copied into "locale" GWT property in > >>> {WebAdmin,UserPortal}.gwt.xml > >> > >> Ok, so I'll remove -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera from > >> build > >> command. > > > > Agreed. > > > >> > >> > >>> > >>>> > >>>> This: > >>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=centos6-host/console > >>>> failed, you can look at it, it has -Dgwt.logLevel=TRACE -X. > >>> > >>> Hm, it contains: > >>> > >>> [DEBUG] (f) logLevel = INFO > >>> > >>> Somehow the TRACE log level for GWT wasn't applied.. not sure why. Let me > >>> check > >>> this on local build. > >> > >> Ok, let me know if I've to change command line. > > > > I wrote upstream patch: http://gerrit.ovirt.org/#/c/24789/ > > > > When it's merged, setting "gwt.logLevel" will work (as you did before) > > and we'll be able to see more details in the build log. > > Here is the new build with your patch merged: > > http://jenkins.ovirt.org/job/manual-build-tarball/228/label=centos6-host/consoleFull
Thanks. I did some more investigation on this. Again it's NullPointerException when rebinding ApplicationResources, but this time for different image file: ApplicationResources.wrenchImage ("images/wrench.png"). Looking at stacktrace & GWT sources, the problematic code seems to be (AbstractResourceContext:74): byte[] bytes = Util.readURLAsBytes(resource); where "resource" is (ImageResourceGenerator:311): localized.getUrl(); // LocalizedImage localized Note: in oVirt UI we don't use localized images (i.e. "wrench_fr_FR.png") so "localized" in this case points to "wrench.png". So the problem seems to be reading this file: frontend/webadmin/modules/webadmin/src/main/resources/org/ovirt/engine/ui/webadmin/images/wrench.png Since we're having many GWT permutations, this file ^^ (along with similar ones) is attempted to be read multiple times *from the same (single) GWT compiler Java process, before actual permutation worker Java sub-processes are spawned*. (More precisely, above NullPointerException happens at deferred binding phase which precedes actual GWT compilation per each permutation.) I think the root cause is Util.readURLAsBytes method used internally by GWT compiler: public static byte[] readURLAsBytes(URL url) { try { URLConnection conn = url.openConnection(); conn.setUseCaches(false); return readURLConnectionAsBytes(conn); } catch (IOException e) { return null; // This causes NullPointerException, but this exception isn't logged (wtf, GWT!) } } To summarize, I'll need to write another patch which replaces Util.readURLAsBytes implementation via AOP and prints IOException details. > > > > > > > >> > >>> > >>>> > >>>> > >>>> > >>>> > >>>>> > >>>>>> > >>>>>> # CI_MAVEN_SETTINGS=$WORKSPACE/artifactory-ovirt-org-settings.xml > >>>>>> # export MAVEN_OPTS="${MAVEN_OPTS} -Xmx2048M -XX:MaxPermSize=1024m > >>>>>> -Dgwt.compiler.localWorkers=1" > >>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -ts "${TARBALL}" > >>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -D "ovirt_build_extra_flags > >>>>>> -gs > >>>>>> ${CI_MAVEN_SETTINGS} -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera > >>>>>> -Dgwt.logLevel=TRACE -X" --rebuild rpmbuild/SRPMS/*.src.rpm > >>>>>> > >>>>>> Let me know if you need anything else. > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> Vojtech > >>>>>>> > >>>>>>> > >>>>>>> ----- Original Message ----- > >>>>>>>> From: "Vojtech Szocs" <vsz...@redhat.com> > >>>>>>>> To: "Sandro Bonazzola" <sbona...@redhat.com> > >>>>>>>> Cc: "Eyal Edri" <ee...@redhat.com>, "Juan Hernandez" > >>>>>>>> <jhern...@redhat.com>, "Kiril Nesenko" <knese...@redhat.com>, > >>>>>>>> "arch" <arch@ovirt.org> > >>>>>>>> Sent: Tuesday, February 18, 2014 6:54:20 PM > >>>>>>>> Subject: Re: ovirt-engine build failure > >>>>>>>> > >>>>>>>> In your log I see following error: > >>>>>>>> > >>>>>>>> ... > >>>>>>>> [INFO] Computing all possible rebind results for > >>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > >>>>>>>> [INFO] Rebinding > >>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources > >>>>>>>> [INFO] Invoking generator > >>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator > >>>>>>>> [INFO] [ERROR] Generator > >>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' > >>>>>>>> threw > >>>>>>>> an exception while rebinding > >>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > >>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect ever computed > >>>>>>>> for > >>>>>>>> removeBrickCommitRequired > >>>>>>>> [INFO] at > >>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) > >>>>>>>> ... > >>>>>>>> > >>>>>>>> I checked again logs at > >>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/212/label=fedora19-host/consoleFull > >>>>>>>> and indeed there is a similar error: > >>>>>>>> > >>>>>>>> ... > >>>>>>>> [INFO] Computing all possible rebind results for > >>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > >>>>>>>> [INFO] Rebinding > >>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources > >>>>>>>> [INFO] Invoking generator > >>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator > >>>>>>>> [INFO] [ERROR] Generator > >>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' > >>>>>>>> threw > >>>>>>>> an exception while rebinding > >>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' > >>>>>>>> [INFO] java.lang.NullPointerException > >>>>>>>> [INFO] at > >>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) > >>>>>>>> ... > >>>>>>>> > >>>>>>>> NullPointerException when processing ApplicationResources but > >>>>>>>> occuring > >>>>>>>> from > >>>>>>>> different places (ImageResourceGenerator vs. InlineResourceContext) > >>>>>>>> > >>>>>>>> Now I understand why Maven build fails without any details: > >>>>>>>> - GWT deferred binding fails for ApplicationResources (i.e. uncaught > >>>>>>>> NPE) > >>>>>>>> - GWT compilation proceeds, fails to link with ApplicationResources > >>>>>>>> - GWT compilation task fails with non-OK status code > >>>>>>>> > >>>>>>>> So the issue here is GWT deferred binding generator > >>>>>>>> (InlineClientBundleGenerator) failure during GWT compilation. > >>>>>>>> > >>>>>>>> GWT compiler writes extra Java files during deferred binding > >>>>>>>> phase, so this could point to issue with local environment.. > >>>>>>>> > >>>>>>>> I will try to investigate this and look at GWT source code. > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Sandro Bonazzola > >>>>>> Better technology. Faster innovation. Powered by community > >>>>>> collaboration. > >>>>>> See how it works at redhat.com > >>>>>> > >>>> > >>>> > >>>> -- > >>>> Sandro Bonazzola > >>>> Better technology. Faster innovation. Powered by community > >>>> collaboration. > >>>> See how it works at redhat.com > >>>> > >> > >> > >> -- > >> Sandro Bonazzola > >> Better technology. Faster innovation. Powered by community collaboration. > >> See how it works at redhat.com > >> > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch