Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Miguel Landaeta]

On Wed, Jan 18, 2017 at 11:43:04AM +0100, Emmanuel Bourg wrote:
> 
> I got another look and I think the root issue is the change I made in
> asm/5.1-1 when I replaced asm-all.jar by a symlink to asm-debug-all.jar.
> I assumed that asm-all was simply a version of the jar stripped from its
> debug symbols, and since our policy recommends to keep these symbols I
> decided to remove the jar and save a few kilobytes in the package.
> 

Hi Emmanuel,

Thanks for the clear explanation of the issue and the quick solution.

I'll keep an eye at the issue you reported in jitescript project.

Cheers,

-- 
Miguel Landaeta, nomadium at debian.org
secure email with PGP 0x6E608B637D8967E9 available at http://miguel.cc/key.
"Faith means not wanting to know what is true." -- Nietzsche


signature.asc
Description: Digital signature

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Emmanuel Bourg]

Le 15/01/2017 à 16:26, Miguel Landaeta a écrit :

> It's probably too late to object this change since it's about to enter
> testing but I suspect this change could trigger FTBFS in libasm-java
> reverse dependencies.

We still have two weeks before the final freeze to examine this actually.

I got another look and I think the root issue is the change I made in
asm/5.1-1 when I replaced asm-all.jar by a symlink to asm-debug-all.jar.
I assumed that asm-all was simply a version of the jar stripped from its
debug symbols, and since our policy recommends to keep these symbols I
decided to remove the jar and save a few kilobytes in the package.

But this is slightly more subtle, the classes in asm-all.jar (and the
other non debug jars) are bytecode processed, the generic signatures are
stripped and the class format is changed from 49 (Java 5) to 46 (Java
1.2). So the jars aren't strictly source compatible.

For example jitescript depends on asm-all, and in FieldDefinition it
uses the org.objectweb.asm.tree.FieldNode class which contains a
visibleAnnotations public field [1] of type List
(stripped to List in asm-all.jar). FieldDefinition puts a type
incompatible with AnnotationNode in this list, so the compilation fails
with asm-debug-all, but not asm-all (I reported this upstream [2]).

The case of android-toolchain-jack is another example of a source
incompatibility. Here the org.objectweb.asm.tree.analysis.Analyzer class
is used with a generic parameter [3], so the code can only build with
asm-debug-all and not asm-all.

So the right solution would be to install both asm-all.jar (shrunk) and
asm-debug-all.jar (normal) to satisfy all needs and match the upstream
distribution. We may refine this and modify the shrinking process to
preserve the debug symbols, what matters is to package two jars with and
without generics.

Emmanuel Bourg

[1]
https://anonscm.debian.org/cgit/pkg-java/asm.git/tree/src/org/objectweb/asm/tree/FieldNode.java?h=upstream/5.2#n77
[2] https://github.com/qmx/jitescript/issues/15
[3]
https://gitlab.com/seamlik/debianpkg_android-toolchain-jack/blob/master/jill/src/com/android/jill/frontend/java/MethodBodyWriter.java#L196

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Emmanuel Bourg]

On Mon, 5 Dec 2016 05:44:46 -0800 tony mancill  wrote:

> I do note that the JAR files produced by
> the "unshrunk" package are much larger (~4x) than what is being
> distributed by Maven Central.
> 
> I'll have a look - there could be another cause.

I noticed that too, the jar files contain the .java sources and other
stuff not found in the jar files available on Maven Central. This
happens because the build was patched to not use bnd to create the jar
files [1], the  task used instead catches too many files.

I don't understand why bnd was disabled, the description of the patch
claims dependency issues but it's not clear what exactly. I tried
disabling the patch and it built fine. So I'll remove it in the next
upload, this will simplify the packaging.

Emmanuel Bourg

[1]
https://anonscm.debian.org/cgit/pkg-java/asm.git/tree/debian/patches/remove_bnd.diff?h=debian/5.2-1

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Miguel Landaeta]

On Sun, Dec 04, 2016 at 09:07:12PM -0800, tony mancill wrote:
> On Sun, Dec 04, 2016 at 07:47:31PM +0800, 殷啟聰 wrote:
> > FYI, I forwarded the bug to ASM's upstream issue tracker [1].
> > 
> > I propose the follwing patch to fix the bug:
> > 
> >  override_dh_auto_build:
> > -   dh_auto_build -- jar jdoc examples
> > +   dh_auto_build -- -Dproduct.noshrink=true jar jdoc examples
> 
> Looks good to me.  I have applied the patch and will upload this week if
> there are no objections.

Hi,

It's probably too late to object this change since it's about to enter
testing but I suspect this change could trigger FTBFS in libasm-java
reverse dependencies.

I said this because I could verify this specific change prevent
jitescript package [1] to be successfully built with libasm-java
5.2-1. When I built it with 5.1-2 or with a new release on 5.2
reverting only this change, everything works on my side.

As workaround, I ended using libasm4-java as B-D for that package but
this is not ideal.

The error messages I got when I use libasm-java/5.2-1 are:

ERROR] 
/home/miguel/packages/jitescript/jitescript-0.4.1/src/main/java/me/qmx/jitescript/MethodDefinition.java:[58,37]
 incompatible types: java.util.ArrayList 
cannot be conv
erted to java.util.List
[ERROR] 
/home/miguel/packages/jitescript/jitescript-0.4.1/src/main/java/me/qmx/jitescript/FieldDefinition.java:[25,35]
 incompatible types: java.util.ArrayList 
cannot be conve
rted to java.util.List

You can inspect those files at upstream if interested at [2] and [3].

Cheers,


1. https://ftp-master.debian.org/new/jitescript_0.4.1-1.html
2. 
https://github.com/qmx/jitescript/blob/master/src/main/java/me/qmx/jitescript/MethodDefinition.java#L58
3. 
https://github.com/qmx/jitescript/blob/master/src/main/java/me/qmx/jitescript/FieldDefinition.java#L25

-- 
Miguel Landaeta, nomadium at debian.org
secure email with PGP 0x6E608B637D8967E9 available at http://miguel.cc/key.
"Faith means not wanting to know what is true." -- Nietzsche


signature.asc
Description: Digital signature

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[殷啟聰]

In version 1.3~b8 that I have been packaging, Jack uses ASM 5.0.4 [1].

The upstream Jack is not affected by this particular problem because
the source of Jack includes the source of all external dependencies
which are compiled using Ant and javac and are included into
`jack.jar` when Jack is being built. Thus, the classes of ASM are not
shrunk and stay in Java 1.5.

You can see the commands compiling ASM in [2].

[1]: 
https://android.googlesource.com/toolchain/jack/+/ub-jack-douarn-b8/asm4/README.android
[2]: 
https://android.googlesource.com/toolchain/jack/+/ub-jack-douarn-b8/build.xml#2215

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Emmanuel Bourg]

Le 5/12/2016 à 16:53, 殷啟聰 a écrit :
> I tried building Jack using the asm-all.jar from jcenter and it failed
> with the same error. And when I checked the classes in this JAR it
> gave me "Java 1.2" as well.
> 
> So looks like the same problem applies to the upstream as well.

Good, at least our asm package is in line with upstream. So the next
question is: what version of asm did Jack use?

Emmanuel Bourg

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[殷啟聰]

I tried building Jack using the asm-all.jar from jcenter and it failed
with the same error. And when I checked the classes in this JAR it
gave me "Java 1.2" as well.

So looks like the same problem applies to the upstream as well.

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[tony mancill]

On Mon, Dec 05, 2016 at 10:09:07AM +0100, Emmanuel Bourg wrote:
> Le 5/12/2016 à 06:07, tony mancill a écrit :
> 
> > Looks good to me.  I have applied the patch and will upload this week if
> > there are no objections.
> 
> There is just one thing unclear to me, does Java Android Compiler Kit
> build fine with the ASM jar from Maven Central? Because if we use the
> same build parameters than upstream we shouldn't see this difference.

Ah, that's a really good idea.  I do note that the JAR files produced by
the "unshrunk" package are much larger (~4x) than what is being
distributed by Maven Central.

I'll have a look - there could be another cause.
Cheers,
tony


signature.asc
Description: PGP signature

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[Emmanuel Bourg]

Le 5/12/2016 à 06:07, tony mancill a écrit :

> Looks good to me.  I have applied the patch and will upload this week if
> there are no objections.

There is just one thing unclear to me, does Java Android Compiler Kit
build fine with the ASM jar from Maven Central? Because if we use the
same build parameters than upstream we shouldn't see this difference.

Emmanuel Bourg

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[tony mancill]

On Sun, Dec 04, 2016 at 07:47:31PM +0800, 殷啟聰 wrote:
> FYI, I forwarded the bug to ASM's upstream issue tracker [1].
> 
> I propose the follwing patch to fix the bug:
> 
>  override_dh_auto_build:
> -   dh_auto_build -- jar jdoc examples
> +   dh_auto_build -- -Dproduct.noshrink=true jar jdoc examples

Looks good to me.  I have applied the patch and will upload this week if
there are no objections.

Cheers,
tony


signature.asc
Description: PGP signature

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[殷啟聰]

FYI, I forwarded the bug to ASM's upstream issue tracker [1].

I propose the follwing patch to fix the bug:

```
diff --git a/debian/rules b/debian/rules
index 734f78c..ed59b80 100755
--- a/debian/rules
+++ b/debian/rules
@@ -6,7 +6,7 @@ VERSION ?= $(shell dpkg-parsechangelog | sed -rne
's/^Version: ([0-9.]+)[-+].*$$
dh $@ --with maven-repo-helper

 override_dh_auto_build:
-   dh_auto_build -- jar jdoc examples
+   dh_auto_build -- -Dproduct.noshrink=true jar jdoc examples

 override_mh_install:
mh_install -e$(VERSION)
```

[1]: 
http://forge.ow2.org/tracker/?func=detail=317779_id=23=100023

Bug#845606: The Bytecode Should Not Be Targeting Java 1.2

[殷啟聰]

Package: libasm-java
Version: 5.1-1
Severity: important

The current package of libasm-java contains classes that targets the
ancient Java 1.2 where generics is not supported. When I was building
a package for Jack (Java Android Compiler Kit) which uses the generics
code in ASM, it yielded the following errors:

```
/android-toolchain-jack/jill/src/com/android/jill/frontend/java/MethodBodyWriter.java:196:
error: type Analyzer does not take parameters
  private final Analyzer analyzer;
^
```

But apparently `org.objectweb.asm.tree.analysis.Analyzer` takes a
type parameter. FYI, `MethodBodyWriter.java` can be found at [1].

So I suspected this was caused by the Java version of the bytecodes,
and I found out that the classes in `asm-all.jar` is for Java 1.2.

By observing the build scripts of ASM I found that it was the
"shrinking" process [2] that changed the classes to Java 1.2 instead
of 1.5 as indicated by `build.xml`. After I applied
`-Dproduct.noshrink=true` to the build command, as the documentation
suggests, Jack was successfully built with the resulting
`asm-all.jar`.

Therefore, I wish the maintainers of this package turn off the
"shrinking" process. According to the ASM documentations [2],
"shrinking" is merely for reducing JAR size, which is not necessary in
this case.

[1]: 
https://android.googlesource.com/toolchain/jack/+/ub-jack-douarn-b8/jill/src/com/android/jill/frontend/java/MethodBodyWriter.java
[2]: http://asm.ow2.org/doc/developer-guide.html#building