Hi,

I think that Seth's case is a general problem with serialization of circular graph(s) with classes containing readResolve() method(s). The issue is being described here in full:

https://bugs.openjdk.java.net/browse/JDK-6785441

In short: readResolve() method is specified to be invoked after the deserialized object's state (fields) are fully initialized and the result of that method is than used instead of the deserialized object as the result of deserialization. But what if deserialized object's fields refer (directly or indirectly) to the object that is to be the result of readResolve method? We then have chicked-egg problem that seems impossible to fix while maintaining the specification/compatibility.

Regards, Peter

On 3/26/19 6:12 AM, David Holmes wrote:
Hi Seth,

I think we've both been chasing red herrings :) The classloader is not relevant. I can run the test with everything loaded as normal by the app loader and it still gets the ClassCastException. Searching JBS it seems that serialization of lambdas is broken - ref:

https://bugs.openjdk.java.net/browse/JDK-8154236
https://bugs.openjdk.java.net/browse/JDK-8174865
https://bugs.openjdk.java.net/browse/JDK-8174864

though I'm not clear if your testcase falls under the same categorization as above.

David
-----

On 26/03/2019 2:08 pm, David Holmes wrote:
On 26/03/2019 2:03 pm, seth lytle wrote:
it's a nested class, but it's loaded with the same class loader as the toplevel. which is what i thought you thought was the problem when

Sorry, hard to see all the diffs inside the email. Not sure what relevance splitting into two files has given the main change was to load the enclosing class in the new loader rather than the nested class.

Anyway, yes this gets rid of the "nested class in a different classloader" problem.

I'll take a look and see what's getting generated under the hood.

David
-----

you wrote:
 > this test loads a nested type into a different classloader to its enclosing type

and:
 > it is a requirement that all nest members be defined in the same
 > package - which implies the same classloader




On Mon, Mar 25, 2019 at 11:40 PM David Holmes <david.hol...@oracle.com <mailto:david.hol...@oracle.com>> wrote:

    On 26/03/2019 1:06 pm, seth lytle wrote:
     > i still get the same stack trace after breaking the example out
    into two
     > files. does that sufficiently address the nesting issue ?

    ??

          public static class MyCode ...

    is still a nested class.

    David

     > file: SerializedLambdaTest
     > package test;
     >
     > import java.io.ByteArrayInputStream;
     > import java.io.ByteArrayOutputStream;
     > import java.io.IOException;
     > import java.io.ObjectInputStream;
     > import java.io.ObjectOutputStream;
     > import java.io.Serializable;
     >
     > // from: https://bugs.openjdk.java.net/browse/JDK-8008770
     > public class SerializedLambdaTest implements Runnable {
     >      public void run() { new MyCode().run(); }
     >      public interface SerializableRunnable extends
    Runnable,Serializable {
     >      }
     >      public static class MyCode implements SerializableRunnable {
     >          SerializableRunnable runnable2 = ()
     >                  -> {
     > System.out.println("HELLO"+this.getClass());
     >          };
     >          private byte[] serialize(Object o) {
     >              ByteArrayOutputStream baos;
     >              try (
     >                       ObjectOutputStream oos
     >                      = new ObjectOutputStream(baos = new
     > ByteArrayOutputStream())) {
     >                  oos.writeObject(o);
     >              } catch (IOException e) {
     >                  throw new RuntimeException(e);
     >              }
     >              return baos.toByteArray();
     >          }
     >          private <T> T deserialize(byte[] bytes) {
     >              try (
     >                       ObjectInputStream ois
     >                      = new ObjectInputStream(new
     > ByteArrayInputStream(bytes))) {
     >                  return (T) ois.readObject();
     >              } catch (IOException|ClassNotFoundException e) {
     >                  throw new RuntimeException(e);
     >              }
     >          }
     >          @Override
     >          public void run() {
     >              System.out.println(" this: "+this);
     >              SerializableRunnable deSerializedThis
     >                      = deserialize(serialize(this));
     >              System.out.println(" deSerializedThis: "
     >                      +deSerializedThis);
     >
     >              SerializableRunnable runnable = runnable2;
     >              System.out.println(" runnable: "+runnable);
     >              SerializableRunnable deSerializedRunnable
     >                      = deserialize(serialize(runnable));
     > System.out.println("deSerializedRunnable: "
     >                      +deSerializedRunnable);
     >          }
     >      }
     > }
     >
     >
     > file: MyClassLoader
     > package test;
     >
     > import java.io.IOException;
     > import java.io.InputStream;
     >
     > class MyClassLoader extends ClassLoader {
     >      MyClassLoader(ClassLoader parent) {
     >          super(parent);
     >      }
     >      @Override
     >      protected Class<?> loadClass(String name,boolean resolve)
    throws
     > ClassNotFoundException {
     >          if (name.startsWith("test."))
     >              synchronized (getClassLoadingLock(name)) {
     >                  Class<?> c = findLoadedClass(name);
     >                  if (c==null) c = findClass(name);
     >                  if (resolve) resolveClass(c);
     >                  return c;
     >              }
     >          else return super.loadClass(name,resolve);
     >      }
     >      @Override
     >      protected Class<?> findClass(String name) throws
     > ClassNotFoundException {
     >          String path = name.replace('.','/').concat(".class");
     >          try (final InputStream is = getResourceAsStream(path)) {
     >              if (is!=null) {
     >                  byte[] bytes = is.readAllBytes();
     >                  return defineClass(name,bytes,0,bytes.length);
     >              }
     >              else throw new ClassNotFoundException(name);
     >          } catch (IOException e) {
     >              throw new ClassNotFoundException(name,e);
     >          }
     >      }
     >      public static void main(String[] args) throws Exception {
     >          ClassLoader myCl = new MyClassLoader(
     > MyClassLoader.class.getClassLoader()
     >          );
     >          Class<?> myCodeClass = Class.forName(
     >                  "test.SerializedLambdaTest",
     >                  true,
     >                  myCl
     >          );
     >          Runnable myCode = (Runnable) myCodeClass.newInstance();
     >          myCode.run();
     >      }
     > }
     >
     >
     >
     >
     > On Mon, Mar 25, 2019 at 10:34 PM David Holmes
    <david.hol...@oracle.com <mailto:david.hol...@oracle.com>
     > <mailto:david.hol...@oracle.com
    <mailto:david.hol...@oracle.com>>> wrote:
     >
     >     Hi Seth,
     >
     >     On 26/03/2019 12:16 pm, seth lytle wrote:
     >      > i haven't changed the assignment from the original test case
     >     (which was
     >      > accepted as valid at the time). i haven't gone through it in
     >     depth, but
     >      > assume that it's ok and used it since people are already
    familiar
     >     with
     >      > it. it appears to me that that example uses reflection to
     >     allocate an
     >      > instance using a definitive class loader and then uses a
    runnable
     >     for
     >      > the serialization/deserialization cycle
     >      >
     >      > the class cast exception happens not at the example level,
    but deep
     >      > inside `readObject` when it's doing an internal assignment
     >      >
     >      > perhaps my choice of words "into a new classloader" was
     >     insufficient or
     >      > misleading. in both examples that i've come up with so
    far, the
     >     class
     >      > loader calls defineClass directly without first delegating to
     >     super (for
     >      > a subset of the classes). so "into a definitive classloader"
     >     might have
     >      > been a better choice
     >
     >     I think the basic problem is that this test loads a nested
    type into a
     >     different classloader to its enclosing type. I can easily
    imagine this
     >     messing up the code that gets generated to implement the lambda
     >     expression - but I'd need to examine that code in detail to
    see exactly
     >     why (others may know this more readily than I do). It's
    unclear to me
     >     whether this would be considered a bug or a "don't do that"
    situation.
     >     As of JDK 11, nested types define "nests" at the VM level
    (see JEP-181)
     >     and it is a requirement that all nest members be defined in
    the same
     >     package - which implies the same classloader.
     >
     >     David
     >
     >      >
     >      >
     >      >
     >      >
     >      >
     >      > On Mon, Mar 25, 2019 at 9:34 PM David Holmes
     >     <david.hol...@oracle.com <mailto:david.hol...@oracle.com>
    <mailto:david.hol...@oracle.com <mailto:david.hol...@oracle.com>>
     >      > <mailto:david.hol...@oracle.com
    <mailto:david.hol...@oracle.com>
     >     <mailto:david.hol...@oracle.com
    <mailto:david.hol...@oracle.com>>>> wrote:
     >      >
     >      >     Hi Seth,
     >      >
     >      >     On 26/03/2019 11:22 am, seth lytle wrote:
     >      >      > if a lambda is a field and captures `this`, and it's
     >     deserialized
     >      >     into a
     >      >      > new class loader, it throws a ClassCastException.
     >      >
     >      >     Not sure I follow. If you load a class into a different
     >     classloader
     >      >     then
     >      >     you get a different type. It might appear the same to
    you but
     >     it is a
     >      >     distinct type, so you can't assign across different
    instances
     >     loaded by
     >      >     different classloaders.
     >      >
     >      >     Cheers,
     >      >     David
     >      >     -----
     >      >
     >      >      > i came across this bug
     >      >      > independently while writing a test, but then found
    an old
     >     openjdk
     >      >     bug with
     >      >      > a similar issue and tweaked the test case (source
    below).
     >     my version
     >      >      > differs only in
     >      >      > 1. moved the lambda to a field
     >      >      > 2. reference `this` in it
     >      >      > 3. uses readAllBytes instead of the internal method
     >      >      > 4. formatting
     >      >      >
     >      >      > running with java 11 or 12 (or java 8 using a
    substitute for
     >      >     readAllBytes)
     >      >      > results in:
     >      >      >                  this:
     >     test.SerializedLambdaTest$MyCode@8efb846
     >      >      >      deSerializedThis:
     >     test.SerializedLambdaTest$MyCode@2b71fc7e
     >      >      >              runnable:
     >      >      >
     >      >
     >  test.SerializedLambdaTest$MyCode$$Lambda$1/0x0000000801188440@37bba400
     >      >      > Exception in thread "main"
    java.lang.ClassCastException:
     >     cannot
     >      >     assign
     >      >      > instance of java.lang.invoke.SerializedLambda to field
     >      >      > test.SerializedLambdaTest$MyCode.runnable2 of type
     >      >      > test.SerializedLambdaTest$SerializableRunnable in
    instance of
     >      >      > test.SerializedLambdaTest$MyCode
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2190)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectStreamClass$FieldReflector.checkObjectFieldValueTypes(ObjectStreamClass.java:2153)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectStreamClass.checkObjFieldValueTypes(ObjectStreamClass.java:1407)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.defaultCheckFieldValues(ObjectInputStream.java:2371)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readSerialData(ObjectInputStream.java:2278)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2087)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readObject0(ObjectInputStream.java:1594)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readArray(ObjectInputStream.java:1993)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readObject0(ObjectInputStream.java:1588)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2355)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readSerialData(ObjectInputStream.java:2249)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2087)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readObject0(ObjectInputStream.java:1594)
     >      >      > at
     >      >      > java.base/java.io <http://java.io> <http://java.io>
     >      >
     >  <http://java.io>.ObjectInputStream.readObject(ObjectInputStream.java:430)
     >      >      > at
     >      >      >
     >      >
     >  test.SerializedLambdaTest$MyCode.deserialize(SerializedLambdaTest.java:35)
     >      >      > at
     >  test.SerializedLambdaTest$MyCode.run(SerializedLambdaTest.java:51)
     >      >      > at
     >  test.SerializedLambdaTest.main(SerializedLambdaTest.java:66)
     >      >      >
     >      >      >
     >      >      >
     >      >      > https://bugs.openjdk.java.net/browse/JDK-8008770
     >      >      >
     >      >      >
     >      >      > package test;
     >      >      >
     >      >      > import java.io.ByteArrayInputStream;
     >      >      > import java.io.ByteArrayOutputStream;
     >      >      > import java.io.IOException;
     >      >      > import java.io.InputStream;
     >      >      > import java.io.ObjectInputStream;
     >      >      > import java.io.ObjectOutputStream;
     >      >      > import java.io.Serializable;
     >      >      >
     >      >      > // from:
    https://bugs.openjdk.java.net/browse/JDK-8008770
     >      >      > public class SerializedLambdaTest {
     >      >      >      public interface SerializableRunnable extends
     >      >     Runnable,Serializable {
     >      >      >      }
     >      >      >      public static class MyCode implements
     >     SerializableRunnable {
     >      >      >          SerializableRunnable runnable2 = ()
     >      >      >                  -> {
     >      >      > System.out.println("HELLO"+this.getClass());
     >      >      >          };
     >      >      >          private byte[] serialize(Object o) {
     >      >      > ByteArrayOutputStream baos;
     >      >      >              try (
     >      >      >  ObjectOutputStream oos
     >      >      >                      = new ObjectOutputStream(baos
    = new
     >      >      > ByteArrayOutputStream())) {
     >      >      > oos.writeObject(o);
     >      >      >              } catch (IOException e) {
     >      >      >                  throw new RuntimeException(e);
     >      >      >              }
     >      >      >              return baos.toByteArray();
     >      >      >          }
     >      >      >          private <T> T deserialize(byte[] bytes) {
     >      >      >              try (
     >      >      >  ObjectInputStream ois
     >      >      >                      = new ObjectInputStream(new
     >      >      > ByteArrayInputStream(bytes))) {
     >      >      >                  return (T) ois.readObject();
     >      >      >              } catch
    (IOException|ClassNotFoundException e) {
     >      >      >                  throw new RuntimeException(e);
     >      >      >              }
     >      >      >          }
     >      >      >          @Override
     >      >      >          public void run() {
     >      >      > System.out.println("                this:
     >     "+this);
     >      >      > SerializableRunnable deSerializedThis
     >      >      >                      = deserialize(serialize(this));
     >      >      > System.out.println("     deSerializedThis: "
     >      >      > +deSerializedThis);
     >      >      >
     >      >      > SerializableRunnable runnable = runnable2;
     >      >      > System.out.println("            runnable:
     >     "+runnable);
     >      >      > SerializableRunnable deSerializedRunnable
     >      >      >                      =
    deserialize(serialize(runnable));
     >      >      > System.out.println("deSerializedRunnable: "
     >      >      > +deSerializedRunnable);
     >      >      >          }
     >      >      >      }
     >      >      >      public static void main(String[] args) throws
    Exception {
     >      >      >          ClassLoader myCl = new MyClassLoader(
     >      >      > SerializedLambdaTest.class.getClassLoader()
     >      >      >          );
     >      >      >          Class<?> myCodeClass = Class.forName(
     >      >      >
     >     SerializedLambdaTest.class.getName()+"$MyCode",
     >      >      >                  true,
     >      >      >                  myCl
     >      >      >          );
     >      >      >          Runnable myCode = (Runnable)
     >     myCodeClass.newInstance();
     >      >      >          myCode.run();
     >      >      >      }
     >      >      >      static class MyClassLoader extends ClassLoader {
     >      >      > MyClassLoader(ClassLoader parent) {
     >      >      >              super(parent);
     >      >      >          }
     >      >      >          @Override
     >      >      >          protected Class<?> loadClass(String
    name,boolean
     >     resolve)
     >      >      >                  throws ClassNotFoundException {
     >      >      >              if (name.startsWith("test."))
     >      >      >                  synchronized
    (getClassLoadingLock(name)) {
     >      >      > Class<?> c =
    findLoadedClass(name);
     >      >      >                      if (c==null)
     >      >      >                          c = findClass(name);
     >      >      >                      if (resolve)
     >      >      > resolveClass(c);
     >      >      >                      return c;
     >      >      >                  }
     >      >      >              else
     >      >      >                  return super.loadClass(name,resolve);
     >      >      >          }
     >      >      >          @Override
     >      >      >          protected Class<?> findClass(String name)
    throws
     >      >      > ClassNotFoundException {
     >      >      >              String path =
     >     name.replace('.','/').concat(".class");
     >      >      >              try ( InputStream is =
     >     getResourceAsStream(path)) {
     >      >      >                  if (is!=null) {
     >      >      >                      byte[] bytes = is.readAllBytes();
     >      >      >                      return
     >     defineClass(name,bytes,0,bytes.length);
     >      >      >                  } else
     >      >      >                      throw new
    ClassNotFoundException(name);
     >      >      >              } catch (IOException e) {
     >      >      >                  throw new
    ClassNotFoundException(name,e);
     >      >      >              }
     >      >      >          }
     >      >      >      }
     >      >      > }
     >      >      >
     >      >
     >


Reply via email to