[jira] [Updated] (SPARK-15910) Schema is not checked when converting DataFrame to Dataset using Kryo encoder
[ https://issues.apache.org/jira/browse/SPARK-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15910: Assignee: Sean Owen > Schema is not checked when converting DataFrame to Dataset using Kryo encoder > - > > Key: SPARK-15910 > URL: https://issues.apache.org/jira/browse/SPARK-15910 > Project: Spark > Issue Type: Bug > Components: SQL >Reporter: Sean Zhong >Assignee: Sean Owen > Fix For: 2.0.0 > > > Here is the case to reproduce it: > {code} > scala> import org.apache.spark.sql.Encoders._ > scala> import org.apache.spark.sql.Encoders > scala> import org.apache.spark.sql.Encoder > scala> case class B(b: Int) > scala> implicit val encoder = Encoders.kryo[B] > encoder: org.apache.spark.sql.Encoder[B] = class[value[0]: binary] > scala> val ds = Seq((1)).toDF("b").as[B].map(identity) > ds: org.apache.spark.sql.Dataset[B] = [value: binary] > scala> ds.show() > 16/06/10 13:46:51 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 45, Column 168: No applicable constructor/method found for actual parameters > "int"; candidates are: "public static java.nio.ByteBuffer > java.nio.ByteBuffer.wrap(byte[])", "public static java.nio.ByteBuffer > java.nio.ByteBuffer.wrap(byte[], int, int)" > ... > {code} > The expected behavior is to report schema check failure earlier when creating > Dataset using {code}dataFrame.as[B]{code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15910) Schema is not checked when converting DataFrame to Dataset using Kryo encoder
[ https://issues.apache.org/jira/browse/SPARK-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15910: Assignee: Sean Zhong (was: Sean Owen) > Schema is not checked when converting DataFrame to Dataset using Kryo encoder > - > > Key: SPARK-15910 > URL: https://issues.apache.org/jira/browse/SPARK-15910 > Project: Spark > Issue Type: Bug > Components: SQL >Reporter: Sean Zhong >Assignee: Sean Zhong > Fix For: 2.0.0 > > > Here is the case to reproduce it: > {code} > scala> import org.apache.spark.sql.Encoders._ > scala> import org.apache.spark.sql.Encoders > scala> import org.apache.spark.sql.Encoder > scala> case class B(b: Int) > scala> implicit val encoder = Encoders.kryo[B] > encoder: org.apache.spark.sql.Encoder[B] = class[value[0]: binary] > scala> val ds = Seq((1)).toDF("b").as[B].map(identity) > ds: org.apache.spark.sql.Dataset[B] = [value: binary] > scala> ds.show() > 16/06/10 13:46:51 ERROR CodeGenerator: failed to compile: > org.codehaus.commons.compiler.CompileException: File 'generated.java', Line > 45, Column 168: No applicable constructor/method found for actual parameters > "int"; candidates are: "public static java.nio.ByteBuffer > java.nio.ByteBuffer.wrap(byte[])", "public static java.nio.ByteBuffer > java.nio.ByteBuffer.wrap(byte[], int, int)" > ... > {code} > The expected behavior is to report schema check failure earlier when creating > Dataset using {code}dataFrame.as[B]{code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org