[ https://issues.apache.org/jira/browse/SPARK-13886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198755#comment-15198755 ]
Rishabh Bhardwaj commented on SPARK-13886: ------------------------------------------ For q2: [~mahmoud.hanafy] You can use List instead of Array. {code} val a = Row( Seq( List(1.toByte) ) ) val b = Row( Seq( List(1.toByte) ) ) a.equals(b) // will return true {code} We can add the support for Array[Byte], Array[Double] etc. but should it be limited to that only because the next requirement can be of Array[Array[Byte]] or Array[Array[Array[Byte]]]. If its only to support Array[Byte],Array[Double] or any other commonly used type,then I can take this task to add support for those in Row. > ArrayType of BinaryType not supported in Row.equals method > ----------------------------------------------------------- > > Key: SPARK-13886 > URL: https://issues.apache.org/jira/browse/SPARK-13886 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: MahmoudHanafy > Priority: Minor > > There are multiple types that are supoprted by Spark SQL, One of them is > ArrayType(Seq) which can be of any element type > So it can be BinaryType(Array\[Byte\]) > In equals method in Row class, there is no handling for ArrayType of > BinaryType. > So for example: > {code:xml} > val a = Row( Seq( Array(1.toByte) ) ) > val b = Row( Seq( Array(1.toByte) ) ) > a.equals(b) // this will return false > {code} > Also, this doesn't work for MapType of BinaryType. > {code:xml} > val a = Row( Map(1 -> Array(1.toByte) ) ) > val b = Row( Map(1 -> Array(1.toByte) ) ) > a.equals(b) // this will return false > {code} > Question1: Can the key in MapType be of BinaryType ? > Question2: Isn't there another way to handle BinaryType by using scala type > instead of Array ? > I want to contribute by fixing this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org