[ https://issues.apache.org/jira/browse/PHOENIX-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034154#comment-15034154 ]
ASF GitHub Bot commented on PHOENIX-2469: ----------------------------------------- GitHub user dawidwys opened a pull request: https://github.com/apache/phoenix/pull/139 PHOENIX-2469 Problem with ARRAYs in Phoenix-spark You can merge this pull request into a Git repository by running: $ git pull https://github.com/dawidwys/phoenix spark-arrays Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/139.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #139 ---- commit fc7404a85964898bf22033f4a011e05f6516d88d Author: dawid <dawid.wysakow...@gmail.com> Date: 2015-12-01T17:33:43Z PHOENIX-2469 Problem with ARRAYs in Phoenix-spark ---- > Problem with ARRAYs in Phoenix-spark > ------------------------------------ > > Key: PHOENIX-2469 > URL: https://issues.apache.org/jira/browse/PHOENIX-2469 > Project: Phoenix > Issue Type: Bug > Reporter: Dawid Wysakowicz > Attachments: phoenix-spark.patch > > > I've recently found some behaviour that I found buggy when working with > phoenix-spark and arrays. > Take a look at those unit tests: > {code:java} > test("Can save arrays from custom dataframes back to phoenix") { > val dataSet = List(Row(2L, Array("String1", "String2", "String3"))) > val sqlContext = new SQLContext(sc) > val schema = StructType( > Seq(StructField("ID", LongType, nullable = false), > StructField("VCARRAY", ArrayType(StringType)))) > val rowRDD = sc.parallelize(dataSet) > // Apply the schema to the RDD. > val df = sqlContext.createDataFrame(rowRDD, schema) > df.write > .format("org.apache.phoenix.spark") > .options(Map("table" -> "ARRAY_TEST_TABLE", "zkUrl" -> quorumAddress)) > .mode(SaveMode.Overwrite) > .save() > } > {code} > {code:java} > test("Can save arrays of AnyVal type back to phoenix") { > val dataSet = List((2L, Array(1, 2, 3), Array(1L, 2L, 3L))) > sc > .parallelize(dataSet) > .saveToPhoenix( > "ARRAY_ANYVAL_TEST_TABLE", > Seq("ID", "INTARRAY", "BIGINTARRAY"), > zkUrl = Some(quorumAddress) > ) > // Load the results back > val stmt = conn.createStatement() > val rs = stmt.executeQuery("SELECT INTARRAY, BIGINTARRAY FROM > ARRAY_ANYVAL_TEST_TABLE WHERE ID = 2") > rs.next() > val intArray = rs.getArray(1).getArray().asInstanceOf[Array[Int]] > val longArray = rs.getArray(2).getArray().asInstanceOf[Array[Long]] > // Verify the arrays are equal > intArray shouldEqual dataSet(0)._2 > longArray shouldEqual dataSet(0)._3 > } > {code} > Both fail with some ClassCastExceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)