[ https://issues.apache.org/jira/browse/SPARK-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944007#comment-14944007 ]
Deborah Siegel edited comment on SPARK-9318 at 10/5/15 9:00 PM: ---------------------------------------------------------------- not sure about the fix. I tried this on 1.5.0 and 1.5.1, same results. regarding the alias column, the issue is that "." in the schema is being converted to "_" behind the scenes. This happens automatically when createDataFrame is used. But it seems that with alias, it is not being converted, however the join is looking for the converted name. this works: ydfsel <- select(ydf, alias(ydf$k1,"k1_y"), alias(ydf$k2,"k2_y"), alias(ydf$data,"data_y")) xdfsel <- select(xdf, alias(xdf$k1,"k1_x"), alias(xdf$k2,"k2_x"), alias(xdf$data,"data_x")) res3 <- join(xdfsel,ydfsel,xdfsel$k1_x==ydfsel$k1_y) was (Author: dsiegel): not sure about the fix. I tried this on 1.5.0 and 1.5.1, same results. regarding the alias column, the issue is that "." in the schema is being converted to "_" behind the scenes. This happens automatically when createDataFrame is used. But it seems that with alias, it is not being converted, however the select is looking for the converted name. this works: ydfsel <- select(ydf, alias(ydf$k1,"k1_y"), alias(ydf$k2,"k2_y"), alias(ydf$data,"data_y")) xdfsel <- select(xdf, alias(xdf$k1,"k1_x"), alias(xdf$k2,"k2_x"), alias(xdf$data,"data_x")) res3 <- join(xdfsel,ydfsel,xdfsel$k1_x==ydfsel$k1_y) > Add `merge` as synonym for join > ------------------------------- > > Key: SPARK-9318 > URL: https://issues.apache.org/jira/browse/SPARK-9318 > Project: Spark > Issue Type: Sub-task > Components: SparkR > Reporter: Shivaram Venkataraman > Assignee: Hossein Falaki > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org