Does anyone tried how to cogroup datasets / join datasets by row num.

e.g
DS 1

43 AA
44 BB
45 CB

DS2

IN india
AU australia


i want to get

rownum   ds1.1 ds1.2   ds2.1 ds2.2

1 43 AA IN india
2 44 BB AU australia
3 45 CB null null

I don’t expect a complete code, some pointers on how to do is sufficient.

I tried row_number function to start

spark.range(100,200).withColumn("id",row_number()).show();

but its throwing error

java.lang.UnsupportedOperationException: Cannot evaluate expression: rownumber()

Thanks
Rohit

Reply via email to