Re: How to search on a Dataset / RDD <Row, Long >

2016-07-22 Thread Pedro Rodriguez
You might look at monotonically_increasing_id() here http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.functions instead of converting it to an RDD. since you pay a performance penalty for that. If you want to change the name you can do something like this (in scala

How to search on a Dataset / RDD <Row, Long >

2016-07-22 Thread VG
Any suggestions here please I basically need an ability to look up *name -> index* and *index -> name* in the code -VG On Fri, Jul 22, 2016 at 6:40 PM, VG wrote: > Hi All, > > I am really confused how to proceed further. Please help. > > I have a dataset created as