Hi,
I am in situation where I want to generate unique Id for each row.
I have use monotonicallyIncreasingId but it is giving increasing values and
start generating from start if it fail.
I have two question here:
Q1. Does this method give me unique id even in failure situation becaue I
want to use that ID in my solr id.
Q2. If answer to previous question is NO. Then Is there way yo generate
UUID for each row which is uniqe and not updatedable.
As I have come up with situation where UUID is updated
val idUDF = udf(() => UUID.randomUUID().toString)
val a = withColumn("alarmUUID", lit(idUDF()))
a.persist(StorageLevel.MEMORY_AND_DISK)
rawDataDf.registerTempTable("rawAlarms")
///
/// I do some joines
but as I reach further below
I do sonthing like
b is transformation of a
sqlContext.sql("""Select a.alarmUUID,b.alarmUUID
from a right outer join b on a.alarmUUID =
b.alarmUUID""")
it give output as
+--------------------+--------------------+
| alarmUUID| alarmUUID|
+--------------------+--------------------+
|7d33a516-5532-410...| null|
| null|2439d6db-16a2-44b...|
+--------------------+--------------------+
--
Thanks and Regards,
Saurav Sinha
Contact: 9742879062
