Hi All,IS NOT NULL is not working in programmatic sql. check below for
input output and code.

Input
====
10,IN
11,PK
12,US
13,UK
14,US
15,IN
16,
17,AS
18,AS
19,IR
20,As

val cntdat = sc.textFile("/user/poc_hortonworks/radha/gsd/sample.txt");
case class CNT (id:Int , code : String)
val cntdf = cntdat.map((f) => { val ff=f.split(",");new
CNT(ff(0).toInt,ff(1))}).toDF
cntdf.registerTempTable("cntids");
sqlContext.sql("SELECT MAX(id),code FROM cntids WHERE code is not null
GROUP BY code").show()

Output
=====
+---+----+
|_c0|code|
+---+----+
| 18|  AS|
| 16|      |
| 13|  UK|
| 14|  US|
| 20|  As|
| 15|  IN|
| 19|  IR|
| 11|  PK|
+---+----+

i am expecting the below one any idea, how to apply IS NOT NULL ?

+---+----+
|_c0|code|
+---+----+
| 18|  AS|
| 13|  UK|
| 14|  US|
| 20|  As|
| 15|  IN|
| 19|  IR|
| 11|  PK|
+---+----+



Thanks & Regards
   Radha krishna

Reply via email to