Spark SQL cannot tolerate regexp with BIGINT

2015-04-29 Thread lonely Feb
Hi all, we are transfer our HIVE job into SparkSQL, but we found a litter difference between HIVE and Spark SQL that our sql has a statement like: select A from B where id regexp '^12345$' in HIVE it works fine but in Spark SQL we got a: java.lang.ClassCastException: java.lang.Long cannot be

Re: Spark SQL cannot tolerate regexp with BIGINT

2015-04-29 Thread Olivier Girardot
I guess you can use cast(id as String) instead of just id in your where clause ? Le mer. 29 avr. 2015 à 12:13, lonely Feb lonely8...@gmail.com a écrit : Hi all, we are transfer our HIVE job into SparkSQL, but we found a litter difference between HIVE and Spark SQL that our sql has a statement

Re: Spark SQL cannot tolerate regexp with BIGINT

2015-04-29 Thread Reynold Xin
Actually I'm doing some cleanups related to type coercion, and I will take care of this. On Wed, Apr 29, 2015 at 5:10 PM, lonely Feb lonely8...@gmail.com wrote: OK, I'll try. On Apr 30, 2015 06:54, Reynold Xin r...@databricks.com wrote: We added ExpectedInputConversion rule recently in