Hi,
I am using SparkSQL 1.1.0.
Actually, I have a table as following:
root
|-- account_id: string (nullable = false)
|-- Birthday: string (nullable = true)
|-- preferstore: string (nullable = true)
|-- registstore: string (nullable = true)
|-- gender: string (nullable = true)
|-- city_name_en: string (nullable = true)
|-- register_date: string (nullable = true)
|-- zip: string (nullable = true)
As you see, all fields have a String type. When creating the table, I did
something like:
*StructType(
StructField(
fields.head, StringType, nullable = false) +: // key can not be
nullable
fields.tail.map(fieldName => StructField(fieldName.trim,
StringType, nullable = true)))*
where *fields* are Array[String] containing field names.
Need to point out that "*preferstore*", "*registstore*" are essentially
numeric, but string in table.
I find that when using these two String fields to do some numeric
comparison, SparkSQL works fine. For example,
*sql("select * from customer where preferstore < 3 limit 5")*
This query returns a correct result.
I don't know well the implementation details on SparkSQL. I worried about
whether there are some side effects here ? Is this an intentional feature ?
Any implicit conversion involved ?
Thank you.
Hao
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-StringType-for-numeric-comparison-tp16295.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
