This conversion is done implicitly anytime you use a string column in an
operation with a numeric column. If you run explain on your query you
should see the cast that is inserted. This is intentional and based on the
type semantics of Apache Hive.
On Mon, Oct 13, 2014 at 9:03 AM, invkrh <inv...@gmail.com> wrote:
> Hi,
>
> I am using SparkSQL 1.1.0.
>
> Actually, I have a table as following:
>
> root
> |-- account_id: string (nullable = false)
> |-- Birthday: string (nullable = true)
> |-- preferstore: string (nullable = true)
> |-- registstore: string (nullable = true)
> |-- gender: string (nullable = true)
> |-- city_name_en: string (nullable = true)
> |-- register_date: string (nullable = true)
> |-- zip: string (nullable = true)
>
> As you see, all fields have a String type. When creating the table, I did
> something like:
>
> *StructType(
> StructField(
> fields.head, StringType, nullable = false) +: // key can not be
> nullable
> fields.tail.map(fieldName => StructField(fieldName.trim,
> StringType, nullable = true)))*
>
> where *fields* are Array[String] containing field names.
>
> Need to point out that "*preferstore*", "*registstore*" are essentially
> numeric, but string in table.
>
> I find that when using these two String fields to do some numeric
> comparison, SparkSQL works fine. For example,
>
> *sql("select * from customer where preferstore < 3 limit 5")*
>
> This query returns a correct result.
>
> I don't know well the implementation details on SparkSQL. I worried about
> whether there are some side effects here ? Is this an intentional feature ?
> Any implicit conversion involved ?
>
> Thank you.
>
> Hao
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-StringType-for-numeric-comparison-tp16295.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
