Why is DecimalType separate from DataType ?

2015-01-30 Thread Manoj Samel 
Spark 1.2

While building schemaRDD using StructType

xxx = new StructField(credit_amount, DecimalType, true) gives error type
mismatch; found : org.apache.spark.sql.catalyst.types.DecimalType.type
required: org.apache.spark.sql.catalyst.types.DataType

From
https://spark.apache.org/docs/1.2.0/api/scala/index.html#org.apache.spark.sql.package,
it seems DecimalType = sql.catalyst.types.DecimalType is separate from
DataType = sql.catalyst.types.DataType

Not sure why that is the case? How does one uses Decimal and other types in
StructField?

Thanks,

Re: Why is DecimalType separate from DataType ?

2015-01-30 Thread Michael Armbrust 
You are grabbing the singleton, not the class.  You need to specify the
precision (i.e. DecimalType.Unlimited or DecimalType(precision, scale))

On Fri, Jan 30, 2015 at 2:23 PM, Manoj Samel manojsamelt...@gmail.com
wrote:

 Spark 1.2

 While building schemaRDD using StructType

 xxx = new StructField(credit_amount, DecimalType, true) gives error
 type mismatch; found :
 org.apache.spark.sql.catalyst.types.DecimalType.type required:
 org.apache.spark.sql.catalyst.types.DataType

 From
 https://spark.apache.org/docs/1.2.0/api/scala/index.html#org.apache.spark.sql.package,
 it seems DecimalType = sql.catalyst.types.DecimalType is separate from
 DataType = sql.catalyst.types.DataType

 Not sure why that is the case? How does one uses Decimal and other types
 in StructField?

 Thanks,