Hi,
We solved this the ugly way, when parsing external column definitions:
private def columnTypeToFieldType(columnType: String): DataType = {
columnType match {
case "IntegerType" => IntegerType
case "StringType" => StringType
case "DateType" => DateType
case "FloatType" =>
Can you share your code and a sample of your data? WIthout seeing it, I
can't give a definitive answer. I can offer some hints. If you have a
column of strings you should either be able to create a new column casted
to Integer. This can be accomplished two ways:
df.withColumn("newColumn",
It seems like its hard to construct a DataType given its String literal
representation.
dataframe.types() return column names and its corresponding Types. for
example say I have an integer column named "sum" doing dataframe.dtypes()
would return "sum" and "IntegerType" but this string
Hi All,
I have a datatype "IntegerType" represented as a String and now I want to
create DataType object out of that. I couldn't find in the DataType or
DataTypes api on how to do that?
Thanks!
