Hello Team, We are building one application using spark where we are creating DataFrame using few parameters, column_name, position and column_datatype. For column_datatype we are doing the mapping from SQL Server Data Type to Spark Struct Data Types. Upon checking from the Datatype documentation: https://spark.apache.org/docs/latest/sql-ref-datatypes.html we found that CharType() and VarcharType() data type are present which accepts the max length of characters for the field as a parameter. However, whenever I am trying to create the Dataframe using I am getting an error related to Char/VarchaType: *java.lang.IllegalStateException: [BUG] logical plan should not have output of char/varchar type: LogicalRDD* The error is resolved when I am using StringType there, but our requirement needs a validation of max characters in the field. Can you please suggest on this?
-- Thanks and Regards, *Souvik Saha* Kolkata, West Bengal, India +91-9674113358
