Intermittently getting "Can not create the managed table error" while creating table from spark 2.4

Hi, We are facing below error in spark 2.4 intermittently when saving the managed table from spark. Error - pyspark.sql.utils.AnalysisException: u"Can not create the managed table('`hive_issue`.`table`'). The associated location('s3://{bucket_name}/EMRFS_WARE_TEST167_new/warehouse/hive_issue.db/t

Error while creating table with space with /wihout partition

I am facing a weird situation wherein the insert overwrite query does not give any error on being executed against a table which contains a column with a space in its name. Following are the queries which give no error: CREATE TABLE TEST_PART (`col1 ` STRING) PARTITIONED BY (`col2` STRING) STORED

Data is not getting written in sorted format on target oracle table through SPARK

Hi, I have a table in hive with below schema emp_id:int emp_name:string I have created data frame from above hive table df = sql_context.sql('SELECT * FROM employee ORDER by emp_id') df.show() After above code is run I see that data is sorted properly on emp_id After this I am trying to write

Re: Datafarme save as table operation is failing when the child columns name contains special characters

.cast(schema)) I have verified the data for much more complex JSON and XML structure and it looks good. Thanks, Abhijeet On Wed, May 16, 2018 at 6:13 PM, abhijeet bedagkar wrote: > Hi, > > I am using SPARK to read the XML / JSON files to create a dataframe and > save it a

Datafarme save as table operation is failing when the child columns name contains special characters

Hi, I am using SPARK to read the XML / JSON files to create a dataframe and save it as a hive table Sample XML file: 101 45 COMMAND Note field 'validation-timeout' under testexecutioncontroller. Below is the schema populated by DF after reading the XML file |-- id: