df = spark.createDataFrame( [("china", "asia"), ("colombia", "south america`")], ["country", "continent`"] ) df.show()
+--------+--------------+ | country| continent`| +--------+--------------+ | china| asia| |colombia|south america`| +--------+--------------+ df.select("continent`").show(1) (...) AnalysisException: Syntax error in attribute name: continent`. clean_df = df.toDF(*(c.replace('`', '_') for c in df.columns)) clean_df.show() +--------+--------------+ | country| continent_| +--------+--------------+ | china| asia| |colombia|south america`| +--------+--------------+ clean_df.select("continent_").show(2) +--------------+ | continent_| +--------------+ | asia| |south america`| +--------------+ Examples are from MungingData Avoiding Dots / Periods in PySpark Column Names <https://mungingdata.com/pyspark/avoid-dots-periods-column-names/> man. 5. des. 2022 kl. 06:56 skrev 한승후 <sim...@naver.com>: > Spark throws an exception if there are backticks in the column name. > > Please help me. > -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297