Hello Experts During a join operation, I see this error below (spark 3.0.2)
Any suggestions on how to debug? Error: java.lang.AssertionError: assertion failed: Found duplicate rewrite attribute Source code: val dfFilteredFinal=dfFiltered .join(dfScenarioSite, Seq("tid","site"), "left_outer") dfFiltered column names below +--------------------+-------------------+------------+--------------+--------------------+--------------------+-----+-------------+---------------+-----------+--------------+---------------+----------+-------------+----------------------+------------------+-------------------+---------------+-----------------+-------------------------+---------------------+---------------------+------------------------+--------------------+-------------------+------------------------------+--------------------------+--------------------------+-------------------------+-------------------+-----------------------+ | tid| site| ap_mac| svrip| swarm_id| ap_name|model| ts|samplesmeasured|anomalycntr| firmware|scenario_number|date_month| date_hour|tid_ap_samplesmeasured|tid_ap_anomalycntr|tid_samplesmeasured|tid_anomalycntr|scenario_ap_array|tid_svrip_samplesmeasured|tid_svrip_anomalycntr|scenario_server_array|tid_site_samplesmeasured|tid_site_anomalycntr|scenario_site_array|tid_site_svrip_samplesmeasured|tid_site_svrip_anomalycntr|site_scenario_server_array|site_scenario_apmac_array| scenario_array|impacted_scenario_count| +--------------------+-------------------+------------+--------------+--------------------+--------------------+-----+-------------+---------------+-----------+--------------+---------------+----------+-------------+----------------------+------------------+-------------------+---------------+-----------------+-------------------------+---------------------+---------------------+------------------------+--------------------+-------------------+------------------------------+--------------------------+--------------------------+-------------------------+-------------------+-----------------------+ dfScenarioSite column names below | tid| site|site_scenario_array|impacted_site_scenario_count|