kongxianghe1234 opened a new issue #4182: URL: https://github.com/apache/carbondata/issues/4182
spark.sql.adaptive.enabled=true spark.sql.adaptive.skewedJoin.enabled=true spark.sql.adaptive.skewedPartitionMaxSplits=5 spark.sql.adaptive.skewedPartitionRowCountThreshold=10000000 spark.sql.adaptive.skewedPartitionSizeThreshold=67108864 spark.sql.adaptive.skewedPartitionFactor : 5 --- In Spark2x JDBC no use for it. t1 left join t2 on t1.id = t2.id column id has one key, for example 0000-00-00 ,has 100,000 records t2 has same key in column id also has 100,000 records ,this will generate 100000*100000 = 10B records!! for only one reducer. carbon solution no use for it,please check it. -- call hw. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
