wangyum commented on issue #27252: [SPARK-29231][SQL] Constraints should be inferred from cast equality constraint URL: https://github.com/apache/spark/pull/27252#issuecomment-576160633 @maropu Before this PR: ``` === Metrics of Analyzer/Optimizer Rules === Total number of runs: 5326085 Total time: 144.747179483 seconds Rule Effective Time / Total Time Effective Runs / Total Runs org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries 16779452101 / 20173139174 748 / 26140 org.apache.spark.sql.catalyst.optimizer.ColumnPruning 1425337133 / 5635754805 5138 / 60806 org.apache.spark.sql.execution.datasources.FindDataSourceTable 4948196097 / 5101730455 2384 / 49272 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery 3341395204 / 3939149301 830 / 49313 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences 1705032936 / 2959703766 9241 / 49384 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions 830752175 / 2570283319 275 / 49294 org.apache.spark.sql.catalyst.optimizer.PushDownPredicates 455566317 / 2312688244 2302 / 47783 org.apache.spark.sql.catalyst.optimizer.CollapseProject 989929053 / 2118855460 5895 / 47762 org.apache.spark.sql.catalyst.optimizer.PruneFilters 14076793 / 1946034212 168 / 47710 org.apache.spark.sql.catalyst.optimizer.ConstantFolding 653505895 / 1904392560 3565 / 34666 org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators 176374899 / 1874860360 2339 / 60754 org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions 738001 / 1856087864 6 / 34666 org.apache.spark.sql.catalyst.optimizer.NullPropagation 45193963 / 1759483772 359 / 34718 org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions 0 / 1743186421 0 / 34666 org.apache.spark.sql.catalyst.optimizer.BooleanSimplification 8531965 / 1689250544 54 / 34666 org.apache.spark.sql.catalyst.optimizer.SimplifyCasts 45594645 / 1557656900 287 / 34666 org.apache.spark.sql.catalyst.optimizer.LikeSimplification 0 / 1536745808 0 / 34666 org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator 0 / 1480669059 0 / 34666 org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals 9875655 / 1456484466 178 / 34666 org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison 2727915 / 1453064933 20 / 34666 org.apache.spark.sql.catalyst.optimizer.EliminateMapObjects 0 / 1424448232 0 / 13044 org.apache.spark.sql.catalyst.optimizer.CombineUnions 6802865 / 1423200337 46 / 47814 org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts 337664736 / 1421112672 1245 / 49272 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions 788751339 / 1396854044 5789 / 49313 org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps 258804 / 1373494442 3 / 34666 org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate 267041 / 1369285011 3 / 34666 org.apache.spark.sql.catalyst.optimizer.OptimizeIn 563702 / 1363849256 4 / 34718 org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints 556287088 / 1322621698 1349 / 13044 ``` After this PR: ``` 23:23:13.592 WARN org.apache.spark.sql.SQLQueryTestSuite: === Metrics of Analyzer/Optimizer Rules === Total number of runs: 5323370 Total time: 148.750443534 seconds Rule Effective Time / Total Time Effective Runs / Total Runs org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries 18733079713 / 22418423535 748 / 26140 org.apache.spark.sql.catalyst.optimizer.ColumnPruning 1457156155 / 6046856332 5137 / 60797 org.apache.spark.sql.execution.datasources.FindDataSourceTable 4908256520 / 5087039927 2384 / 49272 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery 3410581306 / 4001191597 830 / 49313 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences 1731479493 / 3001691281 9241 / 49384 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions 832539494 / 2565833320 275 / 49294 org.apache.spark.sql.catalyst.optimizer.PruneFilters 15101110 / 2212401759 168 / 47701 org.apache.spark.sql.catalyst.optimizer.PushDownPredicates 461342022 / 2065929738 2284 / 47765 org.apache.spark.sql.catalyst.optimizer.SimplifyCasts 44285018 / 1920855654 287 / 34657 org.apache.spark.sql.catalyst.optimizer.ConstantFolding 681578304 / 1905774291 3565 / 34657 org.apache.spark.sql.catalyst.optimizer.NullPropagation 44365153 / 1899594511 359 / 34709 org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators 175903898 / 1869259827 2337 / 60745 org.apache.spark.sql.catalyst.optimizer.CollapseProject 640683938 / 1787207725 5877 / 47753 org.apache.spark.sql.catalyst.optimizer.BooleanSimplification 8551126 / 1690452897 54 / 34657 org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate 247737 / 1667509303 3 / 34657 org.apache.spark.sql.catalyst.optimizer.ConstantPropagation 1252963 / 1640089155 6 / 34709 org.apache.spark.sql.catalyst.optimizer.LikeSimplification 0 / 1519821245 0 / 34657 org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions 633418 / 1510769956 6 / 34657 org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals 11138539 / 1476803222 178 / 34657 org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator 0 / 1461559025 0 / 34657 org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison 2479408 / 1439506402 20 / 34657 org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions 796497708 / 1425108121 5789 / 49313 org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts 338657646 / 1416857363 1245 / 49272 org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions 0 / 1380618071 0 / 34657 org.apache.spark.sql.catalyst.optimizer.OptimizeIn 615040 / 1351329799 4 / 34709 org.apache.spark.sql.catalyst.optimizer.CombineUnions 6596639 / 1351015377 46 / 47805 org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints 533512099 / 1340374908 1246 / 13044 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
