Hi litao, sparkSql function calls withprofiler function method and whenever the queryExecution object and SQLStart is made, this will call the generateDF function, which creates the new DataSet object.
So once the queryExecution object is made from logical plan, we call assertAnalyzed() which executes analyzed(refer dataset class in spark, it is a lazy variable, it will get called when assertAnalyzed is called) by accessing it and throwing the result away. Since analyzed is a lazy value in Scala, it will then get initialized for the first time and stays so forever. basically assertAnalyzed analyzes the rules present in Analyzer and mark the plan as Analyzed. Since it is lazy, only once evaluation happens and result is stored. So As i said, it will create new dataset once QE object is got, again assertAnalyzed is called, but evaluation does not happens. I hope you i have given complete explaination. Please reply if you have doubt -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
