Re: Is spark.sql.codegen.factoryMode property really for tests only?

2018-11-16 Thread Jacek Laskowski
Hi Marco, Many thanks for such a quick response. With that, I'll direct my curiosity into a different direction. Thanks! Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski Mastering Spark SQL https://bit.ly/mastering-spark-sql Spark Structured Streaming

Re: Is spark.sql.codegen.factoryMode property really for tests only?

2018-11-16 Thread Marco Gaido
Hi Jacek, I do believe it is correct. Please check the method you mentioned (CodeGeneratorWithInterpretedFallback.createObject): the value is relevant only if Utils.isTesting. Thanks, Marco Il giorno ven 16 nov 2018 alle ore 13:28 Jacek Laskowski ha scritto: > Hi, > > While reviewing the

Is spark.sql.codegen.factoryMode property really for tests only?

2018-11-16 Thread Jacek Laskowski
Hi, While reviewing the changes in 2.4 I stumbled upon spark.sql.codegen.factoryMode internal configuration property [1]. The doc says: > Note that this config works only for tests. Is that correct? I've got some doubts. I found that it's used in UnsafeProjection.create [2] (through