Hiroshi Inoue created SPARK-15795: ------------------------------------- Summary: Enable more optimizations in whole stage codegen when isNull is a compile-time constant Key: SPARK-15795 URL: https://issues.apache.org/jira/browse/SPARK-15795 Project: Spark Issue Type: Improvement Components: SQL Reporter: Hiroshi Inoue
Whole stage codegen often creates {{isNull}} variable initialized with constant _false_, like {{boolean mapelements_isNull = false || false;}} If there is no further assignment for this {{isNull}} variable, whole stage codegen can do more optimizations by assuming {{isNull}} as a compile-time constant. In the example below, which is generated for a dataset map operation, {{mapelements_isNull}} defined at line 115 can be assumed by a compile-time constant (false). By assuming this as a constant, the whole stage codegen eliminates {{zeroOutNullBytes}} at line 119 and an if-statement at line 121. In addition to the benefits of improved readability of generated code, eliminating {{zeroOutNullBytes}} will give performance advantage since it is difficult to remove for Java JIT compiler. {code} /* 107 */ // CONSUME: Project [id#0L AS l#3L] /* 108 */ // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint /* 109 */ // CONSUME: MapElements <function1>, obj#17: bigint /* 110 */ // CONSUME: SerializeFromObject [input[0, bigint, true] AS value#18L] /* 111 */ // <function1>.apply /* 112 */ Object mapelements_obj = ((Expression) references[1]).eval(null); /* 113 */ scala.Function1 mapelements_value1 = (scala.Function1) mapelements_obj; /* 114 */ /* 115 */ boolean mapelements_isNull = false || false; /* 116 */ final long mapelements_value = mapelements_isNull ? -1L : (Long) mapelements_value1.apply(range_value); /* 117 */ /* 118 */ // CONSUME: WholeStageCodegen /* 119 */ serializefromobject_rowWriter.zeroOutNullBytes(); /* 120 */ /* 121 */ if (mapelements_isNull) { /* 122 */ serializefromobject_rowWriter.setNullAt(0); /* 123 */ } else { /* 124 */ serializefromobject_rowWriter.write(0, mapelements_value); /* 125 */ } /* 126 */ append(serializefromobject_result); {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org