[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

[Apache Spark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-15795:


Assignee: Apache Spark

> Enable more optimizations in whole stage codegen when isNull is a 
> compile-time constant
> ---
>
> Key: SPARK-15795
> URL: https://issues.apache.org/jira/browse/SPARK-15795
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Reporter: Hiroshi Inoue
>Assignee: Apache Spark
>
> Whole stage codegen often creates {{isNull}} variable initialized with 
> constant _false_, like
> {{boolean mapelements_isNull = false || false;}}
> If there is no further assignment for this {{isNull}} variable, whole stage 
> codegen can do more optimizations by assuming {{isNull}} as a compile-time 
> constant.
> In the example below, which is generated for a dataset map operation, 
> {{mapelements_isNull}} defined at line 115 can be assumed by a compile-time 
> constant (false). 
> By assuming this as a constant, the whole stage codegen eliminates 
> {{zeroOutNullBytes}} at line 119 and an if-statement at line 121.
> In addition to the benefits of improved readability of generated code, 
> eliminating {{zeroOutNullBytes}} will give performance advantage since it is 
> difficult to remove for Java JIT compiler.
> {code}
> /* 107 */   // CONSUME: Project [id#0L AS l#3L]
> /* 108 */   // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint
> /* 109 */   // CONSUME: MapElements , obj#17: bigint
> /* 110 */   // CONSUME: SerializeFromObject [input[0, bigint, true] AS 
> value#18L]
> /* 111 */   // .apply
> /* 112 */   Object mapelements_obj = ((Expression) 
> references[1]).eval(null);
> /* 113 */   scala.Function1 mapelements_value1 = (scala.Function1) 
> mapelements_obj;
> /* 114 */
> /* 115 */   boolean mapelements_isNull = false || false;
> /* 116 */   final long mapelements_value = mapelements_isNull ? -1L : 
> (Long) mapelements_value1.apply(range_value);
> /* 117 */
> /* 118 */   // CONSUME: WholeStageCodegen
> /* 119 */   serializefromobject_rowWriter.zeroOutNullBytes();
> /* 120 */
> /* 121 */   if (mapelements_isNull) {
> /* 122 */ serializefromobject_rowWriter.setNullAt(0);
> /* 123 */   } else {
> /* 124 */ serializefromobject_rowWriter.write(0, mapelements_value);
> /* 125 */   }
> /* 126 */   append(serializefromobject_result);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Assigned] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

[Apache Spark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-15795:


Assignee: (was: Apache Spark)

> Enable more optimizations in whole stage codegen when isNull is a 
> compile-time constant
> ---
>
> Key: SPARK-15795
> URL: https://issues.apache.org/jira/browse/SPARK-15795
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Reporter: Hiroshi Inoue
>
> Whole stage codegen often creates {{isNull}} variable initialized with 
> constant _false_, like
> {{boolean mapelements_isNull = false || false;}}
> If there is no further assignment for this {{isNull}} variable, whole stage 
> codegen can do more optimizations by assuming {{isNull}} as a compile-time 
> constant.
> In the example below, which is generated for a dataset map operation, 
> {{mapelements_isNull}} defined at line 115 can be assumed by a compile-time 
> constant (false). 
> By assuming this as a constant, the whole stage codegen eliminates 
> {{zeroOutNullBytes}} at line 119 and an if-statement at line 121.
> In addition to the benefits of improved readability of generated code, 
> eliminating {{zeroOutNullBytes}} will give performance advantage since it is 
> difficult to remove for Java JIT compiler.
> {code}
> /* 107 */   // CONSUME: Project [id#0L AS l#3L]
> /* 108 */   // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint
> /* 109 */   // CONSUME: MapElements , obj#17: bigint
> /* 110 */   // CONSUME: SerializeFromObject [input[0, bigint, true] AS 
> value#18L]
> /* 111 */   // .apply
> /* 112 */   Object mapelements_obj = ((Expression) 
> references[1]).eval(null);
> /* 113 */   scala.Function1 mapelements_value1 = (scala.Function1) 
> mapelements_obj;
> /* 114 */
> /* 115 */   boolean mapelements_isNull = false || false;
> /* 116 */   final long mapelements_value = mapelements_isNull ? -1L : 
> (Long) mapelements_value1.apply(range_value);
> /* 117 */
> /* 118 */   // CONSUME: WholeStageCodegen
> /* 119 */   serializefromobject_rowWriter.zeroOutNullBytes();
> /* 120 */
> /* 121 */   if (mapelements_isNull) {
> /* 122 */ serializefromobject_rowWriter.setNullAt(0);
> /* 123 */   } else {
> /* 124 */ serializefromobject_rowWriter.write(0, mapelements_value);
> /* 125 */   }
> /* 126 */   append(serializefromobject_result);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org