[
https://issues.apache.org/jira/browse/AIRFLOW-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882265#comment-16882265
]
ASF GitHub Bot commented on AIRFLOW-4919:
-----------------------------------------
eladkal commented on pull request #5555: [AIRFLOW-4919] DataProcJobBaseOperator
dataproc_*_properties templated
URL: https://github.com/apache/airflow/pull/5555
Make sure you have checked _all_ steps below.
### Jira
https://issues.apache.org/jira/browse/AIRFLOW-4919
### Description
Same as https://github.com/apache/airflow/pull/5554 for the v1-10 branch.
Add dataproc_*_properties as templated:
dataproc_pig_properties
dataproc_hive_properties
dataproc_spark_properties
dataproc_hadoop_properties
dataproc_pyspark_properties
@potiuk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add 'dataproc_properties' parameter to templated fields of
> DataProcJobBaseOperator child classes
> ------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-4919
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4919
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib, gcp
> Affects Versions: 1.9.0, 1.10.3
> Reporter: Adam Kramer
> Assignee: Elad
> Priority: Major
>
> Properties in Dataproc jobs have a need for templating in cases of default
> values and other labeling properties, such as macro dates.
> dataproc_pig_properties in DataProcPigOperator
> dataproc_hive_properties in DataProcHiveOperator
> dataproc_spark_properties in DataProcSparkSqlOperator
> dataproc_hadoop_properties in DataProcHadoopOperator
> Additionally, should add generic dataproc_properties to each as
> templated_field.
> Update py docs to include that it is templated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
