[ https://issues.apache.org/jira/browse/AIRFLOW-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Kotecha updated AIRFLOW-2661: ---------------------------------- Description: It looks like the configuration options: {code:java} dags_volume_subpath = logs_volume_subpath = {code} don't do anything when set. This is because the {{_constructVolume()}} function is called in [https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124]. It sets subPath on the {{volume}} rather than the mount. This function should be changed to not take the optional parameter {{subPath}} and we should use {{dags_volume_subpath}} if provided in the construction of the {{volumeMount}} If you look at [https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can see that it's a property of {{volumeMount}} not a {{Volume}}. We are correctly handling this for the the {{configMap}} on line 167. My colleague has also commented on the original PR that added this functionality: [https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135] Pull Request: https://github.com/apache/incubator-airflow/pull/3537 was: It looks like the configuration options: {code:java} dags_volume_subpath = logs_volume_subpath = {code} don't do anything when set. This is because the {{_constructVolume()}} function is called in [https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124]. It sets subPath on the {{volume}} rather than the mount. This function should be changed to not take the optional parameter {{subPath}} and we should use {{dags_volume_subpath}} if provided in the construction of the {{volumeMount}} If you look at [https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can see that it's a property of {{volumeMount}} not a {{Volume}}. We are correctly handling this for the the {{configMap}} on line 167. My colleague has also commented on the original PR that added this functionality: [https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135] > Kubernetes Executor - dags_volume_subpath and logs_volume_subpath don't do > anything > ----------------------------------------------------------------------------------- > > Key: AIRFLOW-2661 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2661 > Project: Apache Airflow > Issue Type: Bug > Components: executor > Affects Versions: Airflow 2.0 > Environment: Kubernetes 1.10 (kops) > Reporter: Ravi Kotecha > Assignee: Ravi Kotecha > Priority: Major > Labels: Kubernetes, pull-request-available > Fix For: Airflow 2.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > It looks like the configuration options: > {code:java} > dags_volume_subpath = > logs_volume_subpath = > {code} > don't do anything when set. This is because the > {{_constructVolume()}} > function is called in > [https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124]. > It sets subPath on the {{volume}} rather than the mount. > This function should be changed to not take the optional parameter > {{subPath}} and we should use {{dags_volume_subpath}} if provided in the > construction of the {{volumeMount}} > If you look at > [https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can > see that it's a property of {{volumeMount}} not a {{Volume}}. We are > correctly handling this for the the {{configMap}} on line 167. > My colleague has also commented on the original PR that added this > functionality: > [https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135] > > Pull Request: > https://github.com/apache/incubator-airflow/pull/3537 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)