[jira] [Updated] (AIRFLOW-2695) Support permission of groups of dag with role
[ https://issues.apache.org/jira/browse/AIRFLOW-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng updated AIRFLOW-2695: -- Description: Follow up pr after airflow-2267, this will be done either by enhancing the logic in FAB or by providing a cli command. The approach is still TBD. (was: Follow up pr after airflow-2267) > Support permission of groups of dag with role > - > > Key: AIRFLOW-2695 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2695 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Tao Feng >Assignee: Tao Feng >Priority: Major > > Follow up pr after airflow-2267, this will be done either by enhancing the > logic in FAB or by providing a cli command. The approach is still TBD. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2695) Support assign groups of dag permission to a role
[ https://issues.apache.org/jira/browse/AIRFLOW-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng updated AIRFLOW-2695: -- Summary: Support assign groups of dag permission to a role (was: Support permission of groups of dag with role) > Support assign groups of dag permission to a role > - > > Key: AIRFLOW-2695 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2695 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Tao Feng >Assignee: Tao Feng >Priority: Major > > Follow up pr after airflow-2267, this will be done either by enhancing the > logic in FAB or by providing a cli command. The approach is still TBD. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2694) Allow parsing access control in DAG file
[
https://issues.apache.org/jira/browse/AIRFLOW-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Feng updated AIRFLOW-2694:
--
Issue Type: New Feature (was: Bug)
> Allow parsing access control in DAG file
>
>
> Key: AIRFLOW-2694
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2694
> Project: Apache Airflow
> Issue Type: New Feature
>Reporter: Tao Feng
>Assignee: Tao Feng
>Priority: Major
>
> After AIRFLOW-2267 is checkedin, modify DAG parsing logic to allow user
> passing permission logic through DAG file like the following example:
>
> dag = DAG('mydag', access_control=\{`role`: [ 'custom_role1'], `perm`:
> `can_dag_read` }, ...)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2693) Implement sync_perm model view endpoint
[ https://issues.apache.org/jira/browse/AIRFLOW-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng updated AIRFLOW-2693: -- Issue Type: New Feature (was: Bug) > Implement sync_perm model view endpoint > --- > > Key: AIRFLOW-2693 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2693 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Tao Feng >Assignee: Tao Feng >Priority: Major > > After AIRFLOW-2267 is checked-in, implement sync_perm model view endpoint > which allows airflow scheduler to call upon to sync permission with roles -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2695) Support permission of groups of dag with role
Tao Feng created AIRFLOW-2695: - Summary: Support permission of groups of dag with role Key: AIRFLOW-2695 URL: https://issues.apache.org/jira/browse/AIRFLOW-2695 Project: Apache Airflow Issue Type: New Feature Reporter: Tao Feng Assignee: Tao Feng Follow up pr after airflow-2267 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2694) Allow parsing access control in DAG file
Tao Feng created AIRFLOW-2694:
-
Summary: Allow parsing access control in DAG file
Key: AIRFLOW-2694
URL: https://issues.apache.org/jira/browse/AIRFLOW-2694
Project: Apache Airflow
Issue Type: Bug
Reporter: Tao Feng
Assignee: Tao Feng
After AIRFLOW-2267 is checkedin, modify DAG parsing logic to allow user passing
permission logic through DAG file like the following example:
dag = DAG('mydag', access_control=\{`role`: [ 'custom_role1'], `perm`:
`can_dag_read` }, ...)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-2693) Implement sync_perm model view endpoint
Tao Feng created AIRFLOW-2693: - Summary: Implement sync_perm model view endpoint Key: AIRFLOW-2693 URL: https://issues.apache.org/jira/browse/AIRFLOW-2693 Project: Apache Airflow Issue Type: Bug Reporter: Tao Feng Assignee: Tao Feng After AIRFLOW-2267 is checked-in, implement sync_perm model view endpoint which allows airflow scheduler to call upon to sync permission with roles -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-2692) Allow AWS Batch Operator to use templates in job_name parameter
[ https://issues.apache.org/jira/browse/AIRFLOW-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-2692 started by Craig Forster. -- > Allow AWS Batch Operator to use templates in job_name parameter > --- > > Key: AIRFLOW-2692 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2692 > Project: Apache Airflow > Issue Type: Bug > Components: aws >Reporter: Craig Forster >Assignee: Craig Forster >Priority: Minor > > The AWS Batch operator does not currently apply templates to the job_name > parameter. If this parameter was templated, it'd be easier to trace the > source of jobs in the AWS Batch console. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2692) Allow AWS Batch Operator to use templates in job_name parameter
[ https://issues.apache.org/jira/browse/AIRFLOW-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Forster reassigned AIRFLOW-2692: -- Assignee: Craig Forster > Allow AWS Batch Operator to use templates in job_name parameter > --- > > Key: AIRFLOW-2692 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2692 > Project: Apache Airflow > Issue Type: Bug > Components: aws >Reporter: Craig Forster >Assignee: Craig Forster >Priority: Minor > > The AWS Batch operator does not currently apply templates to the job_name > parameter. If this parameter was templated, it'd be easier to trace the > source of jobs in the AWS Batch console. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2692) Allow AWS Batch Operator to use templates in job_name parameter
Craig Forster created AIRFLOW-2692: -- Summary: Allow AWS Batch Operator to use templates in job_name parameter Key: AIRFLOW-2692 URL: https://issues.apache.org/jira/browse/AIRFLOW-2692 Project: Apache Airflow Issue Type: Bug Components: aws Reporter: Craig Forster The AWS Batch operator does not currently apply templates to the job_name parameter. If this parameter was templated, it'd be easier to trace the source of jobs in the AWS Batch console. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
incubator-airflow git commit: [AIRFLOW-2359] Add set failed for DagRun and task in tree view
Repository: incubator-airflow Updated Branches: refs/heads/master b0061f136 -> 284dbdb60 [AIRFLOW-2359] Add set failed for DagRun and task in tree view Closes #3255 from yrqls21/kevin_yang_add_set_failed Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/284dbdb6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/284dbdb6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/284dbdb6 Branch: refs/heads/master Commit: 284dbdb60ab1fec027dea4871e3013a4727f6041 Parents: b0061f1 Author: Kevin Yang Authored: Thu Jun 28 13:30:36 2018 -0700 Committer: Alex Guziel Committed: Thu Jun 28 13:30:36 2018 -0700 -- airflow/api/common/experimental/mark_tasks.py | 113 +++- airflow/jobs.py | 3 +- airflow/models.py | 4 +- airflow/www/templates/airflow/dag.html| 43 +++ airflow/www/views.py | 295 +++-- airflow/www_rbac/templates/airflow/dag.html | 43 +++ airflow/www_rbac/views.py | 237 - docs/scheduler.rst| 1 + tests/api/common/experimental/mark_tasks.py | 275 --- tests/dags/test_example_bash_operator.py | 5 +- tests/www_rbac/test_views.py | 8 +- 11 files changed, 765 insertions(+), 262 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/284dbdb6/airflow/api/common/experimental/mark_tasks.py -- diff --git a/airflow/api/common/experimental/mark_tasks.py b/airflow/api/common/experimental/mark_tasks.py index e9e4fec..681864d 100644 --- a/airflow/api/common/experimental/mark_tasks.py +++ b/airflow/api/common/experimental/mark_tasks.py @@ -17,15 +17,16 @@ # specific language governing permissions and limitations # under the License. +from sqlalchemy import or_ + from airflow.jobs import BackfillJob from airflow.models import DagRun, TaskInstance from airflow.operators.subdag_operator import SubDagOperator from airflow.settings import Session from airflow.utils import timezone +from airflow.utils.db import provide_session from airflow.utils.state import State -from sqlalchemy import or_ - def _create_dagruns(dag, execution_dates, state, run_id_template): """ @@ -191,15 +192,36 @@ def set_state(task, execution_date, upstream=False, downstream=False, return tis_altered -def set_dag_run_state(dag, execution_date, state=State.SUCCESS, commit=False): +def _set_dag_run_state(dag_id, execution_date, state, session=None): +""" +Helper method that set dag run state in the DB. +:param dag_id: dag_id of target dag run +:param execution_date: the execution date from which to start looking +:param state: target state +:param session: database session """ -Set the state of a dag run and all task instances associated with the dag -run for a specific execution date. +DR = DagRun +dr = session.query(DR).filter( +DR.dag_id == dag_id, +DR.execution_date == execution_date +).one() +dr.state = state +dr.end_date = timezone.utcnow() +session.commit() + + +@provide_session +def set_dag_run_state_to_success(dag, execution_date, commit=False, + session=None): +""" +Set the dag run for a specific execution date and its task instances +to success. :param dag: the DAG of which to alter state :param execution_date: the execution date from which to start looking -:param state: the state to which the DAG need to be set :param commit: commit DAG and tasks to be altered to the database -:return: list of tasks that have been created and updated +:param session: database session +:return: If commit is true, list of tasks that have been updated, + otherwise list of tasks that will be updated :raises: AssertionError if dag or execution_date is invalid """ res = [] @@ -207,18 +229,81 @@ def set_dag_run_state(dag, execution_date, state=State.SUCCESS, commit=False): if not dag or not execution_date: return res -# Mark all task instances in the dag run +# Mark the dag run to success. +if commit: +_set_dag_run_state(dag.dag_id, execution_date, State.SUCCESS, session) + +# Mark all task instances of the dag run to success. for task in dag.tasks: task.dag = dag new_state = set_state(task=task, execution_date=execution_date, - state=state, commit=commit) + state=State.SUCCESS, commit=commit) res.extend(new_state) -# Mark the dag run +retu
[jira] [Commented] (AIRFLOW-2359) Add set failed for DagRun and TaskInstance in tree view
[ https://issues.apache.org/jira/browse/AIRFLOW-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526752#comment-16526752 ] ASF subversion and git services commented on AIRFLOW-2359: -- Commit 284dbdb60ab1fec027dea4871e3013a4727f6041 in incubator-airflow's branch refs/heads/master from [~kevinyang] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=284dbdb ] [AIRFLOW-2359] Add set failed for DagRun and task in tree view Closes #3255 from yrqls21/kevin_yang_add_set_failed > Add set failed for DagRun and TaskInstance in tree view > --- > > Key: AIRFLOW-2359 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2359 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > Fix For: 1.10.0, 2.0.0 > > > User has been requesting to add set failed in tree view for DagRun and > TaskInstance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2687) Move unittest2 from devel_ci to devel in setup.py
[
https://issues.apache.org/jira/browse/AIRFLOW-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kengo Seki reassigned AIRFLOW-2687:
---
Assignee: Kengo Seki
> Move unittest2 from devel_ci to devel in setup.py
> -
>
> Key: AIRFLOW-2687
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2687
> Project: Apache Airflow
> Issue Type: Test
> Components: tests
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> #3549 added {{unittest2}} package to {{devel_ci}}, but I think it should be
> in {{devel}}.
> https://github.com/apache/incubator-airflow/pull/3549/files#diff-2eeaed663bd0d25b7e608891384b7298
> Because without that package, users can't run even a basic test locally.
> {code}
> $ pip install --upgrade -e ".[all]"
> (snip)
> Successfully installed apache-airflow
> $ ./run_unit_tests.sh tests/configuration.py
> (snip)
> ImportError: No module named unittest2
> (snip)
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2687) Move unittest2 from devel_ci to devel in setup.py
[
https://issues.apache.org/jira/browse/AIRFLOW-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526608#comment-16526608
]
Kengo Seki commented on AIRFLOW-2687:
-
Thanks for the comment [~ashb]! I've just submitted a PR.
> Move unittest2 from devel_ci to devel in setup.py
> -
>
> Key: AIRFLOW-2687
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2687
> Project: Apache Airflow
> Issue Type: Test
> Components: tests
>Reporter: Kengo Seki
>Priority: Major
>
> #3549 added {{unittest2}} package to {{devel_ci}}, but I think it should be
> in {{devel}}.
> https://github.com/apache/incubator-airflow/pull/3549/files#diff-2eeaed663bd0d25b7e608891384b7298
> Because without that package, users can't run even a basic test locally.
> {code}
> $ pip install --upgrade -e ".[all]"
> (snip)
> Successfully installed apache-airflow
> $ ./run_unit_tests.sh tests/configuration.py
> (snip)
> ImportError: No module named unittest2
> (snip)
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2691) Make Airflow's JS code (and dependencies) manageable via npm and webpack
[ https://issues.apache.org/jira/browse/AIRFLOW-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Verdan Mahmood updated AIRFLOW-2691: Description: Airflow's JS code is hard to maintain and upgrade. The dependencies are locally existing files making it hard to upgrade versions. Make sure Airflow uses *npm* and *webpack* for the dependencies management. was: Airflow's JS code is hard to maintain and upgrade. The dependencies are locally existing files making it hard to upgrade versions. Make sure Airflow uses `npm` and `webpack` for the dependencies management. > Make Airflow's JS code (and dependencies) manageable via npm and webpack > > > Key: AIRFLOW-2691 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2691 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Verdan Mahmood >Assignee: Verdan Mahmood >Priority: Major > > Airflow's JS code is hard to maintain and upgrade. The dependencies are > locally existing files making it hard to upgrade versions. > Make sure Airflow uses *npm* and *webpack* for the dependencies management. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2691) Make Airflow's JS code (and dependencies) manageable via npm and webpack
Verdan Mahmood created AIRFLOW-2691: --- Summary: Make Airflow's JS code (and dependencies) manageable via npm and webpack Key: AIRFLOW-2691 URL: https://issues.apache.org/jira/browse/AIRFLOW-2691 Project: Apache Airflow Issue Type: New Feature Reporter: Verdan Mahmood Assignee: Verdan Mahmood Airflow's JS code is hard to maintain and upgrade. The dependencies are locally existing files making it hard to upgrade versions. Make sure Airflow uses `npm` and `webpack` for the dependencies management. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2690) Support structured logging in Airflow
David created AIRFLOW-2690: -- Summary: Support structured logging in Airflow Key: AIRFLOW-2690 URL: https://issues.apache.org/jira/browse/AIRFLOW-2690 Project: Apache Airflow Issue Type: Wish Components: logging Affects Versions: 1.9.0 Reporter: David Support structured logging in Airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2689) Add documentation on connection types
Nicholas Pezolano created AIRFLOW-2689: -- Summary: Add documentation on connection types Key: AIRFLOW-2689 URL: https://issues.apache.org/jira/browse/AIRFLOW-2689 Project: Apache Airflow Issue Type: Improvement Reporter: Nicholas Pezolano -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2672) Update documentation for S3 connections and logging
[
https://issues.apache.org/jira/browse/AIRFLOW-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Pezolano updated AIRFLOW-2672:
---
Description:
Documentation currently lacks how to set S3 credentials that point to an s3
credentials file
ex:
{code:java}
{ "profile": "", "s3_config_file":
"/home//creds/s3_credentials", "s3_config_format": "aws" }
{code}
[https://stackoverflow.com/questions/39997714/airflow-s3-connection-using-ui/41985070#41985070]
was:
Documentation currently lacks how to set S3 credentials that point to an s3
credentials file
ex:
{{{ "profile": "", "s3_config_file":
"/home//creds/s3_credentials", "s3_config_format": "aws" }}}
https://stackoverflow.com/questions/39997714/airflow-s3-connection-using-ui/41985070#41985070
> Update documentation for S3 connections and logging
> ---
>
> Key: AIRFLOW-2672
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2672
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Nicholas Pezolano
>Priority: Minor
>
> Documentation currently lacks how to set S3 credentials that point to an s3
> credentials file
> ex:
> {code:java}
> { "profile": "", "s3_config_file":
> "/home//creds/s3_credentials", "s3_config_format": "aws" }
> {code}
>
> [https://stackoverflow.com/questions/39997714/airflow-s3-connection-using-ui/41985070#41985070]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2672) Update documentation for S3 connections and logging
[
https://issues.apache.org/jira/browse/AIRFLOW-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Pezolano updated AIRFLOW-2672:
---
Description:
Documentation currently lacks how to set S3 credentials that point to an s3
credentials file
ex:
{{{ "profile": "", "s3_config_file":
"/home//creds/s3_credentials", "s3_config_format": "aws" }}}
https://stackoverflow.com/questions/39997714/airflow-s3-connection-using-ui/41985070#41985070
> Update documentation for S3 connections and logging
> ---
>
> Key: AIRFLOW-2672
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2672
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Nicholas Pezolano
>Priority: Minor
>
> Documentation currently lacks how to set S3 credentials that point to an s3
> credentials file
> ex:
> {{{ "profile": "", "s3_config_file":
> "/home//creds/s3_credentials", "s3_config_format": "aws" }}}
> https://stackoverflow.com/questions/39997714/airflow-s3-connection-using-ui/41985070#41985070
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-2668) Missing cryptogrpahy dependency on airflow initdb call
[
https://issues.apache.org/jira/browse/AIRFLOW-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Pezolano reassigned AIRFLOW-2668:
--
Assignee: Ash Berlin-Taylor
> Missing cryptogrpahy dependency on airflow initdb call
> --
>
> Key: AIRFLOW-2668
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2668
> Project: Apache Airflow
> Issue Type: Bug
> Components: dependencies
>Affects Versions: 1.10.0
>Reporter: Nicholas Pezolano
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Fix For: 1.10.0
>
>
> The cryptography packages looks to be required now for `airflow initdb` calls
> on a fresh install from master as of commit
> 702a57ec5a96d159105c4f5ca76ddd2229eb2f44.
> $ airflow initdb
> Traceback (most recent call last):
> File "/home/n/git/airflow_testing/env/bin/airflow", line 6, in
> exec(compile(open(__file__).read(), __file__, 'exec'))
> File "/home/n/git/incubator-airflow/airflow/bin/airflow", line 21, in
>
> from airflow import configuration
> File "/home/n/git/incubator-airflow/airflow/__init__.py", line 37, in
>
> from airflow.models import DAG
> File "/home/n/git/incubator-airflow/airflow/models.py", line 31, in
> import cryptography
> ImportError: No module named cryptography
>
> Steps to reproduce:
> {code}
> git clone https://github.com/apache/incubator-airflow
> virtualenv env
> . env/bin/activate
> pip install -e .[s3]
> airflow initdb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2687) Move unittest2 from devel_ci to devel in setup.py
[
https://issues.apache.org/jira/browse/AIRFLOW-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526268#comment-16526268
]
Ash Berlin-Taylor commented on AIRFLOW-2687:
Whops yes, not sure why I put it in devel_ci.
> Move unittest2 from devel_ci to devel in setup.py
> -
>
> Key: AIRFLOW-2687
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2687
> Project: Apache Airflow
> Issue Type: Test
> Components: tests
>Reporter: Kengo Seki
>Priority: Major
>
> #3549 added {{unittest2}} package to {{devel_ci}}, but I think it should be
> in {{devel}}.
> https://github.com/apache/incubator-airflow/pull/3549/files#diff-2eeaed663bd0d25b7e608891384b7298
> Because without that package, users can't run even a basic test locally.
> {code}
> $ pip install --upgrade -e ".[all]"
> (snip)
> Successfully installed apache-airflow
> $ ./run_unit_tests.sh tests/configuration.py
> (snip)
> ImportError: No module named unittest2
> (snip)
> FAILED (errors=1)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (AIRFLOW-2688) Support both 'AES256' and 'aws:kms' for ServerSideEncryption in S3Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-2688 started by Angelos Alexopoulos. > Support both 'AES256' and 'aws:kms' for ServerSideEncryption in S3Hook > -- > > Key: AIRFLOW-2688 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2688 > Project: Apache Airflow > Issue Type: Improvement > Components: logging >Affects Versions: Airflow 2.0, 1.9.0 >Reporter: Angelos Alexopoulos >Assignee: Angelos Alexopoulos >Priority: Major > > Currently in file airflow/hooks/S3_hook.py we support reading and writing > encrypted logs to S3 bucket. However, there are cases that we need to support > both 'AES256' and 'aws:kms' for ServerSideEncryption. So the best thing would > be to add a new setting to airflow.cfg > *server_side_s3_encryption* with default value AES256. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2688) Support both 'AES256' and 'aws:kms' for ServerSideEncryption in S3Hook
Angelos Alexopoulos created AIRFLOW-2688: Summary: Support both 'AES256' and 'aws:kms' for ServerSideEncryption in S3Hook Key: AIRFLOW-2688 URL: https://issues.apache.org/jira/browse/AIRFLOW-2688 Project: Apache Airflow Issue Type: Improvement Components: logging Affects Versions: Airflow 2.0, 1.9.0 Reporter: Angelos Alexopoulos Assignee: Angelos Alexopoulos Currently in file airflow/hooks/S3_hook.py we support reading and writing encrypted logs to S3 bucket. However, there are cases that we need to support both 'AES256' and 'aws:kms' for ServerSideEncryption. So the best thing would be to add a new setting to airflow.cfg *server_side_s3_encryption* with default value AES256. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2687) Move unittest2 from devel_ci to devel in setup.py
Kengo Seki created AIRFLOW-2687:
---
Summary: Move unittest2 from devel_ci to devel in setup.py
Key: AIRFLOW-2687
URL: https://issues.apache.org/jira/browse/AIRFLOW-2687
Project: Apache Airflow
Issue Type: Test
Components: tests
Reporter: Kengo Seki
#3549 added {{unittest2}} package to {{devel_ci}}, but I think it should be in
{{devel}}.
https://github.com/apache/incubator-airflow/pull/3549/files#diff-2eeaed663bd0d25b7e608891384b7298
Because without that package, users can't run even a basic test locally.
{code}
$ pip install --upgrade -e ".[all]"
(snip)
Successfully installed apache-airflow
$ ./run_unit_tests.sh tests/configuration.py
(snip)
ImportError: No module named unittest2
(snip)
FAILED (errors=1)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
