[jira] [Commented] (AIRFLOW-725) Make merge tool use OS' keyring for password storage

2017-02-26 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AIRFLOW-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884940#comment-15884940
 ] 

ASF subversion and git services commented on AIRFLOW-725:
-

Commit 8c169563335371562a617436575b90d3fb4d0882 in incubator-airflow's branch 
refs/heads/master from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=8c16956 ]

[AIRFLOW-725] Use keyring to store credentials for JIRA

Now allows to store the credentials in the keyring
of the OS. Retains backwards compatibility.

Closes #1966 from bolkedebruin/DEV_KEYRING


> Make merge tool use OS' keyring for password storage
> 
>
> Key: AIRFLOW-725
> URL: https://issues.apache.org/jira/browse/AIRFLOW-725
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Bolke de Bruin
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (AIRFLOW-725) Make merge tool use OS' keyring for password storage

2017-02-26 Thread Jeremiah Lowin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Lowin resolved AIRFLOW-725.

   Resolution: Fixed
Fix Version/s: 1.9.0

Issue resolved by pull request #1966
[https://github.com/apache/incubator-airflow/pull/1966]

> Make merge tool use OS' keyring for password storage
> 
>
> Key: AIRFLOW-725
> URL: https://issues.apache.org/jira/browse/AIRFLOW-725
> Project: Apache Airflow
>  Issue Type: Bug
>Reporter: Bolke de Bruin
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

incubator-airflow git commit: [AIRFLOW-725] Use keyring to store credentials for JIRA

2017-02-26 Thread jlowin 
Repository: incubator-airflow
Updated Branches:
  refs/heads/master ef6dd1b29 -> 8c1695633


[AIRFLOW-725] Use keyring to store credentials for JIRA

Now allows to store the credentials in the keyring
of the OS. Retains backwards compatibility.

Closes #1966 from bolkedebruin/DEV_KEYRING


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/8c169563
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/8c169563
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/8c169563

Branch: refs/heads/master
Commit: 8c169563335371562a617436575b90d3fb4d0882
Parents: ef6dd1b
Author: Bolke de Bruin 
Authored: Sun Feb 26 17:41:00 2017 -0500
Committer: Jeremiah Lowin 
Committed: Sun Feb 26 17:41:00 2017 -0500

--
 dev/airflow-pr   | 21 +++--
 dev/requirements.txt |  1 +
 2 files changed, 20 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/8c169563/dev/airflow-pr
--
diff --git a/dev/airflow-pr b/dev/airflow-pr
index cf8c0cb..c7f75a0 100755
--- a/dev/airflow-pr
+++ b/dev/airflow-pr
@@ -51,6 +51,12 @@ except ImportError:
 print("Could not find the click library. Run 'sudo pip install click' to 
install.")
 sys.exit(-1)
 
+try:
+import keyring
+except ImportError:
+print("Could not find the keyring library. Run 'sudo pip install keyring' 
to install.")
+sys.exit(-1)
+
 # Location of your Airflow git development area
 AIRFLOW_GIT_LOCATION = os.environ.get(
 "AIRFLOW_GIT",
@@ -77,6 +83,7 @@ BRANCH_PREFIX = "PR_TOOL"
 
 TMP_CREDENTIALS = {}
 
+
 class PRToolError(Exception):
 pass
 
@@ -460,6 +467,10 @@ def fix_version_from_branch(branch, versions):
 return versions[-1]
 
 
+def register(username, password):
+""" Use this function to register a JIRA account in your OS' keyring """
+keyring.set_password('airflow-pr', username, password)
+
 def validate_jira_id(jira_id):
 if not jira_id:
 return
@@ -542,13 +553,19 @@ def resolve_jira_issue(comment=None, jira_id=None, 
merge_branches=None):
 click.echo(
 'Set a JIRA_USERNAME env var to avoid this prompt in the future.')
 TMP_CREDENTIALS['JIRA_USERNAME'] = JIRA_USERNAME
+if JIRA_USERNAME and not JIRA_PASSWORD:
+JIRA_PASSWORD = keyring.get_password("airflow-pr", JIRA_USERNAME)
+if JIRA_PASSWORD:
+click.echo("Obtained password from keyring. To reset remove it 
there.")
 if not JIRA_PASSWORD:
 JIRA_PASSWORD = click.prompt(
 click.style('Password for Airflow JIRA', fg='blue', bold=True),
 type=str,
 hide_input=True)
-click.echo(
-'Set a JIRA_PASSWORD env var to avoid this prompt in the future.')
+if JIRA_USERNAME and JIRA_PASSWORD:
+if click.confirm(click.style("Would you like to store your 
password "
+ "in your keyring?", fg='blue', 
bold=True)):
+register(JIRA_USERNAME, JIRA_PASSWORD)
 TMP_CREDENTIALS['JIRA_PASSWORD'] = JIRA_PASSWORD
 
 try:

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/8c169563/dev/requirements.txt
--
diff --git a/dev/requirements.txt b/dev/requirements.txt
index e4a3481..29a8941 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -1,2 +1,3 @@
 click>=6.6,<7
 jira>=1.0.7,<2
+keyring==10.1

[jira] [Commented] (AIRFLOW-916) Fix ConfigParser deprecation warning

2017-02-26 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AIRFLOW-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884937#comment-15884937
 ] 

ASF subversion and git services commented on AIRFLOW-916:
-

Commit ef6dd1b29b794c5e0fd4f2bc8422a386395950f5 in incubator-airflow's branch 
refs/heads/master from [~jlowin]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=ef6dd1b ]

[AIRFLOW-916] Remove deprecated readfp function

ConfigParser.readfp() is deprecated in favor of
read_file()

Closes #2108 from jlowin/parser-deprecation


> Fix ConfigParser deprecation warning 
> -
>
> Key: AIRFLOW-916
> URL: https://issues.apache.org/jira/browse/AIRFLOW-916
> Project: Apache Airflow
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Jeremiah Lowin
>Assignee: Jeremiah Lowin
>Priority: Trivial
> Fix For: 1.9.0
>
>
> ConfigParser.readfp() is deprecated in favor of ConfigParser.read_file(), 
> according to warning messages



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (AIRFLOW-916) Fix ConfigParser deprecation warning

2017-02-26 Thread Jeremiah Lowin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AIRFLOW-916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Lowin resolved AIRFLOW-916.

   Resolution: Fixed
Fix Version/s: 1.9.0

Issue resolved by pull request #2108
[https://github.com/apache/incubator-airflow/pull/2108]

> Fix ConfigParser deprecation warning 
> -
>
> Key: AIRFLOW-916
> URL: https://issues.apache.org/jira/browse/AIRFLOW-916
> Project: Apache Airflow
>  Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Jeremiah Lowin
>Assignee: Jeremiah Lowin
>Priority: Trivial
> Fix For: 1.9.0
>
>
> ConfigParser.readfp() is deprecated in favor of ConfigParser.read_file(), 
> according to warning messages



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

incubator-airflow git commit: [AIRFLOW-916] Remove deprecated readfp function

2017-02-26 Thread jlowin 
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 4f52db317 -> ef6dd1b29


[AIRFLOW-916] Remove deprecated readfp function

ConfigParser.readfp() is deprecated in favor of
read_file()

Closes #2108 from jlowin/parser-deprecation


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/ef6dd1b2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/ef6dd1b2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/ef6dd1b2

Branch: refs/heads/master
Commit: ef6dd1b29b794c5e0fd4f2bc8422a386395950f5
Parents: 4f52db3
Author: Jeremiah Lowin 
Authored: Sun Feb 26 17:34:41 2017 -0500
Committer: Jeremiah Lowin 
Committed: Sun Feb 26 17:34:41 2017 -0500

--
 airflow/configuration.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/ef6dd1b2/airflow/configuration.py
--
diff --git a/airflow/configuration.py b/airflow/configuration.py
index cfccbe9..d68b64b 100644
--- a/airflow/configuration.py
+++ b/airflow/configuration.py
@@ -545,7 +545,7 @@ class AirflowConfigParser(ConfigParser):
 # Python 2 requires StringIO buffer
 else:
 import StringIO
-self.readfp(StringIO.StringIO(string))
+self.read_file(StringIO.StringIO(string))
 
 def _validate(self):
 if (

[jira] [Commented] (AIRFLOW-918) Improve bulk_load function for MySqlHook

2017-02-26 Thread Ali Uz (JIRA) 

[ 
https://issues.apache.org/jira/browse/AIRFLOW-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884886#comment-15884886
 ] 

Ali Uz commented on AIRFLOW-918:


It also looks like the function has not changed in version 1.8

> Improve bulk_load function for MySqlHook
> 
>
> Key: AIRFLOW-918
> URL: https://issues.apache.org/jira/browse/AIRFLOW-918
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: hooks
>Affects Versions: Airflow 1.7.1.3
>Reporter: Ali Uz
>Priority: Minor
>  Labels: easyfix, patch
>
> I think we can improve the `bulk_load` function in MySqlHook by adding a few 
> more parameters. For example, if I want to run a LOAD DATA command like the 
> following:
> ```
> LOAD DATA LOCAL INFILE 'abc.csv' INTO TABLE abc
> FIELDS TERMINATED BY ',' 
> ENCLOSED BY '"' 
> LINES TERMINATED BY '\r\n'
> IGNORE 1 LINES
> ```
> I would expect to supply the delimiter parameters, enclosing quotes 
> parameter, line terminating parameter and ignore line number parameter.
> The current function only applies the following command:
> ```
> LOAD DATA LOCAL INFILE 'abc.csv' INTO TABLE abc
> ```
> It would be great if we could extend it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (AIRFLOW-918) Improve bulk_load function for MySqlHook

2017-02-26 Thread Ali Uz (JIRA) 
Ali Uz created AIRFLOW-918:
--

 Summary: Improve bulk_load function for MySqlHook
 Key: AIRFLOW-918
 URL: https://issues.apache.org/jira/browse/AIRFLOW-918
 Project: Apache Airflow
  Issue Type: Improvement
  Components: hooks
Affects Versions: Airflow 1.7.1.3
Reporter: Ali Uz
Priority: Minor


I think we can improve the `bulk_load` function in MySqlHook by adding a few 
more parameters. For example, if I want to run a LOAD DATA command like the 
following:

```
LOAD DATA LOCAL INFILE 'abc.csv' INTO TABLE abc
FIELDS TERMINATED BY ',' 
ENCLOSED BY '"' 
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
```

I would expect to supply the delimiter parameters, enclosing quotes parameter, 
line terminating parameter and ignore line number parameter.
The current function only applies the following command:

```
LOAD DATA LOCAL INFILE 'abc.csv' INTO TABLE abc
```

It would be great if we could extend it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (AIRFLOW-916) Fix ConfigParser deprecation warning

2017-02-26 Thread Jeremiah Lowin (JIRA) 
Jeremiah Lowin created AIRFLOW-916:
--

 Summary: Fix ConfigParser deprecation warning 
 Key: AIRFLOW-916
 URL: https://issues.apache.org/jira/browse/AIRFLOW-916
 Project: Apache Airflow
  Issue Type: Improvement
Affects Versions: 1.8.0
Reporter: Jeremiah Lowin
Assignee: Jeremiah Lowin
Priority: Trivial


ConfigParser.readfp() is deprecated in favor of ConfigParser.read_file(), 
according to warning messages



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)