[ https://issues.apache.org/jira/browse/AIRFLOW-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669459#comment-15669459 ]
ASF subversion and git services commented on AIRFLOW-130: --------------------------------------------------------- Commit 664e63a72d3b6dcacf10ae019a16109fa5c7beea in incubator-airflow's branch refs/heads/master from [~jbhsieh] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=664e63a ] [AIRFLOW-130] Fix ssh operator macosx Copy existing shell environment instead of overwriting Testing Done: - Unit tests previously did not pass on Mac OS X, now pass. Fix typos in comments Fix failing test when setting env for SSHHook on Mac OS X Add logging and use os.environ.copy() to set env in popen fixes test. Move import to top of file Closes #1778 from jbhsieh/fix_ssh_operator_macosx > SSHHook fails in OSX due to mktemp implementation > ------------------------------------------------- > > Key: AIRFLOW-130 > URL: https://issues.apache.org/jira/browse/AIRFLOW-130 > Project: Apache Airflow > Issue Type: Bug > Components: hooks > Affects Versions: Airflow 1.7.0 > Environment: Airflow components: > webserver and scheduler with a mysql database and LocalExecutor > Relevant `airflow.cfg` settings: > ssh_default connection correctly configured > Python Version: > Python 2.7.11 > Operating System: > Darwin Kernel Version 15.4.0: Fri Feb 26 22:08:05 PST 2016; > root:xnu-3248.40.184~3/RELEASE_X86_64 x86_64 > Python packages: > airflow (1.7.0) > alembic (0.8.6) > Babel (1.3) > cffi (1.6.0) > chartkick (0.4.2) > croniter (0.3.12) > cryptography (1.3.2) > dill (0.2.5) > enum34 (1.1.6) > Flask (0.10.1) > Flask-Admin (1.4.0) > Flask-Cache (0.13.1) > Flask-Login (0.2.11) > Flask-WTF (0.12) > future (0.15.2) > gunicorn (19.3.0) > idna (2.1) > ipaddress (1.0.16) > itsdangerous (0.24) > Jinja2 (2.8) > Mako (1.0.4) > Markdown (2.6.6) > MarkupSafe (0.23) > mysqlclient (1.3.7) > netaddr (0.7.18) > numpy (1.11.0) > pandas (0.18.1) > pip (8.1.2) > pyasn1 (0.1.9) > pycparser (2.14) > Pygments (2.1.3) > python-dateutil (2.5.3) > python-editor (1.0) > pytz (2016.4) > requests (2.10.0) > setproctitle (1.1.10) > setuptools (21.0.0) > six (1.10.0) > slackclient (1.0.0) > SQLAlchemy (1.0.13) > thrift (0.9.3) > websocket-client (0.37.0) > Werkzeug (0.11.9) > wheel (0.29.0) > WTForms (2.1) > Reporter: Raul Landa > > h3. Description of Issue > * What did you expect to happen? > In the {{__enter__}} method in the {{SSHTempFileContent}} class > ({{ssh_execute_operator.py}}) a file should be created using {{mktemp}} > * What happened instead? > The call to mktemp fails, because the {{--tmpdir}} command line option is not > supported in OSX > (see > https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/mktemp.1.html) > * Stack trace, if appropriate: > {noformat} > [2016-05-18 17:39:31,735] {models.py:1144} ERROR - Failed to create remote > temp file > Traceback (most recent call last): > File "/Users/rlanda/.virtualenvs/datapath-airflow/bin/airflow", line 15, in > <module> > args.func(args) > File > "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/bin/cli.py", > line 316, in test > ti.run(force=True, ignore_dependencies=True, test_mode=True) > File > "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/models.py", > line 1067, in run > result = task_copy.execute(context=context) > File > "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/contrib/operators/ssh_execute_operator.py", > line 116, in execute > self.task_id) as remote_file_path: > File > "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/contrib/operators/ssh_execute_operator.py", > line 52, in __enter__ > raise AirflowException("Failed to create remote temp file") > airflow.utils.AirflowException: Failed to create remote temp file > {noformat} > The stack trace is not very useful. However looking at the tuple returned by > {{pmktemp.communicate()}} is helpful: > {noformat} > mktemp: illegal option -- - > usage: mktemp [-d] [-q] [-t prefix] [-u] template ... > mktemp [-d] [-q] [-u] -t prefix > {noformat} > h3. Reproducing the Issue > Issue happens with normal usage of {{SSHHook}} when attempting to run > commands on an OSX box. > h3. Fix > Changing {{--tmpdir}} by {{-t}} seems to fix the issue in OSX, but I do not > have access to a linux box to check if it breaks there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)