AFAIK, stdout of the task will be included in the task_instance log, you
might get what you want by just printing it.
Cheers,
Kevin Y
On Fri, Jun 1, 2018 at 12:20 PM Martin Gauthier <gauthiermarti...@gmail.com>
wrote:
> Here is the output i am getting from the log in airflow UI
>
> *** Reading local log.
> [2018-06-01 15:15:55,979] {{cli.py:374}} INFO - Running on host
> airflow-webserver
> [2018-06-01 15:15:56,042] {{models.py:1197}} INFO - Dependencies all
> met for <TaskInstance:
> promenade_fleuve_montagne_dag.write_geo_data_polylines 2018-06-01
> 19:15:16.584033 [queued]>
> [2018-06-01 15:15:56,090] {{models.py:1197}} INFO - Dependencies all
> met for <TaskInstance:
> promenade_fleuve_montagne_dag.write_geo_data_polylines 2018-06-01
> 19:15:16.584033 [queued]>
> [2018-06-01 15:15:56,090] {{models.py:1407}} INFO -
>
> --------------------------------------------------------------------------------
> Starting attempt 1 of 2
>
> --------------------------------------------------------------------------------
>
> [2018-06-01 15:15:56,127] {{models.py:1428}} INFO - Executing
> <Task(PythonOperator): write_geo_data_polylines> on 2018-06-01
> 19:15:16.584033
> [2018-06-01 15:15:56,127] {{base_task_runner.py:115}} INFO - Running:
> ['bash', '-c', 'airflow run promenade_fleuve_montagne_dag
> write_geo_data_polylines 2018-06-01T19:15:16.584033 --job_id 2080
> --raw -sd DAGS_FOLDER/imontagne_dag.py']
> [2018-06-01 15:15:56,970] {{base_task_runner.py:98}} INFO - Subtask:
> [2018-06-01 15:15:56,966] {{__init__.py:45}} INFO - Using executor
> LocalExecutor
> [2018-06-01 15:15:57,015] {{base_task_runner.py:98}} INFO - Subtask:
> [2018-06-01 15:15:57,009] {{models.py:189}} INFO - Filling up the
> DagBag from /usr/local/airflow/dags/montagne_dag.py
> [2018-06-01 15:15:57,808] {{cli.py:374}} INFO - Running on host
> airflow-webserver
> [2018-06-01 15:15:57,946] {{base_task_runner.py:98}} INFO - Subtask:
> Warning 6: dataset
>
> /usr/local/airflow/data/output/montagne_dag/2018-06-01T19:15:16.584033/fleuve_montagne_polylines.geojson
> does not support layer creation option ENCODING
> [2018-06-01 15:15:57,994] {{base_task_runner.py:98}} INFO - Subtask:
> [2018-06-01 15:15:57,976] {{python_operator.py:90}} INFO - Done.
> Returned value was: None
> [2018-06-01 15:15:58,065] {{base_task_runner.py:98}} INFO - Subtask:
> /usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144:
> UserWarning: The psycopg2 wheel package will be renamed from release
> 2.8; in order to keep installing from binary please use "pip install
> psycopg2-binary" instead. For details see:
> <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
> [2018-06-01 15:15:58,065] {{base_task_runner.py:98}} INFO - Subtask: """)
>
>
>
> On Fri, Jun 1, 2018 at 2:33 PM, Martin Gauthier <
> gauthiermarti...@gmail.com>
> wrote:
>
> > For example this python operator
> >
> > write_polylines_geo_data = PythonOperator(
> > task_id='write_geo_data_polylines',
> > provide_context=True,
> > python_callable=generate_geo_data_files_from_wkt_data_file,
> > op_args=[
> > polylines_staging_file_loc,
> > 'pickle',
> > WGS54,
> > 'geometry',
> > None,
> > (polylines_geojson_file_location, 'geojson', WGS54),
> > (polylines_shape_file_location, 'shape', MTM5
> > )
> > ],
> > dag=dag
> > )
> >
> >
> > I would like to log actions done into the function
> > "generate_geo_data_files_from_wkt_data_file" would i need to define a
> > logger in this function and then it should do the trick ?
> >
> >
> >
>
>
> --
> Martin Gauthier
>
> C: 438-499-1749
>
