Here is the output i am getting from the log in airflow UI
*** Reading local log.
[2018-06-01 15:15:55,979] {{cli.py:374}} INFO - Running on host
airflow-webserver
[2018-06-01 15:15:56,042] {{models.py:1197}} INFO - Dependencies all
met for <TaskInstance:
promenade_fleuve_montagne_dag.write_geo_data_polylines 2018-06-01
19:15:16.584033 [queued]>
[2018-06-01 15:15:56,090] {{models.py:1197}} INFO - Dependencies all
met for <TaskInstance:
promenade_fleuve_montagne_dag.write_geo_data_polylines 2018-06-01
19:15:16.584033 [queued]>
[2018-06-01 15:15:56,090] {{models.py:1407}} INFO -
--------------------------------------------------------------------------------
Starting attempt 1 of 2
--------------------------------------------------------------------------------
[2018-06-01 15:15:56,127] {{models.py:1428}} INFO - Executing
<Task(PythonOperator): write_geo_data_polylines> on 2018-06-01
19:15:16.584033
[2018-06-01 15:15:56,127] {{base_task_runner.py:115}} INFO - Running:
['bash', '-c', 'airflow run promenade_fleuve_montagne_dag
write_geo_data_polylines 2018-06-01T19:15:16.584033 --job_id 2080
--raw -sd DAGS_FOLDER/imontagne_dag.py']
[2018-06-01 15:15:56,970] {{base_task_runner.py:98}} INFO - Subtask:
[2018-06-01 15:15:56,966] {{__init__.py:45}} INFO - Using executor
LocalExecutor
[2018-06-01 15:15:57,015] {{base_task_runner.py:98}} INFO - Subtask:
[2018-06-01 15:15:57,009] {{models.py:189}} INFO - Filling up the
DagBag from /usr/local/airflow/dags/montagne_dag.py
[2018-06-01 15:15:57,808] {{cli.py:374}} INFO - Running on host
airflow-webserver
[2018-06-01 15:15:57,946] {{base_task_runner.py:98}} INFO - Subtask:
Warning 6: dataset
/usr/local/airflow/data/output/montagne_dag/2018-06-01T19:15:16.584033/fleuve_montagne_polylines.geojson
does not support layer creation option ENCODING
[2018-06-01 15:15:57,994] {{base_task_runner.py:98}} INFO - Subtask:
[2018-06-01 15:15:57,976] {{python_operator.py:90}} INFO - Done.
Returned value was: None
[2018-06-01 15:15:58,065] {{base_task_runner.py:98}} INFO - Subtask:
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144:
UserWarning: The psycopg2 wheel package will be renamed from release
2.8; in order to keep installing from binary please use "pip install
psycopg2-binary" instead. For details see:
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
[2018-06-01 15:15:58,065] {{base_task_runner.py:98}} INFO - Subtask: """)
On Fri, Jun 1, 2018 at 2:33 PM, Martin Gauthier <gauthiermarti...@gmail.com>
wrote:
> For example this python operator
>
> write_polylines_geo_data = PythonOperator(
> task_id='write_geo_data_polylines',
> provide_context=True,
> python_callable=generate_geo_data_files_from_wkt_data_file,
> op_args=[
> polylines_staging_file_loc,
> 'pickle',
> WGS54,
> 'geometry',
> None,
> (polylines_geojson_file_location, 'geojson', WGS54),
> (polylines_shape_file_location, 'shape', MTM5
> )
> ],
> dag=dag
> )
>
>
> I would like to log actions done into the function
> "generate_geo_data_files_from_wkt_data_file" would i need to define a
> logger in this function and then it should do the trick ?
>
>
>
--
Martin Gauthier
C: 438-499-1749
