+1 (non-binding) Tested the Airflow pool and it is working as expected.
Chao-Han On Mon, Aug 5, 2019 at 1:09 PM Felix Uellendall <[email protected]> wrote: > +1 (non-binding) > > Tested: > - running example dags on Sequential and LocalExecutor > - classic UI and RBAC UI (connections, variables, pools, xcom, logs) There > is a small issue with the logs containing symbols for coloring instead of > actual coloring it in the UI - symbols are for bash only. Maybe we can add > colors to the UI log, too. > > Regards, > Felix > > Sent from ProtonMail mobile > > -------- Original Message -------- > On Aug 5, 2019, 18:41, Kamil Gałuszka wrote: > > > +1 (non-binding) > > > > Works for me and k8s executor is not hanging with that version. > > > > Thanks > > Kamil > > > > --------------------- > > Kamil Gałuszka > > Principal Architect > > FLYR, Inc. www.flyrlabs.com > > > > On Sun, Aug 4, [2019](tel:2019) at 12:30 AM Kaxil Naik < > [email protected]> wrote: > > > >> +1 (binding) > >> > >> No installation issues. Fresh install works fine. Upgrade is smooth. > >> UI Search works > >> Tutorials, Example DAGs & Few of my DAGs run fine > >> > >> > >> > >> On Thu, Aug 1, [2019](tel:2019) at 2:53 PM Ash Berlin-Taylor < > [email protected]> wrote: > >> > >> > Right, 5th time lucky? > >> > > >> > > >> > This email is calling a vote on the release, which will last for 72 > hours > >> > (2019-08-04 14:00 Z - Sunday 3pm UK, 7am PDT), and until three binding > >> > votes have been cast. Consider this my (binding) +1. > >> > > >> > Airflow [1.10.4](tel:1104) RC5 is available at: > >> > https://dist.apache.org/repos/dist/dev/airflow/1.10.4rc5/ < > >> > https://dist.apache.org/repos/dist/dev/airflow/1.10.4rc5/> > >> > > >> > *apache-airflow-[1.10.4](tel:1104)rc5-source.tar.gz* is a source > release that comes > >> > with INSTALL instructions. > >> > *apache-airflow-[1.10.4](tel:1104)rc5-bin.tar.gz* is the binary > Python "sdist" > >> release. > >> > *apache_airflow-[1.10.4](tel:1104)rc5-py2.py3-none-any.whl* is the > binary Python > >> > "wheel" release. > >> > > >> > For connivence of testers the RC is on PYPI too. It can be installed > >> with: > >> > > >> > pip install 'apache-airflow==[1.10.4](tel:1104)rc5' > >> > > >> > Public keys are available at: > >> > https://dist.apache.org/repos/dist/release/airflow/KEYS < > >> > https://dist.apache.org/repos/dist/release/airflow/KEYS> > >> > > >> > Only votes from PMC members are binding (sorry committers), but > members > >> of > >> > the community are encouraged to test the release and vote with > >> > "(non-binding)". > >> > > >> > Please note that the version number excludes the `rcX` string, so it's > >> now > >> > simply [1.10.4](tel:1104). This will allow us to rename the artefact > without > >> modifying > >> > the checksums when we actually release. > >> > > >> > [ ] +1 Release this package as Apache Airflow [1.10.4](tel:1104) > >> > [ ] 0 No opinion > >> > [ ] -1 Do not release this package because... > >> > > >> > > >> > Changes since RC4: > >> > > >> > - a[18283911](tel:18283911) [AIRFLOW-XXX] Update changelog for > [1.10.4](tel:1104)rc5 > >> > - 5a98f9ecb [AIRFLOW-[4883](tel:4883)] Bug-fix to killing hung file > processes (#[5706](tel:5706)) > >> > - bb[3200](tel:3200)e0a [AIRFLOW-[5089](tel:5089)] Spanner > compatibility bug causes missing > >> > imports (#[5703](tel:5703)) > >> > - fd2b[025](tel:025)eb [AIRFLOW-[5091](tel:5091)] Build epoch is > fixed now (#[5704](tel:5704)) > >> > - c[519](tel:519)d[3607](tel:3607) [AIRFLOW-[5086](tel:5086)] Fix DAG > search for partial string in classic > >> UI > >> > (#[5700](tel:5700)) > >> > > >> > > >> > > >> > Full change log since [1.10.3](tel:1103): > >> > > >> > Airflow [1.10.4](tel:1104), - [2019-08-03](tel:20190803) > >> > ---------------------------- > >> > > >> > New Features > >> > """""""""""" > >> > - [AIRFLOW-[4811](tel:4811)] Implement GCP Data Loss Prevention Hook > and Operators > >> > (#[5539](tel:5539)) > >> > - [AIRFLOW-[5035](tel:5035)] Replace multiprocessing.Manager with a > golang-"channel" > >> > style (#[5615](tel:5615)) > >> > - [AIRFLOW-[4883](tel:4883)] Kill hung file process managers > (#[5605](tel:5605)) > >> > - [AIRFLOW-[4929](tel:4929)] Pretty print JSON Variables in UI > (#[5573](tel:5573)) > >> > - [AIRFLOW-[4884](tel:4884)] Roll up import_errors in RBAC UI > (#[5516](tel:5516)) > >> > - [AIRFLOW-[4871](tel:4871)] Allow creating DagRuns via RBAC UI > (#[5507](tel:5507)) > >> > - [AIRFLOW-[4591](tel:4591)] Make default_pool a real pool > (#[5349](tel:5349)) > >> > - [AIRFLOW-[4844](tel:4844)] Add optional is_paused_upon_creation > argument to DAG > >> > (#[5473](tel:5473)) > >> > - [AIRFLOW-[4456](tel:4456)] Add sub-classable BaseBranchOperator > (#[5231](tel:5231)) > >> > - [AIRFLOW-[4343](tel:4343)] Show warning in UI if scheduler is not > running (#[5127](tel:5127)) > >> > - [AIRFLOW-[4739](tel:4739)] Add ability to arbitrarily define > kubernetes worker pod > >> > labels (#[5376](tel:5376)) > >> > - [AIRFLOW-[4348](tel:4348)] Add GCP console link in BigQueryOperator > (#[5195](tel:5195)) > >> > - [AIRFLOW-[4306](tel:4306)] Global operator extra links > (#[5094](tel:5094)) > >> > - [AIRFLOW-[4812](tel:4812)] Add batch images annotation > (#[5433](tel:5433)) > >> > - [AIRFLOW-[4135](tel:4135)] Add Google Cloud Build operator and hook > (#[5251](tel:5251)) > >> > - [AIRFLOW-[4781](tel:4781)] Add the ability to specify ports in > kubernetesOperator > >> > (#[5410](tel:5410)) > >> > - [AIRFLOW-[4521](tel:4521)] Pause dag also pause its subdags > (#[5283](tel:5283)) > >> > - [AIRFLOW-[4738](tel:4738)] Enforce exampleinclude for example DAGs > (#[5375](tel:5375)) > >> > - [AIRFLOW-[4326](tel:4326)] Airflow AWS SQS Operator > (#[5110](tel:5110)) > >> > - [AIRFLOW-[3729](tel:3729)] Support "DownwardAPI" in env variables > for > >> > KubernetesPodOperator (#[4554](tel:4554)) > >> > - [AIRFLOW-[4585](tel:4585)] Implement Kubernetes Pod Mutation Hook > (#[5359](tel:5359)) > >> > - [AIRFLOW-[161](tel:161)] New redirect route and extra links > (#[5059](tel:5059)) > >> > - [AIRFLOW-[4420](tel:4420)] Backfill respects task_concurrency > (#[5221](tel:5221)) > >> > - [AIRFLOW-[4147](tel:4147)] Add Operator to publish event to Redis > (#[4967](tel:4967)) > >> > - [AIRFLOW-[3359](tel:3359)] Add option to pass customer encryption > keys to Dataproc > >> > (#[4200](tel:4200)) > >> > - [AIRFLOW-[4318](tel:4318)] Create Google Cloud Translate Speech > Operator (#[5102](tel:5102)) > >> > - [AIRFLOW-[3960](tel:3960)] Adds Google Cloud Speech operators > (#[4780](tel:4780)) > >> > - [AIRFLOW-[1501](tel:1501)] Add GoogleCloudStorageDeleteOperator > (#[5230](tel:5230)) > >> > - [AIRFLOW-[3672](tel:3672)] Add support for Mongo DB DNS Seedlist > Connection Format > >> > (#[4481](tel:4481)) > >> > - [AIRFLOW-[4397](tel:4397)] add integrations docs manually for gcs > sensors (#[5204](tel:5204)) > >> > - [AIRFLOW-[4251](tel:4251)] Instrument DagRun schedule delay > (#[5050](tel:5050)) > >> > - [AIRFLOW-[4118](tel:4118)] instrument DagRun duration > (#[4946](tel:4946)) > >> > - [AIRFLOW-[4361](tel:4361)] Fix flaky > >> test_integration_run_dag_with_scheduler_failure > >> > (#[5182](tel:5182)) > >> > - [AIRFLOW-[4361](tel:4361)] Fix flaky > >> test_integration_run_dag_with_scheduler_failure > >> > (#[5140](tel:5140)) > >> > - [AIRFLOW-[4168](tel:4168)] Create Google Cloud Video Intelligence > Operators (#[4985](tel:4985)) > >> > - [AIRFLOW-[4397](tel:4397)] Add GCSUploadSessionCompleteSensor > (#[5166](tel:5166)) > >> > - [AIRFLOW-[4335](tel:4335)] Add default num_retries to GCP > connection (#[5117](tel:5117)) > >> > - [AIRFLOW-[3808](tel:3808)] Add cluster_fields to BigQueryHook's > create_empty_table > >> > (#[4654](tel:4654)) > >> > - [AIRFLOW-[4362](tel:4362)] Fix test_execution_limited_parallelism > (#[5141](tel:5141)) > >> > - [AIRFLOW-[4307](tel:4307)] Backfill respects concurrency limit > (#[5128](tel:5128)) > >> > - [AIRFLOW-[4268](tel:4268)] Add MsSqlToGoogleCloudStorageOperator > (#[5077](tel:5077)) > >> > - [AIRFLOW-[4169](tel:4169)] Add Google Cloud Vision Detect Operators > (#[4986](tel:4986)) > >> > - [AIRFLOW-XXX] Fix WS-[2019-0032](tel:20190032) (#[5384](tel:5384)) > >> > - [AIRFLOW-XXX] Fix CVE-[2019-11358](tel:201911358) > (#[5197](tel:5197)) > >> > - [AIRFLOW-XXX] Change allowed version of Jinja2 to fix > CVE-[2019-10906](tel:201910906) > >> > (#[5075](tel:5075)) > >> > > >> > Improvement > >> > """"""""""" > >> > - [AIRFLOW-[5022](tel:5022)] Fix DockerHook for registries with port > numbers (#[5644](tel:5644)) > >> > - [AIRFLOW-[4961](tel:4961)] Insert TaskFail.duration as int match DB > schema column > >> > type (#[5593](tel:5593)) > >> > - [AIRFLOW-[5038](tel:5038)] skip pod deleted log message when pod > deletion is > >> > disabled (#[5656](tel:5656)) > >> > - [AIRFLOW-[5067](tel:5067)] Update pagination symbols > (#[5682](tel:5682)) > >> > - [AIRFLOW-[4981](tel:4981)][AIRFLOW-[4788](tel:4788)] Always use > pendulum DateTimes in task > >> > instance context (#[5654](tel:5654)) > >> > - [AIRFLOW-[4880](tel:4880)] Add success, failure and fail_on_empty > params to > >> > SqlSensor (#[5488](tel:5488)) > >> > - [AIRFLOW-[3617](tel:3617)] Add gpu limits option in configurations > for Kube > >> executor > >> > and pod (#[5643](tel:5643)) > >> > - [AIRFLOW-[4998](tel:4998)] Run multiple queries in BigQueryOperator > (#[5619](tel:5619)) > >> > - [AIRFLOW-[4929](tel:4929)] Improve display of JSON Variables in UI > (#[5641](tel:5641)) > >> > - [AIRFLOW-[4959](tel:4959)] Add .hql support for the > DataProcHiveOperator (#[5591](tel:5591)) > >> > - [AIRFLOW-[4962](tel:4962)] Fix Werkzeug v[0.15](tel:015) > deprecation notice for > >> > DispatcherMiddleware import (#[5595](tel:5595)) > >> > - [AIRFLOW-[4797](tel:4797)] Improve performance and behaviour of > zombie detection > >> > (#[5511](tel:5511)) > >> > - [AIRFLOW-[4911](tel:4911)] Silence the FORBIDDEN errors from the > KubernetesExecutor > >> > (#[5547](tel:5547)) > >> > - [AIRFLOW-[3495](tel:3495)] Validate one of query and query_uri > passed to > >> > DataProcSparkSqlOperator (#[5510](tel:5510)) > >> > - [AIRFLOW-[4925](tel:4925)] Improve css style for Variables Import > file field > >> (#[5552](tel:5552)) > >> > - [AIRFLOW-[4906](tel:4906)] Improve debugging for the > SparkSubmitHook (#[5542](tel:5542)) > >> > - [AIRFLOW-[4904](tel:4904)] unittest.cfg name and path can be > overriden by setting > >> > $AIRFLOW_TEST_CONFIG (#[5540](tel:5540)) > >> > - [AIRFLOW-[4920](tel:4920)] Use html.escape instead of cgi.escape to > fix > >> > DeprecationWarning (#[5551](tel:5551)) > >> > - [AIRFLOW-[4919](tel:4919)] DataProcJobBaseOperator > dataproc_*_properties templated > >> > (#[5555](tel:5555)) > >> > - [AIRFLOW-[4478](tel:4478)] Lazily instantiate default resources > objects. (#[5259](tel:5259)) > >> > - [AIRFLOW-[4564](tel:4564)] AzureContainerInstance bugfixes and > improvements (#[5319](tel:5319)) > >> > - [AIRFLOW-[4237](tel:4237)] Including Try Number of Task in Gantt > Chart (#[5037](tel:5037)) > >> > - [AIRFLOW-[4862](tel:4862)] Allow directly using IP address as > hostname for > >> webserver > >> > logs (#[5501](tel:5501)) > >> > - [AIRFLOW-[4857](tel:4857)] Add templated fields to > SlackWebhookOperator (#[5490](tel:5490)) > >> > - [AIRFLOW-[3502](tel:3502)] Add celery config option for setting > "pool" (#[4308](tel:4308)) > >> > - [AIRFLOW-[3217](tel:3217)] Button to toggle line wrapping in log > and code views > >> > (#[4277](tel:4277)) > >> > - [AIRFLOW-[4491](tel:4491)] Add a "Jump to end" button for logs > (#[5266](tel:5266)) > >> > - [AIRFLOW-[4422](tel:4422)] Pool utilization stats > (#[5453](tel:5453)) > >> > - [AIRFLOW-[4805](tel:4805)] Add py_file as templated field in > DataflowPythonOperator > >> > (#[5451](tel:5451)) > >> > - [AIRFLOW-[4838](tel:4838)] Surface Athena errors in > AWSAthenaOperator (#[5467](tel:5467)) > >> > - [AIRFLOW-[4831](tel:4831)] conf.has_option no longer throws if > section is missing. > >> > (#[5455](tel:5455)) > >> > - [AIRFLOW-[4829](tel:4829)] More descriptive exceptions for EMR > sensors (#[5452](tel:5452)) > >> > - [AIRFLOW-[4414](tel:4414)] AWSAthenaOperator: Push QueryExecutionID > to XCom (#[5276](tel:5276)) > >> > - [AIRFLOW-[4791](tel:4791)] add "schema" keyword arg to > SnowflakeOperator (#[5415](tel:5415)) > >> > - [AIRFLOW-[4759](tel:4759)] Don't error when marking sucessful run > as failed (#[5435](tel:5435)) > >> > - [AIRFLOW-[4716](tel:4716)] Instrument dag loading time duration > (#[5350](tel:5350)) > >> > - [AIRFLOW-[3958](tel:3958)] Support list tasks as upstream in chain > (#[4779](tel:4779)) > >> > - [AIRFLOW-[4409](tel:4409)] Prevent task duration break by null > value (#[5178](tel:5178)) > >> > - [AIRFLOW-[4418](tel:4418)] Add "failed only" option to task modal > (#[5193](tel:5193)) > >> > - [AIRFLOW-[4740](tel:4740)] Accept string ``end_date`` in DAG > default_args (#[5381](tel:5381)) > >> > - [AIRFLOW-[4423](tel:4423)] Improve date handling in mysql to gcs > operator. (#[5196](tel:5196)) > >> > - [AIRFLOW-[4447](tel:4447)] Display task duration as human friendly > format in UI > >> > (#[5218](tel:5218)) > >> > - [AIRFLOW-[4377](tel:4377)] Remove needless object conversion in > DAG.owner() (#[5144](tel:5144)) > >> > - [AIRFLOW-[4766](tel:4766)] Add autoscaling option for > DataprocClusterCreateOperator > >> > (#[5425](tel:5425)) > >> > - [AIRFLOW-[4795](tel:4795)] Upgrade alembic to latest release. > (#[5411](tel:5411)) > >> > - [AIRFLOW-[4793](tel:4793)] Add signature_name to mlengine operator > (#[5417](tel:5417)) > >> > - [AIRFLOW-[3211](tel:3211)] Reattach to GCP Dataproc jobs upon > Airflow restart > >> > (#[4083](tel:4083)) > >> > - [AIRFLOW-[4750](tel:4750)] Log identified zombie task instances > (#[5389](tel:5389)) > >> > - [AIRFLOW-[3870](tel:3870)] STFPOperator: Update log level and > return value (#[4355](tel:4355)) > >> > - [AIRFLOW-[4759](tel:4759)] Batch queries in set_state API. > (#[5403](tel:5403)) > >> > - [AIRFLOW-[2737](tel:2737)] Restore original license header to > >> > airflow.api.auth.backend.kerberos_auth > >> > - [AIRFLOW-[3635](tel:3635)] Fix incorrect logic in detele_dag > (introduced in > >> PR#[4406](tel:4406)) > >> > (#[4445](tel:4445)) > >> > - [AIRFLOW-[3599](tel:3599)] Removed Dagbag from delete dag > (#[4406](tel:4406)) > >> > - [AIRFLOW-[4737](tel:4737)] Increase and document celery queue name > limit (#[5383](tel:5383)) > >> > - [AIRFLOW-[4505](tel:4505)] Correct Tag ALL for PY3 > (#[5275](tel:5275)) > >> > - [AIRFLOW-[4743](tel:4743)] Add environment variables support to > SSHOperator (#[5385](tel:5385)) > >> > - [AIRFLOW-[4725](tel:4725)] Fix setup.py PEP[440](tel:440) & > Sphinx-PyPI-upload dependency > >> > (#[5363](tel:5363)) > >> > - [AIRFLOW-[3370](tel:3370)] Add stdout output options to > Elasticsearch task log > >> > handler (#[5048](tel:5048)) > >> > - [AIRFLOW-[4396](tel:4396)] Provide a link to external Elasticsearch > logs in UI. > >> > (#[5164](tel:5164)) > >> > - [AIRFLOW-[1381](tel:1381)] Allow setting host temporary directory > in DockerOperator > >> > (#[5369](tel:5369)) > >> > - [AIRFLOW-[4598](tel:4598)] Task retries are not exhausted for K8s > executor (#[5347](tel:5347)) > >> > - [AIRFLOW-[4218](tel:4218)] Support to Provide http args to > K8executor while calling > >> > k8 python client lib apis (#[5060](tel:5060)) > >> > - [AIRFLOW-[4159](tel:4159)] Add support for additional static pod > labels for > >> > K8sExecutor (#[5134](tel:5134)) > >> > - [AIRFLOW-[4720](tel:4720)] Allow comments in .airflowignore files. > (#[5355](tel:5355)) > >> > - [AIRFLOW-[4486](tel:4486)] Add AWS IAM authenication in MySqlHook > (#[5334](tel:5334)) > >> > - [AIRFLOW-[4417](tel:4417)] Add AWS IAM authenication for > PostgresHook (#[5223](tel:5223)) > >> > - [AIRFLOW-[3990](tel:3990)] Compile regular expressions. > (#[4813](tel:4813)) > >> > - [AIRFLOW-[4572](tel:4572)] Rename prepare_classpath() to > prepare_syspath() (#[5328](tel:5328)) > >> > - [AIRFLOW-[3869](tel:3869)] Raise consistent exception in > >> > AirflowConfigParser.getboolean (#[4692](tel:4692)) > >> > - [AIRFLOW-[4571](tel:4571)] Add headers to templated field for > SimpleHttpOperator > >> > (#[5326](tel:5326)) > >> > - [AIRFLOW-[3867](tel:3867)] Rename GCP's subpackage > (#[4690](tel:4690)) > >> > - [AIRFLOW-[3725](tel:3725)] Add private_key to bigquery_hook > get_pandas_df (#[4549](tel:4549)) > >> > - [AIRFLOW-[4546](tel:4546)] Upgrade google-cloud-bigtable. > (#[5307](tel:5307)) > >> > - [AIRFLOW-[4519](tel:4519)] Optimise operator classname sorting in > views (#[5282](tel:5282)) > >> > - [AIRFLOW-[4503](tel:4503)] Support fully pig options > (#[5271](tel:5271)) > >> > - [AIRFLOW-[4468](tel:4468)] add sql_alchemy_max_overflow parameter > (#[5249](tel:5249)) > >> > - [AIRFLOW-[4467](tel:4467)] Add dataproc_jars to templated fields in > Dataproc oper… > >> > (#[5248](tel:5248)) > >> > - [AIRFLOW-[4381](tel:4381)] Use get_direct_relative_ids get task > relatives (#[5147](tel:5147)) > >> > - [AIRFLOW-[3624](tel:3624)] Add masterType parameter to > MLEngineTrainingOperator > >> > (#[4428](tel:4428)) > >> > - [AIRFLOW-[3143](tel:3143)] Support Auto-Zone in > DataprocClusterCreateOperator > >> (#[5169](tel:5169)) > >> > - [AIRFLOW-[3874](tel:3874)] Improve > BigQueryHook.run_with_configuration's location > >> > support (#[4695](tel:4695)) > >> > - [AIRFLOW-[4399](tel:4399)] Avoid duplicated os.path.isfile() check > in models.dagbag > >> > (#[5165](tel:5165)) > >> > - [AIRFLOW-[4031](tel:4031)] Allow for key pair auth in snowflake > hook (#[4875](tel:4875)) > >> > - [AIRFLOW-[3901](tel:3901)] add role as optional config parameter > for SnowflakeHook > >> > (#[4721](tel:4721)) > >> > - [AIRFLOW-[3455](tel:3455)] add region in snowflake connector > (#[4285](tel:4285)) > >> > - [AIRFLOW-[4073](tel:4073)] add template_ext for AWS Athena operator > (#[4907](tel:4907)) > >> > - [AIRFLOW-[4093](tel:4093)] AWSAthenaOperator: Throw exception if job > >> > failed/cancelled/reach max retries (#[4919](tel:4919)) > >> > - [AIRFLOW-[4356](tel:4356)] Add extra RuntimeEnvironment keys to > DataFlowHook > >> (#[5149](tel:5149)) > >> > - [AIRFLOW-[4337](tel:4337)] Fix docker-compose deprecation warning > in CI (#[5119](tel:5119)) > >> > - [AIRFLOW-[3603](tel:3603)] QuboleOperator: Remove SQLCommand from > SparkCmd > >> > documentation (#[4411](tel:4411)) > >> > - [AIRFLOW-[4328](tel:4328)] Fix link to task instances from Pool > page (#[5124](tel:5124)) > >> > - [AIRFLOW-[4255](tel:4255)] Make GCS Hook Backwards compatible > (#[5089](tel:5089)) > >> > - [AIRFLOW-[4103](tel:4103)] Allow uppercase letters in dataflow job > names (#[4925](tel:4925)) > >> > - [AIRFLOW-[4255](tel:4255)] Replace Discovery based api with client > based for GCS > >> > (#[5054](tel:5054)) > >> > - [AIRFLOW-[4311](tel:4311)] Remove sleep in localexecutor > (#[5096](tel:5096)) > >> > - [AIRFLOW-[2836](tel:2836)] Minor > improvement-contrib.sensors.FileSensor (#[3674](tel:3674)) > >> > - [AIRFLOW-[4104](tel:4104)] Add type annotations to common classes. > (#[4926](tel:4926)) > >> > - [AIRFLOW-[3910](tel:3910)] Raise exception explicitly in > Connection.get_hook() > >> > (#[4728](tel:4728)) > >> > - [AIRFLOW-[3322](tel:3322)] Update QuboleHook to fetch args > dynamically from qds_sdk > >> > (#[4165](tel:4165)) > >> > - [AIRFLOW-[4565](tel:4565)] instrument celery executor > (#[5321](tel:5321)) > >> > - [AIRFLOW-[4573](tel:4573)] Import airflow_local_settings after > prepare_classpath > >> > (#[5330](tel:5330)) > >> > - [AIRFLOW-[4448](tel:4448)] Don't bake ENV and _cmd into tmp config > for non-sudo > >> > (#[4050](tel:4050)) > >> > - [AIRFLOW-[4295](tel:4295)] Make ``method`` attribute case > insensitive in HttpHook > >> > (#[5313](tel:5313)) > >> > - [AIRFLOW-[3703](tel:3703)] Add dnsPolicy option for > KubernetesPodOperator (#[4520](tel:4520)) > >> > - [AIRFLOW-[3057](tel:3057)] add prev_*_date_success to template > context (#[5372](tel:5372)) > >> > - [AIRFLOW-[4336](tel:4336)] Stop showing entire GCS files bytes in > log for > >> > gcs_download_operator (#[5151](tel:5151)) > >> > - [AIRFLOW-[4528](tel:4528)] Cancel DataProc task on timeout > (#[5293](tel:5293)) > >> > > >> > Bug fixes > >> > """"""""" > >> > - [AIRFLOW-[5089](tel:5089)] Change version requirement on > google-cloudp-spanner to > >> > work around version incompatibility (#[5703](tel:5703)) > >> > - [AIRFLOW-[4289](tel:4289)] fix spark_binary argument being ignored > in > >> > SparkSubmitHook (#[5564](tel:5564)) > >> > - [AIRFLOW-[5075](tel:5075)] Let HttpHook handle connections with > empty host fields > >> > (#[5686](tel:5686)) > >> > - [AIRFLOW-[4822](tel:4822)] Fix bug where parent-dag task instances > are wrongly > >> > cleared when using subdags (#[5444](tel:5444)) > >> > - [AIRFLOW-[5050](tel:5050)] Correctly delete FAB permission m2m > objects in ``airflow > >> > sync_perms`` (#[5679](tel:5679)) > >> > - [AIRFLOW-[5030](tel:5030)] fix env var expansion for config key > contains __ (#[5650](tel:5650)) > >> > - [AIRFLOW-[4590](tel:4590)] changing log level to be proper library > to suppress > >> > warning in WinRM (#[5337](tel:5337)) > >> > - [AIRFLOW-[4451](tel:4451)] Allow named tuples to be templated > (#[5673](tel:5673)) > >> > - [AIRFLOW-XXX] Fix bug where Kube pod limts were not applied > (requests > >> > were, but not limits) (#[5657](tel:5657)) > >> > - [AIRFLOW-[4775](tel:4775)] Fix incorrect parameter order in GceHook > (#[5613](tel:5613)) > >> > - [AIRFLOW-[4995](tel:4995)] Fix DB initialisation on MySQL > >=[8.0.16](tel:8016) (#[5614](tel:5614)) > >> > - [AIRFLOW-[4934](tel:4934)] Fix ProxyFix due to Werkzeug upgrade > (#[5563](tel:5563)) (#[5571](tel:5571)) > >> > - [AIRFLOW-[4136](tel:4136)] fix key_file of hook is overwritten by > SSHHook > >> connection > >> > (#[5558](tel:5558)) > >> > - [AIRFLOW-[4587](tel:4587)] Replace self.conn with self.get_conn() > in AWSAthenaHook > >> > (#[5545](tel:5545)) > >> > - [AIRFLOW-[1740](tel:1740)] Fix xcom creation and update via UI > (#[5530](tel:5530)) (#[5531](tel:5531)) > >> > - [AIRFLOW-[4900](tel:4900)] Resolve incompatible version of Werkzeug > (#[5535](tel:5535)) > >> > - [AIRFLOW-[4510](tel:4510)] Don't mutate default_args during DAG > initialization > >> > (#[5277](tel:5277)) > >> > - [AIRFLOW-[3360](tel:3360)] Make the DAGs search respect other > querystring > >> parameters > >> > with url-search-params-polyfill for IE support (#[5503](tel:5503)) > >> > - [AIRFLOW-[4896](tel:4896)] Make KubernetesExecutorConfig's default > args immutable > >> > (#[5534](tel:5534)) > >> > - [AIRFLOW-[4494](tel:4494)] Remove ``shell=True`` in DaskExecutor > (#[5273](tel:5273)) > >> > - [AIRFLOW-[4890](tel:4890)] Fix Log link in TaskInstance's View for > Non-RBAC (#[5525](tel:5525)) > >> > - [AIRFLOW-[4892](tel:4892)] Fix connection creation via UIs > (#[5527](tel:5527)) > >> > - [AIRFLOW-[4406](tel:4406)] Fix a method name typo: > NullFernet.decrpyt to decrypt > >> > (#[5509](tel:5509)) > >> > - [AIRFLOW-[4849](tel:4849)] Add gcp_conn_id to cloudsqldatabehook > class to use > >> > correctly CloudSqlProxyRunner class (#[5478](tel:5478)) > >> > - [AIRFLOW-[4769](tel:4769)] Pass gcp_conn_id to BigtableHook > (#[5445](tel:5445)) > >> > - [AIRFLOW-[4524](tel:4524)] Fix incorrect field names in view for > Mark > >> > Success/Failure (#[5486](tel:5486)) > >> > - [AIRFLOW-[3671](tel:3671)] Remove arg ``replace`` of > MongoToS3Operator from > >> > ``kwargs`` (#[4480](tel:4480)) > >> > - [AIRFLOW-[4845](tel:4845)] Fix bug where runAsUser 0 doesn't get > set in k8s > >> security > >> > context (#[5474](tel:5474)) > >> > - [AIRFLOW-[4354](tel:4354)] Fix exception in "between" date filter > in classic UI > >> > (#[5480](tel:5480)) > >> > - [AIRFLOW-[4587](tel:4587)] Replace self.conn with self.get_conn() > in AWSAthenaHook > >> > (#[5462](tel:5462)) > >> > - [AIRFLOW-[4516](tel:4516)] K8s runAsUser and fsGroup cannot be > strings (#[5429](tel:5429)) > >> > - [AIRFLOW-[4298](tel:4298)] Stop Scheduler repeatedly warning > "connection > >> > invalidated" (#[5470](tel:5470)) > >> > - [AIRFLOW-[4559](tel:4559)] JenkinsJobTriggerOperator bugfix > (#[5318](tel:5318)) > >> > - [AIRFLOW-[4841](tel:4841)] Pin Sphinx AutoApi to [1.0.0](tel:100) > (#[5468](tel:5468)) > >> > - [AIRFLOW-[4479](tel:4479)] Include s3_overwrite kwarg in load_bytes > method (#[5312](tel:5312)) > >> > - [AIRFLOW-[3746](tel:3746)] Fix DockerOperator missing container > exit (#[4583](tel:4583)) > >> > - [AIRFLOW-[4233](tel:4233)] Remove Template Extension from Bq to GCS > Operator > >> (#[5456](tel:5456)) > >> > - > [AIRFLOW-[2141](tel:2141)][AIRFLOW-[3157](tel:3157)][AIRFLOW-[4170](tel:4170)] > Serialize non-str value by > >> > JSON when importing Variables (#[4991](tel:4991)) > >> > - [AIRFLOW-[4826](tel:4826)] Remove warning from ``airflow resetdb`` > command (#[5447](tel:5447)) > >> > - [AIRFLOW-[4148](tel:4148)] Fix editing DagRuns when clicking state > column (#[5436](tel:5436)) > >> > - [AIRFLOW-[4455](tel:4455)] dag_details broken for subdags in RBAC > UI (#[5234](tel:5234)) > >> > - [AIRFLOW-[2955](tel:2955)] Fix kubernetes pod operator to set > requests and limits > >> on > >> > task pods (#[4551](tel:4551)) > >> > - [AIRFLOW-[4459](tel:4459)] Fix wrong DAG count in /home page when > DAG count is zero > >> > (#[5235](tel:5235)) > >> > - [AIRFLOW-[3876](tel:3876)] AttributeError: module 'distutils' has > no attribute > >> 'util' > >> > - [AIRFLOW-[4146](tel:4146)] Fix CgroupTaskRunner errors > (#[5224](tel:5224)) > >> > - [AIRFLOW-[4524](tel:4524)] Fix bug with "Ignore \*" toggles in RBAC > mode (#[5378](tel:5378)) > >> > - [AIRFLOW-[4765](tel:4765)] Fix DataProcPigOperator execute method > (#[5426](tel:5426)) > >> > - [AIRFLOW-[4798](tel:4798)] obviate interdependencies for dagbag and > TI tests > >> (#[5422](tel:5422)) > >> > - [AIRFLOW-[4800](tel:4800)] fix GKEClusterHook ctor calls > (#[5424](tel:5424)) > >> > - [AIRFLOW-[4799](tel:4799)] don't mutate self.env in BashOperator > execute method > >> > (#[5421](tel:5421)) > >> > - [AIRFLOW-[4393](tel:4393)] Add retry logic when fetching pod status > and/or logs in > >> > KubernetesPodOperator (#[5284](tel:5284)) > >> > - [AIRFLOW-[4174](tel:4174)] Fix HttpHook run with backoff > (#[5213](tel:5213)) > >> > - [AIRFLOW-[4463](tel:4463)] Handle divide-by-zero errors in short > retry intervals > >> > (#[5243](tel:5243)) > >> > - [AIRFLOW-[2614](tel:2614)] Speed up trigger_dag API call when lots > of DAGs in > >> system > >> > - [AIRFLOW-[4756](tel:4756)] add ti.state to ti.start_date as > criteria for gantt > >> > (#[5399](tel:5399)) > >> > - [AIRFLOW-[4760](tel:4760)] Fix zip-packaged DAGs disappearing from > DagBag when > >> > reloaded (#[5404](tel:5404)) > >> > - [AIRFLOW-[4731](tel:4731)] Fix GCS hook with google-storage-client > [1.16](tel:116) (#[5368](tel:5368)) > >> > - [AIRFLOW-[3506](tel:3506)] use match_phrase to query log_id in > elasticsearch > >> (#[4342](tel:4342)) > >> > - [AIRFLOW-[4084](tel:4084)] fix ElasticSearch log download > (#[5177](tel:5177)) > >> > - [AIRFLOW-[4501](tel:4501)] Register pendulum datetime converter for > sqla+pymysql > >> > (#[5190](tel:5190)) > >> > - [AIRFLOW-[986](tel:986)] HiveCliHook ignores 'proxy_user' value in > a connection's > >> > extra parameter (#[5305](tel:5305)) > >> > - [AIRFLOW-[4442](tel:4442)] fix hive_tblproperties in > HiveToDruidTransfer (#[5211](tel:5211)) > >> > - [AIRFLOW-[4557](tel:4557)] Add gcp_conn_id parameter to > get_sqlproxy_runner() of > >> > CloudSqlDatabaseHook (#[5314](tel:5314)) > >> > - [AIRFLOW-[4545](tel:4545)] Upgrade FAB to latest version > (#[4955](tel:4955)) > >> > - [AIRFLOW-[4492](tel:4492)] Change Dataproc Cluster operators to > poll Operations > >> > (#[5269](tel:5269)) > >> > - [AIRFLOW-[4452](tel:4452)] Webserver and Scheduler keep crashing > because of > >> > slackclient update (#[5225](tel:5225)) > >> > - [AIRFLOW-[4450](tel:4450)] Fix request arguments in has_dag_access > (#[5220](tel:5220)) > >> > - [AIRFLOW-[4434](tel:4434)] Support Impala with the HiveServer2Hook > (#[5206](tel:5206)) > >> > - [AIRFLOW-[3449](tel:3449)] Write local dag parsing logs when remote > logging > >> enabled. > >> > (#[5175](tel:5175)) > >> > - [AIRFLOW-[4300](tel:4300)] Fix graph modal call when DAG has not > yet run (#[5185](tel:5185)) > >> > - [AIRFLOW-[4401](tel:4401)] Use managers for Queue synchronization > (#[5200](tel:5200)) > >> > - [AIRFLOW-[3626](tel:3626)] Fixed triggering DAGs contained within > zip files (#[4439](tel:4439)) > >> > - [AIRFLOW-[3720](tel:3720)] Fix missmatch while comparing GCS and S3 > files (#[4766](tel:4766)) > >> > - [AIRFLOW-[4403](tel:4403)] search by ``dag_id`` or ``owners`` in UI > (#[5184](tel:5184)) > >> > - [AIRFLOW-[4308](tel:4308)] Fix TZ-loop around DST on python 3.6+ > (#[5095](tel:5095)) > >> > - [AIRFLOW-[4324](tel:4324)] fix DAG fuzzy search in RBAC UI > (#[5131](tel:5131)) > >> > - [AIRFLOW-[4297](tel:4297)] Temporary hot fix on manage_slas() for > [1.10.4](tel:1104) release > >> > (#[5150](tel:5150)) > >> > - [AIRFLOW-[4299](tel:4299)] Upgrade to Celery [4.3.0](tel:430) to > fix crashing workers (#[5116](tel:5116)) > >> > - [AIRFLOW-[4291](tel:4291)] Correctly render doc_md in DAG graph > page (#[5121](tel:5121)) > >> > - [AIRFLOW-[4310](tel:4310)] Fix incorrect link on Dag Details page > (#[5122](tel:5122)) > >> > - [AIRFLOW-[4331](tel:4331)] Correct filter for Null-state runs from > Dag Detail page > >> > (#[5123](tel:5123)) > >> > - [AIRFLOW-[4294](tel:4294)] Fix missing dag & task runs in UI dag_id > contains a dot > >> > (#[5111](tel:5111)) > >> > - [AIRFLOW-[4332](tel:4332)] Upgrade sqlalchemy to remove security > Vulnerability > >> > (#[5113](tel:5113)) > >> > - [AIRFLOW-[4312](tel:4312)] Add template_fields & template_ext to > BigQueryCheckO… > >> > (#[5097](tel:5097)) > >> > - [AIRFLOW-[4293](tel:4293)] Fix downgrade in > >> > d4ecb8fbee3_add_schedule_interval_to_dag.py (#[5086](tel:5086)) > >> > - [AIRFLOW-[4267](tel:4267)] Fix TI duration in Graph View > (#[5071](tel:5071)) > >> > - [AIRFLOW-[4163](tel:4163)] IntervalCheckOperator supports relative > diff and not > >> > ignore 0 (#[4983](tel:4983)) > >> > - [AIRFLOW-[3938](tel:3938)] QuboleOperator Fixes and Support for > SqlCommand (#[4832](tel:4832)) > >> > - [AIRFLOW-[2903](tel:2903)] Change default owner to ``airflow`` > (#[4151](tel:4151)) > >> > - [aIRFLOW-[4136](tel:4136)] Fix overwrite of key_file by constructor > (#[5155](tel:5155)) > >> > - [AIRFLOW-[3241](tel:3241)] Remove Invalid template ext in GCS > Sensors (#[4076](tel:4076)) > >> > > >> > Misc/Internal > >> > """"""""""""" > >> > - [AIRFLOW-[4338](tel:4338)] Change k8s pod_request_factory to use > yaml safe_load > >> > (#[5120](tel:5120)) > >> > - [AIRFLOW-[4869](tel:4869)] Reorganize sql to gcs operators. > (#[5504](tel:5504)) > >> > - [AIRFLOW-[5021](tel:5021)] move gitpython into setup_requires > (#[5640](tel:5640)) > >> > - [AIRFLOW-[4583](tel:4583)] Fixes type error in GKEPodOperator > (#[5612](tel:5612)) > >> > - [AIRFLOW-[4116](tel:4116)] Dockerfile now supports CI image build > on DockerHub > >> > (#[4937](tel:4937)) > >> > - [AIRFLOW-[4115](tel:4115)] Multi-staging Aiflow Docker image > (#[4936](tel:4936)) > >> > - [AIRFLOW-[4963](tel:4963)] Avoid recreating task context > (#[5596](tel:5596)) > >> > - [AIRFLOW-[4865](tel:4865)] Add context manager to set temporary > config values in > >> > tests. (#[5569](tel:5569)) > >> > - [AIRFLOW-[4937](tel:4937)] Fix lodash security issue with version > below [4.17.13](tel:41713) > >> > (#[5572](tel:5572)) (used only in build-pipeline, not runtime) > >> > - [AIRFLOW-[4868](tel:4868)] Fix typo in kubernetes/docker/build.sh > (#[5505](tel:5505)) > >> > - [AIRFLOW-[4211](tel:4211)] Add tests for WebHDFSHook > (#[5015](tel:5015)) > >> > - [AIRFLOW-[4320](tel:4320)] Add tests for SegmentTrackEventOperator > (#[5104](tel:5104)) > >> > - [AIRFLOW-[4319](tel:4319)] Add tests for Bigquery related Operators > (#[5101](tel:5101)) > >> > - [AIRFLOW-[4014](tel:4014)] Change DatastoreHook and add tests > (#[4842](tel:4842)) > >> > - [AIRFLOW-[4322](tel:4322)] Add test for VerticaOperator > (#[5107](tel:5107)) > >> > - [AIRFLOW-[4323](tel:4323)] Add 2 tests for WinRMOperator > (#[5108](tel:5108)) > >> > - [AIRFLOW-[3677](tel:3677)] Improve CheckOperator test coverage > (#[4756](tel:4756)) > >> > - [AIRFLOW-[4659](tel:4659)] Fix pylint problems for api module > (#[5398](tel:5398)) > >> > - [AIRFLOW-[4358](tel:4358)] Speed up test_jobs by not running tasks > (#[5162](tel:5162)) > >> > - [AIRFLOW-[4394](tel:4394)] Don't test behaviour of BackfillJob from > CLI tests > >> (#[5160](tel:5160)) > >> > - [AIRFLOW-[3471](tel:3471)] Move XCom out of models.py > (#[4629](tel:4629)) > >> > - [AIRFLOW-[4379](tel:4379)] Remove duplicate code & Add validation > in gcs_to_gcs.py > >> > (#[5145](tel:5145)) > >> > - [AIRFLOW-[4259](tel:4259)] Move models out of models.py > (#[5056](tel:5056)) > >> > - [AIRFLOW-XXX] Speed up building of Cassanda module on Travis > (#[5233](tel:5233)) > >> > - [AIRFLOW-[4535](tel:4535)] Break jobs.py into multiple files > (#[5303](tel:5303)) > >> > - [AIRFLOW-[1464](tel:1464)] Batch update task_instance state > (#[5323](tel:5323)) > >> > - [AIRFLOW-[4554](tel:4554)] Test for sudo command, add some other > test docs (#[5310](tel:5310)) > >> > - [AIRFLOW-[4419](tel:4419)] Refine concurrency check in scheduler > (#[5194](tel:5194)) > >> > - [AIRFLOW-[4269](tel:4269)] Minor acceleration of > jobs._process_task_instances() > >> > (#[5076](tel:5076)) > >> > - [AIRFLOW-[4341](tel:4341)] Remove ``View.render()`` already exists > in fab.BaseView > >> > (#[5125](tel:5125)) > >> > - [AIRFLOW-[4342](tel:4342)] Use @cached_property instead of > re-implementing it each > >> > time (#[5126](tel:5126)) > >> > - [AIRFLOW-[4256](tel:4256)] Remove noqa from migrations > (#[5055](tel:5055)) > >> > - [AIRFLOW-[4034](tel:4034)] Remove unnecessary string formatting > with ``**locals()`` > >> > (#[4861](tel:4861)) > >> > - [AIRFLOW-[3944](tel:3944)] Remove code smells (#[4762](tel:4762)) > >> > > >> > Doc-only changes > >> > """""""""""""""" > >> > - [AIRFLOW-XXX] Add missing doc for annotations param of > >> > KubernetesPodOperator (#[5666](tel:5666)) > >> > - [AIRFLOW-XXX] Fix typos in CONTRIBUTING.md (#[5626](tel:5626)) > >> > - [AIRFLOW-XXX] Correct BaseSensorOperator docs (#[5562](tel:5562)) > >> > - [AIRFLOW-[4926](tel:4926)] Fix example dags where its start_date is > >> > datetime.utcnow() (#[5553](tel:5553)) > >> > - [AIRFLOW-[4860](tel:4860)] Remove Redundant Information in Example > Dags (#[5497](tel:5497)) > >> > - [AIRFLOW-[4767](tel:4767)] Fix errors in the documentation of > Dataproc Operator > >> > (#[5487](tel:5487)) > >> > - [AIRFLOW-[1684](tel:1684)] Branching based on XCom variable (Docs) > (#[4365](tel:4365)) > >> > - [AIRFLOW-[3341](tel:3341)] FAQ return DAG object example > (#[4605](tel:4605)) > >> > - [AIRFLOW-[4433](tel:4433)] Add missing type in DockerOperator doc > string (#[5205](tel:5205)) > >> > - [AIRFLOW-[4321](tel:4321)] Replace incorrect info of Max Size limit > of GCS Object > >> > Size (#[5106](tel:5106)) > >> > - [AIRFLOW-XXX] Add information about user list (#[5341](tel:5341)) > >> > - [AIRFLOW-XXX] Clarify documentation related to autodetect parameter > in > >> > GCS_to_BQ Op (#[5294](tel:5294)) > >> > - [AIRFLOW-XXX] Remove mention of pytz compatibility from timezone > >> > documentation (#[5316](tel:5316)) > >> > - [AIRFLOW-XXX] Add missing docs for GoogleCloudStorageDeleteOperator > >> > (#[5274](tel:5274)) > >> > - [AIRFLOW-XXX] Remove incorrect note about Scopes of GCP connection > >> > (#[5242](tel:5242)) > >> > - [AIRFLOW-XXX] Fix mistakes in docs of Dataproc operators > (#[5192](tel:5192)) > >> > - [AIRFLOW-XXX] Link to correct class for timedelta in macros.rst > (#[5226](tel:5226)) > >> > - [AIRFLOW-XXX] Add Kamil as committer (#[5216](tel:5216)) > >> > - [AIRFLOW-XXX] Add Joshua and Kevin as committer (#[5207](tel:5207)) > >> > - [AIRFLOW-XXX] Reduce log spam in tests (#[5174](tel:5174)) > >> > - [AIRFLOW-XXX] Speed up tests for PythonSensor (#[5158](tel:5158)) > >> > - [AIRFLOW-XXX] Add Bas Harenslak to committer list > (#[5157](tel:5157)) > >> > - [AIRFLOW-XXX] Add Jarek Potiuk to commiter list (#[5132](tel:5132)) > >> > - [AIRFLOW-XXX] Update docstring for SchedulerJob (#[5105](tel:5105)) > >> > - [AIRFLOW-XXX] Fix docstrings for > CassandraToGoogleCloudStorageOperator > >> > (#[5103](tel:5103)) > >> > - [AIRFLOW-XXX] update SlackWebhookHook and SlackWebhookOperator > >> docstring > >> > (#[5074](tel:5074)) > >> > - [AIRFLOW-XXX] Ignore python files under node_modules in docs > (#[5063](tel:5063)) > >> > - [AIRFLOW-XXX] Build a universal wheel with LICNESE files > (#[5052](tel:5052)) > >> > - [AIRFLOW-XXX] Fix docstrings of SQSHook (#[5099](tel:5099)) > >> > - [AIRFLOW-XXX] Use Py3.7 on readthedocs > >> > - [AIRFLOW-[4446](tel:4446)] Fix typos (#[5217](tel:5217)) > >> > >> > >> > >> -- > >> *Kaxil Naik* > >> *Big Data Consultant | DevOps Data Engineer* > >> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & > Neo4j > >> Developer > >> *LinkedIn*: https://www.linkedin.com/in/kaxil > >> -- Chao-Han Tsai
