utkarsharma2 commented on issue #36384: URL: https://github.com/apache/airflow/issues/36384#issuecomment-1868220019
Hey Jarek, Would it be possible to have an RC2 to include this commit <https://github.com/apache/airflow/commit/75d74b1f3a535fdc3624077bde3a34d1abcf641e> for the Weavaite Provider? Thanks, Utkarsh Sharma On Sat, Dec 23, 2023 at 10:18 AM Gopal Dirisala ***@***.***> wrote: > I have tested the following Kubernetes provider fixes. They are working as > expected. I am eagerly waiting for the release that has a critical > performance bug. > > - Kubernetes executor running slots leak fix (#36240) > <https://github.com/apache/airflow/pull/36240>: @dirrao > <https://github.com/dirrao> > - list pods performance optimization (#36092) > <https://github.com/apache/airflow/pull/36092>: @dirrao > <https://github.com/dirrao> > > > > > On Sat, Dec 23, 2023 at 7:53 AM Jarek Potiuk ***@***.***> > wrote: > > > Body > > > > I have a kind request for all the contributors to the latest provider > > packages release. > > Could you please help us to test the RC versions of the providers? > > > > The guidelines on how to test providers can be found in > > > > Verify providers by contributors > > < > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors> > > > > > Let us know in the comment, whether the issue is addressed. > > > > Those are providers that require testing as there were some substantial > > changes introduced: > > Provider airbyte: 3.5.1rc1 > > <https://pypi.org/project/apache-airflow-providers-airbyte/3.5.1rc1> > > > > - Cancel airbyte job when timeout exceeded to prevent inconsistency > > amo… (#36241) <https://github.com/apache/airflow/pull/36241>: @shohamy7 > > <https://github.com/shohamy7> > > > > Provider amazon: 8.14.0rc1 > > <https://pypi.org/project/apache-airflow-providers-amazon/8.14.0rc1> > > > > - Add jsonpath_ng.ext.parse support for SqsSensor (#36170) > > <https://github.com/apache/airflow/pull/36170>: @cliebBS > > <https://github.com/cliebBS> > > - Increase ConflictException retries to 4 total (#36337) > > <https://github.com/apache/airflow/pull/36337>: @o-nikolas > > <https://github.com/o-nikolas> > > - Add more parameter support for the copy_object function in > > airflow.providers.amazon.aws.hooks.s3 (#36304) > > <https://github.com/apache/airflow/pull/36304>: @samezapple > > <https://github.com/samezapple> > > - Allow storage options to be passed (#35820) > > <https://github.com/apache/airflow/pull/35820>: @bolkedebruin > > <https://github.com/bolkedebruin> > > - Remove is_authorized_cluster_activity from auth manager (#36175) > > <https://github.com/apache/airflow/pull/36175>: @vincbeck > > <https://github.com/vincbeck> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > - Remove remaining Airflow 2.5 backcompat code from Amazon Provider > > (#36324) <https://github.com/apache/airflow/pull/36324>: @Taragolis > > <https://github.com/Taragolis> > > > > Provider apache.beam: 5.5.0rc1 > > <https://pypi.org/project/apache-airflow-providers-apache-beam/5.5.0rc1> > > > > > - Add ability to run streaming Job for BeamRunPythonPipelineOperator > > in non deferrable mode (#36108) > > <https://github.com/apache/airflow/pull/36108>: @MaksYermak > > <https://github.com/MaksYermak> > > - Implement deferrable mode for BeamRunJavaPipelineOperator (#36122) > > <https://github.com/apache/airflow/pull/36122>: @moiseenkov > > <https://github.com/moiseenkov> > > > > Provider apache.cassandra: 3.4.1rc1 > > < > https://pypi.org/project/apache-airflow-providers-apache-cassandra/3.4.1rc1> > > > > > - Sanitize keyspace and table in the cassandra query to avoid sql > > injection (#36111) <https://github.com/apache/airflow/pull/36111>: > > @hussein-awala <https://github.com/hussein-awala> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > > > Provider apache.hdfs: 4.3.1rc1 > > <https://pypi.org/project/apache-airflow-providers-apache-hdfs/4.3.1rc1> > > > > > - fix connection type webhdfs (#36145) > > <https://github.com/apache/airflow/pull/36145>: @ankurbajaj9 > > <https://github.com/ankurbajaj9> > > > > Provider apache.hive: 6.4.0rc1 > > <https://pypi.org/project/apache-airflow-providers-apache-hive/6.4.0rc1> > > > > > - add param proxy user for hive (#36221) > > <https://github.com/apache/airflow/pull/36221>: @romsharon98 > > <https://github.com/romsharon98> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > > > Provider apache.kafka: 1.3.1rc1 > > <https://pypi.org/project/apache-airflow-providers-apache-kafka/1.3.1rc1> > > > > > - Provide the consumed message to consumer.commit in > > AwaitMessageTrigger (#36272) > > <https://github.com/apache/airflow/pull/36272>: @hussein-awala > > <https://github.com/hussein-awala> > > > > Provider apache.spark: 4.6.0rc1 > > <https://pypi.org/project/apache-airflow-providers-apache-spark/4.6.0rc1> > > > > > - SparkSubmit: Adding propertyfiles option (#36164) > > <https://github.com/apache/airflow/pull/36164>: @pateash > > <https://github.com/pateash> > > - SparkSubmit Connection Extras can be overridden (#36151) > > <https://github.com/apache/airflow/pull/36151>: @pateash > > <https://github.com/pateash> > > > > Provider celery: 3.5.1rc1 > > <https://pypi.org/project/apache-airflow-providers-celery/3.5.1rc1> > > > > - Fix sentinel_kwargs load from ENV (#36318) > > <https://github.com/apache/airflow/pull/36318>: @vaaalik > > <https://github.com/vaaalik> > > > > Provider cncf.kubernetes: 7.12.0rc1 > > < > https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/7.12.0rc1> > > > > > - Add _request_timeout to KPO log fetch calls (#36297) > > <https://github.com/apache/airflow/pull/36297>: @dstandish > > <https://github.com/dstandish> > > - Add pod_template_dict field to KubernetesPodOperator (#33174) > > <https://github.com/apache/airflow/pull/33174>: @sonmezonur > > <https://github.com/sonmezonur> > > - KubernetesPodTrigger: add exception stack trace in TriggerEvent > > (#35716) <https://github.com/apache/airflow/pull/35716>: @functicons > > <https://github.com/functicons> > > - Limit pod_name length to HOST_NAME_MAX (#36332) > > <https://github.com/apache/airflow/pull/36332>: @csp33 > > <https://github.com/csp33> > > - Move KubernetesPodTrigger hook to a cached property (#36290) > > <https://github.com/apache/airflow/pull/36290>: @hussein-awala > > <https://github.com/hussein-awala> > > - Kubernetes executor running slots leak fix (#36240) > > <https://github.com/apache/airflow/pull/36240>: @dirrao > > <https://github.com/dirrao> > > - list pods performance optimization (#36092) > > <https://github.com/apache/airflow/pull/36092>: @dirrao > > <https://github.com/dirrao> > > > > Provider common.sql: 1.10.0rc1 > > <https://pypi.org/project/apache-airflow-providers-common-sql/1.10.0rc1> > > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > - SQLCheckOperator fails if it returns dict with any Falsy values > > (#36273) <https://github.com/apache/airflow/pull/36273>: > > @spencertollefson <https://github.com/spencertollefson> > > > > Provider databricks: 6.0.0rc1 > > <https://pypi.org/project/apache-airflow-providers-databricks/6.0.0rc1> > > > > - Make databricks sql hook return a namedtuple (#36205) > > <https://github.com/apache/airflow/pull/36205>: @Joffreybvn > > <https://github.com/Joffreybvn> > > - Fix: Implement support for fetchone() in the ODBCHook and the > > Databricks SQL Hook (#36161) > > <https://github.com/apache/airflow/pull/36161>: @Joffreybvn > > <https://github.com/Joffreybvn> > > - Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017) > > <https://github.com/apache/airflow/pull/36017>: @eladkal > > <https://github.com/eladkal> > > - fix typos in DatabricksSubmitRunOperator (#36248) > > <https://github.com/apache/airflow/pull/36248>: @adam133 > > <https://github.com/adam133> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > - Prepare docs 1st wave of Providers December 2023 (#36112) > > <https://github.com/apache/airflow/pull/36112>: @eladkal > > <https://github.com/eladkal> > > - Prepare docs 1st wave of Providers December 2023 RC2 (#36190) > > <https://github.com/apache/airflow/pull/36190>: @eladkal > > <https://github.com/eladkal> > > - Re-apply updated version numbers to 2nd wave of providers in > > December (#36380) <https://github.com/apache/airflow/pull/36380>: > > @potiuk <https://github.com/potiuk> > > - Prepare 2nd wave of providers in December (#36373) > > <https://github.com/apache/airflow/pull/36373>: @potiuk > > <https://github.com/potiuk> > > > > Provider docker: 3.9.1rc1 > > <https://pypi.org/project/apache-airflow-providers-docker/3.9.1rc1> > > > > - Allow DockerOperator.skip_on_exit_code to be zero (#36360) > > <https://github.com/apache/airflow/pull/36360>: @dolfinus > > <https://github.com/dolfinus> > > - Remove remaining Airflow 2.5 backcompat code from Docker Provider > > (#36325) <https://github.com/apache/airflow/pull/36325>: @Taragolis > > <https://github.com/Taragolis> > > > > Provider google: 10.13.0rc3 > > <https://pypi.org/project/apache-airflow-providers-google/10.13.0rc3> > > > > - GCP Secrets Backend Impersonation (#36072) > > <https://github.com/apache/airflow/pull/36072>: @nathadfield > > <https://github.com/nathadfield> > > - Add OpenLineage support to GCS operators (#35838) > > <https://github.com/apache/airflow/pull/35838>: @kacpermuda > > <https://github.com/kacpermuda> > > - Add support for service account impersonation with > > computeEngineSSHHook (google provider) and IAP tunnel (#35136) > > <https://github.com/apache/airflow/pull/35136>: @ginolegigot > > <https://github.com/ginolegigot> > > - Added Datascan Profiling (#35696) > > <https://github.com/apache/airflow/pull/35696>: @shourya116 > > <https://github.com/shourya116> > > - Add overrides to template fields of Google Cloud Run Jobs Execute > > Operator (#36133) <https://github.com/apache/airflow/pull/36133>: > > @guillaumeblaquiere <https://github.com/guillaumeblaquiere> > > - Implement deferrable mode for BeamRunJavaPipelineOperator (#36122) > > <https://github.com/apache/airflow/pull/36122>: @moiseenkov > > <https://github.com/moiseenkov> > > - Add ability to run streaming Job for BeamRunPythonPipelineOperator > > in non deferrable mode (#36108) > > <https://github.com/apache/airflow/pull/36108>: @MaksYermak > > <https://github.com/MaksYermak> > > - Add use_glob to GCSObjectExistenceSensor (#34137) > > <https://github.com/apache/airflow/pull/34137>: @A-Costa > > <https://github.com/A-Costa> > > - Fix DataprocSubmitJobOperator to retrieve failed job error message > > (#36053) <https://github.com/apache/airflow/pull/36053>: > > @VladaZakharova <https://github.com/VladaZakharova> > > - Fix CloudRunExecuteJobOperator in deferrable mode (#36012) > > <https://github.com/apache/airflow/pull/36012>: @VladaZakharova > > <https://github.com/VladaZakharova> > > - Fix gcs listing - ensure blobs are loaded (#34919) > > <https://github.com/apache/airflow/pull/34919>: @atrbgithub > > <https://github.com/atrbgithub> > > - allow multiple elements in impersonation chain (#35694) > > <https://github.com/apache/airflow/pull/35694>: @melugoyal > > <https://github.com/melugoyal> > > - Change retry type for Google Clients to async ones where needed > > (#36141) <https://github.com/apache/airflow/pull/36141>: @potiuk > > <https://github.com/potiuk> > > - Minor fix to DataprocCreateClusterOperator operator docs. (#36322) > > <https://github.com/apache/airflow/pull/36322>: @bjankie1 > > <https://github.com/bjankie1> > > - fix(bigquery.py): pass correct project_id to triggerer (#35200) > > <https://github.com/apache/airflow/pull/35200>: @mokshasoul > > <https://github.com/mokshasoul> > > - iterate through blobs before checking prefixes (#36202) > > <https://github.com/apache/airflow/pull/36202>: @Lee-W > > <https://github.com/Lee-W> > > - Fix incompatibility with google-cloud-monitoring 2.18.0 (#36200) > > <https://github.com/apache/airflow/pull/36200>: @potiuk > > <https://github.com/potiuk> > > - Update retry param typing in PubSubAsyncHook (#36198) > > <https://github.com/apache/airflow/pull/36198>: @josh-fell > > <https://github.com/josh-fell> > > - Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017) > > <https://github.com/apache/airflow/pull/36017>: @eladkal > > <https://github.com/eladkal> > > - Deprecate CloudComposerEnvironmentSensor in favor of > > CloudComposerCreateEnvironmentOperator with defer mode (#35775) > > <https://github.com/apache/airflow/pull/35775>: @VladaZakharova > > <https://github.com/VladaZakharova> > > - Allow storage options to be passed (#35820) > > <https://github.com/apache/airflow/pull/35820>: @bolkedebruin > > <https://github.com/bolkedebruin> > > - Add feature to build "chicken-egg" packages from sources (#35890) > > <https://github.com/apache/airflow/pull/35890>: @potiuk > > <https://github.com/potiuk> > > - Remove remaining Airflow 2.5 backcompat code from Google Provider > > (#36366) <https://github.com/apache/airflow/pull/36366>: @Taragolis > > <https://github.com/Taragolis> > > - Move KubernetesPodTrigger hook to a cached property (#36290) > > <https://github.com/apache/airflow/pull/36290>: @hussein-awala > > <https://github.com/hussein-awala> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > > > Provider microsoft.azure: 8.5.0rc1 > > < > https://pypi.org/project/apache-airflow-providers-microsoft-azure/8.5.0rc1> > > > > > - Allow storage options to be passed (#35820) > > <https://github.com/apache/airflow/pull/35820>: @bolkedebruin > > <https://github.com/bolkedebruin> > > - AzureFileShareHook fix with connection type "Azure" (#36309) > > <https://github.com/apache/airflow/pull/36309>: @abhishekbhakat > > <https://github.com/abhishekbhakat> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > > > Provider odbc: 4.4.0rc1 > > <https://pypi.org/project/apache-airflow-providers-odbc/4.4.0rc1> > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > > > Provider openlineage: 1.3.1rc1 > > <https://pypi.org/project/apache-airflow-providers-openlineage/1.3.1rc1> > > > > > - openlineage: set extractors None in config (#36362) > > <https://github.com/apache/airflow/pull/36362>: @JDarDagran > > <https://github.com/JDarDagran> > > > > Provider postgres: 5.10.0rc1 > > <https://pypi.org/project/apache-airflow-providers-postgres/5.10.0rc1> > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > > > Provider presto: 5.4.0rc1 > > <https://pypi.org/project/apache-airflow-providers-presto/5.4.0rc1> > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > > > Provider sftp: 4.8.1rc1 > > <https://pypi.org/project/apache-airflow-providers-sftp/4.8.1rc1> > > > > - Fix sftp warrning log (#36148) > > <https://github.com/apache/airflow/pull/36148>: @romsharon98 > > <https://github.com/romsharon98> > > - Add code snippet formatting in docstrings via Ruff (#36262) > > <https://github.com/apache/airflow/pull/36262>: @josh-fell > > <https://github.com/josh-fell> > > > > Provider slack: 8.5.1rc1 > > <https://pypi.org/project/apache-airflow-providers-slack/8.5.1rc1> > > > > - Slack: Remove parameter token in SlackAPIPostOperator's docstring > > (#36121) <https://github.com/apache/airflow/pull/36121>: @brienzb > > <https://github.com/brienzb> > > > > Provider smtp: 1.6.0rc1 > > <https://pypi.org/project/apache-airflow-providers-smtp/1.6.0rc1> > > > > - Modify SmtpNotifier to accept template with defaults (#36226) > > <https://github.com/apache/airflow/pull/36226>: @vchiapaikeo > > <https://github.com/vchiapaikeo> > > > > Provider sqlite: 3.7.0rc1 > > <https://pypi.org/project/apache-airflow-providers-sqlite/3.7.0rc1> > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > > > Provider ssh: 3.10.0rc1 > > <https://pypi.org/project/apache-airflow-providers-ssh/3.10.0rc1> > > > > - Add skip_on_exit_code to SSHOperator (#36303) > > <https://github.com/apache/airflow/pull/36303>: @dolfinus > > <https://github.com/dolfinus> > > - Allow SSHOperator.skip_on_exit_code to be zero (#36358) > > <https://github.com/apache/airflow/pull/36358>: @dolfinus > > <https://github.com/dolfinus> > > - Review and mark found potential SSH security issues by bandit > > (#36162) <https://github.com/apache/airflow/pull/36162>: @potiuk > > <https://github.com/potiuk> > > > > Provider trino: 5.6.0rc1 > > <https://pypi.org/project/apache-airflow-providers-trino/5.6.0rc1> > > > > - Make "placeholder" of ODBC configurable in UI (#36000) > > <https://github.com/apache/airflow/pull/36000>: @jarnedemunter1 > > <https://github.com/jarnedemunter1> > > > > Provider weaviate: 1.2.0rc1 > > <https://pypi.org/project/apache-airflow-providers-weaviate/1.2.0rc1> > > > > - Add helper function for CRUD operations on weaviate's schema and > > class objects (#35919) <https://github.com/apache/airflow/pull/35919>: > > @utkarsharma2 <https://github.com/utkarsharma2> > > - Add retry mechanism and dataframe support for WeaviateIngestOperator > > (#36085) <https://github.com/apache/airflow/pull/36085>: @utkarsharma2 > > <https://github.com/utkarsharma2> > > - Fixing template_fields for WeaviateIngestOperator (#36359) > > <https://github.com/apache/airflow/pull/36359>: @vatsrahul1001 > > <https://github.com/vatsrahul1001> > > > > Committer > > > > - I acknowledge that I am a maintainer/committer of the Apache Airflow > > project. > > > > — > > Reply to this email directly, view it on GitHub > > <https://github.com/apache/airflow/issues/36384>, or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AJPTQJWWCMFL3PLTC7RXVADYKY6DNAVCNFSM6AAAAABBAO623CVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2TINRSGM4DSOA> > > > . > > You are receiving this because you were mentioned.Message ID: > > ***@***.***> > > > > > -- > > Thanks and regards > Dirisala Gopala Rao > > > Ph: +918861444868 > +919494447656 > alt. email: ***@***.*** > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/airflow/issues/36384#issuecomment-1868204994>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADDLAHIJQIVE7E4OTRHLQUTYKZPATAVCNFSM6AAAAABBAO623CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGIYDIOJZGQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
