utkarsharma2 commented on code in PR #36177:
URL: https://github.com/apache/airflow/pull/36177#discussion_r1435078555
##########
airflow/providers/weaviate/hooks/weaviate.py:
##########
@@ -396,10 +400,25 @@ def batch_data(
.. seealso:: `batch_config_params options
<https://weaviate-python-client.readthedocs.io/en/v3.25.3/weaviate.batch.html#weaviate.batch.Batch.configure>`__
:param vector_col: name of the column containing the vector.
:param retry_attempts_per_object: number of time to try in case of
failure before giving up.
+ :param tenant: The tenant to which the object will be added.
+ :param uuid_col: Name of the column containing the UUID.
+ :param insertion_errors: list to hold errors while inserting.
"""
client = self.conn
if not batch_config_params:
batch_config_params = {}
+
+ # configuration for context manager for __exit__ method to callback on
errors for weaviate
+ # batch ingestion.
+ if not batch_config_params.get("callback"):
+ batch_config_params.update({"callback":
partial(self.process_batch_errors, insertion_errors)})
+
+ if not batch_config_params.get("timeout_retries"):
+ batch_config_params.update({"timeout_retries": 5})
+
+ if not batch_config_params.get("connection_error_retries"):
+ batch_config_params.update({"connection_error_retries": 5})
Review Comment:
Added the suggestion in the code. Thanks for pointing it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
