potiuk commented on code in PR #45962:
URL: https://github.com/apache/airflow/pull/45962#discussion_r1931277176


##########
task_sdk/pyproject.toml:
##########
@@ -27,10 +27,11 @@ dependencies = [
     "httpx>=0.27.0",
     "jinja2>=3.1.4",
     "methodtools>=0.4.7",
-    "msgspec>=0.18.6",
     "psutil>=6.1.0",
     "structlog>=24.4.0",
     "retryhttp>=1.2.0",
+    "orjson>=3.10.11",

Review Comment:
   No, I am just curious - because I was just wondering if adding new 
dependency is needed - every new dependency adds a security risk and 
complexity, and with my Airflow Security beach Cleaning project, I am looking 
closely at every new dependency added. Every single one increases supply chain 
risk, so IMHO, we should become more and more careful and make sure that we add 
new dependency when it's really needed. 
   
   That's mostly it. I am not sure if we can avoid, it by using what 
FastAPI/Pydantic already support - i have not checked it in detail all I know 
serializing back/forth is precisely what Pydantic does, so it would be 
reasonable to assume we could serialize logs with it. 
   
   That's why I am asking question if that's been maybe considered - from what  
I understand Pydantic can already serialize and deserialize arbitrary dicts 
https://docs.pydantic.dev/2.3/usage/types/dicts_mapping/ , so my question is do 
we need yet another library for that ? 
   
   And streaming is more of an optimization, - depending on the use cases for 
that logging API , I guess we now send the logs in chunks, so it might not be 
needed, but theorethically in the future we could just switch to an async 
streaming API for that that could be more efficient. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to