[GitHub] [spark] maheshk114 commented on pull request #41860: [SPARK-44307][SQL] Add Bloom filter for left outer join even if the left side table is smaller than broadcast threshold.

[via GitHub]

maheshk114 commented on PR #41860:
URL: https://github.com/apache/spark/pull/41860#issuecomment-1637344666

   @beliefer 
   I have got some number with varying size for the bloom filter. This decides 
the amount of data filtered by bloom from the application side. Even if the 
data size is reduced to half, I can see some improvements. When number of bits 
to be used are set very low (77864), I can see a regression of around 4 secs as 
 no records are filtered out. 
   
   
![image](https://github.com/apache/spark/assets/19660171/ab599a09-77ae-4d8c-9929-3cb8eb933eeb)
   
![image](https://github.com/apache/spark/assets/19660171/e5a985e6-ec3a-475a-8fc3-5e59b767bda0)
   
   
   
![image](https://github.com/apache/spark/assets/19660171/9ff75fab-b8fa-4f22-a0b0-4291291f39a1)
   
![image](https://github.com/apache/spark/assets/19660171/27d06fbc-3b63-4604-bfe1-aa45f8e9cb6d)
   
![image](https://github.com/apache/spark/assets/19660171/b91b778f-2540-464b-a534-e7c7da004ecc)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org