+1 to doc, seed argument would be great if possible
From: Sean Owen
Sent: Monday, September 26, 2022 5:26:26 PM
To: Nicholas Gustafson
Cc: dev
Subject: Re: Why are hash functions seeded with 42?
Oh yeah I get why we love to pick 42 for random things. I'm
Sorry about that, it made me laugh 6 years ago, I didn't expect this to
come back and haunt me :)...
There are ways out of this, none of them are particularly appealing:
- Add a SQL conf to make the value configurable.
- Add a seed parameter to the function. I am not sure if we can make this
work
Oh yeah I get why we love to pick 42 for random things. I'm guessing it was
a bit of an oversight here as the 'seed' is directly initial state and 0
makes much more sense.
On Mon, Sep 26, 2022, 7:24 PM Nicholas Gustafson
wrote:
> I don’t know the reason, however would offer a hunch that perhaps
I don’t know the reason, however would offer a hunch that perhaps it’s a nod to
Douglas Adams (author of The Hitchhiker’s Guide to the Galaxy).
https://news.mit.edu/2019/answer-life-universe-and-everything-sum-three-cubes-mathematics-0910
> On Sep 26, 2022, at 16:59, Sean Owen wrote:
>
>
>
OK, it came to my attention today that hash functions in spark, like
xxhash64, actually always seed with 42:
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala#L655
This is an issue if you want the hash of some value in
