Re: bug with PYTHONHASHSEED

2017-04-05 Thread Paul Tremblay
I saw the bug fix. I am using the latest Spark available on AWS EMR which I think is 2.01. I am at work and can't check my home config. I don't think AWS merged in this fix. Henry On Tue, Apr 4, 2017 at 4:42 PM, Jeff Zhang wrote: > > It is fixed in

Re: bug with PYTHONHASHSEED

2017-04-04 Thread Jeff Zhang
It is fixed in https://issues.apache.org/jira/browse/SPARK-13330 Holden Karau 于2017年4月5日周三 上午12:03写道： > Which version of Spark is this (or is it a dev build)? We've recently made > some improvements with PYTHONHASHSEED propagation. > > On Tue, Apr 4, 2017 at 7:49 AM Eike

Re: bug with PYTHONHASHSEED

2017-04-04 Thread Holden Karau
Which version of Spark is this (or is it a dev build)? We've recently made some improvements with PYTHONHASHSEED propagation. On Tue, Apr 4, 2017 at 7:49 AM Eike von Seggern wrote: 2017-04-01 21:54 GMT+02:00 Paul Tremblay : When I try to to

Re: bug with PYTHONHASHSEED

2017-04-04 Thread Paul Tremblay
So that means I have to pass that bash variable to the EMR clusters when I spin them up, not afterwards. I'll give that a go. Thanks! Henry On Tue, Apr 4, 2017 at 7:49 AM, Eike von Seggern wrote: > 2017-04-01 21:54 GMT+02:00 Paul Tremblay :

Re: bug with PYTHONHASHSEED

2017-04-04 Thread Eike von Seggern
2017-04-01 21:54 GMT+02:00 Paul Tremblay : > When I try to to do a groupByKey() in my spark environment, I get the > error described here: > > http://stackoverflow.com/questions/36798833/what-does-except > ion-randomness-of-hash-of-string-should-be-disabled-via-pythonh >

bug with PYTHONHASHSEED

2017-04-01 Thread Paul Tremblay
When I try to to do a groupByKey() in my spark environment, I get the error described here: http://stackoverflow.com/questions/36798833/what-does- exception-randomness-of-hash-of-string-should-be-disabled-via-pythonh In order to attempt to fix the problem, I set up my ipython environment with

pyspark bug with PYTHONHASHSEED

2017-04-01 Thread Paul Tremblay
When I try to to do a groupByKey() in my spark environment, I get the error described here: http://stackoverflow.com/questions/36798833/what-does-exception-randomness-of-hash-of-string-should-be-disabled-via-pythonh In order to attempt to fix the problem, I set up my ipython environment with the