spark git commit: [SPARK-12717][PYTHON][BRANCH-2.1] Adding thread-safe broadcast pickle registry

2017-08-02 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.1 b31b30209 -> d93e45b8b [SPARK-12717][PYTHON][BRANCH-2.1] Adding thread-safe broadcast pickle registry ## What changes were proposed in this pull request? When using PySpark broadcast variables in a multi-threaded environment,

spark git commit: [SPARK-12717][PYTHON][BRANCH-2.2] Adding thread-safe broadcast pickle registry

2017-08-02 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.2 467ee8dff -> 690f491f6 [SPARK-12717][PYTHON][BRANCH-2.2] Adding thread-safe broadcast pickle registry ## What changes were proposed in this pull request? When using PySpark broadcast variables in a multi-threaded environment,

spark git commit: [SPARK-21546][SS] dropDuplicates should ignore watermark when it's not a key

2017-08-02 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 9456176da -> 0d26b3aa5 [SPARK-21546][SS] dropDuplicates should ignore watermark when it's not a key ## What changes were proposed in this pull request? When the watermark is not a column of `dropDuplicates`, right now it will crash. This

spark git commit: [SPARK-21546][SS] dropDuplicates should ignore watermark when it's not a key

2017-08-02 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.2 397f90421 -> 467ee8dff [SPARK-21546][SS] dropDuplicates should ignore watermark when it's not a key ## What changes were proposed in this pull request? When the watermark is not a column of `dropDuplicates`, right now it will crash.

spark git commit: [SPARK-21490][CORE] Make sure SparkLauncher redirects needed streams.

2017-08-02 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 7f63e85b4 -> 9456176da [SPARK-21490][CORE] Make sure SparkLauncher redirects needed streams. The code was failing to account for some cases when setting up log redirection. For example, if a user redirected only stdout to a file, the

spark git commit: [SPARK-21597][SS] Fix a potential overflow issue in EventTimeStats

2017-08-02 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.2 67c60d78e -> 397f90421 [SPARK-21597][SS] Fix a potential overflow issue in EventTimeStats ## What changes were proposed in this pull request? This PR fixed a potential overflow issue in EventTimeStats. ## How was this patch tested?

spark git commit: [SPARK-21597][SS] Fix a potential overflow issue in EventTimeStats

2017-08-02 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 845c039ce -> 7f63e85b4 [SPARK-21597][SS] Fix a potential overflow issue in EventTimeStats ## What changes were proposed in this pull request? This PR fixed a potential overflow issue in EventTimeStats. ## How was this patch tested? The

spark git commit: [SPARK-20601][ML] Python API for Constrained Logistic Regression

2017-08-02 Thread yliang
Repository: spark Updated Branches: refs/heads/master 14e75758a -> 845c039ce [SPARK-20601][ML] Python API for Constrained Logistic Regression ## What changes were proposed in this pull request? Python API for Constrained Logistic Regression based on #17922 , thanks for the original