Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85634800
--- Diff: dev/create-release/release-build.sh ---
@@ -162,14 +162,35 @@ if [[ "$1" == "package" ]]; then
export ZINC_PORT=$ZINC_PORT
echo
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85623891
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85538940
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85538754
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85538196
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85536498
--- Diff: dev/create-release/release-build.sh ---
@@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then
# We increment the Zinc port each time to
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85536240
--- Diff: dev/make-distribution.sh ---
@@ -201,6 +201,12 @@ fi
# Copy data files
cp -r "$SPARK_HOME/data" "$DISTDIR"
+# Make pip package
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85533965
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85531031
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85526799
--- Diff: dev/make-distribution.sh ---
@@ -201,6 +201,12 @@ fi
# Copy data files
cp -r "$SPARK_HOME/data" "$DISTDIR"
+# Make pip package
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85525949
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85524567
--- Diff: python/setup.py ---
@@ -0,0 +1,170 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85520992
--- Diff: dev/create-release/release-build.sh ---
@@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then
# We increment the Zinc port each time to
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85509442
--- Diff: pom.xml ---
@@ -26,6 +26,7 @@
org.apache.spark
spark-parent_2.11
+
--- End diff --
So during the binary
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85508101
--- Diff: dev/make-distribution.sh ---
@@ -201,6 +201,12 @@ fi
# Copy data files
cp -r "$SPARK_HOME/data" "$DISTDIR"
+# Make pip package
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85507888
--- Diff: dev/create-release/release-build.sh ---
@@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then
# We increment the Zinc port each time to
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85507910
--- Diff: bin/spark-shell ---
@@ -29,7 +29,7 @@ esac
set -o posix
if [ -z "${SPARK_HOME}" ]; then
- export SPARK_HOME="$(cd "`dirname
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85507670
--- Diff: dev/run-tests.py ---
@@ -583,6 +589,7 @@ def main():
modules_with_python_tests = [m for m in test_modules if
m.python_test_goals]
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85507586
--- Diff: docs/building-spark.md ---
@@ -259,6 +259,12 @@ or
Java 8 tests are automatically enabled when a Java 8 JDK is detected.
If you have JDK
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85491680
--- Diff: dev/run-tests.py ---
@@ -583,6 +589,7 @@ def main():
modules_with_python_tests = [m for m in test_modules if
m.python_test_goals]
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85486513
--- Diff: dev/pip-sanity-check.py ---
@@ -0,0 +1,36 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85486184
--- Diff: dev/make-distribution.sh ---
@@ -201,6 +201,12 @@ fi
# Copy data files
cp -r "$SPARK_HOME/data" "$DISTDIR"
+# Make pip package
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85484770
--- Diff: dev/create-release/release-build.sh ---
@@ -187,10 +208,10 @@ if [[ "$1" == "package" ]]; then
# We increment the Zinc port each time to
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85377223
--- Diff: pom.xml ---
@@ -26,6 +26,7 @@
org.apache.spark
spark-parent_2.11
+
--- End diff --
Not a sticking point
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85372805
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85372583
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85364365
--- Diff: pom.xml ---
@@ -26,6 +26,7 @@
org.apache.spark
spark-parent_2.11
+
--- End diff --
Something along the
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85365186
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85364778
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85361946
--- Diff: bin/spark-class ---
@@ -36,7 +36,7 @@ else
fi
# Find Spark jars.
-if [ -f "${SPARK_HOME}/RELEASE" ]; then
+if [ -d
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85361694
--- Diff: dev/create-release/release-build.sh ---
@@ -162,14 +162,35 @@ if [[ "$1" == "package" ]]; then
export ZINC_PORT=$ZINC_PORT
echo
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85361542
--- Diff: python/MANIFEST.in ---
@@ -0,0 +1,23 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85360766
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85360572
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85359033
--- Diff: python/setup.py ---
@@ -0,0 +1,169 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85355701
--- Diff: python/setup.py ---
@@ -0,0 +1,169 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85352748
--- Diff: pom.xml ---
@@ -26,6 +26,7 @@
org.apache.spark
spark-parent_2.11
+
--- End diff --
Would it be overkill
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85354868
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85355211
--- Diff: python/pyspark/find_spark_home.py ---
@@ -0,0 +1,65 @@
+#!/usr/bin/python
+
+#
+# Licensed to the Apache Software Foundation (ASF)
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85355847
--- Diff: python/setup.cfg ---
@@ -0,0 +1,22 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85350993
--- Diff: bin/spark-class ---
@@ -36,7 +36,7 @@ else
fi
# Find Spark jars.
-if [ -f "${SPARK_HOME}/RELEASE" ]; then
+if [ -d
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85353057
--- Diff: python/MANIFEST.in ---
@@ -0,0 +1,23 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85353699
--- Diff: python/README.md ---
@@ -0,0 +1,32 @@
+# Apache Spark
+
+Spark is a fast and general cluster computing system for Big Data. It
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85351820
--- Diff: dev/create-release/release-build.sh ---
@@ -162,14 +162,35 @@ if [[ "$1" == "package" ]]; then
export ZINC_PORT=$ZINC_PORT
echo
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/15659#discussion_r85326497
--- Diff: python/setup.py ---
@@ -0,0 +1,169 @@
+#!/usr/bin/env python
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or
GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/15659
[WIP][SPARK-1267][SPARK-18129] Allow PySpark to be pip installed
## What changes were proposed in this pull request?
This PR aims to provide a pip installable PySpark package. This does a
46 matches
Mail list logo
