[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742866#comment-14742866
]
holdenk commented on SPARK-10447:
-
I've got a PR for this, and I can start the streaming context in the shell and
do so stuff but I'm not super sure why the streaming tests aren't working with
it (need to do some more digging).
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731896#comment-14731896
]
Apache Spark commented on SPARK-10447:
--
User 'holdenk' has created a pull request for this issue:
https://github.com/apache/spark/pull/8615
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730990#comment-14730990
]
Sean Owen commented on SPARK-10447:
---
I bet there are some upsides to updating, but the question is: do we know if
anything breaks, changes? Worth at least running the tests with this change,
but also skimming the release notes to understand any breaking changes.
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731164#comment-14731164
]
Justin Uang commented on SPARK-10447:
-
Agreed, I'm pretty sure that this will break some APIs and we'll have to fix
those as we do the upgrade =).
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731592#comment-14731592
]
Justin Uang commented on SPARK-10447:
-
Sure, I wouldn't mind doing the code review. Can you add me?
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731591#comment-14731591
]
holdenk commented on SPARK-10447:
-
I can give this a shot if no one else is interested in doing this (I've been
wrangling some py4j bits with Sparkling Pandas).
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731597#comment-14731597
]
holdenk commented on SPARK-10447:
-
Sure, I'll ping you when I've got the PR ready (probably sometime this long
weekend) if that's good for you?
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10447) Upgrade pyspark to use py4j 0.9
[
https://issues.apache.org/jira/browse/SPARK-10447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731598#comment-14731598
]
Justin Uang commented on SPARK-10447:
-
Sound good
> Upgrade pyspark to use py4j 0.9
> ---
>
> Key: SPARK-10447
> URL: https://issues.apache.org/jira/browse/SPARK-10447
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
>Affects Versions: 1.4.1
>Reporter: Justin Uang
>
> This was recently released, and it has many improvements, especially the
> following:
> {quote}
> Python side: IDEs and interactive interpreters such as IPython can now get
> help text/autocompletion for Java classes, objects, and members. This makes
> Py4J an ideal tool to explore complex Java APIs (e.g., the Eclipse API).
> Thanks to @jonahkichwacoders
> {quote}
> Normally we wrap all the APIs in spark, but for the ones that aren't, this
> would make it easier to offroad by using the java proxy objects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
