[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217322#comment-16217322
]
Semet edited comment on SPARK-16367 at 10/24/17 5:29 PM:
-
Yes, I don't use it
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217322#comment-16217322
]
Semet commented on SPARK-16367:
---
Yes, I don't use it because it is a feature of {{pip}}:
{{code}
pip wheel
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217221#comment-16217221
]
Semet edited comment on SPARK-13587 at 10/24/17 4:46 PM:
-
Hello. For me this
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217221#comment-16217221
]
Semet commented on SPARK-13587:
---
Hello. For me this solution is equivalent with my "Wheelhouse"
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815466#comment-15815466
]
Semet commented on SPARK-16367:
---
Yes in our pull request both conda and pip are supported. Wheel allow pip
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714622#comment-15714622
]
Semet commented on SPARK-13587:
---
For myself, I share a NFS folder with all the executors. It works because
[
https://issues.apache.org/jira/browse/SPARK-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557842#comment-15557842
]
Semet commented on SPARK-5864:
--
can I recommend to add .whl and .tar.gz in the file pattern? Python
[
https://issues.apache.org/jira/browse/SPARK-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15538162#comment-15538162
]
Semet commented on SPARK-969:
-
I agree that archiving this page for post mortem analysis is helpful.
>
[
https://issues.apache.org/jira/browse/SPARK-10057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506972#comment-15506972
]
Semet commented on SPARK-10057:
---
Hello
I confirm we have this issue on Spark 1.6.1 when writing a parquet
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470328#comment-15470328
]
Semet commented on SPARK-16367:
---
Blog post about PySpark job deployment:
[
https://issues.apache.org/jira/browse/SPARK-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466855#comment-15466855
]
Semet commented on SPARK-5091:
--
It is a better option to use virtualenv and proper installation with pip,
[
https://issues.apache.org/jira/browse/SPARK-17360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455415#comment-15455415
]
Semet commented on SPARK-17360:
---
Look like it work straigh, without modifying the code, however I propose
Semet created SPARK-17360:
-
Summary: PySpark can create dataframe from a Python generator
Key: SPARK-17360
URL: https://issues.apache.org/jira/browse/SPARK-17360
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416907#comment-15416907
]
Semet commented on SPARK-5160:
--
zip files are already supported, just add zip to --pyfiles and they get added
[
https://issues.apache.org/jira/browse/SPARK-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415127#comment-15415127
]
Semet edited comment on SPARK-16992 at 8/10/16 11:15 AM:
-
For the import
[
https://issues.apache.org/jira/browse/SPARK-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16992:
--
Description:
Add code style checks and auto formating into the Python code.
Features:
- add a
[
https://issues.apache.org/jira/browse/SPARK-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415127#comment-15415127
]
Semet commented on SPARK-16992:
---
For the import statement ordering, this helped us a lot to set up a
Semet created SPARK-16992:
-
Summary: Pep8 code style
Key: SPARK-16992
URL: https://issues.apache.org/jira/browse/SPARK-16992
Project: Spark
Issue Type: Improvement
Reporter: Semet
Add
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375005#comment-15375005
]
Semet edited comment on SPARK-16367 at 7/13/16 1:48 PM:
I have sent a design doc
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375005#comment-15375005
]
Semet edited comment on SPARK-16367 at 7/13/16 1:48 PM:
I have sent a design doc
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375005#comment-15375005
]
Semet commented on SPARK-16367:
---
I have sent a design doc and pullrequest. They are deliberately based on
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365599#comment-15365599
]
Semet commented on SPARK-16367:
---
Wheels are tagged by os, architecture, Python version and it seems to be
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363980#comment-15363980
]
Semet commented on SPARK-16367:
---
Description of the bug updated :)
> Wheelhouse Support for PySpark
>
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363902#comment-15363902
]
Semet commented on SPARK-16367:
---
I am wrong on one point: even if each node fetch independently from the
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363899#comment-15363899
]
Semet commented on SPARK-16367:
---
Yes and you have artifactory that has and automatic mirroring capability
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363898#comment-15363898
]
Semet commented on SPARK-16367:
---
For me, you as a developer are already inside a virtual env when you write
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363189#comment-15363189
]
Semet commented on SPARK-16367:
---
This is where the magic of wheels lies:
- look at
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362294#comment-15362294
]
Semet commented on SPARK-13587:
---
Full proposal is here: https://issues.apache.org/jira/browse/SPARK-16367
[
https://issues.apache.org/jira/browse/SPARK-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361408#comment-15361408
]
Semet edited comment on SPARK-6764 at 7/5/16 9:44 AM:
--
Hello
I am working on a new
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Labels: newbie wh (was: newbie)
> Wheelhouse Support for PySpark
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Labels: newbie python python-wheel wheelhouse (was: newbie wh)
> Wheelhouse Support for PySpark
>
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Issue Type: New Feature (was: Improvement)
> Wheelhouse Support for PySpark
> --
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
[
https://issues.apache.org/jira/browse/SPARK-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semet updated SPARK-16367:
--
Description:
*Rational*
Is it recommended, in order to deploying Scala packages written in Scala, to
build
Semet created SPARK-16367:
-
Summary: Wheelhouse Support for PySpark
Key: SPARK-16367
URL: https://issues.apache.org/jira/browse/SPARK-16367
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361408#comment-15361408
]
Semet commented on SPARK-6764:
--
Hello
I am working on a new proposal for complete wheel support, along with
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355172#comment-15355172
]
Semet edited comment on SPARK-13587 at 6/29/16 12:52 PM:
-
yes it looks cool!
Here
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355172#comment-15355172
]
Semet commented on SPARK-13587:
---
yes it looks cool!
Here is what I have in mind, tell me if it is the wrong
[
https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351074#comment-15351074
]
Semet commented on SPARK-13587:
---
I back this proposal and willing to work on it.
> Support virtualenv in
66 matches
Mail list logo
