Pardon the burst :) Russell Jurney @rjurney <http://twitter.com/rjurney> russell.jur...@gmail.com LI <http://linkedin.com/in/russelljurney> FB <http://facebook.com/jurney> datasyndrome.com
On Mon, Mar 11, 2019 at 7:02 PM Russell Jurney (JIRA) <j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/DATAFU-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790113#comment-16790113 > ] > > Russell Jurney edited comment on DATAFU-148 at 3/12/19 2:01 AM: > ---------------------------------------------------------------- > > I appreciate your work on this and I'm trying to make it work in Python 3. > There was a print statement that crashed previously. I'm using findspark to > see if the files will run locally in Python from the shell. The > pyspark_utils all do. > > I got it to run using Python 3.6.7 on Ubuntu 17.10. This is great! I'll do > some more playing around and then dive into the code before committing it. > The following issues were discovered: > * There was a 2.7 print statement in init_spark_context.py. Can you wrap > it in quotes? > * There is no note about running tests for datafu-spark . in the master > README.md > * Can you remove the '...' from the README? People are going to paste > this code and it makes it hard. > * exporting PYTHONPATH did not work for me, I had to run --jars and > --conf as in the README. I don't know if this is a DataFu issue or a spark > issue. > > I want to thank you for breathing life back into DataFu. With Spark the > project can continue to thrive! This is something I meant to do for years > and never got round to. Thanks again! > > > was (Author: russell.jurney): > I appreciate your work on this and I'm trying to make it work in Python 3. > There was a print statement that crashed previously. I'm using findspark to > see if the files will run locally in Python from the shell. The > pyspark_utils all do. > > I got it to run using Python 3.6.7 on Ubuntu 17.10. This is great! I'll do > some more playing around and then dive into the code before committing it. > The following issues were discovered: > * There was a 2.7 print statement in init_spark_context.py > * There is no note about running tests for datafu-spark . in the master > README.md > * Can you remove the '...' from the README? People are going to paste > this code and it makes it hard. > * exporting PYTHONPATH did not work for me, I had to run --jars and > --conf as in the README. I don't know if this is a DataFu issue or a spark > issue. > > I want to thank you for breathing life back into DataFu. With Spark the > project can continue to thrive! This is something I meant to do for years > and never got round to. Thanks again! > > > Setup Spark sub-project > > ----------------------- > > > > Key: DATAFU-148 > > URL: https://issues.apache.org/jira/browse/DATAFU-148 > > Project: DataFu > > Issue Type: New Feature > > Reporter: Eyal Allweil > > Assignee: Eyal Allweil > > Priority: Major > > Attachments: patch.diff, patch.diff > > > > Time Spent: 20m > > Remaining Estimate: 0h > > > > Create a skeleton Spark sub project for Spark code to be contributed to > DataFu > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005) >
