[
https://issues.apache.org/jira/browse/DATAFU-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790113#comment-16790113
]
Russell Jurney edited comment on DATAFU-148 at 3/12/19 1:59 AM:
----------------------------------------------------------------
I appreciate your work on this and I'm trying to make it work in Python 3.
There was a print statement that crashed previously. I'm using findspark to see
if the files will run locally in Python from the shell. The pyspark_utils all
do.
I got it to run. This is great! I'll do some more playing around and then dive
into the code before committing it. The following issues were discovered:
* There was a 2.7 print statement in init_spark_context.py
* There is no note about running tests for datafu-spark . in the master
README.md
* Can you remove the '...' from the README? People are going to paste this
code and it makes it hard.
* exporting PYTHONPATH did not work for me, I had to run --jars and --conf as
in the README. I don't know if this is a DataFu issue or a spark issue.
was (Author: russell.jurney):
I appreciate your work on this and I'm trying to make it work in Python 3.
There was a print statement that crashed previously. I'm using findspark to see
if the files will run locally in Python from the shell. The pyspark_utils all
do, trying to get tests to as well.
I got it to run. This is great! I'll do some more playing around and then dive
into the code before committing it. The following issues were discovered:
* There was a 2.7 print statement in init_spark_context.py
* There is no note about running tests for datafu-spark . in the master
README.md
* Can you remove the '...' from the README? People are going to paste this
code and it makes it hard.
* exporting PYTHONPATH did not work for me, I had to run --jars and --conf as
in the README. I don't know if this is a DataFu issue or a spark issue.
> Setup Spark sub-project
> -----------------------
>
> Key: DATAFU-148
> URL: https://issues.apache.org/jira/browse/DATAFU-148
> Project: DataFu
> Issue Type: New Feature
> Reporter: Eyal Allweil
> Assignee: Eyal Allweil
> Priority: Major
> Attachments: patch.diff, patch.diff
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Create a skeleton Spark sub project for Spark code to be contributed to DataFu
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
