In yours file /home/spark/real-estate/pullhttp/pull_apartments.py
replace import org.apache.spark.SparkContext with from pyspark import
SparkContext
man. 21. aug. 2023 kl. 15:13 skrev Kal Stevens :
> I am getting a class not found error
> import org.apache.spark.SparkContext
>
> It sounds
I am getting a class not found error
import org.apache.spark.SparkContext
It sounds like this is because pyspark is not installed, but as far as I
can tell it is.
Pyspark is installed in the correct python verison
root@namenode:/home/spark/# pip3.10 install pyspark
Requirement already
This should work
check your path. It should pyspark from
which pyspark
/opt/spark/bin/pyspark
And your installation should contain
cd $SPARK_HOME
/opt/spark> ls
LICENSE NOTICE R README.md RELEASE bin conf data examples jars
kubernetes licenses logs python sbin yarn
You should
Nevermind I was doing something dumb
On Sun, Aug 20, 2023 at 9:53 PM Kal Stevens wrote:
> Are there installation instructions for Spark 3.4.1?
>
> I defined SPARK_HOME as it describes here
>
> https://spark.apache.org/docs/latest/api/python/getting_started/install.html
>
> ls
Are there installation instructions for Spark 3.4.1?
I defined SPARK_HOME as it describes here
https://spark.apache.org/docs/latest/api/python/getting_started/install.html
ls $SPARK_HOME/python/lib
py4j-0.10.9.7-src.zip PY4J_LICENSE.txt pyspark.zip
I am getting a class not found error
