OK with PyCharm itself, i am getting this error
pyspark.sql.utils.AnalysisException: java.lang.RuntimeException: Error
while running command to get file permissions : java.io.IOException: (null)
entry in command string: null ls -F
C:\Users\admin\PycharmProjects\pythonProject\hive-scratchdir
I
You don't have to include all your config and log messages. The error
message would suffice. The java.lang.UnsatisfiedLinkError exception
indicates that the JVM can't find some OS-specific libraries (or
commonly referred as native libraries). On Windows, they would be some
dll files. Look
This is becoming serious pain.
using powershell I am using spark-submit as follows:
PS C:\Users\admin> spark-submit.cmd
C:\Users\admin\PycharmProjects\pythonProject\main.py
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
Apparently this is a OS dynamic lib link error. Make sure you have the
LD_LIBRARY_PATH (in Linux) or PATH (windows) set up properly for the
right .so or .dll file...
On 12/2/20 5:31 PM, Mich Talebzadeh wrote:
Hi,
I have a simple code that tries to create Hive derby database as follows:
Hi,
I have a simple code that tries to create Hive derby database as follows:
from pyspark import SparkContext
from pyspark.sql import SQLContext
from pyspark.sql import HiveContext
from pyspark.sql import SparkSession
from pyspark.sql import Row
from pyspark.sql.types import StringType,
