Hi,
I have made a number of code changes to make the SimpleApp work.
Your error messages are quite good.
package co.uk.backbutton.sparksimpleapp;
/**
*
* @author zahid
*/
/* SimpleApp.java */
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import
Hi,
On page https://spark.apache.org/docs/latest/quick-start.html
if you change one line in pom.xml for SimpleApp
from:
spark-sql_2.12
to:
spark-sql_2.11
Then the ease of use and simplicity would increase ten fold
and the simple app would be even simpler.
Backbutton.co.uk
¯\_(ツ)_/¯
SOLVED
On Sat, 29 Feb 2020 at 17:23, Zahid Rahman wrote:
>
> Hi,
>
> I am running it on linux.
> Is there a programmatic way to get rid of this security error or is it
> configuration file error.
>
> zahid@localhost
> :~/Downloads/apachespark/Apache-Spark-Example/Java-Code-Geek>
> mvn
Hi,
I am running it on linux.
Is there a programmatic way to get rid of this security error or is it
configuration file error.
zahid@localhost:~/Downloads/apachespark/Apache-Spark-Example/Java-Code-Geek>
mvn exec:java -Dexec.mainClass=com.javacodegeek.examples.SparkExampleRDD
Jungtaek,
Thank you for taking a look at this issue. I would be happy to test your
patch - but I am unsure how to do so. Can you help me to understand how I
can:
1. Obtain an updated version of the JAR in question either via a nightly
build or by building myself?
2. Substitute this JAR, ideally
Forgot to mention - it only occurs the SQL type of UDT is having fixed
length. If the UDT is used to represent complex type like array, struct, or
even string, it doesn't trigger the issue. So that's like an edge-case and
the chance of encountering this issue may not be that huge, and that's why
I've investigated a bit, and looks like it's not an issue of
mapGroupsWithState, but an issue of how UDT is handled in UnsafeRow. It
seems to miss handling UDT and the missing spot makes the internal code of
Spark corrupt the value. (So if I'm not mistaken, it's a correctness issue.)
I've filed
