Fwd: Uncaught Exception Handler in master

2019-04-16 Thread Alessandro Liparoti
. Any idea of what I can do? *Alessandro Liparoti*

Uncaught Exception Handler in master

2019-03-28 Thread Alessandro Liparoti
. Any idea of what I can do? *Alessandro Liparoti*

Logging exception of custom spark app

2019-01-04 Thread Alessandro Liparoti
Handler, so probably this code is not effective. The other option would be to try-catch all public methods of my API, log an exception when it happens and then throw it. But I think this is not optimal. Do you have any suggestion? *Alessandro Liparoti*

pySpark and py4j: NoClassDefFoundError when upgrading a jar

2018-12-31 Thread Alessandro Liparoti
log shows when starting up the shell) Do you have any idea of what the problem might be? Does maybe py4j cache something so that when upgrading the classes we get this error? *Alessandro Liparoti*

Package option gets outdated jar when running with "latest"

2018-12-28 Thread Alessandro Liparoti
to solve this? It is also very hard to debug since I couldn't find a pattern to reproduce it. It happens on every release that changes a class name but not for everyone running the job (that's why caching looked like a good hint to me). Thanks, *Alessandro Liparoti*

Checkpointing clarifications

2018-09-20 Thread Alessandro Liparoti
of these 2 apparently different operations? Thanks, *Alessandro Liparoti*

Spark sql syntax checker

2018-08-03 Thread Alessandro Liparoti
me. Which approach do you suggest me to use? Is there a syntactical check in the spark codebase that I can use for my use case? *Alessandro Liparoti*

Spark FAIR Scheduler vs FIFO Scheduler

2018-06-18 Thread Alessandro Liparoti
how it works? Thanks, *Alessandro Liparoti*

Writing to HDFS and cluster utilization

2018-06-15 Thread Alessandro Liparoti
doing something wrong. Thanks, *Alessandro Liparoti*

Spark issue 20236 - overwrite a partitioned data srouce

2018-06-14 Thread Alessandro Liparoti
Good morning, I am trying to see how this bug affects the write in spark 2.2.0, but I cannot reproduce it. Is it ok then using the code df.write.mode(SaveMode.Overwrite).insertInto("table_name") ? Thank you, *Alessandro Liparoti*