Re: log4j custom appender ClassNotFoundException with spark 1.4.1

2015-08-07 Thread mlemay
That starts to smell... When analyzing SparkSubmit.scala, we can see than one of the firsts thing it does is to parse arguments. This uses Utils object and triggers initialization of member variables. One such variable is ShutdownHookManager (which didn't exists in spark 1.3) with the later

Possible bug: JDBC with Speculative mode launches orphan queries

2015-08-07 Thread Saif.A.Ellafi
Hello, When enabling speculation, my first job is to launch a partitioned JDBC dataframe query, in which some partitions take longer than others to respond. This causes speculation and creates new nodes to launch the query. When one of those nodes finish the query, the speculative one remains

<    1   2