Re: Spark SQL doesn't produce output while hive does

2019-03-04 Thread Chunpeng Wang
check the data schema especially for user_mobile On Mon, Mar 4, 2019 at 1:18 AM mayangyang02 wrote: > Hi, > > We have a sql. When we ran it with Hive, it produced the result normally. > But when we ran it with Spark, id didn’t produce any output. > > We found that what caused the problem is the

SGD for pyspark

2018-12-11 Thread Chunpeng Wang
Hi Spark Community, I want to call spark optimizer in pyspark for a self-defined loss function. Do you know if pyspark can have optimization module in the future like org.apache.spark.mllib.optimization in scala? thank you, Chunpeng