ECOS Spark Integration

2017-12-17 Thread Debasish Das 
Hi,

ECOS is a solver for second order conic programs and we showed the Spark
integration at 2014 Spark Summit
https://spark-summit.org/2014/quadratic-programing-solver-for-non-negative-matrix-factorization/.
Right now the examples show how to reformulate matrix factorization as a
SOCP and solve each alternating steps with ECOS:

https://github.com/embotech/ecos-java-scala

For distributed optimization, I expect it will be useful where for each
primary row key (sensor, car, robot :-) we are fitting a constrained
quadratic / cone program. Please try it out and let me know the feedbacks.

Thanks.
Deb

Re: [01/51] [partial] spark-website git commit: 2.2.1 generated doc

2017-12-17 Thread Reynold Xin 
There is an additional step that's needed to update the symlink, and that
step hasn't been done yet.


On Sun, Dec 17, 2017 at 12:32 PM, Jacek Laskowski  wrote:

> Hi Sean,
>
> What does "Not all the pieces are released yet" mean if you don't mind me
> asking? 2.2.1 has already been announced, hasn't it? [1]
>
> [1] http://spark.apache.org/news/spark-2-2-1-released.html
>
> Pozdrawiam,
> Jacek Laskowski
> 
> https://about.me/JacekLaskowski
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark
> Follow me at https://twitter.com/jaceklaskowski
>
> On Sun, Dec 17, 2017 at 4:19 PM, Sean Owen  wrote:
>
>> /latest does not point to 2.2.1 yet. Not all the pieces are released yet,
>> as I understand?
>>
>> On Sun, Dec 17, 2017 at 8:12 AM Jacek Laskowski  wrote:
>>
>>> Hi,
>>>
>>> I saw the following commit, but I can't seem to see 2.2.1 as the version
>>> in the header of the documentation pages under http://spark.apache.org/
>>> docs/latest/ (that is still 2.2.0). Is this being worked on?
>>>
>>> http://spark.apache.org/docs/2.2.1 is available and shows the proper
>>> version, but not http://spark.apache.org/docs/latest :(
>>>
>>> Pozdrawiam,
>>> Jacek Laskowski
>>> 
>>>
>>>
>

Re: [01/51] [partial] spark-website git commit: 2.2.1 generated doc

2017-12-17 Thread Jacek Laskowski 
Hi Sean,

What does "Not all the pieces are released yet" mean if you don't mind me
asking? 2.2.1 has already been announced, hasn't it? [1]

[1] http://spark.apache.org/news/spark-2-2-1-released.html

Pozdrawiam,
Jacek Laskowski

https://about.me/JacekLaskowski
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski

On Sun, Dec 17, 2017 at 4:19 PM, Sean Owen  wrote:

> /latest does not point to 2.2.1 yet. Not all the pieces are released yet,
> as I understand?
>
> On Sun, Dec 17, 2017 at 8:12 AM Jacek Laskowski  wrote:
>
>> Hi,
>>
>> I saw the following commit, but I can't seem to see 2.2.1 as the version
>> in the header of the documentation pages under http://spark.apache.org/
>> docs/latest/ (that is still 2.2.0). Is this being worked on?
>>
>> http://spark.apache.org/docs/2.2.1 is available and shows the proper
>> version, but not http://spark.apache.org/docs/latest :(
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> 
>>
>>

Re: Hinge Gradient

2017-12-17 Thread Debasish Das 
If you can point me to previous benchmarks that are done, I would like to
use smoothing and see if the LBFGS convergence improved while not impacting
linear svc loss.

Thanks.
Deb
On Dec 16, 2017 7:48 PM, "Debasish Das"  wrote:

Hi Weichen,

Traditionally svm are solved using quadratic programming solvers and most
likely that's why this idea is not so popular but since in mllib we are
using smooth methods to optimize linear svm, the idea of smoothing svm loss
become relevant.

The paper also mentions kernel svm using the same idea. In place of full
kernel, we can use random kitchen sink.

http://research.cs.wisc.edu/dmi/svm/ssvm/

I will go through Yuhao's work as well.

Thanks.
Deb


On Dec 16, 2017 6:35 PM, "Weichen Xu"  wrote:

Hi Deb,

Which library or paper do you find to use this loss function in SVM ?

But I prefer the implementation in LIBLINEAR which use coordinate descent
optimizer.

Thanks.

On Sun, Dec 17, 2017 at 6:52 AM, Yanbo Liang  wrote:

> Hello Deb,
>
> To optimize non-smooth function on LBFGS really should be considered
> carefully.
> Is there any literature that proves changing max to soft-max can behave
> well?
> I’m more than happy to see some benchmarks if you can have.
>
> + Yuhao, who did similar effort in this PR: https://github.com/apache/
> spark/pull/17862
>
> Regards
> Yanbo
>
> On Dec 13, 2017, at 12:20 AM, Debasish Das 
> wrote:
>
> Hi,
>
> I looked into the LinearSVC flow and found the gradient for hinge as
> follows:
>
> Our loss function with {0, 1} labels is max(0, 1 - (2y - 1) (f_w(x)))
> Therefore the gradient is -(2y - 1)*x
>
> max is a non-smooth function.
>
> Did we try using ReLu/Softmax function and use that to smooth the hinge
> loss ?
>
> Loss function will change to SoftMax(0, 1 - (2y-1) (f_w(x)))
>
> Since this function is smooth, gradient will be well defined and
> LBFGS/OWLQN should behave well.
>
> Please let me know if this has been tried already. If not I can run some
> benchmarks.
>
> We have soft-max in multinomial regression and can be reused for LinearSVC
> flow.
>
> Thanks.
> Deb
>
>
>

Re: [01/51] [partial] spark-website git commit: 2.2.1 generated doc

2017-12-17 Thread Sean Owen 
/latest does not point to 2.2.1 yet. Not all the pieces are released yet,
as I understand?

On Sun, Dec 17, 2017 at 8:12 AM Jacek Laskowski  wrote:

> Hi,
>
> I saw the following commit, but I can't seem to see 2.2.1 as the version
> in the header of the documentation pages under
> http://spark.apache.org/docs/latest/ (that is still 2.2.0). Is this being
> worked on?
>
> http://spark.apache.org/docs/2.2.1 is available and shows the proper
> version, but not http://spark.apache.org/docs/latest :(
>
> Pozdrawiam,
> Jacek Laskowski
> 
>
>

Re: [01/51] [partial] spark-website git commit: 2.2.1 generated doc

2017-12-17 Thread Jacek Laskowski 
Hi,

I saw the following commit, but I can't seem to see 2.2.1 as the version in
the header of the documentation pages under
http://spark.apache.org/docs/latest/ (that is still 2.2.0). Is this being
worked on?

http://spark.apache.org/docs/2.2.1 is available and shows the proper
version, but not http://spark.apache.org/docs/latest :(

Pozdrawiam,
Jacek Laskowski

https://about.me/JacekLaskowski
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski

On Thu, Dec 14, 2017 at 1:51 PM,  wrote:

> Repository: spark-website
> Updated Branches:
>   refs/heads/asf-site c312ef84e -> 49992614f
>
>
> http://git-wip-us.apache.org/repos/asf/spark-website/blob/49
> 992614/site/docs/2.2.1/api/java/org/apache/spark/api/java/
> function/ForeachFunction.html
> --
> diff --git 
> a/site/docs/2.2.1/api/java/org/apache/spark/api/java/function/ForeachFunction.html
> b/site/docs/2.2.1/api/java/org/apache/spark/api/java/functio
> n/ForeachFunction.html
> new file mode 100644
> index 000..81efb81
> --- /dev/null
> +++ b/site/docs/2.2.1/api/java/org/apache/spark/api/java/functio
> n/ForeachFunction.html
> @@ -0,0 +1,239 @@
> + http://www.w3.org/TR/html4/loose.dtd;>
> +
> +
> +
> +
> +ForeachFunction (Spark 2.2.1 JavaDoc)
> +
> + href="../../../../../../stylesheet.css"
> title="Style">
> +
> +
> +
> +
> +var methods = {"i0":6};
> +var tabs = {65535:["t0","All Methods"],2:["t2","Instance
> Methods"],4:["t3","Abstract Methods"]};
> +var altColor = "altColor";
> +var rowColor = "rowColor";
> +var tableTab = "tableTab";
> +var activeTableTab = "activeTableTab";
> +
> +
> +JavaScript is disabled on your browser.
> +
> +
> +
> +
> +
> +Skip navigation links
> +
> +
> +
> +
> +Overview
> +Package
> +Class
> +Tree
> +Deprecated
> +Index
> +Help
> +
> +
> +
> +
> +Prev
> ;Class
> + href="../../../../../../org/apache/spark/api/java/function/ForeachPartitionFunction.html"
> title="interface in org.apache.spark.api.java.function"> class="typeNameLink">NextClass
> +
> +
> +Frames
> +NoFrames
> +
> +
> +All
> Classes
> +
> +
> +
> +
> +
> +
> +
> +Summary:
> +Nested|
> +Field|
> +Constr|
> +Method
> +
> +
> +Detail:
> +Field|
> +Constr|
> +Method
> +
> +
> +
> +
> +
> +
> +
> +
> +org.apache.spark.api.java.function
> +Interface
> ForeachFunctionT
> +
> +
> +
> +
> +
> +
> +All Superinterfaces:
> +java.io.Serializable
> +
> +
> +Functional Interface:
> +This is a functional interface and can therefore be used as the
> assignment target for a lambda expression or method reference.
> +
> +
> +
> +@FunctionalInterface
> +public interface ForeachF
> unctionT
> +extends java.io.Serializable
> +Base interface for a function used in Dataset's
> foreach function.
> +
> + Spark will invoke the call function on each element in the input
> Dataset.
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +Method Summary
> + summary="Method Summary table, listing methods, and an explanation">
> +All
> Methods class="tableTab">Instance
> Methods class="tableTab">Abstract
> Methods
> +
> +Modifier and Type
> +Method and Description
> +
> +
> +void
> + href="../../../../../../org/apache/spark/api/java/function/
> ForeachFunction.html#call-T-">call( href="../../../../../../org/apache/spark/api/java/function/ForeachFunction.html"
> title="type parameter in ForeachFunction">Tt)
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +Method Detail
> +
> +
> +
> +
> +
> +
> +
> +call
> +voidcall(Tt)
> +   throws Exception
> +
> +Throws:
> +Exception
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +Skip navigation links
> +
> +
> +
> +
> +Overview
> +Package
> +Class
> +Tree
> +Deprecated
> +Index
> +Help
> +
> +
> +
> +
> +Prev
> ;Class
> + href="../../../../../../org/apache/spark/api/java/function/ForeachPartitionFunction.html"
> title="interface in org.apache.spark.api.java.function"> class="typeNameLink">NextClass
> +
> +
> +Frames
> +NoFrames
> +
> +
> +All
> Classes
> +
> +
> +
> +
> +
> +
> +
> +Summary:
> +Nested|
> +Field|
> +Constr|
> +Method
> +
> +
> +Detail:
> +Field|
> +Constr|
> +Method
> +
> +
> +
> +
> +
> +
> + src="../../../../../../lib/jquery.js">