Send real-time alert using Spark

[Priya Ch]

Hi All,

 I am building Real-time Anomaly detection system where I am using k-means
to detect anomaly. Now in-order to send alert to mobile or an email alert
how do i send it using Spark itself ?

Thanks,
Padma CH

Re: Send real-time alert using Spark

[Marcin Tustin]

Priya,

You wouldn't necessarily "use spark" to send the alert. Spark is in an
important sense one library among many. You can have your application use
any other library available for your language to send the alert.

Marcin

On Tue, Jul 12, 2016 at 9:25 AM, Priya Ch 
wrote:

> Hi All,
>
>  I am building Real-time Anomaly detection system where I am using k-means
> to detect anomaly. Now in-order to send alert to mobile or an email alert
> how do i send it using Spark itself ?
>
> Thanks,
> Padma CH
>

-- 
Want to work at Handy? Check out our culture deck and open roles 

Latest news  at Handy
Handy just raised $50m 

 led 
by Fidelity

Re: KEYS file?

[Sean Owen]

PS I've already opened a test PR for the new apache/spark-website repo:

https://github.com/apache/spark-website/pull/1

I guess we'll follow the same process for reviewing there. Next: see
if the main merge script works for this repo!

On Mon, Jul 11, 2016 at 9:52 PM, Sean Owen  wrote:
> Aha, that's landed. OK I'll figure it out tomorrow and push my update
> to verify it all works.
>
> On Mon, Jul 11, 2016 at 8:54 PM, Reynold Xin  wrote:
>> It's related to this apparently:
>> https://issues.apache.org/jira/servicedesk/customer/portal/1/INFRA-12055
>>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Send real-time alert using Spark

[Priya Ch]

I mean model training on incoming data is taken care by Spark. For detected
anomalies, need to send alert. Could we do this with Spark or any other
framework like Akka/REST API would do it ?

Thanks,
Padma CH

On Tue, Jul 12, 2016 at 7:30 PM, Marcin Tustin 
wrote:

> Priya,
>
> You wouldn't necessarily "use spark" to send the alert. Spark is in an
> important sense one library among many. You can have your application use
> any other library available for your language to send the alert.
>
> Marcin
>
> On Tue, Jul 12, 2016 at 9:25 AM, Priya Ch 
> wrote:
>
>> Hi All,
>>
>>  I am building Real-time Anomaly detection system where I am using
>> k-means to detect anomaly. Now in-order to send alert to mobile or an email
>> alert how do i send it using Spark itself ?
>>
>> Thanks,
>> Padma CH
>>
>
>
> Want to work at Handy? Check out our culture deck and open roles
> 
> Latest news  at Handy
> Handy just raised $50m
> 
>  led
> by Fidelity
>
>

Re: ml and mllib persistence

[aka.fe2s]

Okay, I think I found an answer on my question. Some models (for instance
org.apache.spark.mllib.recommendation.MatrixFactorizationModel) hold RDDs,
so just serializing these objects will not work.

--
Oleksiy Dyagilev

On Tue, Jul 12, 2016 at 5:40 PM, aka.fe2s  wrote:

> What is the reason Spark has an individual implementations of read/write
> routines for every model in mllib and ml? (Saveable and MLWritable trait
> impls)
>
> Wouldn't a generic implementation via Java serialization mechanism work? I
> would like to use it to store the models to a custom storage.
>
> --
> Oleksiy
>

Re: ml and mllib persistence

[Reynold Xin]

Also Java serialization isn't great for cross platform compatibility.

On Tuesday, July 12, 2016, aka.fe2s  wrote:

> Okay, I think I found an answer on my question. Some models (for instance
> org.apache.spark.mllib.recommendation.MatrixFactorizationModel) hold RDDs,
> so just serializing these objects will not work.
>
> --
> Oleksiy Dyagilev
>
> On Tue, Jul 12, 2016 at 5:40 PM, aka.fe2s  > wrote:
>
>> What is the reason Spark has an individual implementations of read/write
>> routines for every model in mllib and ml? (Saveable and MLWritable trait
>> impls)
>>
>> Wouldn't a generic implementation via Java serialization mechanism work?
>> I would like to use it to store the models to a custom storage.
>>
>> --
>> Oleksiy
>>
>
>

Re: ml and mllib persistence

[Salih Gedik]

Hi Reynold,

I was wondering if you meant cross language or cross platform? Thanks
On 12.07.2016 19:57, Reynold Xin wrote:

Also Java serialization isn't great for cross platform compatibility.

On Tuesday, July 12, 2016, aka.fe2s > wrote:


Okay, I think I found an answer on my question. Some models (for
instance
org.apache.spark.mllib.recommendation.MatrixFactorizationModel)
hold RDDs, so just serializing these objects will not work.

--
Oleksiy Dyagilev

On Tue, Jul 12, 2016 at 5:40 PM, aka.fe2s > wrote:

What is the reason Spark has an individual implementations of
read/write routines for every model in mllib and ml? (Saveable
and MLWritable trait impls)

Wouldn't a generic implementation via Java serialization
mechanism work? I would like to use it to store the models to
a custom storage.

--
Oleksiy




--
Salih Gedik

Re: ml and mllib persistence

[Reynold Xin]

Platform as a general word, eg language platforms, OS, different JVM
versions, different JVM vendors, different Spark versions...

On Tuesday, July 12, 2016, Salih Gedik  wrote:

> Hi Reynold,
>
> I was wondering if you meant cross language or cross platform? Thanks
> On 12.07.2016 19:57, Reynold Xin wrote:
>
> Also Java serialization isn't great for cross platform compatibility.
>
> On Tuesday, July 12, 2016, aka.fe2s  > wrote:
>
>> Okay, I think I found an answer on my question. Some models (for instance
>> org.apache.spark.mllib.recommendation.MatrixFactorizationModel) hold RDDs,
>> so just serializing these objects will not work.
>>
>> --
>> Oleksiy Dyagilev
>>
>> On Tue, Jul 12, 2016 at 5:40 PM, aka.fe2s  wrote:
>>
>>> What is the reason Spark has an individual implementations of read/write
>>> routines for every model in mllib and ml? (Saveable and MLWritable trait
>>> impls)
>>>
>>> Wouldn't a generic implementation via Java serialization mechanism work?
>>> I would like to use it to store the models to a custom storage.
>>>
>>> --
>>> Oleksiy
>>>
>>
>>
> --
> Salih Gedik
>
>

Re: ml and mllib persistence

[Salih Gedik]

I see. I could not find any explanation about this. Could you tell me 
what sort of portability issue is this? Isn't JVM supposed to give the 
abstraction of that?


Thanks!


On 12.07.2016 20:04, Reynold Xin wrote:
Platform as a general word, eg language platforms, OS, different JVM 
versions, different JVM vendors, different Spark versions...


On Tuesday, July 12, 2016, Salih Gedik > wrote:


Hi Reynold,

I was wondering if you meant cross language or cross platform? Thanks
On 12.07.2016 19:57, Reynold Xin wrote:

Also Java serialization isn't great for cross platform
compatibility.

On Tuesday, July 12, 2016, aka.fe2s > wrote:

Okay, I think I found an answer on my question. Some models
(for instance
org.apache.spark.mllib.recommendation.MatrixFactorizationModel)
hold RDDs, so just serializing these objects will not work.

--
Oleksiy Dyagilev

On Tue, Jul 12, 2016 at 5:40 PM, aka.fe2s
 wrote:

What is the reason Spark has an individual
implementations of read/write routines for every model in
mllib and ml? (Saveable and MLWritable trait impls)

Wouldn't a generic implementation via Java serialization
mechanism work? I would like to use it to store the
models to a custom storage.

--
Oleksiy




-- 
Salih Gedik




--
Salih Gedik

Why isnt spark-yarn module is excluded from the spark parent pom?

[Niranda Perera]

Hi guys,

I could not find the spark-yarn module in the spark parent pom? Is there
any particular reason why this has been excluded?

Best
-- 
Niranda
@n1r44 
+94-71-554-8430
https://pythagoreanscript.wordpress.com/