Re: [jira] [Created] (MAHOUT-1701) Mahout DSL for Flink: implement AtB ABt and AtA operators

[Suneel Marthi]

Maybe it makes sense to first sync up and get a sense for where Mahout is
and what's in the works before we starting out on Flink-Mahout integration.
How does 1pm Eastern Time on Friday work for all?

On Thu, Apr 30, 2015 at 2:48 PM, Alexey Grigorev <
alexey.s.grigor...@gmail.com> wrote:

> Is the code in mahout master up-to-date or there's some other remote I need
> to synchronize with?
>
> On 30 April 2015 at 20:42, Dmitriy Lyubimov  wrote:
>
> > keep in mind the spark code for these things is fairly drastically
> > rewritten for upcoming 0.10.1
> >
> > On Thu, Apr 30, 2015 at 7:45 AM, Alexey Grigorev (JIRA)  >
> > wrote:
> >
> > > Alexey Grigorev created MAHOUT-1701:
> > > ---
> > >
> > >  Summary: Mahout DSL for Flink: implement AtB ABt and AtA
> > > operators
> > >  Key: MAHOUT-1701
> > >  URL:
> https://issues.apache.org/jira/browse/MAHOUT-1701
> > >  Project: Mahout
> > >   Issue Type: Task
> > > Affects Versions: 0.11.0
> > > Reporter: Alexey Grigorev
> > > Priority: Minor
> > >
> > >
> > > as a part of MAHOUT-1570 implement the following operators on Flink:
> > >
> > > - AtB
> > > - ABt
> > > - AtA
> > >
> > >
> > >
> > >
> > >
> > > --
> > > This message was sent by Atlassian JIRA
> > > (v6.3.4#6332)
> > >
> >
>

Re: [jira] [Created] (MAHOUT-1701) Mahout DSL for Flink: implement AtB ABt and AtA operators

[Alexey Grigorev]

Is the code in mahout master up-to-date or there's some other remote I need
to synchronize with?

On 30 April 2015 at 20:42, Dmitriy Lyubimov  wrote:

> keep in mind the spark code for these things is fairly drastically
> rewritten for upcoming 0.10.1
>
> On Thu, Apr 30, 2015 at 7:45 AM, Alexey Grigorev (JIRA) 
> wrote:
>
> > Alexey Grigorev created MAHOUT-1701:
> > ---
> >
> >  Summary: Mahout DSL for Flink: implement AtB ABt and AtA
> > operators
> >  Key: MAHOUT-1701
> >  URL: https://issues.apache.org/jira/browse/MAHOUT-1701
> >  Project: Mahout
> >   Issue Type: Task
> > Affects Versions: 0.11.0
> > Reporter: Alexey Grigorev
> > Priority: Minor
> >
> >
> > as a part of MAHOUT-1570 implement the following operators on Flink:
> >
> > - AtB
> > - ABt
> > - AtA
> >
> >
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>

Re: [jira] [Created] (MAHOUT-1701) Mahout DSL for Flink: implement AtB ABt and AtA operators

[Dmitriy Lyubimov]

keep in mind the spark code for these things is fairly drastically
rewritten for upcoming 0.10.1

On Thu, Apr 30, 2015 at 7:45 AM, Alexey Grigorev (JIRA) 
wrote:

> Alexey Grigorev created MAHOUT-1701:
> ---
>
>  Summary: Mahout DSL for Flink: implement AtB ABt and AtA
> operators
>  Key: MAHOUT-1701
>  URL: https://issues.apache.org/jira/browse/MAHOUT-1701
>  Project: Mahout
>   Issue Type: Task
> Affects Versions: 0.11.0
> Reporter: Alexey Grigorev
> Priority: Minor
>
>
> as a part of MAHOUT-1570 implement the following operators on Flink:
>
> - AtB
> - ABt
> - AtA
>
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

Jenkins build is back to normal : Mahout-Examples-Cluster-Reuters-II #1174

[Apache Jenkins Server]

See 

Re: Brand new Bigtop AWS account

[Andrew Musselman]

BigToppers, over on Mahout we have some AWS credits too; could you share
how you've configured access for team members?

We were thinking smoke tests to start with, spinning up EMR and then
shutting down.

Thanks!

On Wed, Apr 8, 2015 at 10:52 PM, Konstantin Boudnik  wrote:

> Cycling back: today we got a credit coupon from Amazon EMR team so we have
> everything to start building new CI infra. Please contact me directly if
> you
> plan to help with setting it up, so I can create an account with proper
> permissions for you.
>
> Special thanks are going to Tom Zeng and his team for making it happen!
> Thank
> you very much guys!
>
> Cos
>
> On Tue, Mar 24, 2015 at 06:06AM, Konstantin Boudnik wrote:
> > Guys,
> >
> > I want to start a separate thread to track the CI preparations for the
> release
> > next month (fingers crossed). Clearly, we can make a release without CI,
> but
> > it'd way easier to test and create binary artifacts if we have a working
> > environment for official validation. Roman has done a lot in this
> direction
> > (many thanks!), but there are still a few rough edges, which might be
> easy to
> > finish of.
> >
> > I want to figure out a couple of things:
> >  - what's the state of CI and how much still needs to be done (Rvs?
> Could you
> >share any first hand feedback?)
> >  - who would be able to help with the CI completion? I can commit some
> of my
> >cycles, but it'd be great to have few more hands on that. Clearly,
> some
> >Jenkins-foo and prior CI skills won't hurt ;)
> >
> > Please chime in if you can help. Thanks a lot!
> >   Cos
> >
>

[jira] [Commented] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL

[Suneel Marthi (JIRA)]

[ 
https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521614#comment-14521614
 ] 

Suneel Marthi commented on MAHOUT-1570:
---

Welcome Alexy, great to see this happening, was told that u were already on 
this. 

Sent from my iPhone



> Adding support for Apache Flink as a backend for the Mahout DSL
> ---
>
> Key: MAHOUT-1570
> URL: https://issues.apache.org/jira/browse/MAHOUT-1570
> Project: Mahout
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Sebastian Schelter
>  Labels: DSL, flink, scala
>
> With the finalized abstraction of the Mahout DSL plans from the backend 
> operations (MAHOUT-1529), it should be possible to integrate further backends 
> for the Mahout DSL. Apache Flink would be a suitable candidate to act as a 
> good execution backend. 
> With respect to the implementation, the biggest difference between Spark and 
> Flink at the moment is probably the incremental rollout of plans, which is 
> triggered by Spark's actions and which is not supported by Flink yet. 
> However, the Flink community is working on this issue. For the moment, it 
> should be possible to circumvent this problem by writing intermediate results 
> required by an action to HDFS and reading from there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAHOUT-1703) Mahout DSL for Flink: implement cbind and rbind

[Alexey Grigorev (JIRA)]

Alexey Grigorev created MAHOUT-1703:
---

 Summary: Mahout DSL for Flink: implement cbind and rbind
 Key: MAHOUT-1703
 URL: https://issues.apache.org/jira/browse/MAHOUT-1703
 Project: Mahout
  Issue Type: Task
  Components: Math
Affects Versions: 0.11.0
Reporter: Alexey Grigorev
Priority: Minor


as a part of MAHOUT-1570 implement the following operators on Flink:

- cbind
- rbind



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAHOUT-1702) Mahout DSL for Flink: implement element-wise oparators

[Alexey Grigorev (JIRA)]

Alexey Grigorev created MAHOUT-1702:
---

 Summary: Mahout DSL for Flink: implement element-wise oparators
 Key: MAHOUT-1702
 URL: https://issues.apache.org/jira/browse/MAHOUT-1702
 Project: Mahout
  Issue Type: Task
  Components: Math
Affects Versions: 0.11.0
Reporter: Alexey Grigorev
Priority: Minor


as a part of MAHOUT-1570 implement the following operators on Flink:

- AewB
- AewScalar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MAHOUT-1701) Mahout DSL for Flink: implement AtB ABt and AtA operators

[Alexey Grigorev (JIRA)]

Alexey Grigorev created MAHOUT-1701:
---

 Summary: Mahout DSL for Flink: implement AtB ABt and AtA operators
 Key: MAHOUT-1701
 URL: https://issues.apache.org/jira/browse/MAHOUT-1701
 Project: Mahout
  Issue Type: Task
Affects Versions: 0.11.0
Reporter: Alexey Grigorev
Priority: Minor


as a part of MAHOUT-1570 implement the following operators on Flink:

- AtB
- ABt
- AtA 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL

[Sebastian Schelter (JIRA)]

[ 
https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521581#comment-14521581
 ] 

Sebastian Schelter commented on MAHOUT-1570:


great to see this finally happening

> Adding support for Apache Flink as a backend for the Mahout DSL
> ---
>
> Key: MAHOUT-1570
> URL: https://issues.apache.org/jira/browse/MAHOUT-1570
> Project: Mahout
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Sebastian Schelter
>  Labels: DSL, flink, scala
>
> With the finalized abstraction of the Mahout DSL plans from the backend 
> operations (MAHOUT-1529), it should be possible to integrate further backends 
> for the Mahout DSL. Apache Flink would be a suitable candidate to act as a 
> good execution backend. 
> With respect to the implementation, the biggest difference between Spark and 
> Flink at the moment is probably the incremental rollout of plans, which is 
> triggered by Spark's actions and which is not supported by Flink yet. 
> However, the Flink community is working on this issue. For the moment, it 
> should be possible to circumvent this problem by writing intermediate results 
> required by an action to HDFS and reading from there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL

[Alexey Grigorev (JIRA)]

[ 
https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521579#comment-14521579
 ] 

Alexey Grigorev commented on MAHOUT-1570:
-

Hey all, I'm the master student that TU Berlin hired to take care of this 
implementation. 
I've already started working on this, my fork is at 
https://github.com/alexeygrigorev/mahout 

> Adding support for Apache Flink as a backend for the Mahout DSL
> ---
>
> Key: MAHOUT-1570
> URL: https://issues.apache.org/jira/browse/MAHOUT-1570
> Project: Mahout
>  Issue Type: Improvement
>Reporter: Till Rohrmann
>Assignee: Sebastian Schelter
>  Labels: DSL, flink, scala
>
> With the finalized abstraction of the Mahout DSL plans from the backend 
> operations (MAHOUT-1529), it should be possible to integrate further backends 
> for the Mahout DSL. Apache Flink would be a suitable candidate to act as a 
> good execution backend. 
> With respect to the implementation, the biggest difference between Spark and 
> Flink at the moment is probably the incremental rollout of plans, which is 
> triggered by Spark's actions and which is not supported by Flink yet. 
> However, the Flink community is working on this issue. For the moment, it 
> should be possible to circumvent this problem by writing intermediate results 
> required by an action to HDFS and reading from there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)