[jira] [Commented] (BEAM-3147) Nexmark in SQL

2018-07-20 Thread Pablo Estrada (JIRA) 


[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550994#comment-16550994
 ] 

Pablo Estrada commented on BEAM-3147:
-

This is fixed, right?

> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>Priority: Major
>  Labels: newbie,, nexmark, starter
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the same set of queries running on top of Beam SQL.
> References:
> * (Nexmark Paper)[http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf]
> * (Nexmark Queries)[http://datalab.cs.pdx.edu/niagara/NEXMark/]
> * (Beam Java Nexmark 
> Suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-3147) Nexmark in SQL

2017-11-17 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257540#comment-16257540
 ] 

ASF GitHub Bot commented on BEAM-3147:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/4087


> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>  Labels: newbie,, nexmark, starter
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the same set of queries running on top of Beam SQL.
> References:
> * (Nexmark Paper)[http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf]
> * (Nexmark Queries)[http://datalab.cs.pdx.edu/niagara/NEXMark/]
> * (Beam Java Nexmark 
> Suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-3147) Nexmark in SQL

2017-11-09 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246870#comment-16246870
 ] 

ASF GitHub Bot commented on BEAM-3147:
--

GitHub user akedin opened a pull request:

https://github.com/apache/beam/pull/4110

[BEAM-3147] Nexmark launcher refactor

Refactoring Nexmark launcher. Goal is to make it more modular and 
extensible to simplify [SQL 
integration](https://issues.apache.org/jira/browse/BEAM-3147). This PR mostly 
takes pieces out of NexmarkLauncher, like sinks or event sources, into their 
own PTransforms.

This version runs as expected in batch mode on direct runner. Working on 
testing other modes.

Currently this PR also includes commits from the [previous 
PR](https://github.com/apache/beam/pull/4087).


Follow this checklist to help us incorporate your contribution quickly and 
easily:

 - [ ] Make sure there is a [JIRA 
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the 
change (usually before you start working on it).  Trivial changes like typos do 
not require a JIRA issue.  Your pull request should address just this issue, 
without pulling in other changes.
 - [ ] Each commit in the pull request should have a meaningful subject 
line and body.
 - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue.
 - [ ] Write a pull request description that is detailed enough to 
understand what the pull request does, how, and why.
 - [ ] Run `mvn clean verify` to make sure basic checks pass. A more 
thorough check will be performed on your pull request automatically.
 - [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/akedin/beam nexmark-launcher-refactor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/4110.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4110


commit 2b2afd454f584d7995ef87796ae2a37b3431739c
Author: Anton Kedin 
Date:   2017-11-06T22:47:38Z

[Nexmark] Extract GeneratorCheckpoint into a separate class. Move 
getNextEvent() call to the top of the stack.

commit 8792a19e3bc2bf88ce9e23fab49199f74a3c00d7
Author: Anton Kedin 
Date:   2017-11-06T23:01:58Z

[Nexmark] Extract PersonGenerator, StringsGenerator, LongGenerator from 
Generator

commit 7aca5453281b27c292334575e6b9e6527c9c2add
Author: Anton Kedin 
Date:   2017-11-06T23:12:53Z

[Nexmark] Extract AuctionGenerator, PriceGenerator from Generator

commit 2a5c7390b25027947e5bd618026a06d542e53f21
Author: Anton Kedin 
Date:   2017-11-06T23:19:39Z

[Nexmark] Extract BidGenerator from Generator

commit c483f1fdf24d712878ab0e49f06299df625c1527
Author: Anton Kedin 
Date:   2017-11-07T04:39:10Z

fixup! [Nexmark] Extract BidGenerator from Generator

commit 7b25d2ed5e21ae4135cf0615c8616a57eddfa6f6
Author: Anton Kedin 
Date:   2017-11-07T21:44:58Z

[BEAM-3147][NexmarkLauncher] Extract EventSourceFactory and Event sources 
out from NexmarkLauncher

Refactoring to make the launcher smaller.

commit 39cd3170fb5dbd88b4f5bf9066605a63e7efe341
Author: Anton Kedin 
Date:   2017-11-08T18:58:16Z

[BEAM-3147][NexmarkLauncher] Move results Sinks out from NexmarkLauncher

Refactoring to make NexmarkLauncher smaller

commit 61bda79718df3563f7a5e15e58103a534e6dd370
Author: Anton Kedin 
Date:   2017-11-08T21:11:20Z

[BEAM-3147][NexmarkLauncher] Extract NexmarkPerfAnalyzer out of 
NexmarkLauncher

Refactor to make NexmarkLauncher smaller

commit db89bdaca5ec9af98acc9f27feffc2cb3243db8c
Author: Anton Kedin 
Date:   2017-11-08T21:30:25Z

[BEAM-3147][NexmarkLauncher] Move NexmarkQuery creation out of 
NexmarkLauncher

Refactor to make NexmarkLauncher smaller




> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>  Labels: newbie,, nexmark, starter
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the

[jira] [Commented] (BEAM-3147) Nexmark in SQL

2017-11-08 Thread Anton Kedin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16244791#comment-16244791
 ] 

Anton Kedin commented on BEAM-3147:
---

Ah, the other related jiras are under 'extensions' component. Will look for 
them there.

> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>  Labels: newbie,, nexmark, starter
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the same set of queries running on top of Beam SQL.
> References:
> * (Nexmark Paper)[http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf]
> * (Nexmark Queries)[http://datalab.cs.pdx.edu/niagara/NEXMark/]
> * (Beam Java Nexmark 
> Suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-3147) Nexmark in SQL

2017-11-07 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241989#comment-16241989
 ] 

Ismaël Mejía commented on BEAM-3147:


Hi, there was already a JIRA for this one, but since you are already working on 
this I am going to supercede it with this one.

> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the same set of queries running on top of Beam SQL.
> References:
> * (Nexmark Paper)[http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf]
> * (Nexmark Queries)[http://datalab.cs.pdx.edu/niagara/NEXMark/]
> * (Beam Java Nexmark 
> Suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-3147) Nexmark in SQL

2017-11-06 Thread Anton Kedin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241256#comment-16241256
 ] 

Anton Kedin commented on BEAM-3147:
---

Generator refactor: https://github.com/apache/beam/pull/4087

> Nexmark in SQL
> --
>
> Key: BEAM-3147
> URL: https://issues.apache.org/jira/browse/BEAM-3147
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>
> Currently there is a (Nexmark 
> suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] running 
> against Java SDK. It has Java object model and runs a set of PTransofrms 
> replicating the queries specified in Nexmark.
> The task is to have the same set of queries running on top of Beam SQL.
> References:
> * (Nexmark Paper)[http://datalab.cs.pdx.edu/niagara/pstream/nexmark.pdf]
> * (Nexmark Queries)[http://datalab.cs.pdx.edu/niagara/NEXMark/]
> * (Beam Java Nexmark 
> Suite)[https://github.com/apache/beam/tree/master/sdks/java/nexmark] 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)