[GitHub] incubator-gearpump issue #107: Add hbase-example

[manuzhang]

Github user manuzhang commented on the issue:

https://github.com/apache/incubator-gearpump/pull/107
  
please file a [JIRA issue](http://issues.apache.org/jira/browse/GEARPUMP) 
for this 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #107: Add hbase-example

[huafengw]

Github user huafengw commented on a diff in the pull request:

https://github.com/apache/incubator-gearpump/pull/107#discussion_r86284053
  
--- Diff: 
examples/streaming/hbase/src/main/scala/org/apache/gearpump/streaming/hbase/HBaseConn.scala
 ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.gearpump.streaming.hbase
+
+
+import akka.actor.ActorSystem
+import org.apache.gearpump.cluster.UserConfig
+import org.apache.gearpump.cluster.client.ClientContext
+import org.apache.gearpump.cluster.main.{ArgumentsParser, CLIOption, 
ParseResult}
+import org.apache.gearpump.streaming.StreamApplication
+import org.apache.gearpump.streaming.partitioner.HashPartitioner
+import org.apache.gearpump.streaming.sink.DataSinkProcessor
+import org.apache.gearpump.streaming.source.DataSourceProcessor
+import org.apache.gearpump.util.Graph.Node
+import org.apache.gearpump.util.{AkkaApp, Graph, LogUtil}
+import org.slf4j.Logger
+
+object HBaseConn extends AkkaApp with ArgumentsParser {
+  private val LOG: Logger = LogUtil.getLogger(getClass)
+  val RUN_FOR_EVER = -1
+
+  override val options: Array[(String, CLIOption[Any])] = Array(
+"splitNum" -> CLIOption[Int]("", required = false, 
defaultValue = Some(1)),
+"sinkNum" -> CLIOption[Int]("", required = false, 
defaultValue = Some(1))
+  )
+
+  def application(config: ParseResult, system: ActorSystem): 
StreamApplication = {
+implicit val actorSystem = system
+
+val splitNum = config.getInt("splitNum")
+val sinkNum = config.getInt("sinkNum")
+
+val split = new Split
+val sourceProcessor = DataSourceProcessor(split, splitNum, "Split")
+val sink = new Sink
--- End diff --

HBaseSink is already a DataSink so you don't need to implement this one. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Start of release 0.8.2-incubating

[Manu Zhang]

Thanks Kam for volunteering as the release manager. Please shout on the
list if any help is needed.

It's been quite a while since last release and we have put in a lot of bug
fixes and improvements.
Meanwhile, the binary release is required for Gearpump integration with
other projects.

For example,
SAMOA-59: add an adapter for Apache Gearpump

add gearpump benchmark for yahoo streaming benchmarks


We'd better release in shorter cycles in the future.

Now, let's roll it out !

Cheers,
Manu


On Wed, Nov 2, 2016 at 1:10 AM Kam Kasravi  wrote:

> This is notice that we will begin getting 0.8.2-incubating ready for
> release.
> I would like to volunteer to be the release manager for 0.8.2-incubating.
> This next version (0.8.2-incubating) will be a binary release,
> so it will require different LICENSE and NOTICE files.
>
> We would like to minimally include the following JIRA's (pending feedback).
> They have been
> categorized by resolved and open. Our target date is within 3 weeks from
> today to submit a RELEASE vote in dev, followed by the RELEASE vote in
> incubator. We will update the JIRA's below within this thread as feedback
> is provided.
>
> Thanks
> Kam
>
>
> Open
> GEARPUMP-22  Support akka-streams Gearpump Materializer
> GEARPUMP-112 DAG chart in dashboard does not show data, when flow is small
> GEARPUMP-144 Modify Gearpump's yarnclient so it can use provided user for
> HDFS and YARN
> GEARPUMP-190 Add logo with text to site
> GEARPUMP-191 add ability to control rate to TaskActor
> GEARPUMP-193 Need ability to request upstream messages (pull)
> GEARPUMP-198 empty kafka topics show very high receive throughput
> GEARPUMP-199 high throughput results in high GC costs
> GEARPUMP-200 A missing gear.conf in services and other gearpump commands
> should result in an ERROR to the log
> GEARPUMP-206 access to upstream and downstream processors
> GEARPUMP-211 Look at merging or unifying akka-stream DSL and Gearpump DSL
> GEARPUMP-223 Support Redis Cluster Sink
>
> Resolved
> GEARPUMP-15  Define initial Apache branded home website
> GEARPUMP-32  Minimum clock of source Tasks maybe inaccurate
> GEARPUMP-37  Sometimes application clock is "not started" when the data
> stream is very slow
> GEARPUMP-48  Improve Java interoperability
> GEARPUMP-171 Fix some typos
> GEARPUMP-185 Yarn kills Apache Gearpump's worker if the memory usage is too
> high
> GEARPUMP-188 use java.time.Instant for Task start time
> GEARPUMP-189 Update project/Build.scala to enable publishing releases to
>
> https://repository.apache.org/content/repositories/releases/org/apache/gearpump/
> GEARPUMP-197
> 
> Busy loop in FetchThread when incoming queue is full
> GEARPUMP-201 integration test failure
> GEARPUMP-203 Use DataSourceTask / DataSinkTask for DSL
> GEARPUMP-205 remove hdfs dependency from command gear's classpath
> GEARPUMP-207 remove hadoop dependency out of application path from
> dashboard
> GEARPUMP-208 Demo site goes down a lot. Also references gearpump.io when
> connection is refused
> GEARPUMP-212 Make Gearpump's shaded artifacts publishable
> GEARPUMP-215 Gearpump Redis Integration - RedisStorage
> GEARPUMP-222 DFSJarStore is loaded even when local file path is configured
> GEARPUMP-224 Merge gearpump-daemon to gearpump-core
> GEARPUMP-225 move partitioner from gearpump-core to module
> gearpump-streaming
> GEARPUMP-227 remove unneeded license files
>
>
>
>
>-
>

[GitHub] incubator-gearpump pull request #108: [GEARPUMP-226] enforce assembly shaded...

[manuzhang]

GitHub user manuzhang opened a pull request:

https://github.com/apache/incubator-gearpump/pull/108

[GEARPUMP-226] enforce assembly shaded library first



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-gearpump fix_build

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-gearpump/pull/108.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #108


commit 83507287927fa539a147f31436a24d4bc7240479
Author: manuzhang 
Date:   2016-11-02T15:36:42Z

[GEARPUMP-226] enforce assembly shaded library first




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #103: [GEARPUMP-230] Removed unused code fro...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-gearpump/pull/103


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump issue #103: [GEARPUMP-230] Removed unused code from AppMa...

[christophse]

Github user christophse commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump issue #103: [GEARPUMP-230] Removed unused code from AppMa...

[manuzhang]

Github user manuzhang commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
Looks good, merging


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump issue #103: [GEARPUMP-230] Removed unused code from AppMa...

[christophse]

Github user christophse commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
@manuzhang, is there something more to do?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump issue #107: Add hbase-example

[codecov-io]

Github user codecov-io commented on the issue:

https://github.com/apache/incubator-gearpump/pull/107
  
## [Current 
coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/107?src=pr) is 
71.69% (diff: 100%)
> Merging 
[#107](https://codecov.io/gh/apache/incubator-gearpump/pull/107?src=pr) into 
[master](https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr) 
will increase coverage by **0.03%**


```diff
@@ master   #107   diff @@
==
  Files   186186  
  Lines  5985   5985  
  Methods5456   5456  
  Messages  0  0  
  Branches529529  
==
+ Hits   4289   4291 +2   
+ Misses 1696   1694 -2   
  Partials  0  0  
```


![Sunburst](https://codecov.io/gh/apache/incubator-gearpump/pull/107/graphs/sunburst.svg?size=150&src=pr)

> Powered by [Codecov](https://codecov.io?src=pr). Last update 
[7bc5fba...384acfb](https://codecov.io/gh/apache/incubator-gearpump/compare/7bc5fba5b5c5166992e04218071a3da4931e47a9...384acfbb02b8a0def6acf81fc9e1d0ce889e2d4f?src=pr)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #107: Add hbase-example

[Roshanson]

GitHub user Roshanson opened a pull request:

https://github.com/apache/incubator-gearpump/pull/107

Add hbase-example



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Roshanson/incubator-gearpump HBase

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-gearpump/pull/107.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #107


commit d2eed3a6ebc673898e04cae2c1b3f8a4c364d8c8
Author: Roshanson <736781...@qq.com>
Date:   2016-11-02T02:01:46Z

Squashed commit of the following:

commit 68d9027656fd7470c6389cb85ee63181fe64d0f5
Author: Roshanson <736781...@qq.com>
Date:   Mon Oct 31 15:13:38 2016 +0800

Add connecting to HBase example

commit 8a46511c0ded3d5ec8198ce844246255e935b153
Merge: 02b9bac 7bc5fba
Author: Roshanson <736781...@qq.com>
Date:   Mon Oct 31 14:30:10 2016 +0800

Merge branch 'master' of https://github.com/apache/incubator-gearpump 
into InHBase

commit 02b9bac0766f40c7547343084412cf3aef930821
Author: Roshanson <736781...@qq.com>
Date:   Mon Oct 31 13:50:24 2016 +0800

Add example of connecting to HBase

commit 0e1df27b4e78797ced14690980c9066189af7afc
Author: Roshanson <736781...@qq.com>
Date:   Mon Oct 31 13:12:32 2016 +0800

Add example of connecting to HBase

commit b611ef7d2a73bd121c7b3dc38b25ff17c43ae04c
Author: Roshanson <736781...@qq.com>
Date:   Mon Oct 31 13:01:19 2016 +0800

Add example of connection to HBase

commit 6b6f2eee612ce0fe3d5cb5dd25dd1cffd4bc0519
Author: Roshanson <736781...@qq.com>
Date:   Fri Oct 21 15:55:16 2016 +0800

[GEARPUMP-204]INSERT TO HBASE

commit ac1482db04639b9e7101140a8907e988f40c8f92
Merge: 65e13bd c812097
Author: Roshanson <736781...@qq.com>
Date:   Tue Sep 27 20:26:14 2016 +0800

Merge remote-tracking branch 'origin/fix-addHBaseUT' into fix-hbasetest

# Conflicts:
#   
examples/streaming/wordcount/src/main/scala/org/apache/gearpump/streaming/examples/wordcount/Split.scala
#   
examples/streaming/wordcount/src/main/scala/org/apache/gearpump/streaming/examples/wordcount/WordCount.scala
#   
external/hbase/src/main/scala/org/apache/gearpump/external/hbase/HBaseSink.scala
#   
external/hbase/src/test/scala/org/apache/gearpump/external/hbase/HBaseSinkSpec.scala

commit c8120978b8609b41012823707ff04b830f1dfb50
Author: Roshanson <736781...@qq.com>
Date:   Tue Sep 27 19:54:50 2016 +0800

[GEARPUMP-204]Add unit test for external_hbase module

commit 501dca4861549e27beeb9a6ef89f4bb9902a3249
Author: Roshanson <736781...@qq.com>
Date:   Tue Sep 27 18:53:25 2016 +0800

[GEARPUMP-204]Add unit test for external_hbase module

commit d52f51da4ccb797c7ef954b396b88735a2bf969e
Author: Roshanson <736781...@qq.com>
Date:   Fri Sep 23 11:33:33 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 52d4b8d6ff4e1720147b9c088412625f5312f0a0
Author: Roshanson 
Date:   Fri Sep 23 10:41:37 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 1caf488e88253380c01e319d59957f36694784a3
Author: Roshanson <736781...@qq.com>
Date:   Thu Sep 22 19:41:31 2016 +0800

[GEARPUMP-204]Add unit test for external_hbase module

commit 2e6ea2c33fdacddff283ea25d93493d1ef30bb4d
Author: Roshanson 
Date:   Thu Sep 22 16:49:29 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 4920f2598721beede24e182205a87b72b9f13afb
Author: roshanson 
Date:   Thu Sep 22 16:46:08 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit f07f7feba571ba4a2ba830f8ae794b9dcd8f1e54
Author: roshanson 
Date:   Thu Sep 22 15:16:31 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit ff7b930a6d2cd8081bdf2be1f5c2d69c2d144fd8
Author: roshanson 
Date:   Thu Sep 22 14:59:19 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 3086e4535fbbaa1c6e652f8e3cc3fdeb7bf24ead
Author: roshanson 
Date:   Thu Sep 22 13:12:49 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 87364c049fd283c5da6898abf61973eae031a8a7
Author: roshanson 
Date:   Thu Sep 22 11:36:25 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module

commit 8ef7dbb0d9b816a595441e66b0f71a041fc5b9f8
Author: roshanson 
Date:   Wed Sep 21 13:57:24 2016 +0800

[GEARPUMP-204]add unit test for external_hbase module
   

[jira] [Commented] (GEARPUMP-226) run 'sbt clean assembly' failed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEARPUMP-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629331#comment-15629331
 ] 

ASF GitHub Bot commented on GEARPUMP-226:
-

GitHub user manuzhang opened a pull request:

https://github.com/apache/incubator-gearpump/pull/108

[GEARPUMP-226] enforce assembly shaded library first



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-gearpump fix_build

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-gearpump/pull/108.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #108


commit 83507287927fa539a147f31436a24d4bc7240479
Author: manuzhang 
Date:   2016-11-02T15:36:42Z

[GEARPUMP-226] enforce assembly shaded library first




> run 'sbt clean assembly' failed
> ---
>
> Key: GEARPUMP-226
> URL: https://issues.apache.org/jira/browse/GEARPUMP-226
> Project: Apache Gearpump
>  Issue Type: Bug
>Reporter: Huafeng Wang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-230) Removed unused code from AppMasterService

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEARPUMP-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628762#comment-15628762
 ] 

ASF GitHub Bot commented on GEARPUMP-230:
-

Github user christophse commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
Thanks!


> Removed unused code from AppMasterService
> -
>
> Key: GEARPUMP-230
> URL: https://issues.apache.org/jira/browse/GEARPUMP-230
> Project: Apache Gearpump
>  Issue Type: Task
>  Components: restapi
>Affects Versions: 0.8.2
>Reporter: Christoph Seifert
>Assignee: Christoph Seifert
>Priority: Trivial
> Fix For: 0.8.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (GEARPUMP-230) Removed unused code from AppMasterService

[Manu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GEARPUMP-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manu Zhang resolved GEARPUMP-230.
-
   Resolution: Fixed
Fix Version/s: 0.8.2

Issue resolved by pull request 103
[https://github.com/apache/incubator-gearpump/pull/103]

> Removed unused code from AppMasterService
> -
>
> Key: GEARPUMP-230
> URL: https://issues.apache.org/jira/browse/GEARPUMP-230
> Project: Apache Gearpump
>  Issue Type: Task
>  Components: restapi
>Affects Versions: 0.8.2
>Reporter: Christoph Seifert
>Assignee: Christoph Seifert
>Priority: Trivial
> Fix For: 0.8.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-230) Removed unused code from AppMasterService

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEARPUMP-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628740#comment-15628740
 ] 

ASF GitHub Bot commented on GEARPUMP-230:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-gearpump/pull/103


> Removed unused code from AppMasterService
> -
>
> Key: GEARPUMP-230
> URL: https://issues.apache.org/jira/browse/GEARPUMP-230
> Project: Apache Gearpump
>  Issue Type: Task
>  Components: restapi
>Affects Versions: 0.8.2
>Reporter: Christoph Seifert
>Assignee: Christoph Seifert
>Priority: Trivial
> Fix For: 0.8.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-230) Removed unused code from AppMasterService

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEARPUMP-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628738#comment-15628738
 ] 

ASF GitHub Bot commented on GEARPUMP-230:
-

Github user manuzhang commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
Looks good, merging


> Removed unused code from AppMasterService
> -
>
> Key: GEARPUMP-230
> URL: https://issues.apache.org/jira/browse/GEARPUMP-230
> Project: Apache Gearpump
>  Issue Type: Task
>  Components: restapi
>Affects Versions: 0.8.2
>Reporter: Christoph Seifert
>Assignee: Christoph Seifert
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-230) Removed unused code from AppMasterService

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEARPUMP-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628637#comment-15628637
 ] 

ASF GitHub Bot commented on GEARPUMP-230:
-

Github user christophse commented on the issue:

https://github.com/apache/incubator-gearpump/pull/103
  
@manuzhang, is there something more to do?


> Removed unused code from AppMasterService
> -
>
> Key: GEARPUMP-230
> URL: https://issues.apache.org/jira/browse/GEARPUMP-230
> Project: Apache Gearpump
>  Issue Type: Task
>  Components: restapi
>Affects Versions: 0.8.2
>Reporter: Christoph Seifert
>Assignee: Christoph Seifert
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)