[GitHub] incubator-gearpump pull request #228: [GEARPUMP-351] Support executor number...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/228 ---
[GitHub] incubator-gearpump pull request #228: [GEARPUMP-351] Support executor number...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/228 [GEARPUMP-351] Support executor number in submit When `ClientContext` submit application to cluster and must add a argument executors to specifies the executor number use to running job , otherwise it will running in a JVM process. We should support executor number in program API . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-351 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/228.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #228 commit a74960085a55086916a166db84812b96ea1d7289 Author: darion.yaphet Date: 2017-09-29T14:57:04Z [GEARPUMP-351] Support executor number in submit ---
[GitHub] incubator-gearpump issue #200: [GEARPUMP-330] Allow examples to run in `sbt ...
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/200 maybe not only *gearpump-core* and *gearpump-streaming* should change to **compile** . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #199: [GEARPUMP-328] FetchThread fetchSleepM...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/199 [GEARPUMP-328] FetchThread fetchSleepMS from Int to Long currently *KafkaConfig.FETCH_SLEEP_MS_CONFIG* is a INT parameter , move it into a LONG parameter is better . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-328 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/199.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #199 commit 8ec6c65cae6f817aec89ad49bcc7a814633a16ba Author: darionyaphet Date: 2017-07-24T05:17:26Z [GEARPUMP-328] FetchThread fetchSleepMS from Int to Long --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #199: [GEARPUMP-328] FetchThread fetchSleepMS from ...
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/199 cc @manuzhang Could you make a review when you have time. thanks ~~ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #191: [GEARPUMP-319] Support Sorted Set in Redis
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/191 @manuzhang fixed the silly bug --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #191: [GEARPUMP-319] Support Sorted Set in R...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/191 [GEARPUMP-319] Support Sorted Set in Redis Sorted Set is similar with Set in Redis with unique values . Each element storage in Sorted Set is associated with a score that is use to sort the element in order . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-319 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/191.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #191 commit 77d12e97843deae16c2598b5e23bcd4ea0968fed Author: darionyaphet Date: 2017-06-22T08:05:39Z [GEARPUMP-319] Support Sorted Set in Redis --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #125: [GEARPUMP-256] Camel-Case as variable name
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/125 @manuzhang I have repush this one thanks :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #125: [GEARPUMP-256] Camel-Case as variable ...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/125 [GEARPUMP-256] Camel-Case as variable name [Camel-Case as variable name](https://issues.apache.org/jira/browse/GEARPUMP-256) HBaseSink's naming and using Camel-Case as the variable name. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the commit message is formatted like: `[GEARPUMP-] Meaningful description of pull request` - [ ] Make sure tests pass via `sbt clean test`. - [ ] Make sure old documentation affected by the pull request has been updated and new documentation added for new functionality. You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-256 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/125.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #125 commit 26520eff34d00e7a77e0a9ed5e985470947811f9 Author: darionyaphet Date: 2016-12-23T08:14:10Z fix : GEARPUMP-256 Camel-Case as variable name --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #102: [GEARPUMP-228] Message's timestamp should be ...
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/102 thanks @huafengw --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #102: [GEARPUMP-228] Message's timestamp sho...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/102 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #102: [GEARPUMP-228] Message's timestamp should be ...
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/102 @manuzhang using 0L as default could support any useful info , currentTimeMillis will explain when the message be created . what do you think ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #102: [GEARPUMP-228] Message's timestamp sho...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/102 [GEARPUMP-228] Message's timestamp should be current time stamp [GEARPUMP-228 Message's timestamp should be current time stamp](https://issues.apache.org/jira/browse/GEARPUMP-228) Using System.currentTimeMillis() as Message's default timestamp maybe better than 0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-228 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/102.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #102 commit 104c310698660948180ece5b18b5b895331d385b Author: darionyaphet Date: 2016-10-29T12:23:22Z GEARPUMP-228 : Message's timestamp should be current time stamp --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #93: GEARPUMP-215 Gearpump Redis Integration...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/93 GEARPUMP-215 Gearpump Redis Integration - RedisStorage [Gearpump Redis Integration - RedisStorage](https://issues.apache.org/jira/browse/GEARPUMP-215) You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-215 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/93.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #93 commit eee43204480e6b4d539ae0053057ad60a9051b9e Author: darionyaphet Date: 2016-09-25T11:05:42Z GEARPUMP-215 Gearpump Redis Integration - RedisStorage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #89: GEARPUMP-216 Support Delete and Increme...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/89 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #89: GEARPUMP-216 Support Delete and Increme...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/89 GEARPUMP-216 Support Delete and Increment in HBaseSink [GEARPUMP-216](https://issues.apache.org/jira/browse/GEARPUMP-216) Support Delete and Increment in HBase Sink. You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-216 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/89.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #89 commit 591fdf8db5c32a3b118eff6f89677f7043c89fc2 Author: darionyaphet Date: 2016-09-25T13:26:25Z GEARPUMP-216 Support Delete and Increment in HBaseSink --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #56: [GEARPUMP-172] Add timed interval rotat...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/56 [GEARPUMP-172] Add timed interval rotation on HDFS module [GEARPUMP-172](https://issues.apache.org/jira/browse/GEARPUMP-172) The time interval rotation will rolling out a new file when time interval get the limit You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-172 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/56.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #56 commit e7276b5f3e016ce9b0af3d7ace8c3b26471256bb Author: darionyaphet Date: 2016-07-01T14:30:53Z fix GEARPUMP-172 Add timed interval rotation on HDFS module --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/11 @kkasravi @manuzhang `RedisStorage` maybe need more discuss so I have remove it . After we have a discuss , I will repush it :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #55: [GEARPUMP-171] Fix some typos
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/55 [GEARPUMP-171] Fix some typos [GEARPUMP-171](https://issues.apache.org/jira/browse/GEARPUMP-171) Fix some typos You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump typos Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/55.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #55 commit a810fe8cbbef157c6c93ab34dccb131f4a335466 Author: wangcg Date: 2016-07-01T13:32:11Z fix GEARPUMP-171 Fix some typos --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #52: [GEARPUMP-152] Upgrade Storm support to 1.0.x
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/52 @manuzhang I found storm 1.X have add some function about Blob and LogConfig . I'm miss it and working . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #52: [GEARPUMP-152] Upgrade Storm support to 1.0.x
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/52 @manuzhang when I try to implement Nimbus method , I found some of them not implement will throw a UnsupportedOperationException . I want to know which command we are not support . thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #52: [GEARPUMP-152] Upgrade Storm support to...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/52 [GEARPUMP-152] Upgrade Storm support to 1.0.x [GEARPUMP-152](https://issues.apache.org/jira/browse/GEARPUMP-152) Storm 1.0.0 has been released with a lot of changes. We need to upgrade our support for Storm to 1.0.x You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-152 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/52.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #52 commit cb911869f77e21f2518390b79fa09050bd7e018e Author: darionyaphet Date: 2016-06-28T08:36:19Z fix GEARPUMP-152 upgrade Storm support to 1.0.x --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #51: [GEARPUMP-166] Rename sumByValue to sum...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/51 [GEARPUMP-166] Rename sumByValue to sumByKey [GEARPUMP-166](https://issues.apache.org/jira/browse/GEARPUMP-166) Streaming org.apache.gearpump.streaming.dsl.Stream have a function named sumByValue . It rename to sumByKey maybe better . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-166 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/51.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #51 commit a4dc0c0f3395aff3ecf952d70373c75ad6710c2e Author: darionyaphet Date: 2016-06-26T10:47:13Z fix #GEARPUMP-166 Rename sumByValue to sumByKey --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/11 @manuzhang `HBaseSink` seems have the same question . `Sink` and `Source` are support to the data streaming , `Storage` is meaning to put data into a place . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/11 @karol-brejna-i It's seems have a lot things to do . 1. Support more command such as : `time to live` , `exist` and `delete` . 2. Support Redis Cluster and Redis Sharding . 3. Add some Test Case --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on the issue: https://github.com/apache/incubator-gearpump/pull/11 @manuzhang actually storm-redis act as a ORM framework and put the message into a redis instance . So it don't have a full command support . I think we can support more command such as delete , ttl and exists . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r65504318
--- Diff:
experiments/redis/src/main/scala/org/apache/gearpump/streaming/redis/RedisMessage.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gearpump.streaming.redis
+
+import java.nio.charset.Charset
+
+object RedisMessage {
+
+ private def toBytes(string: String,
+ charset: Charset = Charset.forName("UTF8")
+ ): Array[Byte] = string.getBytes(charset)
+
+ case class PublishMessage(message: Array[Byte]) {
+def this(message: String) = this(toBytes(message))
+ }
+
+ case class SetMessage(key: Array[Byte], value: Array[Byte]) {
+def this(key: String, value: String) = this(toBytes(key),
toBytes(value))
+ }
+
+ case class LPushMessage(key: Array[Byte], value: Array[Byte]) {
--- End diff --
`LPushMessage` and `RPushMessage ` are difference message to control redis
client add element at list's head or tail
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #11: fix #106 Gearpump Redis Integration
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r65502913
--- Diff:
experiments/redis/src/main/scala/org/apache/gearpump/streaming/redis/RedisMessage.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gearpump.streaming.redis
+
+import java.nio.charset.Charset
+
+object RedisMessage {
+
+ private def toBytes(string: String,
+ charset: Charset = Charset.forName("UTF8")
+ ): Array[Byte] = string.getBytes(charset)
+
+ case class PublishMessage(message: Array[Byte]) {
--- End diff --
The messages are implemented in `storm-redis` I try to keep they are the
same .
BTW I will add some message both in gearpump and storm .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 Remove useless m...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/20#issuecomment-222503486 @huafengw @manuzhang Could add some comment about this functions ? This functions is very confuse to others . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 Remove useless m...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/20 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r63819150
--- Diff: project/BuildExample.scala ---
@@ -229,4 +229,20 @@ object BuildExample extends sbt.Build {
CrossVersion.binaryScalaVersion(scalaVersion.value)
)
) dependsOn (streaming % "test->test; provided")
+
+ lazy val examples_redis = Project(
+id = "gearpump-examples-redis",
+base = file("examples/streaming/redis"),
+settings = commonSettings ++ noPublish ++ myAssemblySettings ++
+ Seq(
+mainClass in (Compile, packageBin) :=
Some("org.apache.gearpump.streaming.example.redis.RedisSourceSinkExample"),
+target in assembly :=
baseDirectory.value.getParentFile.getParentFile / "target" /
+ CrossVersion.binaryScalaVersion(scalaVersion.value)
+ ) ++
+ Seq(
+mainClass in (Compile, packageBin) :=
Some("org.apache.gearpump.streaming.example.redis.RedisSourceStorageExample"),
--- End diff --
HI @huafengw when I package using SBT , the main class will specifies into
META-INF ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 Remove useless m...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/20#issuecomment-219914766 @manuzhang but the method was decorate with `private` . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 , remove useless...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/19 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 Remove useless m...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/20#issuecomment-219773664 conflicts ?? I can't found any conflicts ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 , remove useless...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/20 fix GEARPUMP-141 , remove useless method [GEARPUMP-141](https://issues.apache.org/jira/browse/GEARPUMP-141) In external-hbase module , writeObject and readObject seems never used . We can remove this methods . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-141 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/20.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20 commit 001ee4ae6074c0bc8e5123f39cb30eb4c431838f Author: darionyaphet Date: 2016-05-17T16:13:17Z fix GEARPUMP-141 , remove useless method --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-141 , remove useless...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/19 fix GEARPUMP-141 , remove useless method [GEARPUMP-141](https://issues.apache.org/jira/browse/GEARPUMP-141) In external-hbase module , writeObject and readObject seems never used . We can remove this methods . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-141 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/19.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #19 commit 11a16edfa8c746f8be6683a277a3b1c1cf56de3e Author: darionyaphet Date: 2016-05-17T16:07:08Z fix GEARPUMP-141 , remove useless method --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/11#issuecomment-219225271 # Sorry to reply it with so long . @manuzhang @huafengw redis examples class name have rename to `RedisSourceSinkExample` and `RedisSourceStorageExample` , also add into `BuildExample.scala` . It seems better ? `external-redis` have move into `experiments` I will add unit test later :) @kkasravi I will fix and add some comment at weekend :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r63278015
--- Diff:
examples/streaming/redis/src/main/scala/org/apache/gearpump/streaming/example/redis/RedisSourceSink.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gearpump.streaming.example.redis
+
+import io.gearpump.Message
+import io.gearpump.cluster.UserConfig
+import io.gearpump.cluster.embedded.EmbeddedCluster
+import io.gearpump.cluster.main.ArgumentsParser
+import org.apache.gearpump.streaming.redis.{RedisSink, RedisSource,
RedisMessage}
+import io.gearpump.streaming.sink.DataSinkProcessor
+import io.gearpump.streaming.source.DataSourceProcessor
+import io.gearpump.streaming.task.{Task, TaskContext}
+import io.gearpump.streaming.{Processor, StreamApplication}
+import io.gearpump.util.Graph._
+import io.gearpump.util.{AkkaApp, Graph}
+
+class RedisSourceSinkUpperProcessor(taskContext: TaskContext, conf:
UserConfig)
+ extends Task(taskContext, conf) {
+
+ import taskContext.output
+
+ override def onNext(message: Message): Unit = {
+val msg = message.msg.asInstanceOf[Option[String]]
--- End diff --
Good ! I will update this :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r62852676
--- Diff:
examples/streaming/redis/src/main/scala/org/apache/gearpump/streaming/example/redis/RedisSourceSink.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gearpump.streaming.example.redis
+
+import io.gearpump.Message
+import io.gearpump.cluster.UserConfig
+import io.gearpump.cluster.embedded.EmbeddedCluster
+import io.gearpump.cluster.main.ArgumentsParser
+import org.apache.gearpump.streaming.redis.{RedisSink, RedisSource,
RedisMessage}
+import io.gearpump.streaming.sink.DataSinkProcessor
+import io.gearpump.streaming.source.DataSourceProcessor
+import io.gearpump.streaming.task.{Task, TaskContext}
+import io.gearpump.streaming.{Processor, StreamApplication}
+import io.gearpump.util.Graph._
+import io.gearpump.util.{AkkaApp, Graph}
+
+class RedisSourceSinkUpperProcessor(taskContext: TaskContext, conf:
UserConfig)
--- End diff --
Yep this processor is change input string to upper case
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/11#discussion_r62852450
--- Diff:
examples/streaming/redis/src/main/scala/org/apache/gearpump/streaming/example/redis/RedisSourceSink.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gearpump.streaming.example.redis
+
+import io.gearpump.Message
+import io.gearpump.cluster.UserConfig
+import io.gearpump.cluster.embedded.EmbeddedCluster
+import io.gearpump.cluster.main.ArgumentsParser
+import org.apache.gearpump.streaming.redis.{RedisSink, RedisSource,
RedisMessage}
+import io.gearpump.streaming.sink.DataSinkProcessor
+import io.gearpump.streaming.source.DataSourceProcessor
+import io.gearpump.streaming.task.{Task, TaskContext}
+import io.gearpump.streaming.{Processor, StreamApplication}
+import io.gearpump.util.Graph._
+import io.gearpump.util.{AkkaApp, Graph}
+
+class RedisSourceSinkUpperProcessor(taskContext: TaskContext, conf:
UserConfig)
+ extends Task(taskContext, conf) {
+
+ import taskContext.output
+
+ override def onNext(message: Message): Unit = {
+val msg = message.msg.asInstanceOf[Option[String]]
+
+if (!msg.isEmpty) {
+ val upper = msg.get.toUpperCase
+ LOG.info("to Upper : " + upper)
+ output(new Message(new PublishMessage(upper), message.timestamp))
+}
+ }
+}
+
+object RedisSourceSink extends AkkaApp with ArgumentsParser {
--- End diff --
`RedisSourceSink ` which is mean reading from RedisSource and write
messages to RedisSink :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/11#issuecomment-218354738 Hi @manuzhang what is the commit format ? I found some rules on `Contribution Guideline` `For all commit log messages, they must contain issue id. Like this: "fix #issueId, comments".` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/11#issuecomment-218355383 Hi @huafengw sure I will rename packages from io.gearpump to org.apache.gearpump :D --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-118 change package n...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/14#issuecomment-218150639 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: GEARPUMP-24, refactor DataSource ...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/7#issuecomment-217859410 Hi @manuzhang why change datasource API from `List[Message]` to `Message` ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #121 DataSourceProcessor and ...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/13#issuecomment-217900186 @manuzhang When I using some processors to build a DAG , it's just to describe how to process the input stream , parallelism is the attribute of this processor . It should have a default value and one is suitable . thank you for you reply :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
Github user darionyaphet commented on the pull request: https://github.com/apache/incubator-gearpump/pull/11#issuecomment-217726672 @manuzhang I have add license headers . @kkasravi I will add some unit tests . thank you :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #121 DataSourceProcessor and ...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/13 fix #121 DataSourceProcessor and DataSinkProcessor default parallelism Set DataSourceProcessor and DataSinkProcessor's default parallelism value to 1. When user startup a processor could don't set parallelism . [GEARPUMP-121](https://issues.apache.org/jira/browse/GEARPUMP-121) You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-121 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/13.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #13 commit c29071fb991e3b6286038dc64618c04f46a4ba04 Author: darionyaphet Date: 2016-05-07T13:00:28Z fix #121 DataSourceProcessor and DataSinkProcessor default parallelism --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #106 Gearpump Redis Integrati...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/11 fix #106 Gearpump Redis Integration [Redis](redis.io) is a hight performance in memory storage , and have widely used in a lot of project . It's should support redis as DataSource and DataSink . You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-106 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/11.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #11 commit 1916b054711ac812b3b2c1603ef70e1a060e3a63 Author: darionyaphet Date: 2016-05-07T12:13:26Z fix #106 Gearpump Redis Integration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #121 DataSourceProcessor and ...
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/12 fix #121 DataSourceProcessor and DataSinkProcessor default parallelism Set DataSourceProcessor and DataSinkProcessor's default parallelism value to 1. When user startup a processor could don't set parallelism . [GEARPUMP-121](https://issues.apache.org/jira/browse/GEARPUMP-121) You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-121 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/12.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12 commit cb3a5c88fd360af71f2ab320ed66ada2dbcdbba9 Author: yaphet Date: 2016-05-06T16:48:25Z Merge pull request #1 from apache/master merge request commit 2194ab289e40b282632028e29244da3f6615b517 Author: darionyaphet Date: 2016-05-07T12:31:35Z fix #106 DataSourceProcessor and DataSinkProcessor default parallelism --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: fix #121 DataSourceProcessor and ...
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/12 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: gearpump redis
Github user darionyaphet closed the pull request at: https://github.com/apache/incubator-gearpump/pull/10 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request: gearpump redis
GitHub user darionyaphet opened a pull request: https://github.com/apache/incubator-gearpump/pull/10 gearpump redis You can merge this pull request into a Git repository by running: $ git pull https://github.com/darionyaphet/incubator-gearpump GEARPUMP-106 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/10.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #10 commit 92b35719a921370bce2030ba81144e0ad9e1a3ec Author: darionyaphet Date: 2016-05-06T17:08:43Z gearpump redis --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
