[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-gearpump/pull/81 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #81: fix GEARPUMP-205 remove hdfs dependency from g...
Github user manuzhang commented on the issue: https://github.com/apache/incubator-gearpump/pull/81 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469480#comment-15469480 ] ASF GitHub Bot commented on GEARPUMP-205: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-gearpump/pull/81 > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-205. - Resolution: Fixed Fix Version/s: 0.8.2 Issue resolved by pull request 81 [https://github.com/apache/incubator-gearpump/pull/81] > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469475#comment-15469475 ] ASF GitHub Bot commented on GEARPUMP-205: - Github user manuzhang commented on the issue: https://github.com/apache/incubator-gearpump/pull/81 +1 > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Shepherd for this month
Hi, > We'll proceed on the recommendations you've made. Great good luck in your incubation journey. > Regarding [9] - Andy also recommended that we' look for additional mentors > that can play an active role. After seeing if any of the less active mentors can find some time, you might try asking on the incubator general list. My workload is currently overflowing so I can’t commit to mentoring any more projects at the moment sorry. Thanks, Justin
Re: Podling Report Reminder - September 2016
Hi Andy We'll be speaking at ReactiveSummit 2016 Oct 4,5 in Austin Texas on integration of Gearpump into akka-streams. On Tuesday, September 6, 2016 10:12 AM, Andrew Purtell wrote: Thanks Vincent, +1 from me. Anyone have anything to add or change? If not I will post it to the incubator wiki tomorrow. On Mon, Sep 5, 2016 at 3:19 AM, Vincent Wang wrote: > Hi All, > > This is the Gearpump podling report for September 2016. Please help to > review. > > Gearpump > > Gearpump is a reactive real-time streaming engine based on the > micro-service actor model. > Gearpump has been incubating since 2016-03-08. > > Three most important issues to address in the move towards graduation: > > 1. Make initial Apache branded release and release on a regular schedule. > 2. Continue to evolve community interest and support. > 3. Integrate with Apache Beam and akka-streams frameworks. > > - Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be > aware > of? > - None > > - How has the community developed since the last report? > - Increased community contribution on integration with Apache Cassandra. > > - How has the project developed since the last report? > - We released Apache Gearpump 0.8.1-incubating, which is Gearpump's first > release > as an Apache incubator project. > - Integration within Apache Beam (Gearpump Runner) has been merged into a > feature branch. > - Continued integration within akka-streams (Gearpump Materializer). Three > PRs committed. > - 28 issues created and 18 resolved. > > Date of last release: 2016-08-16. > > When were the last committers or PMC members elected? No new committers or > PMC members elected yet. > > Signed-off-by: [ ](gearpump) Andrew Purtell [ ](gearpump) Jarek > Jarcec Cecho [ ](gearpump) Reynold Xin [ ](gearpump) Todd Lipcon [ > ](gearpump) Xuefu Zhang > > Andrew Purtell 于2016年9月3日周六 上午8:27写道： > > > Sounds good, thanks! > > > > > > On Thu, Sep 1, 2016 at 10:47 PM, Vincent Wang > wrote: > > > > > Hi Andrew, > > > > > > I'd like to take it this time. > > > > > > Thanks, > > > Huafeng > > > > > > 2016-09-02 5:02 GMT+08:00 Andrew Purtell : > > > > > > > Time to report to the Board and IPMC. Do we have a volunteer from the > > > PPMC > > > > to produce the report this time? > > > > > > > > On Tue, Aug 30, 2016 at 3:46 AM, wrote: > > > > > > > > > Dear podling, > > > > > > > > > > This email was sent by an automated system on behalf of the Apache > > > > > Incubator PMC. It is an initial reminder to give you plenty of time > > to > > > > > prepare your quarterly board report. > > > > > > > > > > The board meeting is scheduled for Wed, 21 September 2016, 10:30 am > > > PDT. > > > > > The report for your podling will form a part of the Incubator PMC > > > > > report. The Incubator PMC requires your report to be submitted 2 > > weeks > > > > > before the board meeting, to allow sufficient time for review and > > > > > submission (Wed, September 07). > > > > > > > > > > Please submit your report with sufficient time to allow the > Incubator > > > > > PMC, and subsequently board members to review and digest. Again, > the > > > > > very latest you should submit your report is 2 weeks prior to the > > board > > > > > meeting. > > > > > > > > > > Thanks, > > > > > > > > > > The Apache Incubator PMC > > > > > > > > > > Submitting your Report > > > > > > > > > > -- > > > > > > > > > > Your report should contain the following: > > > > > > > > > > * Your project name > > > > > * A brief description of your project, which assumes no knowledge > > of > > > > > the project or necessarily of its field > > > > > * A list of the three most important issues to address in the > move > > > > > towards graduation. > > > > > * Any issues that the Incubator PMC or ASF Board might wish/need > to > > > be > > > > > aware of > > > > > * How has the community developed since the last report > > > > > * How has the project developed since the last report. > > > > > > > > > > This should be appended to the Incubator Wiki page at: > > > > > > > > > > http://wiki.apache.org/incubator/September2016 > > > > > > > > > > Note: This is manually populated. You may need to wait a little > > before > > > > > this page is created from a template. > > > > > > > > > > Mentors > > > > > --- > > > > > > > > > > Mentors should review reports for their project(s) and sign them > off > > on > > > > > the Incubator wiki page. Signing off reports shows that you are > > > > > following the project - projects that are not signed may raise > alarms > > > > > for the Incubator PMC. > > > > > > > > > > Incubator PMC > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > > > > > - Andy > > > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > > Hein > > > > (via Tom White) > > > > > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. -
[jira] [Commented] (GEARPUMP-202) DSL improvement
[ https://issues.apache.org/jira/browse/GEARPUMP-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467971#comment-15467971 ] Kam Kasravi commented on GEARPUMP-202: -- Manu I think we need an ability to directly access both downstream and upstream tasks. We do have output(index, message) but I don't believe this is accessible from Task and the index is opaque. > DSL improvement > > > Key: GEARPUMP-202 > URL: https://issues.apache.org/jira/browse/GEARPUMP-202 > Project: Apache Gearpump > Issue Type: Improvement > Components: streaming >Affects Versions: 0.8.1 >Reporter: Manu Zhang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-gearpump issue #80: [GEARPUMP-203] Use DataSourceTask and DataSink...
Github user huafengw commented on the issue: https://github.com/apache/incubator-gearpump/pull/80 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
GitHub user huafengw opened a pull request: https://github.com/apache/incubator-gearpump/pull/81 fix GEARPUMP-205 remove hdfs dependency from gear's classpath raise the pr to use travis UT You can merge this pull request into a Git repository by running: $ git pull https://github.com/huafengw/incubator-gearpump blob Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/81.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #81 commit acc150941d0dc0a4bae31aa6ea8949c79fcca0b1 Author: huafengw Date: 2016-08-09T11:34:35Z fix GEARPUMP-205 remove hdfs dependency from gear's classpath --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #80: [GEARPUMP-203] Use DataSourceTask and D...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-gearpump/pull/80 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #55: [GEARPUMP-171] Fix some typos
Github user huafengw commented on the issue: https://github.com/apache/incubator-gearpump/pull/55 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Podling Report Reminder - September 2016
Thanks Vincent, +1 from me. Anyone have anything to add or change? If not I will post it to the incubator wiki tomorrow. On Mon, Sep 5, 2016 at 3:19 AM, Vincent Wang wrote: > Hi All, > > This is the Gearpump podling report for September 2016. Please help to > review. > > Gearpump > > Gearpump is a reactive real-time streaming engine based on the > micro-service actor model. > Gearpump has been incubating since 2016-03-08. > > Three most important issues to address in the move towards graduation: > > 1. Make initial Apache branded release and release on a regular schedule. > 2. Continue to evolve community interest and support. > 3. Integrate with Apache Beam and akka-streams frameworks. > > - Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be > aware > of? > - None > > - How has the community developed since the last report? > - Increased community contribution on integration with Apache Cassandra. > > - How has the project developed since the last report? > - We released Apache Gearpump 0.8.1-incubating, which is Gearpump's first > release > as an Apache incubator project. > - Integration within Apache Beam (Gearpump Runner) has been merged into a > feature branch. > - Continued integration within akka-streams (Gearpump Materializer). Three > PRs committed. > - 28 issues created and 18 resolved. > > Date of last release: 2016-08-16. > > When were the last committers or PMC members elected? No new committers or > PMC members elected yet. > > Signed-off-by: [ ](gearpump) Andrew Purtell [ ](gearpump) Jarek > Jarcec Cecho [ ](gearpump) Reynold Xin [ ](gearpump) Todd Lipcon [ > ](gearpump) Xuefu Zhang > > Andrew Purtell 于2016年9月3日周六 上午8:27写道： > > > Sounds good, thanks! > > > > > > On Thu, Sep 1, 2016 at 10:47 PM, Vincent Wang > wrote: > > > > > Hi Andrew, > > > > > > I'd like to take it this time. > > > > > > Thanks, > > > Huafeng > > > > > > 2016-09-02 5:02 GMT+08:00 Andrew Purtell : > > > > > > > Time to report to the Board and IPMC. Do we have a volunteer from the > > > PPMC > > > > to produce the report this time? > > > > > > > > On Tue, Aug 30, 2016 at 3:46 AM, wrote: > > > > > > > > > Dear podling, > > > > > > > > > > This email was sent by an automated system on behalf of the Apache > > > > > Incubator PMC. It is an initial reminder to give you plenty of time > > to > > > > > prepare your quarterly board report. > > > > > > > > > > The board meeting is scheduled for Wed, 21 September 2016, 10:30 am > > > PDT. > > > > > The report for your podling will form a part of the Incubator PMC > > > > > report. The Incubator PMC requires your report to be submitted 2 > > weeks > > > > > before the board meeting, to allow sufficient time for review and > > > > > submission (Wed, September 07). > > > > > > > > > > Please submit your report with sufficient time to allow the > Incubator > > > > > PMC, and subsequently board members to review and digest. Again, > the > > > > > very latest you should submit your report is 2 weeks prior to the > > board > > > > > meeting. > > > > > > > > > > Thanks, > > > > > > > > > > The Apache Incubator PMC > > > > > > > > > > Submitting your Report > > > > > > > > > > -- > > > > > > > > > > Your report should contain the following: > > > > > > > > > > * Your project name > > > > > * A brief description of your project, which assumes no knowledge > > of > > > > > the project or necessarily of its field > > > > > * A list of the three most important issues to address in the > move > > > > > towards graduation. > > > > > * Any issues that the Incubator PMC or ASF Board might wish/need > to > > > be > > > > > aware of > > > > > * How has the community developed since the last report > > > > > * How has the project developed since the last report. > > > > > > > > > > This should be appended to the Incubator Wiki page at: > > > > > > > > > > http://wiki.apache.org/incubator/September2016 > > > > > > > > > > Note: This is manually populated. You may need to wait a little > > before > > > > > this page is created from a template. > > > > > > > > > > Mentors > > > > > --- > > > > > > > > > > Mentors should review reports for their project(s) and sign them > off > > on > > > > > the Incubator wiki page. Signing off reports shows that you are > > > > > following the project - projects that are not signed may raise > alarms > > > > > for the Incubator PMC. > > > > > > > > > > Incubator PMC > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > > > > >- Andy > > > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > > Hein > > > > (via Tom White) > > > > > > > > > > > > > > > -- > > Best regards, > > > >- Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628699
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/FileDirective.scala ---
@@ -96,24 +105,47 @@ object FileDirective {
}
// Downloads file from server
- def downloadFile(file: File): Route = {
+ def downloadFileFrom(jarStore: JarStore, filePath: String): Route = {
val responseEntity = HttpEntity(
MediaTypes.`application/octet-stream`,
- file.length,
- FileIO.fromFile(file, CHUNK_SIZE))
+ StreamConverters.fromInputStream(
+() => jarStore.getFile(filePath), CHUNK_SIZE
+ ))
complete(responseEntity)
}
- private def uploadFileImpl(rootDirectory: File)(implicit mat:
Materializer, ec: ExecutionContext)
+ private def uploadFileImpl(jarStore: JarStore)
+(implicit mat: Materializer, ec: ExecutionContext):
Directive1[Future[Map[Name, FilePath]]] = {
+Directive[Tuple1[Future[Map[Name, FilePath { inner =>
+ entity(as[Multipart.FormData]) { (formdata: Multipart.FormData) =>
+val fileNameMap = formdata.parts.mapAsync(1) { p =>
+ if (p.filename.isDefined) {
+val path = Instant.now() + p.filename.get
+val sink = StreamConverters.fromOutputStream(() =>
jarStore.createFile(path), true)
+val written = p.entity.dataBytes.runWith(sink)
+written.map(written =>
--- End diff --
better use a different local name
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628288
--- Diff:
core/src/main/scala/org/apache/gearpump/cluster/client/ClientContext.scala ---
@@ -140,7 +139,7 @@ class ClientContext(config: Config, sys: ActorSystem,
_master: ActorRef) {
private def loadFile(jarPath: String): AppJar = {
val jarFile = new java.io.File(jarPath)
-val path = jarStoreService.copyFromLocal(jarFile)
+val path = jarStoreClient.copyFromLocal(jarFile)
--- End diff --
Can we reuse `Util.uploadJar` here ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628447
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/FileDirective.scala ---
@@ -67,25 +69,32 @@ object FileDirective {
type FormField = Either[FileInfo, String]
/**
- * directive to uploadFile, it store the uploaded files
- * to temporary directory, and return a Map from form field name
+ * Store the uploaded files to temporary directory, and return a Map
from form field name
* to FileInfo.
*/
def uploadFile: Directive1[Form] = {
-uploadFileTo(null)
+Directive[Tuple1[Form]] { inner =>
+ extractMaterializer {implicit mat =>
+extractExecutionContext {implicit ec =>
+ uploadFileImpl(mat, ec) { filesFuture =>
+ctx => {
+ filesFuture.map(map => inner(Tuple1(map))).flatMap(route =>
route(ctx))
--- End diff --
Can we improve code readability here ? It's hard to comprehend what it is
doing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77626721
--- Diff: core/src/main/scala/org/apache/gearpump/jarstore/JarStore.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.gearpump.jarstore
+
+import java.io.{InputStream, OutputStream}
+import java.net.URI
+import java.util.ServiceLoader
+
+import com.typesafe.config.Config
+import org.apache.gearpump.util.Util
+
+import scala.collection.JavaConverters._
+
+case class FilePath(path: String)
+
+/**
+ * JarStoreService is used to manage the upload/download of binary files,
--- End diff --
`JarStoreService` ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77626272
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/JarStoreService.scala ---
@@ -15,72 +15,39 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.gearpump.jarstore
-import java.io.File
-import java.net.URI
-import java.util.ServiceLoader
-import scala.collection.JavaConverters._
-
-import akka.actor.ActorSystem
-import com.typesafe.config.Config
-
-import org.apache.gearpump.util.{Constants, Util}
-
-case class FilePath(path: String)
-
-/**
- * JarStoreService is used to manage the upload/download of binary files,
- * like user submitted application jar.
- */
-trait JarStoreService {
- /**
- * The scheme of the JarStoreService.
- * Like "hdfs" for HDFS file system, and "file" for a local
- * file system.
- */
- val scheme: String
-
- /**
- * Init the Jar Store.
- */
- def init(config: Config, system: ActorSystem)
-
- /**
- * This function will copy the local file to the remote JarStore, called
from client side.
- * @param localFile The local file
- */
- def copyFromLocal(localFile: File): FilePath
-
- /**
- * This function will copy the remote file to local file system, called
from client side.
- *
- * @param localFile The destination of file path
- * @param remotePath The remote file path from JarStore
- */
- def copyToLocalFile(localFile: File, remotePath: FilePath)
-}
-
-object JarStoreService {
-
- /**
- * Get a active JarStoreService by specifying a scheme.
- *
- * Please see config
[[org.apache.gearpump.util.Constants#GEARPUMP_APP_JAR_STORE_ROOT_PATH]] for
- * more information.
- */
- def get(config: Config): JarStoreService = {
-val jarStoreRootPath =
config.getString(Constants.GEARPUMP_APP_JAR_STORE_ROOT_PATH)
-get(jarStoreRootPath)
+import akka.actor.{Actor, Stash}
+import akka.pattern.pipe
+import org.slf4j.Logger
+
+import org.apache.gearpump.cluster.ClientToMaster.{GetJarStoreServer,
JarStoreServerAddress}
+import org.apache.gearpump.util._
+
+class JarStoreService(jarStoreRootPath: String) extends Actor with Stash {
+ val host =
context.system.settings.config.getString(Constants.GEARPUMP_HOSTNAME)
--- End diff --
Are these values accessed elsewhere ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77625961
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/JarStoreService.scala ---
@@ -15,72 +15,39 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.gearpump.jarstore
-import java.io.File
-import java.net.URI
-import java.util.ServiceLoader
-import scala.collection.JavaConverters._
-
-import akka.actor.ActorSystem
-import com.typesafe.config.Config
-
-import org.apache.gearpump.util.{Constants, Util}
-
-case class FilePath(path: String)
-
-/**
- * JarStoreService is used to manage the upload/download of binary files,
- * like user submitted application jar.
- */
-trait JarStoreService {
- /**
- * The scheme of the JarStoreService.
- * Like "hdfs" for HDFS file system, and "file" for a local
- * file system.
- */
- val scheme: String
-
- /**
- * Init the Jar Store.
- */
- def init(config: Config, system: ActorSystem)
-
- /**
- * This function will copy the local file to the remote JarStore, called
from client side.
- * @param localFile The local file
- */
- def copyFromLocal(localFile: File): FilePath
-
- /**
- * This function will copy the remote file to local file system, called
from client side.
- *
- * @param localFile The destination of file path
- * @param remotePath The remote file path from JarStore
- */
- def copyToLocalFile(localFile: File, remotePath: FilePath)
-}
-
-object JarStoreService {
-
- /**
- * Get a active JarStoreService by specifying a scheme.
- *
- * Please see config
[[org.apache.gearpump.util.Constants#GEARPUMP_APP_JAR_STORE_ROOT_PATH]] for
- * more information.
- */
- def get(config: Config): JarStoreService = {
-val jarStoreRootPath =
config.getString(Constants.GEARPUMP_APP_JAR_STORE_ROOT_PATH)
-get(jarStoreRootPath)
+import akka.actor.{Actor, Stash}
+import akka.pattern.pipe
+import org.slf4j.Logger
+
+import org.apache.gearpump.cluster.ClientToMaster.{GetJarStoreServer,
JarStoreServerAddress}
+import org.apache.gearpump.util._
+
+class JarStoreService(jarStoreRootPath: String) extends Actor with Stash {
--- End diff --
Do you think `JarStoreServer` is better mapping to `JarStoreClient` ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #81: fix GEARPUMP-205 remove hdfs dependency...
Github user manuzhang commented on a diff in the pull request: https://github.com/apache/incubator-gearpump/pull/81#discussion_r77625222 --- Diff: .gitignore --- @@ -1,5 +1,5 @@ # Gearpump -jarstore/ +#jarstore/ --- End diff -- why comment this line ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #81: fix GEARPUMP-205 remove hdfs dependency from g...
Github user codecov-io commented on the issue: https://github.com/apache/incubator-gearpump/pull/81 ## [Current coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/81?src=pr) is 70.90% (diff: 56.41%)  > No coverage report found for **master** at 584a2ca. > Powered by [Codecov](https://codecov.io?src=pr). Last update [584a2ca...acc1509](https://codecov.io/gh/apache/incubator-gearpump/compare/584a2ca23180add9f7454334907a10c8144565de...acc150941d0dc0a4bae31aa6ea8949c79fcca0b1?src=pr) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #55: [GEARPUMP-171] Fix some typos
Github user huafengw commented on the issue: https://github.com/apache/incubator-gearpump/pull/55 Sorry it took so long. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump pull request #55: [GEARPUMP-171] Fix some typos
Github user asfgit closed the pull request at: https://github.com/apache/incubator-gearpump/pull/55 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-gearpump issue #80: [GEARPUMP-203] Use DataSourceTask and DataSink...
Github user codecov-io commented on the issue: https://github.com/apache/incubator-gearpump/pull/80 ## [Current coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/80?src=pr) is 69.95% (diff: 88.88%) > Merging [#80](https://codecov.io/gh/apache/incubator-gearpump/pull/80?src=pr) into [master](https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr) will increase coverage by **1.65%** ```diff @@ master#80 diff @@ == Files 177177 Lines 5927 5907-20 Methods5606 5591-15 Messages 0 0 Branches321316 -5 == + Hits 4048 4132+84 + Misses 1879 1775 -104 Partials 0 0 ```  > Powered by [Codecov](https://codecov.io?src=pr). Last update [5bf7c7c...6ce5b1f](https://codecov.io/gh/apache/incubator-gearpump/compare/5bf7c7cb606aca8c8ffc93375b8c6316f8f6624c...6ce5b1fcc4f011f8c1518acf499a12a76ae32df0?src=pr) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Shepherd for this month
Thanks Justin Your notes are always very nicely annotated with references - thanks! We'll proceed on the recommendations you've made. Regarding [9] - Andy also recommended that we' look for additional mentors that can play an active role. Regarding [3], [7] - we'll have the old site redirect to the apache incubating one. Thanks again Kam On Mon, Sep 5, 2016 at 7:16 PM, Justin Mclean wrote: > Hi, > > I’ve been assigned as a shepherd and was talking a look at your web site > and noticed a couple of minor things you might want to do something about > before graduation. Of course I don’t know much about the history of this > project and these may already been looked into or already being dealt with > somewhere, so apologies if I bring up anything that is known or anything > that I’m mistaken about. > > Looking at the website I think it could do with a little more branding [1] > and make it clearer that this is an incubating site.[2] There seems to be > two websites which may be adding to the confusion here [3][4]. I notice [4] > is better re branding/disclaimer etc etc but [3] comes up a google search > first. > > The download link on [5] is not using the mirror scripts [6]. The download > page here [7] includes a binary that is hosted outside of apache [8][10] > > Perhaps a redirect from the old site to the new is in order as that would > fix most of the above? > > I also noticed an unanswered question re releases that your mentors could > help with [9], but it seems (from what I can see from brief look) you only > have one currently active mentor. Is this correct? Idk so do you think you > may need to add an additional mentors or prompt existing mentors to see if > they still have time to mentor? > > Thanks, > Justin > > PS I’m not subscribed to this list so please CC me on any replies. > > 1. http://incubator.apache.org/guides/branding.html > 2. http://incubator.apache.org/guides/branding.html#disclaimers > 3. http://www.gearpump.io/overview.html > 4. http://gearpump.incubator.apache.org/overview.html > 5. https://dist.apache.org/repos/dist/release/incubator/ > gearpump/0.8.1-incubating/ > 6. http://www.apache.org/dev/release-download-pages > 7. http://www.gearpump.io/downloads.html > 8. https://github.com/gearpump/gearpump/releases/download/0. > 8.0/gearpump-2.11-0.8.0.zip > 9. https://lists.apache.org/thread.html/53c45abd3c188b06068c0d424a5510 > a33a622e9cc5535075da20d564@%3Cdev.gearpump.apache.org%3E > 10. http://www.apache.org/dev/release.html#host-GA
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467355#comment-15467355
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628699
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/FileDirective.scala ---
@@ -96,24 +105,47 @@ object FileDirective {
}
// Downloads file from server
- def downloadFile(file: File): Route = {
+ def downloadFileFrom(jarStore: JarStore, filePath: String): Route = {
val responseEntity = HttpEntity(
MediaTypes.`application/octet-stream`,
- file.length,
- FileIO.fromFile(file, CHUNK_SIZE))
+ StreamConverters.fromInputStream(
+() => jarStore.getFile(filePath), CHUNK_SIZE
+ ))
complete(responseEntity)
}
- private def uploadFileImpl(rootDirectory: File)(implicit mat:
Materializer, ec: ExecutionContext)
+ private def uploadFileImpl(jarStore: JarStore)
+(implicit mat: Materializer, ec: ExecutionContext):
Directive1[Future[Map[Name, FilePath]]] = {
+Directive[Tuple1[Future[Map[Name, FilePath { inner =>
+ entity(as[Multipart.FormData]) { (formdata: Multipart.FormData) =>
+val fileNameMap = formdata.parts.mapAsync(1) { p =>
+ if (p.filename.isDefined) {
+val path = Instant.now() + p.filename.get
+val sink = StreamConverters.fromOutputStream(() =>
jarStore.createFile(path), true)
+val written = p.entity.dataBytes.runWith(sink)
+written.map(written =>
--- End diff --
better use a different local name
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467351#comment-15467351
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628447
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/FileDirective.scala ---
@@ -67,25 +69,32 @@ object FileDirective {
type FormField = Either[FileInfo, String]
/**
- * directive to uploadFile, it store the uploaded files
- * to temporary directory, and return a Map from form field name
+ * Store the uploaded files to temporary directory, and return a Map
from form field name
* to FileInfo.
*/
def uploadFile: Directive1[Form] = {
-uploadFileTo(null)
+Directive[Tuple1[Form]] { inner =>
+ extractMaterializer {implicit mat =>
+extractExecutionContext {implicit ec =>
+ uploadFileImpl(mat, ec) { filesFuture =>
+ctx => {
+ filesFuture.map(map => inner(Tuple1(map))).flatMap(route =>
route(ctx))
--- End diff --
Can we improve code readability here ? It's hard to comprehend what it is
doing.
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467349#comment-15467349
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77628288
--- Diff:
core/src/main/scala/org/apache/gearpump/cluster/client/ClientContext.scala ---
@@ -140,7 +139,7 @@ class ClientContext(config: Config, sys: ActorSystem,
_master: ActorRef) {
private def loadFile(jarPath: String): AppJar = {
val jarFile = new java.io.File(jarPath)
-val path = jarStoreService.copyFromLocal(jarFile)
+val path = jarStoreClient.copyFromLocal(jarFile)
--- End diff --
Can we reuse `Util.uploadJar` here ?
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467331#comment-15467331
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77626721
--- Diff: core/src/main/scala/org/apache/gearpump/jarstore/JarStore.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.gearpump.jarstore
+
+import java.io.{InputStream, OutputStream}
+import java.net.URI
+import java.util.ServiceLoader
+
+import com.typesafe.config.Config
+import org.apache.gearpump.util.Util
+
+import scala.collection.JavaConverters._
+
+case class FilePath(path: String)
+
+/**
+ * JarStoreService is used to manage the upload/download of binary files,
--- End diff --
`JarStoreService` ?
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467320#comment-15467320
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77626272
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/JarStoreService.scala ---
@@ -15,72 +15,39 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.gearpump.jarstore
-import java.io.File
-import java.net.URI
-import java.util.ServiceLoader
-import scala.collection.JavaConverters._
-
-import akka.actor.ActorSystem
-import com.typesafe.config.Config
-
-import org.apache.gearpump.util.{Constants, Util}
-
-case class FilePath(path: String)
-
-/**
- * JarStoreService is used to manage the upload/download of binary files,
- * like user submitted application jar.
- */
-trait JarStoreService {
- /**
- * The scheme of the JarStoreService.
- * Like "hdfs" for HDFS file system, and "file" for a local
- * file system.
- */
- val scheme: String
-
- /**
- * Init the Jar Store.
- */
- def init(config: Config, system: ActorSystem)
-
- /**
- * This function will copy the local file to the remote JarStore, called
from client side.
- * @param localFile The local file
- */
- def copyFromLocal(localFile: File): FilePath
-
- /**
- * This function will copy the remote file to local file system, called
from client side.
- *
- * @param localFile The destination of file path
- * @param remotePath The remote file path from JarStore
- */
- def copyToLocalFile(localFile: File, remotePath: FilePath)
-}
-
-object JarStoreService {
-
- /**
- * Get a active JarStoreService by specifying a scheme.
- *
- * Please see config
[[org.apache.gearpump.util.Constants#GEARPUMP_APP_JAR_STORE_ROOT_PATH]] for
- * more information.
- */
- def get(config: Config): JarStoreService = {
-val jarStoreRootPath =
config.getString(Constants.GEARPUMP_APP_JAR_STORE_ROOT_PATH)
-get(jarStoreRootPath)
+import akka.actor.{Actor, Stash}
+import akka.pattern.pipe
+import org.slf4j.Logger
+
+import org.apache.gearpump.cluster.ClientToMaster.{GetJarStoreServer,
JarStoreServerAddress}
+import org.apache.gearpump.util._
+
+class JarStoreService(jarStoreRootPath: String) extends Actor with Stash {
+ val host =
context.system.settings.config.getString(Constants.GEARPUMP_HOSTNAME)
--- End diff --
Are these values accessed elsewhere ?
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[
https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467315#comment-15467315
]
ASF GitHub Bot commented on GEARPUMP-205:
-
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/81#discussion_r77625961
--- Diff:
core/src/main/scala/org/apache/gearpump/jarstore/JarStoreService.scala ---
@@ -15,72 +15,39 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.gearpump.jarstore
-import java.io.File
-import java.net.URI
-import java.util.ServiceLoader
-import scala.collection.JavaConverters._
-
-import akka.actor.ActorSystem
-import com.typesafe.config.Config
-
-import org.apache.gearpump.util.{Constants, Util}
-
-case class FilePath(path: String)
-
-/**
- * JarStoreService is used to manage the upload/download of binary files,
- * like user submitted application jar.
- */
-trait JarStoreService {
- /**
- * The scheme of the JarStoreService.
- * Like "hdfs" for HDFS file system, and "file" for a local
- * file system.
- */
- val scheme: String
-
- /**
- * Init the Jar Store.
- */
- def init(config: Config, system: ActorSystem)
-
- /**
- * This function will copy the local file to the remote JarStore, called
from client side.
- * @param localFile The local file
- */
- def copyFromLocal(localFile: File): FilePath
-
- /**
- * This function will copy the remote file to local file system, called
from client side.
- *
- * @param localFile The destination of file path
- * @param remotePath The remote file path from JarStore
- */
- def copyToLocalFile(localFile: File, remotePath: FilePath)
-}
-
-object JarStoreService {
-
- /**
- * Get a active JarStoreService by specifying a scheme.
- *
- * Please see config
[[org.apache.gearpump.util.Constants#GEARPUMP_APP_JAR_STORE_ROOT_PATH]] for
- * more information.
- */
- def get(config: Config): JarStoreService = {
-val jarStoreRootPath =
config.getString(Constants.GEARPUMP_APP_JAR_STORE_ROOT_PATH)
-get(jarStoreRootPath)
+import akka.actor.{Actor, Stash}
+import akka.pattern.pipe
+import org.slf4j.Logger
+
+import org.apache.gearpump.cluster.ClientToMaster.{GetJarStoreServer,
JarStoreServerAddress}
+import org.apache.gearpump.util._
+
+class JarStoreService(jarStoreRootPath: String) extends Actor with Stash {
--- End diff --
Do you think `JarStoreServer` is better mapping to `JarStoreClient` ?
> remove hdfs dependency from command gear's classpath
>
>
> Key: GEARPUMP-205
> URL: https://issues.apache.org/jira/browse/GEARPUMP-205
> Project: Apache Gearpump
> Issue Type: Improvement
> Components: core
>Affects Versions: 0.8.2
>Reporter: Huafeng Wang
>Assignee: Huafeng Wang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467303#comment-15467303 ] ASF GitHub Bot commented on GEARPUMP-205: - Github user manuzhang commented on a diff in the pull request: https://github.com/apache/incubator-gearpump/pull/81#discussion_r77625222 --- Diff: .gitignore --- @@ -1,5 +1,5 @@ # Gearpump -jarstore/ +#jarstore/ --- End diff -- why comment this line ? > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467187#comment-15467187 ] ASF GitHub Bot commented on GEARPUMP-205: - Github user codecov-io commented on the issue: https://github.com/apache/incubator-gearpump/pull/81 ## [Current coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/81?src=pr) is 70.90% (diff: 56.41%)  > No coverage report found for **master** at 584a2ca. > Powered by [Codecov](https://codecov.io?src=pr). Last update [584a2ca...acc1509](https://codecov.io/gh/apache/incubator-gearpump/compare/584a2ca23180add9f7454334907a10c8144565de...acc150941d0dc0a4bae31aa6ea8949c79fcca0b1?src=pr) > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (GEARPUMP-171) Fix some typos
[ https://issues.apache.org/jira/browse/GEARPUMP-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated GEARPUMP-171: Assignee: darion yaphet > Fix some typos > -- > > Key: GEARPUMP-171 > URL: https://issues.apache.org/jira/browse/GEARPUMP-171 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.0 >Reporter: darion yaphet >Assignee: darion yaphet > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-171) Fix some typos
[ https://issues.apache.org/jira/browse/GEARPUMP-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467151#comment-15467151 ] ASF GitHub Bot commented on GEARPUMP-171: - Github user huafengw commented on the issue: https://github.com/apache/incubator-gearpump/pull/55 Sorry it took so long. > Fix some typos > -- > > Key: GEARPUMP-171 > URL: https://issues.apache.org/jira/browse/GEARPUMP-171 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.0 >Reporter: darion yaphet >Assignee: darion yaphet > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (GEARPUMP-171) Fix some typos
[ https://issues.apache.org/jira/browse/GEARPUMP-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-171. - Resolution: Fixed Fix Version/s: 0.8.2 Issue resolved by pull request 55 [https://github.com/apache/incubator-gearpump/pull/55] > Fix some typos > -- > > Key: GEARPUMP-171 > URL: https://issues.apache.org/jira/browse/GEARPUMP-171 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.0 >Reporter: darion yaphet > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-171) Fix some typos
[ https://issues.apache.org/jira/browse/GEARPUMP-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467147#comment-15467147 ] ASF GitHub Bot commented on GEARPUMP-171: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-gearpump/pull/55 > Fix some typos > -- > > Key: GEARPUMP-171 > URL: https://issues.apache.org/jira/browse/GEARPUMP-171 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.0 >Reporter: darion yaphet > Fix For: 0.8.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
[ https://issues.apache.org/jira/browse/GEARPUMP-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467141#comment-15467141 ] ASF GitHub Bot commented on GEARPUMP-205: - GitHub user huafengw opened a pull request: https://github.com/apache/incubator-gearpump/pull/81 fix GEARPUMP-205 remove hdfs dependency from gear's classpath raise the pr to use travis UT You can merge this pull request into a Git repository by running: $ git pull https://github.com/huafengw/incubator-gearpump blob Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-gearpump/pull/81.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #81 commit acc150941d0dc0a4bae31aa6ea8949c79fcca0b1 Author: huafengw Date: 2016-08-09T11:34:35Z fix GEARPUMP-205 remove hdfs dependency from gear's classpath > remove hdfs dependency from command gear's classpath > > > Key: GEARPUMP-205 > URL: https://issues.apache.org/jira/browse/GEARPUMP-205 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (GEARPUMP-205) remove hdfs dependency from command gear's classpath
Huafeng Wang created GEARPUMP-205: - Summary: remove hdfs dependency from command gear's classpath Key: GEARPUMP-205 URL: https://issues.apache.org/jira/browse/GEARPUMP-205 Project: Apache Gearpump Issue Type: Improvement Components: core Affects Versions: 0.8.2 Reporter: Huafeng Wang Assignee: Huafeng Wang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEARPUMP-171) Fix some typos
[ https://issues.apache.org/jira/browse/GEARPUMP-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15467129#comment-15467129 ] ASF GitHub Bot commented on GEARPUMP-171: - Github user huafengw commented on the issue: https://github.com/apache/incubator-gearpump/pull/55 +1 > Fix some typos > -- > > Key: GEARPUMP-171 > URL: https://issues.apache.org/jira/browse/GEARPUMP-171 > Project: Apache Gearpump > Issue Type: Improvement > Components: core >Affects Versions: 0.8.0 >Reporter: darion yaphet > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (GEARPUMP-203) Use DataSourceTask / DataSinkTask for DSL
[
https://issues.apache.org/jira/browse/GEARPUMP-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manu Zhang resolved GEARPUMP-203.
-
Resolution: Fixed
Fix Version/s: 0.8.2
> Use DataSourceTask / DataSinkTask for DSL
> --
>
> Key: GEARPUMP-203
> URL: https://issues.apache.org/jira/browse/GEARPUMP-203
> Project: Apache Gearpump
> Issue Type: Sub-task
> Components: streaming
>Affects Versions: 0.8.1
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Fix For: 0.8.2
>
>
> Currently, {{source}} operation in DSL will be translated to {{SourceTask}}
> allowing for a passed-in operation processing input data while
> {{DataSourceTask}} is default container running {{DataSource}} API without
> any processing. {{SinkTask}} and {{DataSinkTask}} are almost identical. We
> can improve the latter to account for both cases such that we don't keep two
> sets of implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (GEARPUMP-203) Use DataSourceTask / DataSinkTask for DSL
[
https://issues.apache.org/jira/browse/GEARPUMP-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466684#comment-15466684
]
ASF GitHub Bot commented on GEARPUMP-203:
-
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-gearpump/pull/80
> Use DataSourceTask / DataSinkTask for DSL
> --
>
> Key: GEARPUMP-203
> URL: https://issues.apache.org/jira/browse/GEARPUMP-203
> Project: Apache Gearpump
> Issue Type: Sub-task
> Components: streaming
>Affects Versions: 0.8.1
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Fix For: 0.8.2
>
>
> Currently, {{source}} operation in DSL will be translated to {{SourceTask}}
> allowing for a passed-in operation processing input data while
> {{DataSourceTask}} is default container running {{DataSource}} API without
> any processing. {{SinkTask}} and {{DataSinkTask}} are almost identical. We
> can improve the latter to account for both cases such that we don't keep two
> sets of implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
