[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208323#comment-15208323 ] Sean Owen commented on SPARK-13843: --- ... and per http://apache-spark-developers-list.1001551.n3.nabble.com/SPARK-13843-Next-steps-td16783.html it looks like the resolution is: streaming-flume comes back in; everything else stays as is: SPARK-14073 > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200267#comment-15200267 ] Reynold Xin commented on SPARK-13843: - BTW I think we should give all committers write access to the github repo. I worry much more about these repos not having enough maintainers rather than the opposite of too many companies wanting to "own" them. > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202138#comment-15202138 ] Chris A. Mattmann commented on SPARK-13843: --- Hi - what is being done about concerns from at least one committer here e.g., based on recent dev list discussion about moving code out of the ASF and into a Github repo without more discussion from the PMC? > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202143#comment-15202143 ] Sean Owen commented on SPARK-13843: --- This has forked to dev@: http://apache-spark-developers-list.1001551.n3.nabble.com/SPARK-13843-and-future-of-streaming-backends-td16711.html > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197147#comment-15197147 ] Sean Owen commented on SPARK-13843: --- Yeah same question about Ganglia support > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196661#comment-15196661 ] Liwei Lin commented on SPARK-13843: --- hi [~zsxwing], we didn't move streaming-kinesis (which is also under external) out -- why is that please? > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195811#comment-15195811 ] Shixiong Zhu commented on SPARK-13843: -- They are all public. And definitely, any contributions are welcome. For now I'm just helping set up these projects. We will add people if they want to maintain these projects. For docs, I'm going to go through the official docs and remove all related to these projects. > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195760#comment-15195760 ] Sean Owen commented on SPARK-13843: --- It's a little different in that it started as part of the ASF project. It has been removed from the project (OK), it's been forked and maintained by others outside the project (OK), and nobody has now less access to it (i.e. I assume any committer would be added as a project member if they wanted to). We have to make sure it's not presented from official docs as still part of Spark, and can't release it together in a way that suggests it's official (I assume we won't). We also have to be careful this doesn't add up to appearing to take a part of a community project "private". These modules are so ancillary that I can't imagine it's controversial. It's going to be more of an issue for the Kafka integration (which we are discussing on the other JIRA) > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195734#comment-15195734 ] Shixiong Zhu commented on SPARK-13843: -- Sure. These projects are just like other SQL data source spark packages. > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195723#comment-15195723 ] Sean Owen commented on SPARK-13843: --- That's great -- what's the theory about who maintains them? they're no longer part of Spark itself. I'm guessing it's whoever wants to contribute and/or asks nicely to be a contributor can just do so? > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195703#comment-15195703 ] Shixiong Zhu commented on SPARK-13843: -- They are in https://github.com/spark-packages I'm working on moving docs to them and will add their links to Spark docs. > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13843) Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
[ https://issues.apache.org/jira/browse/SPARK-13843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195485#comment-15195485 ] Sean Owen commented on SPARK-13843: --- Where did this code end up, by the way? we probably need a link to it. > Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, > streaming-twitter to Spark packages > --- > > Key: SPARK-13843 > URL: https://issues.apache.org/jira/browse/SPARK-13843 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu > Fix For: 2.0.0 > > > Currently there are a few sub-projects, each for integrating with different > external sources for Streaming. Now that we have better ability to include > external libraries (Spark packages) and with Spark 2.0 coming up, we can move > the following projects out of Spark to https://github.com/spark-packages > - streaming-flume > - streaming-akka > - streaming-mqtt > - streaming-zeromq > - streaming-twitter > They are just some ancillary packages and considering the overhead of > maintenance, running tests and PR failures, it's better to maintain them out > of Spark. In addition, these projects can have their different release cycles > and we can release them faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
