When people have questions about Spark, there are 2 main places (as far as I can tell) where they ask them:
- Stack Overflow, under the apache-spark tag <http://stackoverflow.com/questions/tagged/apache-spark> - This mailing list The mailing list is valuable as an independent place for discussion that is part of the Spark project itself. Furthermore, it allows for a broader range of discussions than would be allowed on Stack Overflow <http://stackoverflow.com/help/dont-ask>. As the Spark project has grown in popularity, I see that a few problems have emerged with this mailing list: - It’s hard to follow topics (e.g. Streaming vs. SQL) that you’re interested in, and it’s hard to know when someone has mentioned you specifically. - It’s hard to search for existing threads and link information across disparate threads. - It’s hard to format code and log snippets nicely, and by extension, hard to read other people’s posts with this kind of information. There are existing solutions to all these (and other) problems based around straight-up discipline or client-side tooling, which users have to conjure up for themselves. I’d like us as a community to consider using Discourse <http://www.discourse.org/> as an alternative to, or overlay on top of, this mailing list, that provides better out-of-the-box solutions to these problems. Discourse is a modern discussion platform built by some of the same people who created Stack Overflow. It has many neat features <http://v1.discourse.org/about/> that I believe this community would benefit from. For example: - When a user starts typing up a new post, they get a panel *showing existing conversations that look similar*, just like on Stack Overflow. - It’s easy to search for posts and link between them. - *Markdown support* is built-in to composer. - You can *specifically mention people* and they will be notified. - Posts can be categorized (e.g. Streaming, SQL, etc.). - There is a built-in option for mailing list support which forwards all activity on the forum to a user’s email address and which allows for creation of new posts via email. What do you think of Discourse as an alternative, more manageable way to discus Spark? There are a few options we can consider: 1. Work with the ASF as well as the Discourse team to allow Discourse to act as an overlay on top of this mailing list <https://meta.discourse.org/t/discourse-as-a-front-end-for-existing-asf-mailing-lists/23167?u=nicholaschammas>, allowing people to continue to use the mailing list as-is if they want. (This is the toughest but perhaps most attractive option.) 2. Create a new Discourse forum for Spark that is not bound to this user list. This is relatively easy but will effectively fork the community on this list. (We cannot shut down this mailing in favor of one managed by Discourse.) 3. Don’t use Discourse. Just encourage people on this list to post instead on Stack Overflow whenever possible. 4. Something else. What does everyone think? Nick -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Discourse-A-proposed-alternative-to-the-Spark-User-list-tp20851.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
