On Fri, Apr 29, 2016 at 6:22 PM Steve Blackmon <sblack...@apache.org> wrote:
> The new website is now up at streams.incubator.apache.org. It’s not > perfect but it’s good enough and way better than it’s ever been. It’s got > a basic connection to Google Analytics - if you’d like to be added to the > organization just ask. This looks great! I agree with Ate on a diagram or other visually engaging component. > > I’m going to put together a blog post about website revamp and everything > that there for the project blog at https://blogs.apache.org/streams/ > > Hopefully we’ll also be able to tweet it out from @ApacheStreams if the > PMC is able to get control of it (WIP) > > > Steve Blackmon > sblack...@apache.org > > On Wed, Apr 27, 2016 at 3:28 PM Steve Blackmon <Steve Blackmon > <steve+blackmon+%3csblack...@apache.org%3E>> wrote: > >> All, >> >> I think we are pretty close to pulling all of the work on STREAMS-401 >> (new web page) together into a pretty slick package. Thanks for your >> patience and feedback. >> >> Some new additions include diagrams on the architecture page and within >> many modules depicting streams components and data flow among them and with >> third-party systems, a 5-step tutorial to run a useful stream, consistent >> look-and-feel across all three repos, integrated breadcrumbs, and >> rudimentary github and twitter integrations. >> >> Feel free to browse these links to see where we’re at. >> >> >> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html >> >> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/index.html >> >> http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/index.html >> >> All of the menu link now resolve, or will resolve as soon as everything >> is promoted. If you get any Not Available errors that you CAN’T resolve by >> substituting ‘latest’ for ‘0.3-incubating-SNAPSHOT’ in the url, please let >> me know. >> >> Additionally if you identify any major issues or have any last >> suggestions please let me know before I kickoff the process to merge the >> STREAMS-401 branches and promote the site later this week. >> >> Steve Blackmon >> sblack...@apache.org >> >> On Thu, Mar 03, 2016 at 9:48 AM Ate Douma <Ate Douma >> <ate+douma+%3c...@douma.nu%3E>> wrote: >> >>> +1 to everything below :) >>> >>> Good stuff Steve! >>> >>> On 2016-03-01 23:50, Steve Blackmon wrote: >>> > Following up: >>> > >>> > I took a pass at cleaning up the release markdown files and they now >>> look >>> > mostly the same as they did under apache cms. >>> > >>> > I also confirmed that site-deploy can write to a sub-directory of >>> > streams.incubator.apache.org using scm:svn capability. >>> > >>> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html >>> >>> > >>> > Recall that streams-project and streams-examples each publish schema >>> > artifacts to their maven sites, so maintaining prior versions is >>> important. >>> > streams-master doesn't currently, but since it's a parent of those >>> other >>> > two and prior experience has led to believe we're better off using the >>> same >>> > scheme and setting the streams-master in the pom to be a sibling of >>> > streams-project and streams-example rather that publishing it to a >>> > different URL entirely. This is why the version an artifactId are >>> being >>> > included in the above url. >>> > >>> > Brings up two issues - >>> > resolving the latest release and/or snapshot version at any given time >>> > ensuring a visitor to streams.incubator.apache.org lands on the >>> > streams-master site for that version >>> > >>> > While symlinks are a viable way to resolving the latest release and/or >>> > snapshot version while keeping previous release versions around >>> > >>> > i.e. forwarding >>> > http://streams.incubator.apache.org/site/latest/streams-master/index.html >>> >>> > to >>> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html >>> >>> > >>> > they won't help us forward http://streams.incubator.apache.org to a >>> > subdirectory of itself. >>> > >>> > Good news is apache cms supports .htaccess and the following snippet >>> can. >>> > >>> > RewriteEngine *on* >>> > >>> > Redirect / >>> > http://streams.incubator.apache.org/site/latest/streams-master/index.html >>> >>> > Feel free to confirm by visiting streams.staging.incubator.org >>> (you'll be >>> > redirected to the output of the maven site-deploy) >>> > >>> > *I ask that everyone on the list review and send me any concerns or >>> > feedback they want incorporated before this goes live on >>> > streams.incubator.apache.org <http://streams.incubator.apache.org>.* >>> > >>> > Procedurally, the way I expect that to happen is: >>> > - I'll submit a PR to streams-master/master >>> > - That PR will merge >>> > - Update the streams-master jenkins job to run site-deploy, and do the >>> > needful to get it the necessary credentials >>> > - Going forward all builds of streams-master/master will publish to >>> > site/${version}/streams-master >>> > - Add a few more instructions to jenkins to maintain the symlinks and >>> > .htaccess >>> > - If there are any manual steps required to oversee, document that on >>> the >>> > site itself. >>> > >>> > Then, we'll move on to hooking streams-project and streams-examples up >>> to >>> > this new site (as children with an integrated breadcrumb) and to >>> jenkins CD. >>> > >>> > Steve Blackmon >>> > sblack...@apache.org >>> > >>> > On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblack...@apache.org> >>> >>> > wrote: >>> > >>> >> One thing which I think can 'spice up' the overview page a bit more >>> would >>> >>> be a >>> >>> diagram providing some very high-level interaction between data >>> sources >>> >>> and >>> >>> destinations and the streams role between them. >>> >> >>> >> >>> >> I agree - a diagram depicting streams mediating between sources and >>> >> destination on the landing page is appropriate. I'll work on this >>> soon. >>> >> >>> >> And before replacing the current site several existing pages need to >>> be >>> >>> moved to >>> >>> the new one. But once those are done I think this is great >>> improvement. >>> >> >>> >> >>> >> I've pushed a new version (to the source and site links from my prior >>> >> message) that brings most of the existing content along, using the >>> >> reporting-info plugin to generate the HTML where possible. >>> >> >>> >> On a few of the pages - SCM, CI, and Dependency Info - the output of >>> the >>> >> maven plugin simply isn't adequate for a multi-repository project >>> such as >>> >> this, so those are now managed within src/site/markdown >>> >> >>> >> Release Setup and Release Process need some cleanup but all of the >>> content >>> >> has been migrated. >>> >> >>> >> Other report-info pages - such as Dependencies, Dependency >>> Management, >>> >> Dependency Convergence, JavaDocs, Test JavaDocs, - don't make sense >>> at the >>> >> streams-master level but do exist and are useful within >>> streams-project, >>> >> streams-examples, and many of their respective sub-modules. >>> >> >>> >> In theory those additional report items will show up when navigating >>> into >>> >> the site hierarchy of the sub-projects (links to each sub-project are >>> >> present along the header), which will inherit the structure, >>> look-and-feel, >>> >> and breadcrumb of this site revision. Naturally these changes must be >>> >> deployed to streams.incubator.apache.org and sub-project pom.xml and >>> >> site_en.xml will need some tweaking but I'm optimistic it will all >>> fit >>> >> together nicely. >>> >> >>> >> Steve Blackmon >>> >> sblack...@apache.org >>> >> >>> >> On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <a...@douma.nu> wrote: >>> >> >>> >>> On 2016-02-25 23:50, Steve Blackmon wrote: >>> >>> >>> >>>> Hello streamers, >>> >>>> >>> >>>> I finally consolidated some materials I've been working on into a >>> >>>> nice-looking thing that is a viable candidate to replace the >>> current >>> >>>> streams web page. Please take a look and let me know what you >>> think. >>> >>>> Sometime >>> >>>> soon I expect to submit a vote to take down the previous site and >>> replace >>> >>>> it with something derived from this version. >>> >>>> >>> >>>> The proposed new site is temporarily being hosted at: >>> >>>> >>> >>>> >>> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html >>> >>>> >>> >>>> I placed all of the content into markdown files which are checked >>> into a >>> >>>> branch of streams-master. This approach allows us to author the >>> site >>> >>>> using >>> >>>> markdown, package and deploy it with maven-site, and manage changes >>> using >>> >>>> git. >>> >>>> >>> >>> >>> >>> This looks cool! >>> >>> >>> >>> I like the new overview and faq pages, definitely more concrete >>> about >>> >>> Streams. >>> >>> >>> >>> One thing which I think can 'spice up' the overview page a bit more >>> would >>> >>> be a >>> >>> diagram providing some very high-level interaction between data >>> sources >>> >>> and >>> >>> destinations and the streams role between them. >>> >>> >>> >>> And before replacing the current site several existing pages need to >>> be >>> >>> moved to >>> >>> the new one. But once those are done I think this is great >>> improvement. >>> >>> >>> >>> >>> >>>> If you'd like to submit specific changes feel free to do so as >>> >>>> pull-requests to: >>> >>>> https://github.com/steveblackmon/incubator-streams-master.git >>> >>>> >>> >>>> Referring back to the original list of goals, some are addressed in >>> whole >>> >>>> or in part with this revision, others not just yet. >>> >>>> >>> >>>> ADDRESSED IN WHOLE: >>> >>>> - the website isn't providing any practical guidance *why* or *how* >>> to >>> >>>> use >>> >>>> Streams >>> >>>> - there is no public javadoc or other technical documentation >>> >>>> linked/hosted >>> >>>> - there is a 'wiki (coming soon)' link which never has been >>> activated >>> >>>> >>> >>>> ADDRESSED IN PART: >>> >>>> - its missing concrete examples and recognizable use-cases for >>> which >>> >>>> Streams might >>> >>>> be used, nor comparison against other solutions, and why Streams >>> would >>> >>>> be a >>> >>>> good/better solution >>> >>>> - the Architecture page only has some basic wording but provides no >>> >>>> insight >>> >>>> whatsoever about the concrete implementation and certainly not its >>> >>>> architecture >>> >>>> >>> >>>> This is just a start. Communication on the list has been dead >>> lately, >>> >>>> but >>> >>>> I'm hoping to change that beginning with this effort to explain >>> what is >>> >>>> unique and valuable about Streams, and why. Hopefully this take on >>> the >>> >>>> project will inspire others to get on-board and help out in the >>> coming >>> >>>> months. >>> >>>> >>> >>> >>> >>> +1, good work! >>> >>> >>> >>> >>> >>> >>> >>>> Steve Blackmon >>> >>>> sblack...@apache.org >>> >>>> >>> >>>> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <a...@douma.nu> wrote: >>> >>>> >>> >>>> Hi Steve, >>> >>>>> >>> >>>>> Cool. I don't have much extra input or ideas right now, but >>> looking >>> >>>>> forward to the steps taken you proposed below. >>> >>>>> And about getting out a next release first soon, if it is not >>> >>>>> side-tracking these other steps much, sure +1 >>> >>>>> >>> >>>>> Ate >>> >>>>> >>> >>>>> On 2016-01-05 15:35, Steve Blackmon wrote: >>> >>>>> >>> >>>>> All, >>> >>>>>> >>> >>>>>> Ate gave us a reality check during our board report in December >>> and >>> >>>>>> enumerated some of the most prominent ways our documentation >>> currently >>> >>>>>> falls short - of Apache community standards and of any reasonable >>> >>>>>> useful-ness test. I've responded in-line below - please kick in >>> any >>> >>>>>> ideas >>> >>>>>> you have. >>> >>>>>> >>> >>>>>> >>> >>>>>> - the website isn't providing any practical guidance *why* or >>> *how* to >>> >>>>>> use >>> >>>>>> Streams >>> >>>>>> >>> >>>>>> >>> >>>>>> I agree. This is the content that belongs on the streams website >>> >>>>>> landing >>> >>>>>> page (high-level) or linked from the body (the details). I will >>> take a >>> >>>>>> first shot at this and circulate for feedback. >>> >>>>>> >>> >>>>>> - its missing concrete examples and recognizable use-cases for >>> which >>> >>>>>> Streams might be used, nor comparison against other solutions, >>> and why >>> >>>>>> Streams would be a good/better solution >>> >>>>>> >>> >>>>>> >>> >>>>>> The examples documentation site [1] are meant to serve as both >>> concrete >>> >>>>>> examples and recognizable use-cases, but there are a few problems >>> with >>> >>>>>> the >>> >>>>>> current setup. >>> >>>>>> >>> >>>>>> 1. The examples site lacks plain-language READMEs for the root >>> >>>>>> module >>> >>>>>> and the first children of the root module (the runtime-specific >>> >>>>>> aggregators). >>> >>>>>> 2. The examples site isn't linked from the landing page of the >>> >>>>>> website. >>> >>>>>> 3. There is no permanent url for latest/current. >>> >>>>>> 4. There hasn't been a formal release of this repo. >>> >>>>>> 5. There are broken links within the markdown of some examples. >>> >>>>>> 6. The plain-language descriptions of the examples themselves >>> >>>>>> could be >>> >>>>>> >>> >>>>>> improved. >>> >>>>>> >>> >>>>>> All of these can be addressed (easily I think) and I'm happy to >>> take >>> >>>>>> them >>> >>>>>> on. >>> >>>>>> >>> >>>>>> Additionally, a page (or several) comparing streams to frameworks >>> such >>> >>>>>> as >>> >>>>>> Storm, Spark, Samza, Flink, etc... and describing how they are >>> >>>>>> different/complementary would be valuable. These questions come >>> up a >>> >>>>>> lot. >>> >>>>>> >>> >>>>>> - the Architecture page only has some basic wording but provides >>> no >>> >>>>>> insight >>> >>>>>> whatsoever about the concrete implementation and certainly not >>> its >>> >>>>>> architecture >>> >>>>>> >>> >>>>>> >>> >>>>>> I think this page needs either an multi-page outline format >>> heavily >>> >>>>>> linked >>> >>>>>> into READMEs and interfaces throughout the streams repos, or a >>> system >>> >>>>>> diagram depicting a simple streams deployment alongside the >>> vocabulary. >>> >>>>>> Maybe both as several pages. >>> >>>>>> >>> >>>>>> - there is no public javadoc or other technical documentation >>> >>>>>> linked/hosted >>> >>>>>> >>> >>>>>> >>> >>>>>> We do generate java-docs for releases and snapshots. We should >>> add a >>> >>>>>> link >>> >>>>>> to the root page. We've made progress but there are still many >>> classes >>> >>>>>> lacking any javadoc header. We should reiterate a commitment to >>> fixing >>> >>>>>> that and a schedule for deprecating and deleting any class that >>> doesn't >>> >>>>>> still serve a distinct purpose. >>> >>>>>> >>> >>>>>> An index of the many resource files (json/xml schemas, conversion >>> >>>>>> rules, >>> >>>>>> shell scripts, etc...) published to the project and examples >>> release >>> >>>>>> should >>> >>>>>> also placed on the main website. Ideally comprehensive, but if >>> not at >>> >>>>>> least enough to show a developer that they exist and can be >>> >>>>>> hard-linked to >>> >>>>>> within their own project resources. >>> >>>>>> >>> >>>>>> >>> >>>>>> - there is a 'wiki (coming soon)' link which never has been >>> activated >>> >>>>>> >>> >>>>>> >>> >>>>>> This link (actually most of streams.incubator.apache.org) >>> predates the >>> >>>>>> regular publication of the streams-project maven site where >>> READMEs and >>> >>>>>> other resources are hosted. Personally I'd prefer to see >>> documentation >>> >>>>>> improving within the code-base rather than in a separate wiki in >>> the >>> >>>>>> near-term and suggest we just delete this link. >>> >>>>>> >>> >>>>>> - the mailing list is NOT used for any form of >>> discussion/information >>> >>>>>> other >>> >>>>>> than JIRA and Git(hub) notifications >>> >>>>>> - especially the latter doesn't give a newbie any indication what >>> is >>> >>>>>> going >>> >>>>>> on, nor how to (start) interact >>> >>>>>> >>> >>>>>> This is unfortunate and must change for the community to grow. I >>> know >>> >>>>>> I'm >>> >>>>>> guilty of having ad-hoc off-list discussions with my team that >>> lead to >>> >>>>>> new >>> >>>>>> stories and pull requests. Mea culpa, I resolve to stop doing >>> this in >>> >>>>>> 2016. >>> >>>>>> >>> >>>>>> - there are no blog posts (that I am aware of) around >>> using/trying out >>> >>>>>> Streams either >>> >>>>>> >>> >>>>>> >>> >>>>>> This is a critical missing piece of the puzzle. Static HTML via >>> apache >>> >>>>>> httpd is not the ideal platform for hosting a blog but some >>> other >>> >>>>>> projects >>> >>>>>> have made it work. I'd like us to replace 'Latest News' with a >>> >>>>>> navigable, >>> >>>>>> indexable set of blog posts going forward, where release >>> summaries and >>> >>>>>> new >>> >>>>>> examples can be published, and 'in-the-wild' mentions of the >>> project >>> >>>>>> can >>> >>>>>> be >>> >>>>>> aggregated. >>> >>>>>> >>> >>>>>> >>> >>>>>> And I personally would prefer the dev@ list to be much more / >>> >>>>>> primarily >>> >>>>>> used for actual human interaction, not just these notifications >>> from >>> >>>>>> JIRA >>> >>>>>> and Github. >>> >>>>>> Maybe change the configuration to delegate (some of) these to the >>> >>>>>> commit@ >>> >>>>>> list instead? >>> >>>>>> >>> >>>>>> >>> >>>>>> I agree. AFAICT though com...@streams.incubator.apache.org does >>> not >>> >>>>>> exist. Per http://apache.org/dev/committers.html#mail we should >>> hold >>> >>>>>> a >>> >>>>>> vote to create it. I'll open that vote today. >>> >>>>>> >>> >>>>>> What I suggest is to for a while stop (or largely postpone) >>> working on >>> >>>>>> code >>> >>>>>> but instead working on any/all of the above points first... >>> >>>>>> >>> >>>>>> >>> >>>>>> I agree that making the website helpful is the top project >>> priority. >>> >>>>>> >>> >>>>>> Even working on a next release IMO is far less important than >>> first >>> >>>>>> building up some explanation *why* a release of Stream is needed. >>> >>>>>> >>> >>>>>> >>> >>>>>> I agree that these concerns should be addressed before the next >>> >>>>>> release. >>> >>>>>> The main reason for releasing soon is that per [4] there are 36 >>> issues >>> >>>>>> resolved since the 0.2 release, including quite a few bugs and >>> >>>>>> improvements >>> >>>>>> related to stability, deployment flexibility, or likely to impact >>> the >>> >>>>>> experience of a new developer. >>> >>>>>> >>> >>>>>> [1] >>> >>>>>> >>> >>>>>> >>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/ >>> >>> >>>>>> [2] >>> >>>>>> >>> >>>>>> >>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html >>> >>> >>>>>> [3] >>> >>>>>> >>> >>>>>> >>> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html >>> >>> >>>>>> [4] >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions() >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>> >>> >>>> >>> >>> >>> >> >>> > >>> >>> >> >
