Following up: I took a pass at cleaning up the release markdown files and they now look mostly the same as they did under apache cms.
I also confirmed that site-deploy can write to a sub-directory of streams.incubator.apache.org using scm:svn capability. http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html Recall that streams-project and streams-examples each publish schema artifacts to their maven sites, so maintaining prior versions is important. streams-master doesn't currently, but since it's a parent of those other two and prior experience has led to believe we're better off using the same scheme and setting the streams-master <url/> in the pom to be a sibling of streams-project and streams-example rather that publishing it to a different URL entirely. This is why the version an artifactId are being included in the above url. Brings up two issues - resolving the latest release and/or snapshot version at any given time ensuring a visitor to streams.incubator.apache.org lands on the streams-master site for that version While symlinks are a viable way to resolving the latest release and/or snapshot version while keeping previous release versions around i.e. forwarding http://streams.incubator.apache.org/site/latest/streams-master/index.html to http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html they won't help us forward http://streams.incubator.apache.org to a subdirectory of itself. Good news is apache cms supports .htaccess and the following snippet can. RewriteEngine *on* Redirect / http://streams.incubator.apache.org/site/latest/streams-master/index.html Feel free to confirm by visiting streams.staging.incubator.org (you'll be redirected to the output of the maven site-deploy) *I ask that everyone on the list review and send me any concerns or feedback they want incorporated before this goes live on streams.incubator.apache.org <http://streams.incubator.apache.org>.* Procedurally, the way I expect that to happen is: - I'll submit a PR to streams-master/master - That PR will merge - Update the streams-master jenkins job to run site-deploy, and do the needful to get it the necessary credentials - Going forward all builds of streams-master/master will publish to site/${version}/streams-master - Add a few more instructions to jenkins to maintain the symlinks and .htaccess - If there are any manual steps required to oversee, document that on the site itself. Then, we'll move on to hooking streams-project and streams-examples up to this new site (as children with an integrated breadcrumb) and to jenkins CD. Steve Blackmon sblack...@apache.org On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblack...@apache.org> wrote: > One thing which I think can 'spice up' the overview page a bit more would >> be a >> diagram providing some very high-level interaction between data sources >> and >> destinations and the streams role between them. > > > I agree - a diagram depicting streams mediating between sources and > destination on the landing page is appropriate. I'll work on this soon. > > And before replacing the current site several existing pages need to be >> moved to >> the new one. But once those are done I think this is great improvement. > > > I've pushed a new version (to the source and site links from my prior > message) that brings most of the existing content along, using the > reporting-info plugin to generate the HTML where possible. > > On a few of the pages - SCM, CI, and Dependency Info - the output of the > maven plugin simply isn't adequate for a multi-repository project such as > this, so those are now managed within src/site/markdown > > Release Setup and Release Process need some cleanup but all of the content > has been migrated. > > Other report-info pages - such as Dependencies, Dependency Management, > Dependency Convergence, JavaDocs, Test JavaDocs, - don't make sense at the > streams-master level but do exist and are useful within streams-project, > streams-examples, and many of their respective sub-modules. > > In theory those additional report items will show up when navigating into > the site hierarchy of the sub-projects (links to each sub-project are > present along the header), which will inherit the structure, look-and-feel, > and breadcrumb of this site revision. Naturally these changes must be > deployed to streams.incubator.apache.org and sub-project pom.xml and > site_en.xml will need some tweaking but I'm optimistic it will all fit > together nicely. > > Steve Blackmon > sblack...@apache.org > > On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <a...@douma.nu> wrote: > >> On 2016-02-25 23:50, Steve Blackmon wrote: >> >>> Hello streamers, >>> >>> I finally consolidated some materials I've been working on into a >>> nice-looking thing that is a viable candidate to replace the current >>> streams web page. Please take a look and let me know what you think. >>> Sometime >>> soon I expect to submit a vote to take down the previous site and replace >>> it with something derived from this version. >>> >>> The proposed new site is temporarily being hosted at: >>> >>> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html >>> >>> I placed all of the content into markdown files which are checked into a >>> branch of streams-master. This approach allows us to author the site >>> using >>> markdown, package and deploy it with maven-site, and manage changes using >>> git. >>> >> >> This looks cool! >> >> I like the new overview and faq pages, definitely more concrete about >> Streams. >> >> One thing which I think can 'spice up' the overview page a bit more would >> be a >> diagram providing some very high-level interaction between data sources >> and >> destinations and the streams role between them. >> >> And before replacing the current site several existing pages need to be >> moved to >> the new one. But once those are done I think this is great improvement. >> >> >>> If you'd like to submit specific changes feel free to do so as >>> pull-requests to: >>> https://github.com/steveblackmon/incubator-streams-master.git >>> >>> Referring back to the original list of goals, some are addressed in whole >>> or in part with this revision, others not just yet. >>> >>> ADDRESSED IN WHOLE: >>> - the website isn't providing any practical guidance *why* or *how* to >>> use >>> Streams >>> - there is no public javadoc or other technical documentation >>> linked/hosted >>> - there is a 'wiki (coming soon)' link which never has been activated >>> >>> ADDRESSED IN PART: >>> - its missing concrete examples and recognizable use-cases for which >>> Streams might >>> be used, nor comparison against other solutions, and why Streams would >>> be a >>> good/better solution >>> - the Architecture page only has some basic wording but provides no >>> insight >>> whatsoever about the concrete implementation and certainly not its >>> architecture >>> >>> This is just a start. Communication on the list has been dead lately, >>> but >>> I'm hoping to change that beginning with this effort to explain what is >>> unique and valuable about Streams, and why. Hopefully this take on the >>> project will inspire others to get on-board and help out in the coming >>> months. >>> >> >> +1, good work! >> >> >> >>> Steve Blackmon >>> sblack...@apache.org >>> >>> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <a...@douma.nu> wrote: >>> >>> Hi Steve, >>>> >>>> Cool. I don't have much extra input or ideas right now, but looking >>>> forward to the steps taken you proposed below. >>>> And about getting out a next release first soon, if it is not >>>> side-tracking these other steps much, sure +1 >>>> >>>> Ate >>>> >>>> On 2016-01-05 15:35, Steve Blackmon wrote: >>>> >>>> All, >>>>> >>>>> Ate gave us a reality check during our board report in December and >>>>> enumerated some of the most prominent ways our documentation currently >>>>> falls short - of Apache community standards and of any reasonable >>>>> useful-ness test. I've responded in-line below - please kick in any >>>>> ideas >>>>> you have. >>>>> >>>>> >>>>> - the website isn't providing any practical guidance *why* or *how* to >>>>> use >>>>> Streams >>>>> >>>>> >>>>> I agree. This is the content that belongs on the streams website >>>>> landing >>>>> page (high-level) or linked from the body (the details). I will take a >>>>> first shot at this and circulate for feedback. >>>>> >>>>> - its missing concrete examples and recognizable use-cases for which >>>>> Streams might be used, nor comparison against other solutions, and why >>>>> Streams would be a good/better solution >>>>> >>>>> >>>>> The examples documentation site [1] are meant to serve as both concrete >>>>> examples and recognizable use-cases, but there are a few problems with >>>>> the >>>>> current setup. >>>>> >>>>> 1. The examples site lacks plain-language READMEs for the root >>>>> module >>>>> and the first children of the root module (the runtime-specific >>>>> aggregators). >>>>> 2. The examples site isn't linked from the landing page of the >>>>> website. >>>>> 3. There is no permanent url for latest/current. >>>>> 4. There hasn't been a formal release of this repo. >>>>> 5. There are broken links within the markdown of some examples. >>>>> 6. The plain-language descriptions of the examples themselves >>>>> could be >>>>> >>>>> improved. >>>>> >>>>> All of these can be addressed (easily I think) and I'm happy to take >>>>> them >>>>> on. >>>>> >>>>> Additionally, a page (or several) comparing streams to frameworks such >>>>> as >>>>> Storm, Spark, Samza, Flink, etc... and describing how they are >>>>> different/complementary would be valuable. These questions come up a >>>>> lot. >>>>> >>>>> - the Architecture page only has some basic wording but provides no >>>>> insight >>>>> whatsoever about the concrete implementation and certainly not its >>>>> architecture >>>>> >>>>> >>>>> I think this page needs either an multi-page outline format heavily >>>>> linked >>>>> into READMEs and interfaces throughout the streams repos, or a system >>>>> diagram depicting a simple streams deployment alongside the vocabulary. >>>>> Maybe both as several pages. >>>>> >>>>> - there is no public javadoc or other technical documentation >>>>> linked/hosted >>>>> >>>>> >>>>> We do generate java-docs for releases and snapshots. We should add a >>>>> link >>>>> to the root page. We've made progress but there are still many classes >>>>> lacking any javadoc header. We should reiterate a commitment to fixing >>>>> that and a schedule for deprecating and deleting any class that doesn't >>>>> still serve a distinct purpose. >>>>> >>>>> An index of the many resource files (json/xml schemas, conversion >>>>> rules, >>>>> shell scripts, etc...) published to the project and examples release >>>>> should >>>>> also placed on the main website. Ideally comprehensive, but if not at >>>>> least enough to show a developer that they exist and can be >>>>> hard-linked to >>>>> within their own project resources. >>>>> >>>>> >>>>> - there is a 'wiki (coming soon)' link which never has been activated >>>>> >>>>> >>>>> This link (actually most of streams.incubator.apache.org) predates the >>>>> regular publication of the streams-project maven site where READMEs and >>>>> other resources are hosted. Personally I'd prefer to see documentation >>>>> improving within the code-base rather than in a separate wiki in the >>>>> near-term and suggest we just delete this link. >>>>> >>>>> - the mailing list is NOT used for any form of discussion/information >>>>> other >>>>> than JIRA and Git(hub) notifications >>>>> - especially the latter doesn't give a newbie any indication what is >>>>> going >>>>> on, nor how to (start) interact >>>>> >>>>> This is unfortunate and must change for the community to grow. I know >>>>> I'm >>>>> guilty of having ad-hoc off-list discussions with my team that lead to >>>>> new >>>>> stories and pull requests. Mea culpa, I resolve to stop doing this in >>>>> 2016. >>>>> >>>>> - there are no blog posts (that I am aware of) around using/trying out >>>>> Streams either >>>>> >>>>> >>>>> This is a critical missing piece of the puzzle. Static HTML via apache >>>>> httpd is not the ideal platform for hosting a blog but some other >>>>> projects >>>>> have made it work. I'd like us to replace 'Latest News' with a >>>>> navigable, >>>>> indexable set of blog posts going forward, where release summaries and >>>>> new >>>>> examples can be published, and 'in-the-wild' mentions of the project >>>>> can >>>>> be >>>>> aggregated. >>>>> >>>>> >>>>> And I personally would prefer the dev@ list to be much more / >>>>> primarily >>>>> used for actual human interaction, not just these notifications from >>>>> JIRA >>>>> and Github. >>>>> Maybe change the configuration to delegate (some of) these to the >>>>> commit@ >>>>> list instead? >>>>> >>>>> >>>>> I agree. AFAICT though com...@streams.incubator.apache.org does not >>>>> exist. Per http://apache.org/dev/committers.html#mail we should hold >>>>> a >>>>> vote to create it. I'll open that vote today. >>>>> >>>>> What I suggest is to for a while stop (or largely postpone) working on >>>>> code >>>>> but instead working on any/all of the above points first... >>>>> >>>>> >>>>> I agree that making the website helpful is the top project priority. >>>>> >>>>> Even working on a next release IMO is far less important than first >>>>> building up some explanation *why* a release of Stream is needed. >>>>> >>>>> >>>>> I agree that these concerns should be addressed before the next >>>>> release. >>>>> The main reason for releasing soon is that per [4] there are 36 issues >>>>> resolved since the 0.2 release, including quite a few bugs and >>>>> improvements >>>>> related to stability, deployment flexibility, or likely to impact the >>>>> experience of a new developer. >>>>> >>>>> [1] >>>>> >>>>> >>>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/ >>>>> [2] >>>>> >>>>> >>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html >>>>> [3] >>>>> >>>>> >>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html >>>>> [4] >>>>> >>>>> >>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions() >>>>> >>>>> >>>>> >>>> >>> >> >
