+1 to everything below :)
Good stuff Steve!
On 2016-03-01 23:50, Steve Blackmon wrote:
Following up:
I took a pass at cleaning up the release markdown files and they now look
mostly the same as they did under apache cms.
I also confirmed that site-deploy can write to a sub-directory of
streams.incubator.apache.org using scm:svn capability.
http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
Recall that streams-project and streams-examples each publish schema
artifacts to their maven sites, so maintaining prior versions is important.
streams-master doesn't currently, but since it's a parent of those other
two and prior experience has led to believe we're better off using the same
scheme and setting the streams-master <url/> in the pom to be a sibling of
streams-project and streams-example rather that publishing it to a
different URL entirely. This is why the version an artifactId are being
included in the above url.
Brings up two issues -
resolving the latest release and/or snapshot version at any given time
ensuring a visitor to streams.incubator.apache.org lands on the
streams-master site for that version
While symlinks are a viable way to resolving the latest release and/or
snapshot version while keeping previous release versions around
i.e. forwarding
http://streams.incubator.apache.org/site/latest/streams-master/index.html
to
http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
they won't help us forward http://streams.incubator.apache.org to a
subdirectory of itself.
Good news is apache cms supports .htaccess and the following snippet can.
RewriteEngine *on*
Redirect /
http://streams.incubator.apache.org/site/latest/streams-master/index.html
Feel free to confirm by visiting streams.staging.incubator.org (you'll be
redirected to the output of the maven site-deploy)
*I ask that everyone on the list review and send me any concerns or
feedback they want incorporated before this goes live on
streams.incubator.apache.org <http://streams.incubator.apache.org>.*
Procedurally, the way I expect that to happen is:
- I'll submit a PR to streams-master/master
- That PR will merge
- Update the streams-master jenkins job to run site-deploy, and do the
needful to get it the necessary credentials
- Going forward all builds of streams-master/master will publish to
site/${version}/streams-master
- Add a few more instructions to jenkins to maintain the symlinks and
.htaccess
- If there are any manual steps required to oversee, document that on the
site itself.
Then, we'll move on to hooking streams-project and streams-examples up to
this new site (as children with an integrated breadcrumb) and to jenkins CD.
Steve Blackmon
sblack...@apache.org
On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblack...@apache.org>
wrote:
One thing which I think can 'spice up' the overview page a bit more would
be a
diagram providing some very high-level interaction between data sources
and
destinations and the streams role between them.
I agree - a diagram depicting streams mediating between sources and
destination on the landing page is appropriate. I'll work on this soon.
And before replacing the current site several existing pages need to be
moved to
the new one. But once those are done I think this is great improvement.
I've pushed a new version (to the source and site links from my prior
message) that brings most of the existing content along, using the
reporting-info plugin to generate the HTML where possible.
On a few of the pages - SCM, CI, and Dependency Info - the output of the
maven plugin simply isn't adequate for a multi-repository project such as
this, so those are now managed within src/site/markdown
Release Setup and Release Process need some cleanup but all of the content
has been migrated.
Other report-info pages - such as Dependencies, Dependency Management,
Dependency Convergence, JavaDocs, Test JavaDocs, - don't make sense at the
streams-master level but do exist and are useful within streams-project,
streams-examples, and many of their respective sub-modules.
In theory those additional report items will show up when navigating into
the site hierarchy of the sub-projects (links to each sub-project are
present along the header), which will inherit the structure, look-and-feel,
and breadcrumb of this site revision. Naturally these changes must be
deployed to streams.incubator.apache.org and sub-project pom.xml and
site_en.xml will need some tweaking but I'm optimistic it will all fit
together nicely.
Steve Blackmon
sblack...@apache.org
On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <a...@douma.nu> wrote:
On 2016-02-25 23:50, Steve Blackmon wrote:
Hello streamers,
I finally consolidated some materials I've been working on into a
nice-looking thing that is a viable candidate to replace the current
streams web page. Please take a look and let me know what you think.
Sometime
soon I expect to submit a vote to take down the previous site and replace
it with something derived from this version.
The proposed new site is temporarily being hosted at:
http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html
I placed all of the content into markdown files which are checked into a
branch of streams-master. This approach allows us to author the site
using
markdown, package and deploy it with maven-site, and manage changes using
git.
This looks cool!
I like the new overview and faq pages, definitely more concrete about
Streams.
One thing which I think can 'spice up' the overview page a bit more would
be a
diagram providing some very high-level interaction between data sources
and
destinations and the streams role between them.
And before replacing the current site several existing pages need to be
moved to
the new one. But once those are done I think this is great improvement.
If you'd like to submit specific changes feel free to do so as
pull-requests to:
https://github.com/steveblackmon/incubator-streams-master.git
Referring back to the original list of goals, some are addressed in whole
or in part with this revision, others not just yet.
ADDRESSED IN WHOLE:
- the website isn't providing any practical guidance *why* or *how* to
use
Streams
- there is no public javadoc or other technical documentation
linked/hosted
- there is a 'wiki (coming soon)' link which never has been activated
ADDRESSED IN PART:
- its missing concrete examples and recognizable use-cases for which
Streams might
be used, nor comparison against other solutions, and why Streams would
be a
good/better solution
- the Architecture page only has some basic wording but provides no
insight
whatsoever about the concrete implementation and certainly not its
architecture
This is just a start. Communication on the list has been dead lately,
but
I'm hoping to change that beginning with this effort to explain what is
unique and valuable about Streams, and why. Hopefully this take on the
project will inspire others to get on-board and help out in the coming
months.
+1, good work!
Steve Blackmon
sblack...@apache.org
On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <a...@douma.nu> wrote:
Hi Steve,
Cool. I don't have much extra input or ideas right now, but looking
forward to the steps taken you proposed below.
And about getting out a next release first soon, if it is not
side-tracking these other steps much, sure +1
Ate
On 2016-01-05 15:35, Steve Blackmon wrote:
All,
Ate gave us a reality check during our board report in December and
enumerated some of the most prominent ways our documentation currently
falls short - of Apache community standards and of any reasonable
useful-ness test. I've responded in-line below - please kick in any
ideas
you have.
- the website isn't providing any practical guidance *why* or *how* to
use
Streams
I agree. This is the content that belongs on the streams website
landing
page (high-level) or linked from the body (the details). I will take a
first shot at this and circulate for feedback.
- its missing concrete examples and recognizable use-cases for which
Streams might be used, nor comparison against other solutions, and why
Streams would be a good/better solution
The examples documentation site [1] are meant to serve as both concrete
examples and recognizable use-cases, but there are a few problems with
the
current setup.
1. The examples site lacks plain-language READMEs for the root
module
and the first children of the root module (the runtime-specific
aggregators).
2. The examples site isn't linked from the landing page of the
website.
3. There is no permanent url for latest/current.
4. There hasn't been a formal release of this repo.
5. There are broken links within the markdown of some examples.
6. The plain-language descriptions of the examples themselves
could be
improved.
All of these can be addressed (easily I think) and I'm happy to take
them
on.
Additionally, a page (or several) comparing streams to frameworks such
as
Storm, Spark, Samza, Flink, etc... and describing how they are
different/complementary would be valuable. These questions come up a
lot.
- the Architecture page only has some basic wording but provides no
insight
whatsoever about the concrete implementation and certainly not its
architecture
I think this page needs either an multi-page outline format heavily
linked
into READMEs and interfaces throughout the streams repos, or a system
diagram depicting a simple streams deployment alongside the vocabulary.
Maybe both as several pages.
- there is no public javadoc or other technical documentation
linked/hosted
We do generate java-docs for releases and snapshots. We should add a
link
to the root page. We've made progress but there are still many classes
lacking any javadoc header. We should reiterate a commitment to fixing
that and a schedule for deprecating and deleting any class that doesn't
still serve a distinct purpose.
An index of the many resource files (json/xml schemas, conversion
rules,
shell scripts, etc...) published to the project and examples release
should
also placed on the main website. Ideally comprehensive, but if not at
least enough to show a developer that they exist and can be
hard-linked to
within their own project resources.
- there is a 'wiki (coming soon)' link which never has been activated
This link (actually most of streams.incubator.apache.org) predates the
regular publication of the streams-project maven site where READMEs and
other resources are hosted. Personally I'd prefer to see documentation
improving within the code-base rather than in a separate wiki in the
near-term and suggest we just delete this link.
- the mailing list is NOT used for any form of discussion/information
other
than JIRA and Git(hub) notifications
- especially the latter doesn't give a newbie any indication what is
going
on, nor how to (start) interact
This is unfortunate and must change for the community to grow. I know
I'm
guilty of having ad-hoc off-list discussions with my team that lead to
new
stories and pull requests. Mea culpa, I resolve to stop doing this in
2016.
- there are no blog posts (that I am aware of) around using/trying out
Streams either
This is a critical missing piece of the puzzle. Static HTML via apache
httpd is not the ideal platform for hosting a blog but some other
projects
have made it work. I'd like us to replace 'Latest News' with a
navigable,
indexable set of blog posts going forward, where release summaries and
new
examples can be published, and 'in-the-wild' mentions of the project
can
be
aggregated.
And I personally would prefer the dev@ list to be much more /
primarily
used for actual human interaction, not just these notifications from
JIRA
and Github.
Maybe change the configuration to delegate (some of) these to the
commit@
list instead?
I agree. AFAICT though com...@streams.incubator.apache.org does not
exist. Per http://apache.org/dev/committers.html#mail we should hold
a
vote to create it. I'll open that vote today.
What I suggest is to for a while stop (or largely postpone) working on
code
but instead working on any/all of the above points first...
I agree that making the website helpful is the top project priority.
Even working on a next release IMO is far less important than first
building up some explanation *why* a release of Stream is needed.
I agree that these concerns should be addressed before the next
release.
The main reason for releasing soon is that per [4] there are 36 issues
resolved since the 0.2 release, including quite a few bugs and
improvements
related to stability, deployment flexibility, or likely to impact the
experience of a new developer.
[1]
http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/
[2]
http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html
[3]
http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
[4]
https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions()
