On Wed, May 29, 2024 at 1:01 PM Robert Burke <rob...@frantil.com> wrote: > > Honestly, it's less about how we propose things and more to do with actually > making "completing" the documentation work to some permanent place. > > That can be as simple as "migrate the final implemented design to a markdown > file in the GitHub repo at *foo* or published on the beam website at *bar*. > > The big issue is backfilling our existing set of proposals.
I actually don't think there's a big difference between past proposals and future ones--unless we can make the process really easy (aka essentially automated) I think the motivation to manually migrate will be low in the future too (and note that the doc remains a living doc generally until a short while after its implementation). The "two sources of truth" doesn't work well until it's static and ready to be archived, at which point additional work is harder to motivate. > I do like the google docs for commentary and dispersed discussions, but it's > not a long term solution (per the initial message in this thread). I'm a big fan of markdown in a repo for long-term storage. Anyone know of any good google docs -> markdown converters? That might be harder to do if there are images (and of course comment threads, etc. would be less likely to be preserved). Even snapshotting them as PDF or HTML might not be that bad, if the primary purpose is for archiving. > On Wed, May 29, 2024, 11:56 AM Robert Bradshaw via dev <dev@beam.apache.org> > wrote: >> >> While I'm not opposed to having a more formal process for proposals to >> go from idea to consensus to implementation, I'm not sure how much >> this would solve the primary issues we face (discoverability and >> durability). But maybe that could be built into the process? At the >> very least we could have an "index" which would give identifiers (and >> hopefully good titles) to all the proposals, and maybe have an offline >> process to snapshot such docs (even just periodically pulling the >> content to a repo like I do with >> https://github.com/cython/cython-issues ). I have yet to find a medium >> (not even wikis) that facilitates conversation/collaborative editing >> to the extent that docs does, but I agree with the downside that >> ownership by random individuals can pose a problem. >> >> On Wed, May 29, 2024 at 7:07 AM Jan Lukavský <je...@seznam.cz> wrote: >> > >> > Hi, >> > >> > regarding changing the way we document past (and more importantly >> > future) changes, I've always been a big fan of the FLIP analogy [1]. I >> > would love if we could make this work for Beam as well, while preserving >> > the 'informal' part that I believe all of us want to keep. On the other >> > hand, this could make the design decisions more searchable, transparent >> > and get more people involved in the process. Having design documents >> > durable is of course a highly important part of it. >> > >> > Jan >> > >> > [1] https://lists.apache.org/thread/whfy3706w2d0q6rdk4kwyrzvhfd4b5kg >> > >> > On 5/29/24 15:04, Kenneth Knowles wrote: >> > > Hi all, >> > > >> > > Yesterday someone asked me about the design doc linked from >> > > https://github.com/apache/beam/issues/18297 because it is now a 404. >> > > >> > > There are plenty of reasons a Google Doc might no longer be >> > > accessible. They exist outside the project's control. This is part of >> > > why ASF projects emphasize having discussions on the dev@ list and >> > > often put all their designs directly onto some ASF-hosted >> > > infrastructure, such as a Wiki (Example: >> > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals). >> > > In the early days we had a project-owned shared folder but it fell >> > > into disuse. >> > > >> > > In my opinion, Google Docs are still the best place for design docs to >> > > get feedback and be revised, but the lack of durability is a downside >> > > to stay aware of. I've also gotten lots of complaints of lack of >> > > discoverability and lack of systematization of design docs, neither of >> > > which would be addressed by a shared folder. >> > > >> > > I don't have a proposal or suggestion. I don't think this is super >> > > urgent, certainly not my personal highest priority, but I thought I'd >> > > just share this as food for thought. >> > > >> > > Kenn
