I wonder if we could make it easier to maintain the page? If design docs were in the Wiki, it would be trivial to maintain such a list. Obviously the Google Docs are great for commenting/collaboration, but they are hard to discover.
On 25.01.19 18:37, Alexey Romanenko wrote:
Hi Alex,Indeed, some of them are not shared with everyone. I asked on Slack channel to fix this.Thank you for the list!On 18 Jan 2019, at 19:57, Alex Van Boxel <a...@vanboxel.be <mailto:a...@vanboxel.be>> wrote:typically me... I just click on 2 of the 3 that were not shared. I went over all of the proposals to see where I needed to get access, here is the list:SQL / Schema * Pubsub to Beam SQL [doc <https://docs.google.com/document/d/1wIXTxh-nQ3u694XbF0iEZX_7-b3yi4ad0ML2pcAxYfE>] * Calcite/Beam SQL Windowing [doc <https://docs.google.com/document/d/1RmyV9e1Qab-axsLI1WWpw5oGAJDv0X7y9OSnPnrZWJk>] * Reject Unsupported Windowing Strategies in JOIN [doc <https://docs.google.com/document/d/1V-ZgKVTwHdNSGlQWncWIzcf_Rw2oLKZFSkU43scLff4>] the ones that I had access to, from the past I don't know anymore of course. _/ _/ Alex Van BoxelOn Fri, Jan 18, 2019 at 6:13 PM Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote:Hi Alex, Hmm, afaik, this is mostly google docs file which shared with anyone who knows the link. Could you send here the names of proposals that required an access approval? Thanks.On 18 Jan 2019, at 16:58, Alex Van Boxel <a...@vanboxel.be <mailto:a...@vanboxel.be>> wrote: Hey Alexey, I see that a lot (well, I tried 2) proposals require access approval. Should that be the case? _/ _/ Alex Van Boxel On Fri, Jan 18, 2019 at 4:51 PM Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote: I’m sorry but I forgot to mention that the whole list could be found here: https://beam.apache.org/contribute/design-documents/On 18 Jan 2019, at 16:49, Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote: FYI: I updated the list of design documents to make it up-to-date. PR: https://github.com/apache/beam/pull/7560 Please, feel free to add new ones if I missed something. Also, I’d like to remind that it would be very helpful to add design document to this page in the same time as it was created and finalised after all discussions. It should help us to keep this list consistent and always up-to-date. Thank you in advance! On 11 Jul 2018, at 17:03, Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote:Thank you for this link, Etienne. I agree that it doesn’t fit well for design documents page. So, I think it makes sense to add either on wiki or as a part of Nexmark documentation on web site: https://beam.apache.org/documentation/sdks/java/nexmark/. On 11 Jul 2018, at 11:16, Etienne Chauchot <echauc...@apache.org <mailto:echauc...@apache.org>> wrote:Alexey, One doc that can be interesting that I forgot to point out is https://docs.google.com/document/d/1VgnGiVu8vSfm7Et-xAtQYv0PlEpqeyfmhpQUNPmWRJs/edit?usp=sharing It is the doc I wrote when I submitted Nexmark PR to ease the reading of the code. It is not a design doc, I don't know if it belongs to the website page or to the wiki for beam devs. Etienne Le mercredi 06 juin 2018 à 17:48 +0200, Alexey Romanenko a écrit :FYI: Finally, it was merged and you can find this page here: https://beam.apache.org/contribute/design-documents/ Thank you everybody who helped me to compile this list! I’ll try to do my best to update this with new coming docs. In the same time, please, feel free to add your new docs (or notify me if I missed this) once they are finished and ready to be published. WBR, AlexeyOn 31 May 2018, at 18:52, Eugene Kirpichov <kirpic...@google.com <mailto:kirpic...@google.com>> wrote: Thank you! On Thu, May 31, 2018 at 8:30 AM Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote:Thank you everybody for provided links. I collected all of them (please, correct me if I missed something), categorized and created a dedicated page for Beam website. Here is a PR for that (please, review): https://github.com/apache/beam-site/pull/456 WBR, AlexeyOn 30 May 2018, at 13:17, Łukasz Gajowy <lukasz.gaj...@gmail.com <mailto:lukasz.gaj...@gmail.com>> wrote: Hi, I just wanted to add those two (sorry for being kinda late with this): https://docs.google.com/document/d/1dA-5s6OHiP_cz-NRAbwapoKF5MEC1wKps4A5tFbIPKE/edit?usp=sharing https://docs.google.com/document/d/1Cb7XVmqe__nA_WCrriAifL-3WCzbZzV4Am5W_SkQLeA/edit?usp=sharing Thanks, Łukasz 2018-05-29 22:42 GMT+02:00 Lukasz Cwik <lc...@google.com <mailto:lc...@google.com>>:Providing ownership to the PMC account allows others to take over ownership of the document once a contributor stops being active. This allows docs to be updated (even if just to point to a newer doc). On Tue, May 29, 2018 at 1:20 PM Kenneth Knowles <k...@google.com <mailto:k...@google.com>> wrote:My position on ownership is design docs are really documents "of the moment" and authored by a particular individual or group. Experience shows that even if you try, keeping it fresh is not likely to happen. Anything that needs freshness (like end-user docs) should be in a different medium. I would just date the gdoc so readers know how to interpret it (the automated "last edit" date is not sufficient for understanding how stale something is). So it seems like it makes little difference if the project or PMC has ownership or even write access. Of course I have no objections if someone wants to transfer ownership, but is there a reason to encourage it? Kenn On Tue, May 29, 2018 at 1:11 PM Lukasz Cwik <lc...@google.com <mailto:lc...@google.com>> wrote:I transferred ownership of the docs that I owned to the apacheb...@gmail.com <mailto:apacheb...@gmail.com> PMC account and put the ones that I owned into the drive folder. Would it be a good idea for others to follow suit? Instructions on how to transfer ownership are here: http://support.it.mtu.edu/Accounts/E-Mail/75946047/How-do-I-transfer-ownership-of-a-Google-Doc.htm On Tue, May 29, 2018 at 11:23 AM Lukasz Cwik <lc...@google.com <mailto:lc...@google.com>> wrote:I created a PR for the beam-site to link to the design docs and template from the contribution guide: https://github.com/apache/beam-site/pull/454 On Fri, May 25, 2018 at 10:23 AM Lukasz Cwik <lc...@google.com <mailto:lc...@google.com>> wrote:Here are some more links related to portability efforts: https://s.apache.org/beam-fn-api https://s.apache.org/beam-fn-api-processing-a-bundle https://s.apache.org/beam-fn-api-send-and-receive-data https://s.apache.org/beam-fn-state-api-and-bundle-processing https://s.apache.org/beam-fn-api-progress-reporting https://s.apache.org/beam-fn-api-container-contract https://s.apache.org/beam-breaking-fusion https://s.apache.org/beam-runner-api-combine-model https://s.apache.org/beam-fn-api-metrics On Thu, May 24, 2018 at 2:11 PM Scott Wegner <sweg...@google.com <mailto:sweg...@google.com>> wrote:Thanks for sharing these. I also put together a design doc template based on common styling / sections I saw in the docs listed above. Others are free to use it as they'd like. https://docs.google.com/document/d/1kVePqjt2daZd0bQHGUwghlcLbhvrny7VpflAzk9sjUg/edit?usp=sharing On Thu, May 24, 2018 at 6:23 AM Kenneth Knowles <k...@google.com <mailto:k...@google.com>> wrote:OK, I will also put a list here of those I know off the top of my head. Some are redundant with Etienne's but short links that I can think of: https://s.apache.org/a-new-dofn https://s.apache.org/beam-triggers https://s.apache.org/beam-sink-triggers https://s.apache.org/beam-runner-composites https://s.apache.org/beam-lateness https://s.apache.org/beam-runner-api https://s.apache.org/beam-state https://s.apache.org/beam-side-inputs-1-pager Kenn On Thu, May 24, 2018 at 6:08 AM Etienne Chauchot <echauc...@apache.org <mailto:echauc...@apache.org>> wrote:Great that you take this action Alexey ! Here are the links I have, there is duplicates with the ones you already received and maybe old docs as well: https://docs.google.com/document/d/1MtBZYV7NAcfbwyy9Op8STeFNBxtljxgy69FkHMvhTMA/edit https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit https://docs.google.com/document/d/1Fl_LM918j7ZxAmCSkm43GBjV8knsZAIA1tRhvJ4DneM/edit#heading=h.1lcfuwfvxg2 https://docs.google.com/document/d/1zcF4ZGtq8pxzLZxgD_JMWAouSszIf9LnFANWHKBsZlg/edit https://docs.google.com/document/d/1AQmx-T9XjSi1PNoEp5_L-lT0j7BkgTbmQnc6uFEMI4c/edit#heading=h.dtl8cwoybr2y https://docs.google.com/document/d/1u-4o_0uj8uKa2SVNPBNxIKfvcJ4t66ecCoU1M2yVoDA/edit#heading=h.c1deqkr0bp31 https://docs.google.com/document/d/17H2sBEtnoTSxjzlrz7rmKtX5E3F0mW1NpFQzWzSYOpY/edit#heading=h.1lcfuwfvxg2 https://docs.google.com/document/d/1tnG2DPHZYbsomvihIpXruUmQ12pHGK0QIvXS1FOTgRc/edit#heading=h.puuotbien1gf https://docs.google.com/document/d/1KEtUPE5d6oCZFvFlR7PtSDvHpx5kDj2dTlIxF7UXa5w/edit#heading=h.p6lvszfbmyj6 https://docs.google.com/document/d/1IGduUqmhWDi_69l9nG8kw73HZ5WI5wOps9Tshl5wpQA/edit#heading=h.gh88g5y0rekp https://docs.google.com/document/d/1NehB__CxJTmjxkt7hDYIJWNDLJyEoE0b-AUJvInHPCI/edit#heading=h.gh88g5y0rekp https://docs.google.com/document/d/1BOozW0bzBuz4oHJEuZNDOHdzaV5Y56ix58Ozrqm2jFg/edit#heading=h.gh88g5y0rekp https://docs.google.com/document/d/1XYzb1Fnt2sam7u2MsGFaZp-2qSIGxUn66VLer-bcXAk/edit#heading=h.p6lvszfbmyj6 https://docs.google.com/document/d/1voyUIQ2DrWkoY-BsJwM8YvF4gGKB76CDG8BYL8XBc7A/edit#heading=h.vv2fbulkp7t Etienne Le jeudi 24 mai 2018 à 14:53 +0200, Alexey Romanenko a écrit :Thank you everybody for positive feedback and sending me links to design docs. I’m going to complete the full list and create PR for review. *Griselda* Perhaps, it was a bit of misunderstanding here, let me explain what I assume under creating of such page. Actually, for now I’m going to collect all known design docs under one umbrella (the page on Beam web site) to make it easy to discover. For example, as a new contributor, I’d like to know more details what was initial design of new FileIO, SDF, Metrics, etc. I'm _not_ going to copy the content of this docs to one page or even web site, let’s keep this as it is, no changes here for the moment. I think, moving to something else than Google docs is a tough question and requires another discussion. So, in this case, this task seems not so hard since we don’t add such docs too often - I'll just have to update this index page on web site. In addition, the authors will be always welcomed to update this page by themselves. In my turn, I’ll try to keep an eye on this to keep it synced. And of course, any help will be welcomed too =) WBR, AlexeyOn 24 May 2018, at 00:01, Griselda Cuevas <g...@google.com <mailto:g...@google.com>> wrote: Hi Everyone, @Alexey, I think this is a great idea, I'd like to understand more of the motivation behind having all the designs doc under a single page. In my opinion it could become a challenge to maintain a page, so knowing what you want to accomplish could help us think of alternative solutions? On Wed, 23 May 2018 at 14:08, Daniel Oliveira <danolive...@google.com <mailto:danolive...@google.com>> wrote:+1 to web site page (not Google Doc). Definitely agree that a common entry point would be excellent. I don't like the idea of the Google Doc so much because it's not very good for having changes reviewed and keeping track of who added what, unlike Github. Adding an entry to the list in the website would require reviews and leave behind a commit history, which I think is important for an authoritative source like this. PS: I also have a doc I proposed that I didn't see in the lists: https://s.apache.org/beam-runner-api-combine-model On Wed, May 23, 2018 at 12:52 PM Lukasz Cwik <lc...@google.com <mailto:lc...@google.com>> wrote:+1, Thanks for picking this up Alexey On Wed, May 23, 2018 at 10:41 AM Huygaa Batsaikhan <bat...@google.com <mailto:bat...@google.com>> wrote:+1. That is great, Alexey. Robin and I are working on documenting some missing pieces of Java SDK. We will let you know when we create polished documents. On Wed, May 23, 2018 at 9:28 AM Ismaël Mejía <ieme...@gmail.com <mailto:ieme...@gmail.com>> wrote:+1 and thanks for volunteering for this Alexey. We really need to make this more accesible. On Wed, May 23, 2018 at 6:00 PM Alexey Romanenko <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote: > Joseph, Eugene - thank you very much for the links! > All, regarding one common entry point for all design documents. Could we just have a dedicated page on Beam web site with a list of links to every proposed document? Every entry (optionally) might contain, in addition, short abstract and list of author(s). In this case, it would be easily searchable and available for those who are interested in this. > In the same time, using a Google doc for writing/discussing the documents seems more than reasonable since it’s quite native and easy to use. I only propose to have a common entry point to fall of them. > If this idea looks feasible, I’d propose myself to collect the links to already created documents, create such page and update this list in the future. > WBR, > Alexey > On 22 May 2018, at 21:34, Eugene Kirpichov <kirpic...@google.com <mailto:kirpic...@google.com>> wrote: > Making it easier to manage indeed would be good. Could someone from PMC please add the following documents of mine to it? > SDF related documents: > http://s.apache.org/splittable-do-fn > http://s.apache.org/sdf-via-source > http://s.apache.org/textio-sdf > http://s.apache.org/beam-watch-transform > http://s.apache.org/beam-breaking-fusion > Non SDF related: > http://s.apache.org/context-fn > http://s.apache.org/fileio-write > A suggestion: maybe we can establish a convention to send design document proposals to dev+desi...@beam.apache.org <mailto:dev%2bdesi...@beam.apache.org>? Does the Apache mailing list management software support this kind of stuff? Then they'd be quite easy to find and filter. > On Tue, May 22, 2018 at 10:57 AM Kenneth Knowles <k...@google.com <mailto:k...@google.com>> wrote: >> It is owned by the Beam PMC collectively. Any PMC member can add things to it. Ideas for making it easy to manage are welcome. >> Probably easier to have a markdown file somewhere with a list of docs so we can issue and review PRs. Not sure the web site is the right place for it - we have a history of porting docs to markdown but really that is high overhead and users/community probably don't gain from it so much. Some have suggested a wiki. >> Kenn >> On Tue, May 22, 2018 at 10:22 AM Scott Wegner <sweg...@google.com <mailto:sweg...@google.com>> wrote: >>> Thanks for the links. Any details on that Google drive folder? Who maintains it? Is it possible for any contributor to add their design doc? >>> On Mon, May 21, 2018 at 8:15 AM Joseph PENG <josephtengp...@gmail.com <mailto:josephtengp...@gmail.com>> wrote: >>>> Alexey, >>>> I do not know where you can find all design docs, but I know a blog that has collected some of the major design docs. Hope it helps. >>>> https://wtanaka.com/beam/design-doc >>>> https://drive.google.com/drive/folders/0B-IhJZh9Ab52OFBVZHpsNjc4eXc >>>> On Mon, May 21, 2018 at 9:28 AM Alexey Romanenko < aromanenko....@gmail.com <mailto:aromanenko....@gmail.com>> wrote: >>>>> Hi all, >>>>> Is it possible to obtain somewhere a list of all proposals / prototype documents that have been published as a technical / design documents for new features? I have links to only some of them (found in mail list discussions by chance) but I’m not aware of others. >>>>> If yes, could someone share it or point me out where it is located in case if I missed this? >>>>> If not, don’t you think it would make sense to have such index of these documents? I believe it can be useful for Beam contributors since these proposals contain information which is absent or not so detailed on Beam web site documentation. >>>>> WBR, >>>>> Alexey
