On Mon, May 4, 2020 at 6:58 PM Aizhamal Nurmamat kyzy <aizha...@apache.org>
wrote:
> Thanks everyone for your feedback and support with the review. Please add
> any other comments so we can address them soon, if not please share your
> LGTMs.
>
> @Robert, thanks for separating the PR!
>
> @Thomas, regarding your question "There are some changes missing though
> (for example [2]), are you planning to add more recent commits later?" -
> yes, after merging the PR we will update all of the recent changes that are
> missing.
>
If we are going to do this, could people continue editing website after
Wednesday if PR is still not merged?
>
> @Nam Bui <nam....@polidea.com> , can we look into using the feature from
> this PR [1] that Brian mentioned to keep dates in blog post file names?
>
> @everyone, Nam also had a question regarding staging functionality - it
> keeps showing the errors like below:
>
> RAT ("Run RAT PreCommit") — FAILURE
>
There are files without the license header. We either need to modify RAT
config or add license headers. See for example:
*13:01:08* Printing headers for text files without a valid
license header...*13:01:08* *13:01:08*
=====================================================*13:01:08*
== File:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_RAT_Commit/src/website/www/site/static/js/bootstrap.min.js
> Website_Stage_GCS ("Run Website_Stage_GCS PreCommit") — FAILURE
>
Some links are failing with 404 errors. We need to update those urls.
Example:
*13:22:47* [=================================================> ] 1366
/ 1397 curl: (22) The requested URL returned error: 404 *13:22:47*
*13:22:47*
https://www.talend.com/blog/2017/01/13/future-apache-beam-now-top-level-apache-software-foundation-project/*13:22:50*
[=================================================> ] 1367 / 1397
curl: (22) The requested URL returned error: 404 *13:22:50* *13:22:50*
https://www.talend.com/blog/2017/01/23/apache-beam-way-greater-data-agility/?utm_medium=socialpost&utm_source=twitter&utm_campaign=blog
> Website_Stage_GCS ("Run Website_Stage_GCS PreCommit") — FAILURE
>
Probably the same as above. Take a look at the logs, they usually have
sufficient information.
>
> The staging is working, but the jobs show up as failed. Does anyone have
> an idea what the failures are related to and how we can fix it?
>
> [1] https://github.com/gohugoio/hugo/pull/4494
>
>
>
> On Mon, May 4, 2020 at 6:30 PM Robert Bradshaw <rober...@google.com>
> wrote:
>
>> I took the massive commit and split it up into:
>>
>> (1) Infrastructure changes (basically everything outside of
>> (website/www/site/content)
>> (2) Sed script changes, and
>> (3) Manual changes (everything not in (1) and (2)).
>>
>
Thank you Robert. This makes it much easier. What is the source of the sed
script? I am not sure why some of those lines are there. It would be much
easier for us to comment on the script source if it is reviewable somewhere.
>
>> It does seem that (3) has a number of unintentional changes, some
>> stylistic (e.g. lost of removal of end-of-file newlines) and some
>> actual content that's not up to date. This cuts down the number of
>> lines to be reviewed by more than half (and, notably, the more
>> substantial ones).
>>
>> [1]
>> https://github.com/apache/beam/pull/11608/commits/1bcf519a0f041607dfa401f167164301acbca2ac
>> 72 files changed, 3546 insertions(+), 1472 deletions(-)
>> [2]
>> https://github.com/apache/beam/pull/11608/commits/8b9f488c519b97a11ca4c7e3b644bb9ffe12cb98
>> 252 files changed, 4136 insertions(+), 4684 deletions(-)
>> [3]
>> https://github.com/apache/beam/pull/11608/commits/f9d8bc13a0fda0a60a436aa56186139d0f71de4e
>> 228 files changed, 1859 insertions(+), 2370 deletions(-)
>>
>> I also separated out the compatibility matrix move, which was ~1700
>> lines.
>> https://github.com/apache/beam/pull/11608/commits/16516d036af047493445654d61940dea8d04eaaa
>>
>> On Mon, May 4, 2020 at 6:15 PM Robert Bradshaw <rober...@google.com>
>> wrote:
>> >
>> > On Mon, May 4, 2020 at 6:02 PM Thomas Weise <thomas.we...@gmail.com>
>> wrote:
>> > >
>> > > I took a brief look at [1] and looks good overall.
>> > >
>> > > There are some changes missing though (for example [2]), are you
>> planning to add more recent commits later?
>> > >
>> > > Also, there was an earlier question from Brian regarding the
>> possibility to retain the post dates in blog file names. I would second
>> that, it would make the posts significantly easier to navigate.
>> >
>> > I'm OK with removing them from the URL if they're to distracting, but
>> > generally agree here. (If it's too difficult, it's not a huge issue.)
>> >
>> > > [1]
>> http://apache-beam-website-pull-requests.storage.googleapis.com/11554/index.html
>> > > [2]
>> http://apache-beam-website-pull-requests.storage.googleapis.com/11554/documentation/runners/flink/index.html
>> > >
>> > > Thomas
>> > >
>> > > On Mon, May 4, 2020 at 11:06 AM Hannah Jiang <hannahji...@google.com>
>> wrote:
>> > >>
>> > >> Hi Aizhamal,, yes, Wednesday sounds good to me. Thank you.
>> > >>
>> > >>
>> > >> On Mon, May 4, 2020 at 10:40 AM Aizhamal Nurmamat kyzy <
>> aizha...@apache.org> wrote:
>> > >>>
>> > >>> Hannah,
>> > >>>
>> > >>> We don't have an exact date, but we are hoping to address all the
>> comments and merge the PR by Wednesday. Will it be possible for you to wait
>> until then?
>> > >>>
>> > >>> On Thu, Apr 30, 2020 at 4:29 PM Hannah Jiang <
>> hannahji...@google.com> wrote:
>> > >>>>>
>> > >>>>> Since we want to move forward with the PR, I would like to ask
>> the community to hold off changes to the current Beam website for a week,
>> until we are able to review and merge the PR. Is this acceptable to
>> everyone?
>> > >>>>
>> > >>>> Do we have an exact date when we can push changes to the website?
>> I have PRs to update documents so would like to plan ahead.
>> > >>>>
>> > >>>> On Thu, Apr 30, 2020 at 1:17 PM Nam Bui <nam....@polidea.com>
>> wrote:
>> > >>>>>
>> > >>>>> Hey guys,
>> > >>>>>
>> > >>>>> I tried my best to handle renamed files in Git. I have no clue
>> why GitHub doesn't show it, but finally, I made this commit [1] (thanks for
>> your idea @bhulette) so you guys can review changes with ease (there is no
>> bunch of deleted markdown files anymore :D). Also, new staged version is
>> deployed, you could check it out [2].
>> > >>>>>
>> > >>>>> In case you are interested in translation, here is the proof of
>> concept [3] (the earth icon on the right corner is temporarily used for
>> switching languages). You can take a look at the translation guide for this
>> PoC [4].
>> > >>>>>
>> > >>>>> [1]
>> https://github.com/apache/beam/pull/11554/commits/b267bb360866a723ac2536f408f23de648c7cd4d
>> > >>>>> [2]
>> http://apache-beam-website-pull-requests.storage.googleapis.com/11554/index.html
>> > >>>>> [3] https://safe-relation.surge.sh/
>> > >>>>> [4]
>> https://github.com/PolideaInternal/beam/blob/website-develop/website/CONTRIBUTE.md#translation-guide
>> > >>>>>
>> > >>>>>
>> > >>>>> On Thu, Apr 30, 2020 at 7:24 PM Brian Hulette <
>> bhule...@google.com> wrote:
>> > >>>>>>
>> > >>>>>> Changing the URLs is fine with me as long as the old urls will
>> work too.
>> > >>>>>>
>> > >>>>>> But do we need to change the filenames for the blog posts to
>> accomplish that? It's nice that the blog post markdown files start with a
>> date so they naturally sort chronologically. It looks like this hugo PR [1]
>> made it possible to extract date metadata and slug (i.e.
>> dataflow-python-sdk-is-now-public) separately from the filename.
>> > >>>>>>
>> > >>>>>> [1] https://github.com/gohugoio/hugo/pull/4494
>> > >>>>>>
>> > >>>>>> On Thu, Apr 30, 2020 at 10:06 AM Ahmet Altay <al...@google.com>
>> wrote:
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Thu, Apr 30, 2020 at 9:55 AM Thomas Weise <t...@apache.org>
>> wrote:
>> > >>>>>>>>
>> > >>>>>>>> For changed URLs, will previous URLs be mapped to avoid broken
>> external links?
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> I believe the answer is yes from Nam's response "For now, we
>> keep the old URLs working in terms of redirecting them". I very much agree
>> that this is very important and should work for all existing urls.
>> > >>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Thu, Apr 30, 2020 at 9:34 AM Aizhamal Nurmamat kyzy <
>> aizha...@apache.org> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>> Hi,
>> > >>>>>>>>>
>> > >>>>>>>>> To give a little more context regarding the URLs, the date
>> should still appear on the blog post, but not on the URL.
>> > >>>>>>>>> For example, we'd have:
>> > >>>>>>>>>
>> https://beam.apache.org/beam/python/sdk/2016/02/25/python-sdk-now-public.html
>> become https://beam.apache.org/blog/dataflow-python-sdk-is-now-public/.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> I am not a content marketer. IMO, this is a good change. In the
>> past, a few times, we edited dates on posts (e.g. a release date was
>> entered incorrectly) and we had to either have a mismatch between dates in
>> the url and the date in the blog, or change the url. This change
>> simplifies, by having date only in place (in content metadata).
>> > >>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> The blog posts would have a small header showing the title,
>> author and publish date. But the URL would not have it.
>> > >>>>>>>>> Thoughts?
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> On Thu, Apr 30, 2020 at 9:23 AM Nam Bui <nam....@polidea.com>
>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>> Hi,
>> > >>>>>>>>>>
>> > >>>>>>>>>> @altay: Hey hey. Yeah, I didn't expect the baseUrl of
>> staging version is "
>> http://apache-beam-website-pull-requests.storage.googleapis.com/11554/";
>> which also includes "/11554", and Hugo considers it as a path so it breaks
>> the path of "static files" (like images). We made a fix. Now I'm working on
>> "getting git to recognize files as renames" as you suggested.
>> > >>>>>>>>>>
>> > >>>>>>>>>> @robert: The dates are nice but it causes verbose/long/ugly
>> URLs. We discussed with Aizhamal in the development stage and agreed to get
>> rid of this. For now, we keep the old URLs working in terms of redirecting
>> them. However, from now on, we should change the name convention on blog
>> posts to have a fancy URL like "beam.apache.org/blog/myblogpost.md". :)
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Thu, Apr 30, 2020 at 2:57 AM Robert Bradshaw <
>> rober...@google.com> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Wed, Apr 29, 2020 at 5:08 PM Ahmet Altay <
>> al...@google.com> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Nam, this looks better. At least links are working, and
>> the website visually looks similar and generally in good shape. I think
>> there are still issues. For example, I do not see any of the images (e.g.
>> the beam logo on top left is missing.)
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Wed, Apr 29, 2020 at 3:11 PM Brian Hulette <
>> bhule...@google.com> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> I left a comment on the PR [1]. I think the reason all of
>> the website content is not being tracked as file renames is because there
>> was a series of commits that created files in the new directory, and then
>> one commit that deleted the old directory. If there were a single commit
>> with all of the deleted and new files, git would surely recognize they are
>> effectively renameds and mark them as such. Maybe we just need to get all
>> these commits squashed into one?
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> [1]
>> https://github.com/apache/beam/pull/11554#issuecomment-621489844
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Nam, could you try this? If we can get git to recognize
>> these as renames, review process would be much easier.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> +1.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Alternatively, create a commit that just moves the files
>> into a new location (which git can always detect), then sit the edits on
>> top of that (which should preserve history better).
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Also, is there a reason the dates were removed from the
>> blog post filenames? For content like that, the dates are nice.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Wed, Apr 29, 2020 at 10:39 AM Nam Bui <
>> nam....@polidea.com> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hi guys,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> I'm Nam - from the responsible team of Apache Beam
>> website migration. I am pleased to answer some of the questions here.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> @aizhamal: Thanks for informing to the community. :)
>> > >>>>>>>>>>>>>> @altay, @robertwb: Yes. there is a problem with the
>> staged version at the moment. We didn't expect some behaviours on the build
>> process. So, we fixed it today and been waiting for @pablo to re-run it
>> again. The purpose of this PR is to migrate completely Beam site from
>> Jekyll to Hugo. Therefore, a bunch of deleted markdown files are from
>> Jekyll which was located at `beam/website/src`, and Hugo is located at
>> `beam/website/www` now. In `beam/website/README.md`, I wrote down about
>> running the Hugo website locally, although it is actually same as Jekyll
>> (because it's also set up with Docker & Gradle). In
>> `beam/website/CONTRIBUTE.md`, I guided people on how to get started with
>> Hugo on the Beam website. There is also a link in the "Translation Guide"
>> section which points to a branch of multilingual provenance, and it will
>> become a next PR soon.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Please let me know if you need more details. Feel free
>> to ask any questions and I will get back to you with answers. I'm so sorry
>> if I answer a little bit due to the timezone. :)
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Best regards,
>> > >>>>>>>>>>>>>> Nam
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Tue, Apr 28, 2020 at 8:49 PM Aizhamal Nurmamat kyzy <
>> aizha...@apache.org> wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Adding +Nam Bui and +Karolina Rosół to follow up on
>> questions.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Tue, Apr 28, 2020 at 11:34 AM Ahmet Altay <
>> al...@google.com> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> I am having trouble reviewing the staged version. What
>> is the best way to review this change?
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Do we expect any changes to markdown files, beyond
>> some metadata?
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Tue, Apr 28, 2020 at 10:45 AM Robert Bradshaw <
>> rober...@google.com> wrote:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Thanks. It'll be great to better support more
>> languages.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I looked at the PR and there seems to be no
>> provenance/history. E.g. all the content seems to be entirely new files
>> rather than diffs from the old. (There also seems to be a huge amount of
>> auto-generated js code as well.)
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> I agree. This makes it very hard to review. I also see
>> a bunch of deleted markdown files. Are they not getting migrated?
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> On Tue, Apr 28, 2020 at 10:23 AM Aizhamal Nurmamat
>> kyzy <aizha...@apache.org> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Hello everybody,
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> We are almost done migrating the Apache Beam website
>> from Jekyll to Hugo. You can see the PR in [1], and we'd love to hear your
>> feedback/comments on the PR. It includes detailed guidelines on
>> contributing to the new Hugo-based website and adding translations to pages
>> [2]. For those who are curious about adding new languages, we will provide
>> a proof of concept in the next couple of days in this thread.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Since we want to move forward with the PR, I would
>> like to ask the community to hold off changes to the current Beam website
>> for a week, until we are able to review and merge the PR. Is this
>> acceptable to everyone?
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> In case anyone missed my previous email with the
>> background for the website migration, you can find more context here [3].
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>>>>>>>> Aizhamal
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11554
>> > >>>>>>>>>>>>>>>>>> [2]
>> https://github.com/apache/beam/blob/256b7042bf504b94f161ca03b388a2ba247918d9/website/CONTRIBUTE.md
>> > >>>>>>>>>>>>>>>>>> [3]
>> https://lists.apache.org/thread.html/r7fa6d710c0a1959cce5108e460d71c306ce5756cf96af818b41cb7ca%40%3Cdev.beam.apache.org%3E
>> > >>
>> > >> --
>> > >> Please help me know how I am doing: go/hannahjiang-feedback
>> <https://goto.google.com/hannahjiang-feedback>
>>
>
