[ https://issues.apache.org/jira/browse/BEAM-9876?focusedWorklogId=432253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-432253 ]
ASF GitHub Bot logged work on BEAM-9876: ---------------------------------------- Author: ASF GitHub Bot Created on: 08/May/20 20:29 Start Date: 08/May/20 20:29 Worklog Time Spent: 10m Work Description: TheNeuralBit commented on a change in pull request #11554: URL: https://github.com/apache/beam/pull/11554#discussion_r422326464 ########## File path: website/README.md ########## @@ -17,58 +17,50 @@ under the License. --> +# Apache Beam website + These are the main sources of the website for Apache Beam, hosted at https://beam.apache.org/. -## About this site +## About -The Beam website is built using [Jekyll](https://jekyllrb.com/). Additionally, +The Beam website is built using [Hugo](https://gohugo.io/). Additionally, for additional formatting capabilities, this website uses [Twitter Bootstrap](https://getbootstrap.com/). Documentation generated from source code, such as Javadoc and Pydoc, is stored separately on the [beam-site repository](https://github.com/apache/beam-site/tree/release-docs). -## Active development +## Getting started Website development requires Docker installed if you wish to preview changes and run website tests. -The following command is used to build and serve the website locally. +The Docsy theme required for the site to work properly is included as a git submodule. This means that after you already cloned the repository, you need to update submodules at `<ROOT_DIRECTORY>`. + +`$ git submodule update --init --recursive` + +The following command is used to build and serve the website locally. Note: you should run the command at `<ROOT_DIRECTORY>`. - $ ./gradlew :website:serveWebsite +`$ ./gradlew :website:serveWebsite` Any changes made locally will trigger a rebuild of the website. Websites tests may be run using this command: - $ ./gradlew :website:testWebsite +`$ ./gradlew :website:testWebsite` -## Website push +For more detailed description, please refer to [contribution guide](CONTRIBUTE.md). Review comment: nit: "For a more" .. "refer to the contribution guide" ########## File path: website/build.gradle ########## @@ -130,55 +139,41 @@ class BuildTaskConfiguration { def createBuildTask = { BuildTaskConfiguration config = it as BuildTaskConfiguration task "build${config.name}Website" (type:Exec) { - dependsOn setupDockerContainer, setupBuildDir + dependsOn setupDockerContainer finalizedBy stopAndRemoveDockerContainer - inputs.files 'Gemfile.lock', '_config.yml' - inputs.dir 'src' - outputs.dir "$buildDir/.sass-cache" - outputs.dir buildContentDir(config.name) - def configs = "${config.dockerWorkDir}/website/_config.yml" - if (config.useTestConfig) { - configs += ",${config.dockerWorkDir}/website/_config_test.yml" - } + + def configs = "$dockerSourceDir/site/config.toml" if (config.useBranchRepoConfig) { - configs += ",/tmp/_config_branch_repo.yml" + configs += ",/tmp/_config_branch_repo.toml" } - def baseUrlFlag = config.baseUrl ? "--baseurl=/${config.baseUrl}" : "" + def baseUrlFlag = config.baseUrl ? "--baseURL /${config.baseUrl}" : "" commandLine 'docker', 'exec', "${->setupDockerContainer.containerId()}", '/bin/bash', '-c', - """cd ${config.dockerWorkDir}/build/website && \ - bundle exec jekyll build \ - --destination generated-${config.name.toLowerCase()}-content \ - --config ${configs} \ - --incremental ${baseUrlFlag} \ - --source ${config.dockerWorkDir}/website/src + """cd $dockerSourceDir && \ + yarn build \ + -d $dockerBuildDir/generated-${config.name.toLowerCase()}-content \ + --config $configs \ + $baseUrlFlag """ } } // task buildLocalWebsite createBuildTask( name:'Local', - useTestConfig: true, - useBranchRepoConfig: true, - dockerWorkDir: dockerWorkDir, ) task buildWebsite(dependsOn:buildLocalWebsite) build.dependsOn buildWebsite // task buildGcsWebsite createBuildTask( name:'Gcs', - useTestConfig: true, - useBranchRepoConfig: true, Review comment: It looks like here and `buildLocalWebsite` were the only places where `useBranchRepoConfig` was true. Should it still be true here? Or should we just remove that config option altogether? ########## File path: website/README.md ########## @@ -17,58 +17,50 @@ under the License. --> +# Apache Beam website + These are the main sources of the website for Apache Beam, hosted at https://beam.apache.org/. -## About this site +## About -The Beam website is built using [Jekyll](https://jekyllrb.com/). Additionally, +The Beam website is built using [Hugo](https://gohugo.io/). Additionally, for additional formatting capabilities, this website uses [Twitter Bootstrap](https://getbootstrap.com/). Documentation generated from source code, such as Javadoc and Pydoc, is stored separately on the [beam-site repository](https://github.com/apache/beam-site/tree/release-docs). -## Active development +## Getting started Website development requires Docker installed if you wish to preview changes and run website tests. -The following command is used to build and serve the website locally. +The Docsy theme required for the site to work properly is included as a git submodule. This means that after you already cloned the repository, you need to update submodules at `<ROOT_DIRECTORY>`. + +`$ git submodule update --init --recursive` + +The following command is used to build and serve the website locally. Note: you should run the command at `<ROOT_DIRECTORY>`. - $ ./gradlew :website:serveWebsite +`$ ./gradlew :website:serveWebsite` Any changes made locally will trigger a rebuild of the website. Websites tests may be run using this command: - $ ./gradlew :website:testWebsite +`$ ./gradlew :website:testWebsite` -## Website push +For more detailed description, please refer to [contribution guide](CONTRIBUTE.md). + +## Deployment After a PR is merged, a background Jenkins job will automatically generate and push [website content](https://github.com/apache/beam/tree/asf-site/website/generated-content) to the asf-site branch. This content is later picked up and pushed to https://beam.apache.org/. -## Additional Information - -### Writing blog posts - -Blog posts are created in the `_posts` directory. - -If this is your first post, make sure to add yourself to `_data\authors.yml`. - -While you a working on your post before the publishing time listed in its header, -add `--future` when running Jekyll in order to view your draft on your local copy of -the site. - -### Adding Jekyll plugins - -If you modify the site to use additional Jekyll plugins, add them in `Gemfile` -and then run `bundle update`, which will regenerate the complete `Gemfile.lock`. -Make sure that the updated `Gemfile.lock` is included in your pull request. For more information, -see the Bundler [documentation](https://bundler.io/v1.3/rationale.html). +## Contribution guide +If you'd like to contribute to the Apache Airflow website project, read our [contribution guide](CONTRIBUTE.md) where you can find detailed instructions on how to work with the website. Review comment: Beam not Airflow :) (to be safe I searched the diff and didn't find any other Airflow references) I'd also drop "project" and just say "If you'd like to contribute to the Apache Beam website, ..." ########## File path: website/build.gradle ########## @@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) { "${->createDockerContainer.containerId()}" // Lazily evaluate containerId. } +task initGitSubmodules(type: Exec) { + commandLine 'docker', 'exec', + "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive' +} + +task installDependencies(type: Exec) { + commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir", + "${->startDockerContainer.containerId()}", 'yarn', 'install' +} + +task buildGithubSamples(type: Exec) { + commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir", + "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples' +} + Review comment: nit: do these need to be separate tasks? Maybe they could be folded into setupDockerContainer ########## File path: website/CONTRIBUTE.md ########## @@ -0,0 +1,336 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Contribution Guide + +This guide consists of: + +- [Project structure](#project-structure) +- [Configuration walkthrough](#configuration-walkthrough) +- [How to add a new doc](#how-to-add-a-new-doc) +- [How to add a new blogpost](#how-to-add-a-new-blogpost) +- [How to add a new landing page](#how-to-add-a-new-landing-page) +- [How to write in Hugo way](#how-to-write-in-hugo-way) + - [Define TableOfContents](#define-tableofcontents) + - [Language switching](#language-switching) + - [Code highlighting](#code-highlighting) + - [Adding class to markdown text](#paragraph) + - [Table](#table) + - [Github sample](#github-sample) + - [Others](#others) +- [Translation guide](#translation-guide) + +## Project structure + +``` +www/ +├── dist # bundle files +├── site +│ ├── archetypes # frontmatter template +│ ├── assets +│ │ └── scss # styles +│ ├── content # pages +│ │ └── en +│ │ ├── blog +│ │ ├── community +│ │ ├── contribute +│ │ ├── documentation +│ │ ├── get-started +│ │ ├── privacy_policy +│ │ ├── roadmap +│ │ └── security +│ │ └── _index.md +│ ├── data +│ ├── layouts # content template +│ ├── static +│ │ ├── downloads # downloaded files +│ │ └── fonts +│ │ └── images +│ │ └── js +│ └── themes +│ └── docsy +├── build_github_samples.sh +├── check-links.sh # links checker +└── package.json +``` + +## Configuration walkthrough + +When we mention the `config file` in this documentation, we mean the Hugo configuration file located at `/www/site/config.toml`. + +Most of the setup are self-explained in the comments. Please refer to [Hugo documentation](https://gohugo.io/getting-started/configuration/) for more details. + +You should notice at `[params]`, they are considered as global variables. For instance, when you'd like to replace the release latest version, make a change on `release_latest = ""` to replace it everywhere in the project. Review comment: Could you add a link to https://gohugo.io/getting-started/installing/ or something similar since people need to install hugo for commands in the rest of the doc ########## File path: website/build.gradle ########## @@ -66,8 +67,8 @@ task createDockerContainer(type: Exec) { gradle.taskGraph.whenReady { def extraOptions = '' if (gradle.taskGraph.hasTask(":${project.name}:serveWebsite")) { - // Publish port 4000 where Jekyll serves website from - extraOptions = '--publish 127.0.0.1:4000:4000' + // Publish port 1313 where Hugo serves website from + extraOptions = "--publish 1313:1313" Review comment: Can you keep the `127.0.0.1:`? I'm not sure about the default behavior of docker's "--publish" arg but its good to be clear we only want the port to be available internally. ########## File path: website/www/package.json ########## @@ -0,0 +1,19 @@ +{ + "name": "apache-beam-website", + "version": "1.0.0", + "description": "Apache Beam website", + "repository": "apache/beam", + "license": "MIT", + "scripts": { + "build_github_samples": "./build_github_samples.sh", + "develop": "cd site && hugo server", + "build": "cross-env HUGO_ENV=production hugo -d ../dist -s site -v", + "start": "hugo -d ../dist -s site -vw" + }, + "dependencies": {}, + "devDependencies": { + "autoprefixer": "^9.7.4", + "cross-env": "^7.0.2", + "postcss-cli": "^7.1.0" Review comment: Is postcss/autoprefixer actually used anywhere? I can't find any other references to them ########## File path: website/build.gradle ########## @@ -78,7 +79,7 @@ task createDockerContainer(type: Exec) { extraOptions += " -u \$(id -u):\$(id -g)" } commandLine '/bin/bash', '-c', - "docker create -v $project.rootDir:$dockerWorkDir $extraOptions $dockerImageTag" + "docker create -v $project.rootDir:$dockerWorkDir $extraOptions $dockerImageTag sh -c 'trap \"exit 0\" INT; while true; do sleep 30; done;'" Review comment: I'm not clear on why you had to add this trap and while loop, could you explain that? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 432253) Time Spent: 9.5h (was: 9h 20m) > Migrate the Beam website from Jekyll to Hugo to enable localization of the > site content > --------------------------------------------------------------------------------------- > > Key: BEAM-9876 > URL: https://issues.apache.org/jira/browse/BEAM-9876 > Project: Beam > Issue Type: Task > Components: website > Reporter: Aizhamal Nurmamat kyzy > Assignee: Aizhamal Nurmamat kyzy > Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > Enable internationalization of the Apache Beam website to increase the reach > of the project, and facilitate adoption and growth of its community. > The proposal was to do this by migrating the current Apache Beam website from > Jekyll do Hugo [1]. Hugo supports internationalization out-of-the-box, making > it easier both for contributors and maintainers support the > internationalization effort. > The further discussion on implementation can be viewed here [2] > [1] > [https://lists.apache.org/thread.html/rfab4cc1411318c3f4667bee051df68f37be11846ada877f3576c41a9%40%3Cdev.beam.apache.org%3E] > [2] > [https://lists.apache.org/thread.html/r6b999b6d7d1f6cbb94e16bb2deed2b65098a6b14c4ac98707fe0c36a%40%3Cdev.beam.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005)