[ https://issues.apache.org/jira/browse/AIRFLOW-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642197#comment-16642197 ]
ASF GitHub Bot commented on AIRFLOW-3078: ----------------------------------------- sprzedwojski opened a new pull request #4022: [AIRFLOW-3078] Basic operators for Google Compute Engine URL: https://github.com/apache/incubator-airflow/pull/4022 ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW-3078) issue and references it in the PR title. ### Description - [x] Here are some details about my PR, including screenshots of any UI changes: Add GceInstanceStartOperator, GceInstanceStopOperator and GceSetMachineTypeOperator. Each operator includes: - core logic - input params validation - unit tests - presence in the example DAG - docstrings - How-to and Integration documentation Additionally, in GceHook error checking if response is 200 OK was added: Some types of errors are only visible in the response's "error" field and the overall HTTP response is 200 OK. That is why apart from checking if status is "done" we also check if "error" is empty, and if not an exception is raised with error message extracted from the "error" field of the response. In this commit we also separated out Body Field Validator to separate module in tools - this way it can be reused between various GCP operators, it has proven to be usable in at least two of them now. ### Tests - [x] My PR adds the following unit tests: `test_gcp_function_operator` ### Commits - [x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": 1. Subject is separated from body by a blank line 1. Subject is limited to 50 characters (not including Jira issue reference) 1. Subject does not end with a period 1. Subject uses the imperative mood ("add", not "adding") 1. Body wraps at 72 characters 1. Body explains "what" and "why", not "how" ### Documentation - [x] In case of new functionality, my PR adds documentation that describes how to use it. - When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added. ### Code Quality - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Basic operators for Google Compute Engine > ----------------------------------------- > > Key: AIRFLOW-3078 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3078 > Project: Apache Airflow > Issue Type: New Feature > Components: contrib, gcp > Reporter: Jarek Potiuk > Assignee: Jarek Potiuk > Priority: Trivial > > In order to be able to interact with raw Google Compute Engine, we need an > operator that should be able to: > For managing individual machines: > * Start Instance: > ([https://cloud.google.com/compute/docs/reference/rest/v1/instances/start]) > * Set Machine Type > ([https://cloud.google.com/compute/docs/reference/rest/v1/instances/setMachineType]) > > * Stop Instance: > ([https://cloud.google.com/compute/docs/reference/rest/v1/instances/stop]) > Also we should be able to manipulate instance groups: > * Get instance group: > ([https://cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers/get]) > * Insert Group: > ([https://cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers/insert]) > * Update Group: > ([https://cloud.google.com/compute/docs/reference/rest/beta/instanceGroupManagers/update]) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)