[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new fc22e2a [DLAB-1467]: Added terraform scripts for deploying DLab endpoint fc22e2a is described below commit fc22e2a60e5bddd1aa764da2fda2e4e66f1a596b Author: Oleh Martushevskyi AuthorDate: Thu Feb 6 11:01:01 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index 72ff1d7..9b4bb7e 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -326,7 +326,7 @@ def configure_supervisor_endpoint(endpoint_keystore_password): cloud_properties = [ { 'key': "OS_USER", -'value': args.os_userd +'value': args.os_user }, { 'key': "KEYNAME", - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new d42fe44 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint new ca182bc Merge remote-tracking branch 'origin/DLAB-1467' into DLAB-1467 d42fe44 is described below commit d42fe44fcf5c61e7147527e7c6e4b551130beabd Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 17:17:38 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/src/general/conf/dlab.ini | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infrastructure-provisioning/src/general/conf/dlab.ini b/infrastructure-provisioning/src/general/conf/dlab.ini index e2055e0..5416291 100644 --- a/infrastructure-provisioning/src/general/conf/dlab.ini +++ b/infrastructure-provisioning/src/general/conf/dlab.ini @@ -163,7 +163,7 @@ locale = en-US ### Azure region code # region_info = ### Azure datalake to create -# datalake_enable = +datalake_enable = false ### Azure login application ID # application_id = - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new fc05b59 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint fc05b59 is described below commit fc05b597cb249a6018cdc69e8ec07dfca925ec2a Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 14:15:49 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- services/provisioning-service/provisioning.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/services/provisioning-service/provisioning.yml b/services/provisioning-service/provisioning.yml index 814ef89..d234e43 100644 --- a/services/provisioning-service/provisioning.yml +++ b/services/provisioning-service/provisioning.yml @@ -123,6 +123,7 @@ cloudProperties: azureClientId: AZURE_CLIENT_ID gcpProjectId: GCP_PROJECT_ID imageEnabled: CONF_IMAGE_ENABLED + azureAuthFile: AZURE_AUTH_FILE_PATH ldap: host: LDAP_HOST dn: LDAP_DN - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 0b16e34 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 0b16e34 is described below commit 0b16e34cae0d1856e8e84c780381170ed643618f Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 14:12:42 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- .../src/general/scripts/aws/ssn_configure.py| 4 .../src/general/scripts/azure/ssn_configure.py | 4 .../src/general/scripts/gcp/ssn_configure.py| 4 .../terraform/bin/deploy/endpoint_fab.py| 13 - .../terraform/bin/deploy/provisioning.yml | 1 + 5 files changed, 25 insertions(+), 1 deletion(-) diff --git a/infrastructure-provisioning/src/general/scripts/aws/ssn_configure.py b/infrastructure-provisioning/src/general/scripts/aws/ssn_configure.py index 7d3a328..a4d6dbc 100644 --- a/infrastructure-provisioning/src/general/scripts/aws/ssn_configure.py +++ b/infrastructure-provisioning/src/general/scripts/aws/ssn_configure.py @@ -472,6 +472,10 @@ if __name__ == "__main__": { 'key': 'CONF_IMAGE_ENABLED', 'value': os.environ['conf_image_enabled'] +}, +{ +'key': "AZURE_AUTH_FILE_PATH", +'value': "" } ] if os.environ['conf_duo_vpc_enable'] == 'true': diff --git a/infrastructure-provisioning/src/general/scripts/azure/ssn_configure.py b/infrastructure-provisioning/src/general/scripts/azure/ssn_configure.py index 2faa7ed..406eef4 100644 --- a/infrastructure-provisioning/src/general/scripts/azure/ssn_configure.py +++ b/infrastructure-provisioning/src/general/scripts/azure/ssn_configure.py @@ -336,6 +336,10 @@ if __name__ == "__main__": { 'key': 'CONF_IMAGE_ENABLED', 'value': os.environ['conf_image_enabled'] +}, +{ +'key': "AZURE_AUTH_FILE_PATH", +'value': "" } ] diff --git a/infrastructure-provisioning/src/general/scripts/gcp/ssn_configure.py b/infrastructure-provisioning/src/general/scripts/gcp/ssn_configure.py index 9915f5e..cc9f170 100644 --- a/infrastructure-provisioning/src/general/scripts/gcp/ssn_configure.py +++ b/infrastructure-provisioning/src/general/scripts/gcp/ssn_configure.py @@ -407,6 +407,10 @@ if __name__ == "__main__": { 'key': 'CONF_IMAGE_ENABLED', 'value': os.environ['conf_image_enabled'] +}, +{ +'key': "AZURE_AUTH_FILE_PATH", +'value': "" } ] if os.environ['conf_stepcerts_enabled'] == 'true': diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index 47ead50..72ff1d7 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -253,6 +253,10 @@ def create_key_dir_endpoint(): try: if not exists(conn, '/home/{}/keys'.format(args.os_user)): conn.run('mkdir /home/{}/keys'.format(args.os_user)) +if args.auth_file_path: +conn.put(args.auth_file_path, '/tmp/azure_auth.json') +conn.sudo('mv /tmp/azure_auth.json /home/{}/keys/'.format(args.os_user)) +args.auth_file_path = '/home/{}/keys/azure_auth.json'.format(args.os_user) except Exception as err: logging.error('Failed create keys directory as ~/keys: ', str(err)) traceback.print_exc() @@ -317,10 +321,12 @@ def configure_supervisor_endpoint(endpoint_keystore_password): .format(dlab_conf_dir)) if args.resource_group_name == '': args.resource_group_name = args.service_base_name +if args.cloud_provider == 'azure': +args.region = args.region.lower().replace(' ', '') cloud_properties = [ { 'key': "OS_USER", -'value': args.os_user +'value': args.os_userd }, { 'key': "KEYNAME", @@ -501,6 +507,10 @@ def configure_supervisor_endpoint(endpoint_keystore_password): { 'key': "KEYCLOAK_PASSWORD", 'value': args.keycloak_user_password +}, +{ +'key': "AZURE_AUTH_FILE_PATH", +'value': args.auth_file_path } ] for param in
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 3f06079 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 3f06079 is described below commit 3f060791e9481b35ea15241a5d0a3e3240d09df5 Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 13:29:05 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index a7c9a34..47ead50 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -315,7 +315,8 @@ def configure_supervisor_endpoint(endpoint_keystore_password): .format(args.dlab_path, supervisor_conf)) conn.put('./provisioning.yml', '{}provisioning.yml' .format(dlab_conf_dir)) - +if args.resource_group_name == '': +args.resource_group_name = args.service_base_name cloud_properties = [ { 'key': "OS_USER", @@ -415,7 +416,7 @@ def configure_supervisor_endpoint(endpoint_keystore_password): }, { 'key': "AZURE_RESOURCE_GROUP_NAME", -'value': args.azure_resource_group_name +'value': args.resource_group_name }, { 'key': "AZURE_SSN_STORAGE_ACCOUNT_TAG", @@ -743,7 +744,7 @@ def init_args(): parser.add_argument('--conf_key_dir', type=str, default='/root/keys/', help='Should end by symbol /') parser.add_argument('--vpc_id', type=str, default='') parser.add_argument('--peering_id', type=str, default='') -parser.add_argument('--azure_resource_group_name', type=str, default='') +parser.add_argument('--resource_group_name', type=str, default='') parser.add_argument('--azure_ssn_storage_account_tag', type=str, default='') parser.add_argument('--azure_shared_storage_account_tag', type=str, default='') parser.add_argument('--azure_datalake_tag', type=str, default='') - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new b20d09d [DLAB-1467]: Added terraform scripts for deploying DLab endpoint b20d09d is described below commit b20d09d6a75d53be4c9d29d7ae5554acc788d68c Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 10:41:57 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index 0c5cc6b..3b7df79 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -130,11 +130,12 @@ def ensure_step_certs(): elif args.cloud_provider == 'azure': local_ip_address = conn.sudo('curl -s -H Metadata:true "http://169.254.169.254/metadata/' 'instance?api-version=2017-08-01=json" | jq -r ".network.' - 'interface[].ipv4.ipAddress[].privateIpAddress"').stdout + 'interface[].ipv4.ipAddress[].privateIpAddress"').stdout.replace('\n', '') try: public_ip_address = conn.sudo('curl -s -H Metadata:true "http://169.254.169.254/metadata/' 'instance?api-version=2017-08-01=json" | jq -r ".network.' - 'interface[].ipv4.ipAddress[].publicIpAddress"').stdout + 'interface[].ipv4.ipAddress[].publicIpAddress"').stdout.replace('\n', + '') except: public_ip_address = None else: - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new f5c21a5 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint f5c21a5 is described below commit f5c21a5d45c43cde580e99f129627d682bb571d3 Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 10:25:31 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/dlab.py | 5 - 1 file changed, 5 deletions(-) diff --git a/infrastructure-provisioning/terraform/bin/dlab.py b/infrastructure-provisioning/terraform/bin/dlab.py index b7f0dcd..07214c8 100644 --- a/infrastructure-provisioning/terraform/bin/dlab.py +++ b/infrastructure-provisioning/terraform/bin/dlab.py @@ -1233,11 +1233,6 @@ class AzureEndpointBuilder(AbstractDeployBuilder): .add_str('--endpoint_shape', 'Instance shape of Endpoint.', default='Standard_DS2_v2', group='endpoint') .add_str('--endpoint_volume_size', 'Endpoint disk size', default='30', group='endpoint') .add_str('--additional_tag', 'Additional tag.', default='product:dlab', group='endpoint') - .add_str('--ldap_host', 'ldap host', required=True, group='endpoint') - .add_str('--ldap_dn', 'ldap dn', required=True, group='endpoint') - .add_str('--ldap_user', 'ldap user', required=True, group='endpoint') - .add_str('--ldap_bind_creds', 'ldap bind creds', required=True, group='endpoint') - .add_str('--ldap_users_group', 'ldap users group', required=True, group='endpoint') ) return params.build() - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 4200b8b [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 4200b8b is described below commit 4200b8bd806ee7c05bedaf7fb855d5672f4c275b Author: Oleh Martushevskyi AuthorDate: Wed Feb 5 10:20:20 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/dlab.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/infrastructure-provisioning/terraform/bin/dlab.py b/infrastructure-provisioning/terraform/bin/dlab.py index e1aebd7..b7f0dcd 100644 --- a/infrastructure-provisioning/terraform/bin/dlab.py +++ b/infrastructure-provisioning/terraform/bin/dlab.py @@ -1220,6 +1220,8 @@ class AzureEndpointBuilder(AbstractDeployBuilder): .add_str('--service_base_name', 'Service base name', group='endpoint') .add_str('--resource_group_name', 'Resource group name', group='endpoint') .add_str('--vpc_id', 'ID of VPC if you already have VPC created.', group='endpoint') + .add_str('--vpc_cidr', 'CIDR for VPC creation. Conflicts with vpc_id.', default='172.31.0.0/16', + group='endpoint') .add_str('--subnet_cidr', 'CIDR for Subnet creation. Conflicts with vpc_id.', default='172.31.0.0/24', group='endpoint') .add_str('--ssn_subnet', 'ID of AWS Subnet if you already have subnet created.', group='endpoint') - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 668ed46 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 668ed46 is described below commit 668ed464239fe27544ad98af23726a806ed4263d Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 16:16:17 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- services/billing-azure/Dockerfile | 28 1 file changed, 28 insertions(+) diff --git a/services/billing-azure/Dockerfile b/services/billing-azure/Dockerfile new file mode 100644 index 000..dc19faf --- /dev/null +++ b/services/billing-azure/Dockerfile @@ -0,0 +1,28 @@ +# * +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# ** + +FROM openjdk:8-alpine + +USER root + +COPY billing-azure-2.2.jar /root/ + +CMD java -Xmx1024M -jar -Duser.timezone=UTC -Dfile.encoding=UTF-8 /root/billing-azure-2.2.jar --conf /root/billing.yml \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 15a5288 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 15a5288 is described below commit 15a5288f2482566778409531a5791efd626f14f2 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 16:03:00 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/bin/dlab.py | 24 +++ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/infrastructure-provisioning/terraform/bin/dlab.py b/infrastructure-provisioning/terraform/bin/dlab.py index 2d37683..e1aebd7 100644 --- a/infrastructure-provisioning/terraform/bin/dlab.py +++ b/infrastructure-provisioning/terraform/bin/dlab.py @@ -1150,17 +1150,23 @@ class GCPEndpointBuilder(AbstractDeployBuilder): .add_str('--ssn_subnet', 'ID of AWS Subnet if you already have subnet created.', group='endpoint') .add_str('--subnet_id', 'ID of subnet', group='endpoint') .add_str('--ami', 'ID of EC2 AMI.', group='endpoint') - .add_str('--key_path', 'Path to public key', required=True, group='endpoint') + .add_str('--path_to_pub_key', 'Path to public key', required=True, group='endpoint') .add_str('--endpoint_id', 'Endpoint id.', required=True, group='endpoint') .add_str('--region', 'Name of region.', group='endpoint') - .add_str('--endpoint_shape', 'Instance shape of Endpoint.', default='Standard_DS2_v2', group='endpoint') - .add_str('--endpoint_volume_size', 'Endpoint disk size', default='30', group='endpoint') + .add_str('--zone', 'Name of zone.', group='endpoint') + .add_str('--endpoint_shape', 'Instance shape of Endpoint.', group='endpoint') + .add_str('--endpoint_volume_size', 'Endpoint disk size', group='endpoint') .add_str('--additional_tag', 'Additional tag.', default='product:dlab', group='endpoint') .add_str('--ldap_host', 'ldap host', required=True, group='endpoint') .add_str('--ldap_dn', 'ldap dn', required=True, group='endpoint') .add_str('--ldap_user', 'ldap user', required=True, group='endpoint') .add_str('--ldap_bind_creds', 'ldap bind creds', required=True, group='endpoint') .add_str('--ldap_users_group', 'ldap users group', required=True, group='endpoint') + .add_str('--firewall_ing_cidr_range', 'Ingress range', group='endpoint') + .add_str('--firewall_eg_cidr_range', 'Egress range', group='endpoint') + .add_str('--endpoint_policies', 'Endpoint policies list', group='endpoint') + .add_str('--endpoint_roles', 'Endpoint roles list', group='endpoint') + .add_str('--bucket_region', 'Bucket region', group='endpoint') ) return params.build() @@ -1219,23 +1225,17 @@ class AzureEndpointBuilder(AbstractDeployBuilder): .add_str('--ssn_subnet', 'ID of AWS Subnet if you already have subnet created.', group='endpoint') .add_str('--subnet_id', 'ID of subnet', group='endpoint') .add_str('--ami', 'ID of EC2 AMI.', group='endpoint') - .add_str('--path_to_pub_key', 'Path to public key', required=True, group='endpoint') + .add_str('--key_path', 'Path to public key', required=True, group='endpoint') .add_str('--endpoint_id', 'Endpoint id.', required=True, group='endpoint') .add_str('--region', 'Name of region.', group='endpoint') - .add_str('--zone', 'Name of zone.', group='endpoint') - .add_str('--endpoint_shape', 'Instance shape of Endpoint.', group='endpoint') - .add_str('--endpoint_volume_size', 'Endpoint disk size', group='endpoint') + .add_str('--endpoint_shape', 'Instance shape of Endpoint.', default='Standard_DS2_v2', group='endpoint') + .add_str('--endpoint_volume_size', 'Endpoint disk size', default='30', group='endpoint') .add_str('--additional_tag', 'Additional tag.', default='product:dlab', group='endpoint') .add_str('--ldap_host', 'ldap host', required=True, group='endpoint') .add_str('--ldap_dn', 'ldap dn', required=True, group='endpoint') .add_str('--ldap_user', 'ldap user', required=True, group='endpoint') .add_str('--ldap_bind_creds', 'ldap bind creds', required=True, group='endpoint') .add_str('--ldap_users_group', 'ldap users group', required=True, group='endpoint') - .add_str('--firewall_ing_cidr_range', 'Ingress range', group='endpoint') - .add_str('--firewall_eg_cidr_range', 'Egress range', group='endpoint') - .add_str('--endpoint_policies', 'Endpoint policies list', group='endpoint') - .add_str('--endpoint_roles', 'Endpoint roles list',
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new b39ee4c [DLAB-1467]: Added terraform scripts for deploying DLab endpoint b39ee4c is described below commit b39ee4cc8251f7e6bcfbd0a068f74dcd3f862a77 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 15:54:38 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- .../terraform/bin/deploy/endpoint_fab.py| 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index 0b9c5b6..0c5cc6b 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -128,12 +128,13 @@ def ensure_step_certs(): except: public_ip_address = None elif args.cloud_provider == 'azure': -local_ip_address = conn.sudo('curl -s ' - 'http://169.254.169.254/latest/meta-data/local-ipv4').stdout.replace('\n', - '') +local_ip_address = conn.sudo('curl -s -H Metadata:true "http://169.254.169.254/metadata/' + 'instance?api-version=2017-08-01=json" | jq -r ".network.' + 'interface[].ipv4.ipAddress[].privateIpAddress"').stdout try: -public_ip_address = conn.sudo('curl -s http://169.254.169.254/latest/meta-data/' - 'public-ipv4').stdout.replace('\n', '') +public_ip_address = conn.sudo('curl -s -H Metadata:true "http://169.254.169.254/metadata/' + 'instance?api-version=2017-08-01=json" | jq -r ".network.' + 'interface[].ipv4.ipAddress[].publicIpAddress"').stdout except: public_ip_address = None else: @@ -229,7 +230,7 @@ def ensure_docker_endpoint(): .stdout.rstrip("\n\r")) conn.sudo("sed -i 's|DNS_IP_RESOLVE|\"dns\": [\"{0}\"],|g' {1}/tmp/daemon.json" .format(dns_ip_resolve, args.dlab_path)) -elif args.cloud_provider == "gcp": +elif args.cloud_provider == "gcp" or args.cloud_provider == "azure": dns_ip_resolve = "" conn.sudo('sed -i "s|DNS_IP_RESOLVE||g" {1}/tmp/daemon.json' .format(dns_ip_resolve, args.dlab_path)) - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new f57ea18 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint f57ea18 is described below commit f57ea18f75ecff944edf030a272fe670710dadf9 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 13:46:29 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- .../terraform/azure/endpoint/main/instance.tf | 2 +- .../terraform/azure/endpoint/main/outputs.tf | 2 +- .../terraform/azure/endpoint/main/variables.tf | 2 +- .../terraform/bin/deploy/endpoint_fab.py | 12 +++- infrastructure-provisioning/terraform/bin/dlab.py | 75 +- 5 files changed, 88 insertions(+), 5 deletions(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf index c5bc89d..989c32c 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf @@ -33,7 +33,7 @@ resource "azurerm_virtual_machine" "endpoint_instance" { location = data.azurerm_resource_group.data-endpoint-resource-group.location resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name network_interface_ids = [azurerm_network_interface.endpoint-nif.id] - vm_size = var.endpoint_instance_shape + vm_size = var.endpoint_shape delete_os_disk_on_termination = true storage_image_reference { diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf index bc5e20d..793917d 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf @@ -32,5 +32,5 @@ output "vpc_id" { } output "ssn_k8s_sg_id" { - value = azurerm_network_security_group.enpoint-sg.id + value = azurerm_network_security_group.enpoint-sg.name } \ No newline at end of file diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/variables.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/variables.tf index b7b855f..a00ce18 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/variables.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/variables.tf @@ -53,7 +53,7 @@ variable "subnet_id" { variable "subnet_cidr" {} -variable "endpoint_instance_shape" {} +variable "endpoint_shape" {} variable "ami" { default = "Canonical_UbuntuServer_16.04-LTS" diff --git a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py index 05077c5..0b9c5b6 100644 --- a/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py +++ b/infrastructure-provisioning/terraform/bin/deploy/endpoint_fab.py @@ -126,7 +126,16 @@ def ensure_step_certs(): 'http://metadata/computeMetadata/v1/instance/network-interfaces/0/' 'ip').stdout.replace('\n', '') except: -public_ip_address = None +public_ip_address = None +elif args.cloud_provider == 'azure': +local_ip_address = conn.sudo('curl -s ' + 'http://169.254.169.254/latest/meta-data/local-ipv4').stdout.replace('\n', + '') +try: +public_ip_address = conn.sudo('curl -s http://169.254.169.254/latest/meta-data/' + 'public-ipv4').stdout.replace('\n', '') +except: +public_ip_address = None else: local_ip_address = None public_ip_address = None @@ -755,6 +764,7 @@ def init_args(): def update_system(): conn.sudo('apt-get update') +conn.sudo('apt-get install -y jq') def init_dlab_connection(ip=None, user=None, diff --git a/infrastructure-provisioning/terraform/bin/dlab.py b/infrastructure-provisioning/terraform/bin/dlab.py index 6100930..2d37683 100644 --- a/infrastructure-provisioning/terraform/bin/dlab.py +++ b/infrastructure-provisioning/terraform/bin/dlab.py @@ -1150,6 +1150,75 @@ class GCPEndpointBuilder(AbstractDeployBuilder): .add_str('--ssn_subnet', 'ID of AWS Subnet if you already have subnet created.', group='endpoint')
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 882a9e6 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 882a9e6 is described below commit 882a9e685ff2ed008a994b3e2530cefaeac77ef3 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:57:42 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf index c48eea7..c5bc89d 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf @@ -54,7 +54,7 @@ resource "azurerm_virtual_machine" "endpoint_instance" { admin_username = "ubuntu" } os_profile_linux_config { -disable_password_authentication = false +disable_password_authentication = true ssh_keys { key_data = data.tls_public_key.enpoint_key.public_key_openssh path = "/home/ubuntu/.ssh/authorized_keys" - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 07e713c [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 07e713c is described below commit 07e713c56447d283572674939a7cea41daa50783 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:50:00 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/azure/endpoint/main/network.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf index fa157e3..ac782f9 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf @@ -84,7 +84,7 @@ resource "azurerm_network_interface" "endpoint-nif" { ip_configuration { name = "configuration" subnet_id = data.azurerm_subnet.data-endpoint-subnet.id -private_ip_address_allocation = "Static" +private_ip_address_allocation = "Dynamic" public_ip_address_id = azurerm_public_ip.endpoint-static-ip.id private_ip_address_version= "IPv4" } - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new d07e834 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint d07e834 is described below commit d07e834b453c08867278d2e298e95a4fee25ccf5 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:42:19 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf index 545def7..c48eea7 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf @@ -32,7 +32,7 @@ resource "azurerm_virtual_machine" "endpoint_instance" { name = local.endpoint_instance_name location = data.azurerm_resource_group.data-endpoint-resource-group.location resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name - network_interface_ids = azurerm_network_interface.endpoint-nif.id + network_interface_ids = [azurerm_network_interface.endpoint-nif.id] vm_size = var.endpoint_instance_shape delete_os_disk_on_termination = true - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new db9264d [DLAB-1467]: Added terraform scripts for deploying DLab endpoint db9264d is described below commit db9264d9ae57ca9fed61a362704cb5060ce4b0bb Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:41:06 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- .../terraform/azure/endpoint/main/instance.tf | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf index 6cadd36..545def7 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf @@ -22,7 +22,6 @@ locals { endpoint_instance_name = "${var.service_base_name}-${var.endpoint_id}-endpoint" endpoint_instance_disk_name = "${var.service_base_name}-${var.endpoint_id}-endpoint-disk" - endpoimt_image = split(var.ami, "_") } data "tls_public_key" "enpoint_key" { @@ -38,9 +37,9 @@ resource "azurerm_virtual_machine" "endpoint_instance" { delete_os_disk_on_termination = true storage_image_reference { -publisher = local.endpoimt_image[0] -offer = local.endpoimt_image[1] -sku = local.endpoimt_image[2] +publisher = element(split("_", var.ami),0) +offer = element(split("_", var.ami),1) +sku = element(split("_", var.ami),2) version = "latest" } storage_os_disk { - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new fbc2ed2 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint fbc2ed2 is described below commit fbc2ed2819dc62430ca3ffd0afa983082294 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:38:21 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf | 2 +- infrastructure-provisioning/terraform/azure/endpoint/main/network.tf | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf index 546cb10..6cadd36 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/instance.tf @@ -58,7 +58,7 @@ resource "azurerm_virtual_machine" "endpoint_instance" { disable_password_authentication = false ssh_keys { key_data = data.tls_public_key.enpoint_key.public_key_openssh - path = "/home/${var.dlab_user_name}/.ssh/authorized_keys" + path = "/home/ubuntu/.ssh/authorized_keys" } } diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf index c14be83..fa157e3 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/network.tf @@ -79,7 +79,7 @@ resource "azurerm_network_interface" "endpoint-nif" { name = local.endpoint_nif_name location = data.azurerm_resource_group.data-endpoint-resource-group.location resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name - network_security_group_id = azure_security_group.enpoint-sg.id + network_security_group_id = azurerm_network_security_group.enpoint-sg.id ip_configuration { name = "configuration" - To unsubscribe, e-mail: commits-unsubscr...@dlab.apache.org For additional commands, e-mail: commits-h...@dlab.apache.org
[incubator-dlab] branch DLAB-1467 updated: [DLAB-1467]: Added terraform scripts for deploying DLab endpoint
This is an automated email from the ASF dual-hosted git repository. omartushevskyi pushed a commit to branch DLAB-1467 in repository https://gitbox.apache.org/repos/asf/incubator-dlab.git The following commit(s) were added to refs/heads/DLAB-1467 by this push: new 28dbde9 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint 28dbde9 is described below commit 28dbde9cb40c6804b6c19dca22f53ce7d41a5a92 Author: Oleh Martushevskyi AuthorDate: Tue Feb 4 11:36:55 2020 +0200 [DLAB-1467]: Added terraform scripts for deploying DLab endpoint --- .../terraform/azure/endpoint/main/outputs.tf | 2 +- .../terraform/azure/endpoint/main/sg.tf| 76 -- 2 files changed, 41 insertions(+), 37 deletions(-) diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf index fe09d3e..bc5e20d 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/outputs.tf @@ -32,5 +32,5 @@ output "vpc_id" { } output "ssn_k8s_sg_id" { - value = azure_security_group.enpoint-sg.id + value = azurerm_network_security_group.enpoint-sg.id } \ No newline at end of file diff --git a/infrastructure-provisioning/terraform/azure/endpoint/main/sg.tf b/infrastructure-provisioning/terraform/azure/endpoint/main/sg.tf index 63f11c5..9ab63cc 100644 --- a/infrastructure-provisioning/terraform/azure/endpoint/main/sg.tf +++ b/infrastructure-provisioning/terraform/azure/endpoint/main/sg.tf @@ -23,46 +23,50 @@ locals { endpoint_sg_name = "${var.service_base_name}-${var.endpoint_id}-sg" } -resource "azure_security_group" "enpoint-sg" { - name = local.endpoint_sg_name - location = var.region +resource "azurerm_network_security_group" "enpoint-sg" { + location= data.azurerm_resource_group.data-endpoint-resource-group.location + resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name + name= local.endpoint_sg_name } -resource "azure_security_group_rule" "inbound-1" { - name = "inbound-1" - security_group_names = [azure_security_group.enpoint-sg.name] - type = "Inbound" - action = "Allow" - priority = 100 - source_address_prefix = "*" - source_port_range = "*" - destination_address_prefix = "*" - destination_port_range = "22" - protocol = "TCP" +resource "azurerm_network_security_rule" "inbound-1" { + resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name + network_security_group_name = azurerm_network_security_group.enpoint-sg.name + name= "inbound-1" + direction = "Inbound" + access = "Allow" + priority= 100 + source_address_prefix = "*" + source_port_range = "*" + destination_address_prefix = "*" + destination_port_range = "22" + protocol= "TCP" } -resource "azure_security_group_rule" "inbound-2" { - name = "inbound-2" - security_group_names = [azure_security_group.enpoint-sg.name] - type = "Inbound" - action = "Allow" - priority = 200 - source_address_prefix = "*" - source_port_range = "*" - destination_address_prefix = "*" - destination_port_range = "8084" - protocol = "TCP" +resource "azurerm_network_security_rule" "inbound-2" { + resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name + network_security_group_name = azurerm_network_security_group.enpoint-sg.name + name= "inbound-2" + direction = "Inbound" + access = "Allow" + priority= 200 + source_address_prefix = "*" + source_port_range = "*" + destination_address_prefix = "*" + destination_port_range = "8084" + protocol= "TCP" } -resource "azure_security_group_rule" "outbound-1" { - name = "outbound-1" - security_group_names = [azure_security_group.enpoint-sg.name] - type = "Outbound" - action = "Allow" - priority = 100 - source_address_prefix = "*" - source_port_range = "*" - destination_address_prefix = "*" - destination_port_range = "*" - protocol = "*" +resource "azurerm_network_security_rule" "outbound-1" { + resource_group_name = data.azurerm_resource_group.data-endpoint-resource-group.name + network_security_group_name = azurerm_network_security_group.enpoint-sg.name + name= "outbound-1" + direction