I think amazon/aws is better for 2 reasons:
- Consistent with what we are doing with google - There can be API services for Amazon platform like https://developer.amazon.com/apps-and-games/incentives-api | https://developer.amazon.com/docs/app-submission-api/overview.html etc On Thu, Oct 31, 2019 at 10:44 AM Ash Berlin-Taylor <a...@apache.org> wrote: > Personally I would prefer not amazon. - it feels unnecessary as there's > unlikely to be anything else but amazon.aws > > -a > > On 31 October 2019 09:29:18 GMT, MinJae Kwon <mingram...@gmail.com> wrote: > >I love all those rules too :) > > > >On 2019/10/30 08:13:27, Bas Harenslak <basharens...@godatadriven.com> > >wrote: > >> Hi, > >> > >> How about the following conventions? > >> > >> * No AWS in whatever notation in the class name, since this is > >already in the module (airflow.providers.aws….) > >> * Only exception to above is the AwsHook, which is a “base” hook > >which most other hooks inherit from > >> * No “Transfer” in class names, since all transfer operators > >already have “to” in the name, e.g. RedshiftToS3Transfer becomes > >RedshiftToS3Operator > >> * For all class names, the naming convention is > >{service}{component}, e.g. “AthenaHook" > >> * For all “transfer” names, the naming convention is > >{service}To{service}{component}, e.g. “RedshiftToS3Operator" > >> * Migrate all AWS related components to /airflow/providers/aws/… > >> * Sidenote: lambda is a reserved Python keyword, so name the > >Lambda service files "lambda_.py", according to Python conventions to > >avoid name clashes > >> > >> Applying these rules, I get the following changes (done some regex, > >might have missed a few): > >> > >> ./contrib/hooks/aws_dynamodb_hook.py:AwsDynamoDBHook -> > >./providers/aws/hooks/dynamodb.py:DynamoDBHook > >> ./contrib/hooks/aws_firehose_hook.py:AwsFirehoseHook -> > >./providers/aws/hooks/firehose.py:FirehoseHook > >> ./contrib/hooks/aws_glue_catalog_hook.py:AwsGlueCatalogHook -> > >./providers/aws/hooks/glue.py:GlueHook > >> ./contrib/hooks/aws_hook.py:AwsHook -> > >./providers/aws/hooks/base/base.py:AwsHook > >> ./contrib/hooks/aws_lambda_hook.py:AwsLambdaHook -> > >./providers/aws/hooks/lambda_.py:LambdaHook > >> ./contrib/hooks/aws_logs_hook.py:AwsLogsHook -> > >./providers/aws/hooks/cloudwatch.py:CloudwatchLogsHook > >> ./contrib/hooks/aws_sns_hook.py:AwsSnsHook -> > >./providers/aws/hooks/sns.py:SnsHook > >> ./contrib/hooks/aws_sqs_hook.py:SQSHook -> > >./providers/aws/hooks/sqs.py:SqsHook > >> ./contrib/hooks/emr_hook.py:EmrHook -> > >./providers/aws/hooks/emr.py:EmrHook > >> ./contrib/hooks/redshift_hook.py:RedshiftHook -> > >./providers/aws/hooks/redshift.py:RedshiftHook > >> ./contrib/hooks/sagemaker_hook.py:SageMakerHook -> > >./providers/aws/hooks/sagemaker.py:SagemakerHook > >> ./hooks/S3_hook.py:S3Hook -> ./providers/aws/hooks/s3.py:S3Hook > >> ./providers/aws/hooks/athena.py:AWSAthenaHook -> > >./providers/aws/hooks/athena.py:AthenaHook > >> > >> ./contrib/operators/aws_sqs_publish_operator.py:SQSPublishOperator -> > >./providers/aws/operators/sqs.py:SQSPublishOperator > >> ./contrib/operators/dynamodb_to_s3.py:DynamoDBToS3Operator -> > >./providers/aws/operators/s3.py:DynamoDBToS3Operator > >> > >./contrib/operators/hive_to_dynamodb.py:HiveToDynamoDBTransferOperator > >-> ./providers/aws/operators/dynamodb.py:HiveToDynamoDBOperator > >> > > >./contrib/operators/imap_attachment_to_s3_operator.py:ImapAttachmentToS3Operator > >-> ./providers/aws/operators/s3.py:ImapAttachmentToS3Operator > >> ./contrib/operators/mongo_to_s3.py:MongoToS3Operator -> > >./providers/aws/operators/s3.py:MongoToS3Operator > >> ./contrib/operators/s3_copy_object_operator.py:S3CopyObjectOperator > >-> ./providers/aws/operators/s3.py:S3CopyObjectOperator > >> > >./contrib/operators/s3_delete_objects_operator.py:S3DeleteObjectsOperator > >-> ./providers/aws/operators/s3.py:S3DeleteObjectsOperator > >> ./contrib/operators/s3_list_operator.py:S3ListOperator -> > >./providers/aws/operators/s3.py:S3ListOperator > >> > >./contrib/operators/s3_to_gcs_operator.py:S3ToGoogleCloudStorageOperator > >-> ./providers/aws/operators/s3.py:S3ToGoogleCloudStorageOperator > >> ./contrib/operators/s3_to_sftp_operator.py:S3ToSFTPOperator -> > >./providers/aws/operators/s3.py:S3ToSFTPOperator > >> ./contrib/operators/sftp_to_s3_operator.py:SFTPToS3Operator -> > >./providers/aws/operators/s3.py:SFTPToS3Operator > >> ./operators/gcs_to_s3.py:GoogleCloudStorageToS3Operator -> > >./providers/aws/operators/s3.py:GoogleCloudStorageToS3Operator > >> ./operators/google_api_to_s3_transfer.py:GoogleApiToS3Transfer -> > >./providers/aws/operators/s3.py:GoogleApiToS3Operator > >> ./operators/redshift_to_s3_operator.py:RedshiftToS3Transfer -> > >./providers/aws/operators/s3.py:RedshiftToS3Operator > >> ./operators/s3_file_transform_operator.py:S3FileTransformOperator -> > >./providers/aws/operators/s3.py:S3FileTransformOperator > >> ./operators/s3_to_hive_operator.py:S3ToHiveTransfer -> > >./providers/aws/operators/s3.py:S3ToHiveOperator > >> ./operators/s3_to_redshift_operator.py:S3ToRedshiftTransfer -> > >./providers/aws/operators/s3.py:S3ToRedshiftOperator > >> ./providers/aws/operators/athena.py:AWSAthenaOperator -> > >./providers/aws/operators/athena.py:AthenaOperator > >> ./providers/aws/operators/batch.py:AWSBatchOperator -> > >./providers/aws/operators/batch.py:BatchOperator > >> > >> > > >./contrib/sensors/aws_glue_catalog_partition_sensor.py:AwsGlueCatalogPartitionSensor > >-> ./providers/aws/sensors/glue.py:GlueCatalogPartitionSensor > >> > >./contrib/sensors/aws_redshift_cluster_sensor.py:AwsRedshiftClusterSensor > >-> ./providers/aws/sensors/redshift.py:RedshiftClusterSensor > >> ./contrib/sensors/aws_sqs_sensor.py:SQSSensor -> > >./providers/aws/sensors/sqs.py:SQSSensor > >> ./providers/aws/sensors/athena.py:AthenaSensor -> > >./providers/aws/sensors/athena.py:AthenaSensor > >> ./sensors/s3_key_sensor.py:S3KeySensor -> > >./providers/aws/sensors/s3.py:S3KeySensor > >> ./sensors/s3_prefix_sensor.py:S3PrefixSensor -> > >./providers/aws/sensors/s3.py:S3PrefixSensor > >> > >> WDYT? > >> > >> Bas > >> > >> On 30 Oct 2019, at 08:52, Tomasz Urbaszek > ><tomasz.urbas...@polidea.com<mailto:tomasz.urbas...@polidea.com>> > >wrote: > >> > >> I think it's a great idea and I am willing to help. > >> > >> Bests, > >> Tomek > >> > >> On Wed, Oct 30, 2019 at 3:17 AM MinJae Kwon > ><mingram...@gmail.com<mailto:mingram...@gmail.com>> wrote: > >> > >> Hi all. > >> > >> While I was working on AIRFLOW-5803, I found there are > >inconsistencies > >> between class names for each AWS operators/hooks. > >> > >> For example, Athena hook is named as 'AWSAthenaHook', but Dynamodb > >hook is > >> named as 'AwsDynamoDBHook', and S3 hook is just 'S3Hook' (So, > >AIRFLOW-5803 > >> is trying to rename S3Hook to AWSS3Hook) > >> > >> So we might need to have some conventions or guidelines for AWS > >> integrations like GCP ( > >> > > > https://docs.google.com/document/d/1_rTdJSLCt0eyrAylmmgYc3yZr-_h51fVlnvMmWqhCkY/edit?ts=5bb72dfd# > >> ) > >> > >> What do you think about it? > >> > >> Thank you. > >> > >> > >> > >> -- > >> > >> Tomasz Urbaszek > >> Polidea <https://www.polidea.com/> | Junior Software Engineer > >> > >> M: +48 505 628 493 <+48505628493> > >> E: tomasz.urbas...@polidea.com<mailto:tomasz.urbas...@polidea.com> > ><tomasz.urbasz...@polidea.com<mailto:tomasz.urbasz...@polidea.com>> > >> > >> Unique Tech > >> Check out our projects! <https://www.polidea.com/our-work> > >> > >> >
