Hi Sumit, I was actually referring to what Robin (thank you) pointed out which is reason enough for having region in S3Hook.
There are even other situations having the region would be beneficial, that's why the AwsHook has that definition, although I would agree that their goals are different they typically refer to the same credentials, just different use cases. This makes me feel that it becomes redundant to have both an S3 connection and an AWS connection with the same credentials just be able to access region. Wouldn't it be beneficial to implement S3Hook (and S3 connection) as a special case of AwsHook (and Aws connection) and be able to use both interchangeably ? I'm also aware that one uses boto and the other uses boto3, and it seems like the project is naturally slowly migrating towards the later. Maybe there is even some discussion regarding this that I'm not aware. If you think this is a desirable move, I may start to design this move, making sure nothing gets broken in the process. WDYT? On Thu, Mar 23, 2017 at 2:46 PM Robin Bartholdson < [email protected]> wrote: > I _think_ that’s a different issue (do correct me if I’m wrong) > > For a number of regions (see the list on > http://docs.aws.amazon.com/general/latest/gr/signature-version-2.html), > you have to specify an extra host-parameter in the boto-S3Connection as > they only support signature V4 and the host-param is required to have this > working ( > http://boto.cloudhackers.com/en/latest/ref/s3.html#boto.s3.connection.HostRequiredError > ) > > -Robin > > On 23 Mar 2017, at 15:10, Sumit Maheshwari <[email protected]> wrote: > > Are we talking about this > https://github.com/apache/incubator-airflow/pull/2176 ? > > > > > On Thu, Mar 23, 2017 at 4:32 PM, Robin Bartholdson < > [email protected]> wrote: > > Hi, > > We actually have a patch for this that allows you to specify the host > parameter in the S3 config in the connections. > > Been thinking about submitting this for a while but have not had the time, > maybe now’s the time to do it :-) > > If you want to open a JIRA I can submit the patch we are using in > production! > > -Robin > > > On 23 Mar 2017, at 11:55, Jorge Alpedrinha Ramos < > [email protected]> wrote: > > Hi, > > I've been using S3Hook for a while, but I've run into some issues for not > having a defined region. I can't really say why I can't access this bucket > without defining the region, but I know that there is no way to define > region in S3Hook via the connection extra parameter unless a config_file is > used, which is not always an available method. > > I haven't found any ticket regarding this, is there any alternative ? > > Regards, > Jorge > > > > >
