Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-27 Thread Casey Bodley
s3cmd does have special handling for 'US' and 'us-east-1' that skips the LocationConstraint on bucket creation: https://github.com/s3tools/s3cmd/blob/master/S3/S3.py#L380 On 02/26/2018 05:16 PM, David Turner wrote: I just realized the difference between the internal realm, local realm, and lo

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
I just realized the difference between the internal realm, local realm, and local-atl realm. local-atl is a Luminous cluster while the other 2 are Jewel. It looks like that option was completely ignored in Jewel and now Luminous is taking it into account (which is better imo). I think you're rig

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread Yehuda Sadeh-Weinraub
I don't know why 'us' works for you, but it could be that s3cmd is just not sending any location constraint when 'us' is set. You can try looking at the capture for this. You can try using wireshark for the capture (assuming http endpoint and not https). Yehuda On Mon, Feb 26, 2018 at 1:21 PM, Da

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
I set it to that for randomness. I don't have a zonegroup named 'us' either, but that works fine. I don't see why 'cn' should be any different. The bucket_location that triggered me noticing this was 'gd1'. I don't know where that one came from, but I don't see why we should force people setting

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread Yehuda Sadeh-Weinraub
If that's what you set in the config file, I assume that's what passed in. Why did you set that in your config file? You don't have a zonegroup named 'cn', right? On Mon, Feb 26, 2018 at 1:10 PM, David Turner wrote: > I'm also not certain how to do the tcpdump for this. Do you have any > pointer

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
I'm also not certain how to do the tcpdump for this. Do you have any pointers to how to capture that for you? On Mon, Feb 26, 2018 at 4:09 PM David Turner wrote: > That's what I set it to in the config file. I probably should have > mentioned that. > > On Mon, Feb 26, 2018 at 4:07 PM Yehuda Sa

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
That's what I set it to in the config file. I probably should have mentioned that. On Mon, Feb 26, 2018 at 4:07 PM Yehuda Sadeh-Weinraub wrote: > According to the log here, it says that the location constraint it got > is "cn", can you take a look at a tcpdump, see if that's actually > what's p

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread Yehuda Sadeh-Weinraub
According to the log here, it says that the location constraint it got is "cn", can you take a look at a tcpdump, see if that's actually what's passed in? On Mon, Feb 26, 2018 at 12:02 PM, David Turner wrote: > I run with `debug rgw = 10` and was able to find these lines at the end of a > request

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
I run with `debug rgw = 10` and was able to find these lines at the end of a request to create the bucket. Successfully creating a bucket with `bucket_location = US` looks like [1]this. Failing to create a bucket has "ERROR: S3 error: 400 (InvalidLocationConstraint): The specified location-constr

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread Yehuda Sadeh-Weinraub
I'm not sure if the rgw logs (debug rgw = 20) specify explicitly why a bucket creation is rejected in these cases, but it might be worth trying to look at these. If not, then a tcpdump of the specific failed request might shed some light (would be interesting to look at the generated LocationConstr

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-26 Thread David Turner
Our problem only appeared to be present in bucket creation. Listing, putting, etc objects in a bucket work just fine regardless of the bucket_location setting. I ran this test on a few different realms to see what would happen and only 1 of them had a problem. There isn't an obvious thing that s

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread F21
I am using the official ceph/daemon docker image. It starts RGW and creates a zonegroup and zone with their names set to an empty string: https://github.com/ceph/ceph-container/blob/master/ceph-releases/luminous/ubuntu/16.04/daemon/start_rgw.sh#L36:54 $RGW_ZONEGROUP and $RGW_ZONE are both empty

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread Yehuda Sadeh-Weinraub
What is the name of your zonegroup? On Mon, Feb 19, 2018 at 3:29 PM, F21 wrote: > I've done some debugging and the LocationConstraint is not being set by the > SDK by default. > > I do, however, need to set the region on the client to us-east-1 for it to > work. Anything else will return an Inval

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread F21
I've done some debugging and the LocationConstraint is not being set by the SDK by default. I do, however, need to set the region on the client to us-east-1 for it to work. Anything else will return an InvalidLocationConstraint error. Francis On 20/02/2018 8:40 AM, Yehuda Sadeh-Weinraub wrot

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread David Turner
Specifically my issue was having problems without this set in the .s3cfg file. `bucket_location = US` On Mon, Feb 19, 2018 at 5:04 PM David Turner wrote: > I wasn't using the Go SDK. I was using s3cmd when I came across this. > > On Mon, Feb 19, 2018 at 4:42 PM Yehuda Sadeh-Weinraub > wrote: >

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread David Turner
I wasn't using the Go SDK. I was using s3cmd when I came across this. On Mon, Feb 19, 2018 at 4:42 PM Yehuda Sadeh-Weinraub wrote: > Sounds like the go sdk adds a location constraint to requests that > don't go to us-east-1. RGW itself is definitely isn't tied to > us-east-1, and does not know

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-19 Thread Yehuda Sadeh-Weinraub
Sounds like the go sdk adds a location constraint to requests that don't go to us-east-1. RGW itself is definitely isn't tied to us-east-1, and does not know anything about it (unless you happen to have a zonegroup named us-east-1). Maybe there's a way to configure the sdk to avoid doing that? Yeh

Re: [ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-18 Thread David Turner
I recently I came across this as well. It is an odd requirement. On Sun, Feb 18, 2018, 4:54 PM F21 wrote: > I am using the AWS Go SDK v2 (https://github.com/aws/aws-sdk-go-v2) to > talk to my RGW instance using the s3 interface. I am running ceph in > docker using the ceph/daemon docker images i

[ceph-users] Significance of the us-east-1 region when using S3 clients to talk to RGW

2018-02-18 Thread F21
I am using the AWS Go SDK v2 (https://github.com/aws/aws-sdk-go-v2) to talk to my RGW instance using the s3 interface. I am running ceph in docker using the ceph/daemon docker images in demo mode. The RGW is started with a zonegroup and zone with their names set to an empty string by the script