Re: [ceph-users] Ceph on Centos 7

[Marco Garcês]

I had problems in CentOS 7, with the normal Ceph's mirrors... try
using the eu.ceph.com ones... it helped me at the time!
Good luck!

Marco Garcês
#sysadmin
Maputo - Mozambique

On Thu, Jan 8, 2015 at 1:09 PM, John Spray john.sp...@redhat.com wrote:
 On Tue, Jan 6, 2015 at 7:40 AM, Nur Aqilah aqi...@impact-multimedia.com
 wrote:

 I was wondering if anyone can give me some guidelines in installing ceph
 on Centos 7. I followed the guidelines on ceph.com on how to do the Quick
 Installation. But there was always this one particular error. When i typed
 in this command sudo yum update  sudo yum install ceph-deploy a long
 error pops up. I later checked and found out that el7/CentOS 7 is not listed
 in here http://ceph.com/packages/ceph-extras/rpm/


 FWIW I recently installed a CentOS7 cluster without problems.  However, I
 was only installing ceph itself using yum -- I got ceph-deploy by cloning it
 from git and running the bootstrap.sh script.

 Cheers,
 John


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] buckets and users

[Marco Garcês]

So I really need to create the region also? I thought it was using the
default region, so I didn't have to create extra regions.
Let me try to figure this out, the docs are a little bit confusing.

Marco Garcês



On Thu, Nov 6, 2014 at 6:39 PM, Craig Lewis cle...@centraldesktop.com wrote:
 You need to tell each radosgw daemon which zone to use.  In ceph.conf, I
 have:
 [client.radosgw.ceph3c]
   host = ceph3c
   rgw socket path = /var/run/ceph/radosgw.ceph3c
   keyring = /etc/ceph/ceph.client.radosgw.ceph3c.keyring
   log file = /var/log/ceph/radosgw.log
   admin socket = /var/run/ceph/radosgw.asok
   rgw dns name = us-central-1.ceph.cdlocal
   rgw region = us
   rgw region root pool = .us.rgw.root
   rgw zone = us-central-1
   rgw zone root pool = .us-central-1.rgw.root




 On Thu, Nov 6, 2014 at 6:35 AM, Marco Garcês ma...@garces.cc wrote:

 Update:

 I was able to fix the authentication error, and I have 2 radosgw
 running on the same host.
 The problem now, is, I believe I have created the zone wrong, or, I am
 doing something wrong, because I can login with the user I had before,
 and I can access his buckets. I need to have everything separated.

 Here are my zone info:

 default zone:
 { domain_root: .rgw,
   control_pool: .rgw.control,
   gc_pool: .rgw.gc,
   log_pool: .log,
   intent_log_pool: .intent-log,
   usage_log_pool: .usage,
   user_keys_pool: .users,
   user_email_pool: .users.email,
   user_swift_pool: .users.swift,
   user_uid_pool: .users.uid,
   system_key: { access_key: ,
   secret_key: },
   placement_pools: [
 { key: default-placement,
   val: { index_pool: .rgw.buckets.index,
   data_pool: .rgw.buckets,
   data_extra_pool: .rgw.buckets.extra}}]}

 env2 zone:
 { domain_root: .rgw,
   control_pool: .rgw.control,
   gc_pool: .rgw.gc,
   log_pool: .log,
   intent_log_pool: .intent-log,
   usage_log_pool: .usage,
   user_keys_pool: .users,
   user_email_pool: .users.email,
   user_swift_pool: .users.swift,
   user_uid_pool: .users.uid,
   system_key: { access_key: ,
   secret_key: },
   placement_pools: [
 { key: default-placement,
   val: { index_pool: .rgw.buckets.index,
   data_pool: .rgw.buckets,
   data_extra_pool: .rgw.buckets.extra}}]}

 Could you guys help me?



 Marco Garcês


 On Thu, Nov 6, 2014 at 3:56 PM, Marco Garcês ma...@garces.cc wrote:
  By the way,
  Is it possible to run 2 radosgw on the same host?
 
  I think I have created the zone, not sure if it was correct, because
  it used the default pool names, even though I had changed them in the
  json file I had provided.
 
  Now I am trying to run ceph-radosgw with two different entries in the
  ceph.conf file, but without sucess. Example:
 
  [client.radosgw.gw]
  host = GATEWAY
  keyring = /etc/ceph/keyring.radosgw.gw
  rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
  log file = /var/log/ceph/client.radosgw.gateway.log
  rgw print continue = false
  rgw dns name = gateway.local
  rgw enable ops log = false
  rgw enable usage log = true
  rgw usage log tick interval = 30
  rgw usage log flush threshold = 1024
  rgw usage max shards = 32
  rgw usage max user shards = 1
  rgw cache lru size = 15000
  rgw thread pool size = 2048
 
  #[client.radosgw.gw.env2]
  #host = GATEWAY
  #keyring = /etc/ceph/keyring.radosgw.gw
  #rgw socket path = /var/run/ceph/ceph.env2.radosgw.gateway.fastcgi.sock
  #log file = /var/log/ceph/client.env2.radosgw.gateway.log
  #rgw print continue = false
  #rgw dns name = cephppr.local
  #rgw enable ops log = false
  #rgw enable usage log = true
  #rgw usage log tick interval = 30
  #rgw usage log flush threshold = 1024
  #rgw usage max shards = 32
  #rgw usage max user shards = 1
  #rgw cache lru size = 15000
  #rgw thread pool size = 2048
  #rgw zone = ppr
 
  It fails to create the socket:
  2014-11-06 15:39:08.862364 7f80cc670880  0 ceph version 0.80.5
  (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 7930
  2014-11-06 15:39:08.870429 7f80cc670880  0 librados:
  client.radosgw.gw.env2 authentication error (1) Operation not
  permitted
  2014-11-06 15:39:08.870889 7f80cc670880 -1 Couldn't init storage
  provider (RADOS)
 
 
  What am I doing wrong?
 
  Marco Garcês
  #sysadmin
  Maputo - Mozambique
  [Skype] marcogarces
 
 
  On Thu, Nov 6, 2014 at 10:11 AM, Marco Garcês ma...@garces.cc wrote:
  Your solution of pre-pending the environment name to the bucket, was
  my first choice, but at the moment I can't ask the devs to change the
  code to do that. For now I have to stick with the zones solution.
  Should I follow the federated zones docs
  (http://ceph.com/docs/master/radosgw/federated-config/) but skip the
  sync step?
 
  Thank you,
 
  Marco Garcês
 
  On Wed, Nov 5, 2014 at 8:13 PM, Craig Lewis cle...@centraldesktop.com
  wrote:
  You could setup dedicated zones for each environment, and not
  replicate between them.
 
  Each zone would have it's own URL

Re: [ceph-users] buckets and users

[Marco Garcês]

Your solution of pre-pending the environment name to the bucket, was
my first choice, but at the moment I can't ask the devs to change the
code to do that. For now I have to stick with the zones solution.
Should I follow the federated zones docs
(http://ceph.com/docs/master/radosgw/federated-config/) but skip the
sync step?

Thank you,

Marco Garcês

On Wed, Nov 5, 2014 at 8:13 PM, Craig Lewis cle...@centraldesktop.com wrote:
 You could setup dedicated zones for each environment, and not
 replicate between them.

 Each zone would have it's own URL, but you would be able to re-use
 usernames and bucket names.  If different URLs are a problem, you
 might be able to get around that in the load balancer or the web
 servers.  I wouldn't really recommend that, but it's possible.


 I have a similar requirement.  I was able to pre-pending the
 environment name to the bucket in my client code, which made things
 much easier.


 On Wed, Nov 5, 2014 at 8:52 AM, Marco Garcês ma...@garces.cc wrote:
 Hi there,

 I have this situation, where I'm using the same Ceph cluster (with
 radosgw), for two different environments, QUAL and PRE-PRODUCTION.

 I need different users for each environment, but I need to create the
 same buckets, with the same name; I understand there is no way to have
 2 buckets with the same name, but how can I go around this? Perhaps
 creating a different pool for each user?

 Can you help me? Thank you in advance, my best regards,

 Marco Garcês
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] buckets and users

[Marco Garcês]

By the way,
Is it possible to run 2 radosgw on the same host?

I think I have created the zone, not sure if it was correct, because
it used the default pool names, even though I had changed them in the
json file I had provided.

Now I am trying to run ceph-radosgw with two different entries in the
ceph.conf file, but without sucess. Example:

[client.radosgw.gw]
host = GATEWAY
keyring = /etc/ceph/keyring.radosgw.gw
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/ceph/client.radosgw.gateway.log
rgw print continue = false
rgw dns name = gateway.local
rgw enable ops log = false
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024
rgw usage max shards = 32
rgw usage max user shards = 1
rgw cache lru size = 15000
rgw thread pool size = 2048

#[client.radosgw.gw.env2]
#host = GATEWAY
#keyring = /etc/ceph/keyring.radosgw.gw
#rgw socket path = /var/run/ceph/ceph.env2.radosgw.gateway.fastcgi.sock
#log file = /var/log/ceph/client.env2.radosgw.gateway.log
#rgw print continue = false
#rgw dns name = cephppr.local
#rgw enable ops log = false
#rgw enable usage log = true
#rgw usage log tick interval = 30
#rgw usage log flush threshold = 1024
#rgw usage max shards = 32
#rgw usage max user shards = 1
#rgw cache lru size = 15000
#rgw thread pool size = 2048
#rgw zone = ppr

It fails to create the socket:
2014-11-06 15:39:08.862364 7f80cc670880  0 ceph version 0.80.5
(38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 7930
2014-11-06 15:39:08.870429 7f80cc670880  0 librados:
client.radosgw.gw.env2 authentication error (1) Operation not
permitted
2014-11-06 15:39:08.870889 7f80cc670880 -1 Couldn't init storage
provider (RADOS)


What am I doing wrong?

Marco Garcês
#sysadmin
Maputo - Mozambique
[Skype] marcogarces


On Thu, Nov 6, 2014 at 10:11 AM, Marco Garcês ma...@garces.cc wrote:
 Your solution of pre-pending the environment name to the bucket, was
 my first choice, but at the moment I can't ask the devs to change the
 code to do that. For now I have to stick with the zones solution.
 Should I follow the federated zones docs
 (http://ceph.com/docs/master/radosgw/federated-config/) but skip the
 sync step?

 Thank you,

 Marco Garcês

 On Wed, Nov 5, 2014 at 8:13 PM, Craig Lewis cle...@centraldesktop.com wrote:
 You could setup dedicated zones for each environment, and not
 replicate between them.

 Each zone would have it's own URL, but you would be able to re-use
 usernames and bucket names.  If different URLs are a problem, you
 might be able to get around that in the load balancer or the web
 servers.  I wouldn't really recommend that, but it's possible.


 I have a similar requirement.  I was able to pre-pending the
 environment name to the bucket in my client code, which made things
 much easier.


 On Wed, Nov 5, 2014 at 8:52 AM, Marco Garcês ma...@garces.cc wrote:
 Hi there,

 I have this situation, where I'm using the same Ceph cluster (with
 radosgw), for two different environments, QUAL and PRE-PRODUCTION.

 I need different users for each environment, but I need to create the
 same buckets, with the same name; I understand there is no way to have
 2 buckets with the same name, but how can I go around this? Perhaps
 creating a different pool for each user?

 Can you help me? Thank you in advance, my best regards,

 Marco Garcês
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] buckets and users

[Marco Garcês]

Update:

I was able to fix the authentication error, and I have 2 radosgw
running on the same host.
The problem now, is, I believe I have created the zone wrong, or, I am
doing something wrong, because I can login with the user I had before,
and I can access his buckets. I need to have everything separated.

Here are my zone info:

default zone:
{ domain_root: .rgw,
  control_pool: .rgw.control,
  gc_pool: .rgw.gc,
  log_pool: .log,
  intent_log_pool: .intent-log,
  usage_log_pool: .usage,
  user_keys_pool: .users,
  user_email_pool: .users.email,
  user_swift_pool: .users.swift,
  user_uid_pool: .users.uid,
  system_key: { access_key: ,
  secret_key: },
  placement_pools: [
{ key: default-placement,
  val: { index_pool: .rgw.buckets.index,
  data_pool: .rgw.buckets,
  data_extra_pool: .rgw.buckets.extra}}]}

env2 zone:
{ domain_root: .rgw,
  control_pool: .rgw.control,
  gc_pool: .rgw.gc,
  log_pool: .log,
  intent_log_pool: .intent-log,
  usage_log_pool: .usage,
  user_keys_pool: .users,
  user_email_pool: .users.email,
  user_swift_pool: .users.swift,
  user_uid_pool: .users.uid,
  system_key: { access_key: ,
  secret_key: },
  placement_pools: [
{ key: default-placement,
  val: { index_pool: .rgw.buckets.index,
  data_pool: .rgw.buckets,
  data_extra_pool: .rgw.buckets.extra}}]}

Could you guys help me?



Marco Garcês


On Thu, Nov 6, 2014 at 3:56 PM, Marco Garcês ma...@garces.cc wrote:
 By the way,
 Is it possible to run 2 radosgw on the same host?

 I think I have created the zone, not sure if it was correct, because
 it used the default pool names, even though I had changed them in the
 json file I had provided.

 Now I am trying to run ceph-radosgw with two different entries in the
 ceph.conf file, but without sucess. Example:

 [client.radosgw.gw]
 host = GATEWAY
 keyring = /etc/ceph/keyring.radosgw.gw
 rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
 log file = /var/log/ceph/client.radosgw.gateway.log
 rgw print continue = false
 rgw dns name = gateway.local
 rgw enable ops log = false
 rgw enable usage log = true
 rgw usage log tick interval = 30
 rgw usage log flush threshold = 1024
 rgw usage max shards = 32
 rgw usage max user shards = 1
 rgw cache lru size = 15000
 rgw thread pool size = 2048

 #[client.radosgw.gw.env2]
 #host = GATEWAY
 #keyring = /etc/ceph/keyring.radosgw.gw
 #rgw socket path = /var/run/ceph/ceph.env2.radosgw.gateway.fastcgi.sock
 #log file = /var/log/ceph/client.env2.radosgw.gateway.log
 #rgw print continue = false
 #rgw dns name = cephppr.local
 #rgw enable ops log = false
 #rgw enable usage log = true
 #rgw usage log tick interval = 30
 #rgw usage log flush threshold = 1024
 #rgw usage max shards = 32
 #rgw usage max user shards = 1
 #rgw cache lru size = 15000
 #rgw thread pool size = 2048
 #rgw zone = ppr

 It fails to create the socket:
 2014-11-06 15:39:08.862364 7f80cc670880  0 ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 7930
 2014-11-06 15:39:08.870429 7f80cc670880  0 librados:
 client.radosgw.gw.env2 authentication error (1) Operation not
 permitted
 2014-11-06 15:39:08.870889 7f80cc670880 -1 Couldn't init storage
 provider (RADOS)


 What am I doing wrong?

 Marco Garcês
 #sysadmin
 Maputo - Mozambique
 [Skype] marcogarces


 On Thu, Nov 6, 2014 at 10:11 AM, Marco Garcês ma...@garces.cc wrote:
 Your solution of pre-pending the environment name to the bucket, was
 my first choice, but at the moment I can't ask the devs to change the
 code to do that. For now I have to stick with the zones solution.
 Should I follow the federated zones docs
 (http://ceph.com/docs/master/radosgw/federated-config/) but skip the
 sync step?

 Thank you,

 Marco Garcês

 On Wed, Nov 5, 2014 at 8:13 PM, Craig Lewis cle...@centraldesktop.com 
 wrote:
 You could setup dedicated zones for each environment, and not
 replicate between them.

 Each zone would have it's own URL, but you would be able to re-use
 usernames and bucket names.  If different URLs are a problem, you
 might be able to get around that in the load balancer or the web
 servers.  I wouldn't really recommend that, but it's possible.


 I have a similar requirement.  I was able to pre-pending the
 environment name to the bucket in my client code, which made things
 much easier.


 On Wed, Nov 5, 2014 at 8:52 AM, Marco Garcês ma...@garces.cc wrote:
 Hi there,

 I have this situation, where I'm using the same Ceph cluster (with
 radosgw), for two different environments, QUAL and PRE-PRODUCTION.

 I need different users for each environment, but I need to create the
 same buckets, with the same name; I understand there is no way to have
 2 buckets with the same name, but how can I go around this? Perhaps
 creating a different pool for each user?

 Can you help me? Thank you in advance, my best regards,

 Marco Garcês
 ___
 ceph-users

[ceph-users] buckets and users

[Marco Garcês]

Hi there,

I have this situation, where I'm using the same Ceph cluster (with
radosgw), for two different environments, QUAL and PRE-PRODUCTION.

I need different users for each environment, but I need to create the
same buckets, with the same name; I understand there is no way to have
2 buckets with the same name, but how can I go around this? Perhaps
creating a different pool for each user?

Can you help me? Thank you in advance, my best regards,

Marco Garcês
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] logging, radosgw and pools questions

[Marco Garcês]

Hi there,

I have a few questions regarding pools, radosgw and logging:

1) How do I turn on radosgw logs for a specific pool?
I have this in my config:

rgw enable ops log = false
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024

but when I do radosgw-admin log list I get an empty list.
Any suggestions?

2) Is it possible to have different swift/s3 users accessing radosgw,
but placing data on different pools? At the moment, it is all going on
.rgw.buckets, I would like to separate this.

3) Is it possible to have 2 different swift/s3 users, sharing buckets?

4) How to setup radosgw on 3 different hosts? Do I need to had 3
different radosgw configs inside my configs, or can I have the host
part, with these 3 hosts?

I think this is all for now.. thank you for your help in advance!

Cheers,
Marco Garcês
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ssh; cannot resolve hostname errors

[Marco Garcês]

The best way to setup SSH is to use a ~/.ssh/config file It solves
a lot of issues!
Example:

~/.ssh/config


Host ceph1 cephosd1
   HostName 192.168.1.10
   User ceph

Host ceph2 cephosd2
   HostName 192.168.1.11
   User ceph


With that you can just do a ssh ceph1 for example... All other SSH
options are available in the config file.
You can also spread this file with automation tools like ansible.

Marco Garcês
#sysadmin
Maputo - Mozambique

On Thu, Oct 16, 2014 at 1:15 AM, JIten Shah jshah2...@me.com wrote:
 Please send your /etc/hosts contents here.

 --Jiten

 On Oct 15, 2014, at 7:27 AM, Support - Avantek supp...@avantek.co.uk
 wrote:

 I may be completely overlooking something here but I keep getting “ssh;
 cannot resolve hostname” when I try to contact my OSD node’s from my monitor
 node. I have set the ipaddress’s of the 3 nodes in /etc/hosts as suggested
 on the website.

 Thanks in advance

 James
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph packages being blocked by epel packages on Centos6

[Marco Garcês]

I am getting this to, but in CentOS 7... fighting to get the 0.80.6
update, but EPEL blocks Ceph repos.

Marco Garcês
#sysadmin
Maputo - Mozambique


On Mon, Oct 13, 2014 at 8:20 AM, 10 minus t10te...@gmail.com wrote:
 Hi ,

 I have observed that the latest ceph packages from ceph are being blocked by
 ceph packages from epel on cEntos6 is it just me or are others observing
 this too.

 Cheers,

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

Hi guys, thanks for the hints...
I was able to fix it, by adding the line to nginx.conf (or fastcgi_params file):

fastcgi_param  SERVER_PORT_SECURE $server_port;


Thank you so much!

Marco Garcês
#sysadmin
Maputo - Mozambique


On Wed, Oct 8, 2014 at 6:25 PM, Yehuda Sadeh yeh...@redhat.com wrote:
 On Wed, Oct 8, 2014 at 9:21 AM, Marco Garcês ma...@garces.cc wrote:
 I believe so:
 2014-10-08 18:19:38.438133 7f9119b90700  2
 RGWDataChangesLog::ChangesRenewThread: start
 2014-10-08 18:19:44.151527 7f90ea7fc700 20 enqueued request req=0x1b9e400
 2014-10-08 18:19:44.151558 7f90ea7fc700 20 RGWWQ:
 2014-10-08 18:19:44.151561 7f90ea7fc700 20 req: 0x1b9e400
 2014-10-08 18:19:44.151569 7f90ea7fc700 10 allocated request req=0x1b9e6f0
 2014-10-08 18:19:44.151595 7f90e97fa700 20 dequeued request req=0x1b9e400
 2014-10-08 18:19:44.151600 7f90e97fa700 20 RGWWQ: empty
 2014-10-08 18:19:44.151655 7f90e97fa700 20 CONTENT_LENGTH=
 2014-10-08 18:19:44.151659 7f90e97fa700 20 CONTENT_TYPE=
 2014-10-08 18:19:44.151660 7f90e97fa700 20 
 DOCUMENT_ROOT=/usr/local/nginx/html
 2014-10-08 18:19:44.151662 7f90e97fa700 20 DOCUMENT_URI=/auth
 2014-10-08 18:19:44.151663 7f90e97fa700 20 FCGI_ROLE=RESPONDER
 2014-10-08 18:19:44.151665 7f90e97fa700 20 GATEWAY_INTERFACE=CGI/1.1
 2014-10-08 18:19:44.151666 7f90e97fa700 20 HTTP_ACCEPT=*/*
 2014-10-08 18:19:44.151668 7f90e97fa700 20 HTTP_HOST=gateway.local
 2014-10-08 18:19:44.151669 7f90e97fa700 20 HTTP_SERVER_PORT_SECURE=443

 This is not what we expect. The server translates it into
 HTTP_SERVER_PORT_SECURE, whereas we need it to be SERVER_PORT_SECURE.
 Maybe there's a way to configure the web server to send the needed
 header?

 Yehuda

 2014-10-08 18:19:44.151670 7f90e97fa700 20 HTTP_USER_AGENT=curl/7.30.0
 2014-10-08 18:19:44.151672 7f90e97fa700 20
 HTTP_X_AUTH_KEY=QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3
 2014-10-08 18:19:44.151673 7f90e97fa700 20 HTTP_X_AUTH_USER=frontend:swf0002
 2014-10-08 18:19:44.151675 7f90e97fa700 20 HTTPS=on
 2014-10-08 18:19:44.151676 7f90e97fa700 20 QUERY_STRING=
 2014-10-08 18:19:44.151677 7f90e97fa700 20 REDIRECT_STATUS=200
 2014-10-08 18:19:44.151678 7f90e97fa700 20 REMOTE_ADDR=10.5.5.222
 2014-10-08 18:19:44.151679 7f90e97fa700 20 REMOTE_PORT=64145
 2014-10-08 18:19:44.151680 7f90e97fa700 20 REQUEST_METHOD=GET
 2014-10-08 18:19:44.151681 7f90e97fa700 20 REQUEST_URI=/auth
 2014-10-08 18:19:44.151682 7f90e97fa700 20 SCRIPT_NAME=/auth
 2014-10-08 18:19:44.151683 7f90e97fa700 20 SERVER_ADDR=10.2.27.80
 2014-10-08 18:19:44.151684 7f90e97fa700 20 SERVER_NAME=gateway.local
 2014-10-08 18:19:44.151685 7f90e97fa700 20 SERVER_PORT=443
 2014-10-08 18:19:44.151686 7f90e97fa700 20 SERVER_PROTOCOL=HTTP/1.1
 2014-10-08 18:19:44.151687 7f90e97fa700 20 SERVER_SOFTWARE=nginx/1.4.7
 2014-10-08 18:19:44.151690 7f90e97fa700  1 == starting new request
 req=0x1b9e400 =
 2014-10-08 18:19:44.151711 7f90e97fa700  2 req 2:0.22::GET
 /auth::initializing
 2014-10-08 18:19:44.151718 7f90e97fa700 10 host=gateway.local
 rgw_dns_name=gateway.local
 2014-10-08 18:19:44.151757 7f90e97fa700  2 req
 2:0.68:swift-auth:GET /auth::getting op
 2014-10-08 18:19:44.151763 7f90e97fa700  2 req
 2:0.75:swift-auth:GET /auth:swift_auth_get:authorizing
 2014-10-08 18:19:44.151767 7f90e97fa700  2 req
 2:0.78:swift-auth:GET /auth:swift_auth_get:reading permissions
 2014-10-08 18:19:44.151770 7f90e97fa700  2 req
 2:0.82:swift-auth:GET /auth:swift_auth_get:init op
 2014-10-08 18:19:44.151773 7f90e97fa700  2 req
 2:0.85:swift-auth:GET /auth:swift_auth_get:verifying op mask
 2014-10-08 18:19:44.151797 7f90e97fa700 20 required_mask= 0 user.op_mask=7
 2014-10-08 18:19:44.151799 7f90e97fa700  2 req
 2:0.000111:swift-auth:GET /auth:swift_auth_get:verifying op
 permissions
 2014-10-08 18:19:44.151803 7f90e97fa700  2 req
 2:0.000115:swift-auth:GET /auth:swift_auth_get:verifying op params
 2014-10-08 18:19:44.151806 7f90e97fa700  2 req
 2:0.000117:swift-auth:GET /auth:swift_auth_get:executing
 2014-10-08 18:19:44.151874 7f90e97fa700 20 get_obj_state:
 rctx=0x7f90d8018380 obj=.users.swift:frontend:swf0002
 state=0x7f90d8022c18 s-prefetch_data=0
 2014-10-08 18:19:44.151895 7f90e97fa700 10 cache get:
 name=.users.swift+frontend:swf0002 : type miss (requested=6, cached=3)
 2014-10-08 18:19:44.153757 7f90e97fa700 10 cache put:
 name=.users.swift+frontend:swf0002
 2014-10-08 18:19:44.153763 7f90e97fa700 10 moving
 .users.swift+frontend:swf0002 to cache LRU end
 2014-10-08 18:19:44.153770 7f90e97fa700 20 get_obj_state: s-obj_tag
 was set empty
 2014-10-08 18:19:44.153780 7f90e97fa700 10 cache get:
 name=.users.swift+frontend:swf0002 : hit
 2014-10-08 18:19:44.153828 7f90e97fa700 20 get_obj_state:
 rctx=0x7f90d8018380 obj=.users.uid:frontend state=0x7f90d8023578
 s-prefetch_data=0
 2014-10-08 18:19:44.153837 7f90e97fa700 10 cache get:
 name=.users.uid+frontend : type miss (requested=6, cached=3)
 2014-10-08 18:19:44.154943 7f90e97fa700 10 cache put: 
 name=.users.uid+frontend
 2014-10-08 18

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

I spoke to soon...
Now if I use HTTP I get errors!
Let me try to debug, and post back.

Thanks,

Marco Garcês
#sysadmin
Maputo - Mozambique
[Phone] +258 84 4105579
[Skype] marcogarces


On Thu, Oct 9, 2014 at 10:38 AM, Marco Garcês ma...@garces.cc wrote:
 Hi guys, thanks for the hints...
 I was able to fix it, by adding the line to nginx.conf (or fastcgi_params 
 file):

 fastcgi_param  SERVER_PORT_SECURE $server_port;


 Thank you so much!

 Marco Garcês
 #sysadmin
 Maputo - Mozambique


 On Wed, Oct 8, 2014 at 6:25 PM, Yehuda Sadeh yeh...@redhat.com wrote:
 On Wed, Oct 8, 2014 at 9:21 AM, Marco Garcês ma...@garces.cc wrote:
 I believe so:
 2014-10-08 18:19:38.438133 7f9119b90700  2
 RGWDataChangesLog::ChangesRenewThread: start
 2014-10-08 18:19:44.151527 7f90ea7fc700 20 enqueued request req=0x1b9e400
 2014-10-08 18:19:44.151558 7f90ea7fc700 20 RGWWQ:
 2014-10-08 18:19:44.151561 7f90ea7fc700 20 req: 0x1b9e400
 2014-10-08 18:19:44.151569 7f90ea7fc700 10 allocated request req=0x1b9e6f0
 2014-10-08 18:19:44.151595 7f90e97fa700 20 dequeued request req=0x1b9e400
 2014-10-08 18:19:44.151600 7f90e97fa700 20 RGWWQ: empty
 2014-10-08 18:19:44.151655 7f90e97fa700 20 CONTENT_LENGTH=
 2014-10-08 18:19:44.151659 7f90e97fa700 20 CONTENT_TYPE=
 2014-10-08 18:19:44.151660 7f90e97fa700 20 
 DOCUMENT_ROOT=/usr/local/nginx/html
 2014-10-08 18:19:44.151662 7f90e97fa700 20 DOCUMENT_URI=/auth
 2014-10-08 18:19:44.151663 7f90e97fa700 20 FCGI_ROLE=RESPONDER
 2014-10-08 18:19:44.151665 7f90e97fa700 20 GATEWAY_INTERFACE=CGI/1.1
 2014-10-08 18:19:44.151666 7f90e97fa700 20 HTTP_ACCEPT=*/*
 2014-10-08 18:19:44.151668 7f90e97fa700 20 HTTP_HOST=gateway.local
 2014-10-08 18:19:44.151669 7f90e97fa700 20 HTTP_SERVER_PORT_SECURE=443

 This is not what we expect. The server translates it into
 HTTP_SERVER_PORT_SECURE, whereas we need it to be SERVER_PORT_SECURE.
 Maybe there's a way to configure the web server to send the needed
 header?

 Yehuda

 2014-10-08 18:19:44.151670 7f90e97fa700 20 HTTP_USER_AGENT=curl/7.30.0
 2014-10-08 18:19:44.151672 7f90e97fa700 20
 HTTP_X_AUTH_KEY=QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3
 2014-10-08 18:19:44.151673 7f90e97fa700 20 HTTP_X_AUTH_USER=frontend:swf0002
 2014-10-08 18:19:44.151675 7f90e97fa700 20 HTTPS=on
 2014-10-08 18:19:44.151676 7f90e97fa700 20 QUERY_STRING=
 2014-10-08 18:19:44.151677 7f90e97fa700 20 REDIRECT_STATUS=200
 2014-10-08 18:19:44.151678 7f90e97fa700 20 REMOTE_ADDR=10.5.5.222
 2014-10-08 18:19:44.151679 7f90e97fa700 20 REMOTE_PORT=64145
 2014-10-08 18:19:44.151680 7f90e97fa700 20 REQUEST_METHOD=GET
 2014-10-08 18:19:44.151681 7f90e97fa700 20 REQUEST_URI=/auth
 2014-10-08 18:19:44.151682 7f90e97fa700 20 SCRIPT_NAME=/auth
 2014-10-08 18:19:44.151683 7f90e97fa700 20 SERVER_ADDR=10.2.27.80
 2014-10-08 18:19:44.151684 7f90e97fa700 20 SERVER_NAME=gateway.local
 2014-10-08 18:19:44.151685 7f90e97fa700 20 SERVER_PORT=443
 2014-10-08 18:19:44.151686 7f90e97fa700 20 SERVER_PROTOCOL=HTTP/1.1
 2014-10-08 18:19:44.151687 7f90e97fa700 20 SERVER_SOFTWARE=nginx/1.4.7
 2014-10-08 18:19:44.151690 7f90e97fa700  1 == starting new request
 req=0x1b9e400 =
 2014-10-08 18:19:44.151711 7f90e97fa700  2 req 2:0.22::GET
 /auth::initializing
 2014-10-08 18:19:44.151718 7f90e97fa700 10 host=gateway.local
 rgw_dns_name=gateway.local
 2014-10-08 18:19:44.151757 7f90e97fa700  2 req
 2:0.68:swift-auth:GET /auth::getting op
 2014-10-08 18:19:44.151763 7f90e97fa700  2 req
 2:0.75:swift-auth:GET /auth:swift_auth_get:authorizing
 2014-10-08 18:19:44.151767 7f90e97fa700  2 req
 2:0.78:swift-auth:GET /auth:swift_auth_get:reading permissions
 2014-10-08 18:19:44.151770 7f90e97fa700  2 req
 2:0.82:swift-auth:GET /auth:swift_auth_get:init op
 2014-10-08 18:19:44.151773 7f90e97fa700  2 req
 2:0.85:swift-auth:GET /auth:swift_auth_get:verifying op mask
 2014-10-08 18:19:44.151797 7f90e97fa700 20 required_mask= 0 user.op_mask=7
 2014-10-08 18:19:44.151799 7f90e97fa700  2 req
 2:0.000111:swift-auth:GET /auth:swift_auth_get:verifying op
 permissions
 2014-10-08 18:19:44.151803 7f90e97fa700  2 req
 2:0.000115:swift-auth:GET /auth:swift_auth_get:verifying op params
 2014-10-08 18:19:44.151806 7f90e97fa700  2 req
 2:0.000117:swift-auth:GET /auth:swift_auth_get:executing
 2014-10-08 18:19:44.151874 7f90e97fa700 20 get_obj_state:
 rctx=0x7f90d8018380 obj=.users.swift:frontend:swf0002
 state=0x7f90d8022c18 s-prefetch_data=0
 2014-10-08 18:19:44.151895 7f90e97fa700 10 cache get:
 name=.users.swift+frontend:swf0002 : type miss (requested=6, cached=3)
 2014-10-08 18:19:44.153757 7f90e97fa700 10 cache put:
 name=.users.swift+frontend:swf0002
 2014-10-08 18:19:44.153763 7f90e97fa700 10 moving
 .users.swift+frontend:swf0002 to cache LRU end
 2014-10-08 18:19:44.153770 7f90e97fa700 20 get_obj_state: s-obj_tag
 was set empty
 2014-10-08 18:19:44.153780 7f90e97fa700 10 cache get:
 name=.users.swift+frontend:swf0002 : hit
 2014-10-08 18:19:44.153828 7f90e97fa700 20 get_obj_state:
 rctx=0x7f90d8018380 obj

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

Fixed.. I attach the server part, for nginx/tengine config file:

server {
listen 80;
server_name gateway.local;
error_log logs/error_http.log debug;
client_max_body_size 100m;

fastcgi_request_buffering off;

location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;

if ($request_method  = PUT ) {
rewrite ^ /PUT$request_uri;
 }
 include fastcgi_params;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

 location /PUT/ {
 internal;
 fastcgi_pass_header Authorization;
 fastcgi_pass_request_headers on;

 include fastcgi_params;
 fastcgi_param  CONTENT_LENGTH   $content_length;
 fastcgi_param HTTPS on;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

}
server {
listen 10.2.27.80:443 ssl default;

server_name gateway.local;
error_log logs/error_https.log debug;
client_max_body_size 100m;

fastcgi_request_buffering off;

ssl_certificate  /etc/pki/tls/certs/ca_rgw.crt;
ssl_certificate_key  /etc/pki/tls/private/ca_rgw.key;

ssl_session_timeout  5m;

ssl_protocols  SSLv2 SSLv3 TLSv1;
ssl_ciphers  HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers   on;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
fastcgi_param HTTPS on;
fastcgi_param  SERVER_PORT_SECURE $server_port;

if ($request_method  = PUT ) {
rewrite ^ /PUT$request_uri;
 }
 include fastcgi_params;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

 location /PUT/ {
 internal;
 fastcgi_pass_header Authorization;
 fastcgi_pass_request_headers on;

 include fastcgi_params;
 fastcgi_param  CONTENT_LENGTH   $content_length;
 fastcgi_param HTTPS on;
 fastcgi_param  SERVER_PORT_SECURE $server_port;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

}
}

I had the /server listening on 80 and 443 together, and I just had to
separate everything, and include the fastcgi_param
SERVER_PORT_SECURE $server_port; on the 443 listener.
I hope this helps someone same day! :)

Thank you once again!


Marco Garcês
#sysadmin
Maputo - Mozambique

On Thu, Oct 9, 2014 at 10:52 AM, Marco Garcês ma...@garces.cc wrote:
 I spoke to soon...
 Now if I use HTTP I get errors!
 Let me try to debug, and post back.

 Thanks,

 Marco Garcês
 #sysadmin
 Maputo - Mozambique
 [Phone] +258 84 4105579
 [Skype] marcogarces


 On Thu, Oct 9, 2014 at 10:38 AM, Marco Garcês ma...@garces.cc wrote:
 Hi guys, thanks for the hints...
 I was able to fix it, by adding the line to nginx.conf (or fastcgi_params 
 file):

 fastcgi_param  SERVER_PORT_SECURE $server_port;


 Thank you so much!

 Marco Garcês
 #sysadmin
 Maputo - Mozambique


 On Wed, Oct 8, 2014 at 6:25 PM, Yehuda Sadeh yeh...@redhat.com wrote:
 On Wed, Oct 8, 2014 at 9:21 AM, Marco Garcês ma...@garces.cc wrote:
 I believe so:
 2014-10-08 18:19:38.438133 7f9119b90700  2
 RGWDataChangesLog::ChangesRenewThread: start
 2014-10-08 18:19:44.151527 7f90ea7fc700 20 enqueued request req=0x1b9e400
 2014-10-08 18:19:44.151558 7f90ea7fc700 20 RGWWQ:
 2014-10-08 18:19:44.151561 7f90ea7fc700 20 req: 0x1b9e400
 2014-10-08 18:19:44.151569 7f90ea7fc700 10 allocated request req=0x1b9e6f0
 2014-10-08 18:19:44.151595 7f90e97fa700 20 dequeued request req=0x1b9e400
 2014-10-08 18:19:44.151600 7f90e97fa700 20 RGWWQ: empty
 2014-10-08 18:19:44.151655 7f90e97fa700 20 CONTENT_LENGTH=
 2014-10-08 18:19:44.151659 7f90e97fa700 20 CONTENT_TYPE=
 2014-10-08 18:19:44.151660 7f90e97fa700 20 
 DOCUMENT_ROOT=/usr/local/nginx/html
 2014-10-08 18:19:44.151662 7f90e97fa700 20 DOCUMENT_URI=/auth
 2014-10-08 18:19:44.151663 7f90e97fa700 20 FCGI_ROLE=RESPONDER
 2014-10-08 18:19:44.151665 7f90e97fa700 20 GATEWAY_INTERFACE=CGI/1.1
 2014-10-08 18:19:44.151666 7f90e97fa700 20 HTTP_ACCEPT=*/*
 2014-10-08 18:19:44.151668 7f90e97fa700 20 HTTP_HOST=gateway.local
 2014-10-08 18:19:44.151669 7f90e97fa700 20 HTTP_SERVER_PORT_SECURE=443

 This is not what we expect. The server translates it into
 HTTP_SERVER_PORT_SECURE, whereas we need it to be SERVER_PORT_SECURE.
 Maybe there's a way to configure the web server to send the needed
 header?

 Yehuda

 2014-10-08 18:19:44.151670 7f90e97fa700 20 HTTP_USER_AGENT=curl/7.30.0
 2014-10-08 18:19:44.151672 7f90e97fa700 20
 HTTP_X_AUTH_KEY=QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3
 2014-10-08 18:19:44.151673

Re: [ceph-users] How to restore a Ceph cluster from its cluster map？

[Marco Garcês]

Im in on this thread.


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces

On Wed, Oct 8, 2014 at 11:00 AM, Aegeaner xih...@gmail.com wrote:


 Hi all!

 For production use, I want to use two ceph clusters at the same time.
 One is the master cluster, and the other is the replication cluster,
 which syncs RBD snapshots with master cluster at fixed time (every day,
 e.g.), by the way this article describes:
 http://ceph.com/dev-notes/incremental-snapshots-with-rbd/ . In case the
 master cluster is down, I mean, there is some problem with ceph so that
 the whole cluster is down, I can switch from master cluster to slave
 cluster.

 Now the question is, if the master cluster is down, and if I have backed
 up all the metadata before: the monitor map, the osd map, the pg map,
 the crush map. How can I restore the master Ceph cluster from these
 cluster maps? Is there a tool or certain way to do it?

 Thanks!

 ===
 Aegeaner





 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] RadosGW over HTTPS

[Marco Garcês]

Hi there,

I am using RadosGW over NGINX, with Swift API, and everything is
working great, over HTTP, but with HTTPS, I keep getting errors, and
I'm guessing is something on the gateway itself.

Does anyone have a working HTTPS gateway with nginx? Can you provide
it, so I can compare to mine?

If I do a HTTP request, using Swift client from my machine, I get the
response ok, but If I try it with HTTPS, I get:

Account HEAD failed: http://gateway.local/swift/v1 400 Bad Request

and on nginx side:

2014/10/08 13:37:34 [info] 18198#0: *50 client sent plain HTTP request
to HTTPS port while reading client request headers, client:
10.5.5.222, server: *.gatew
ay.local, request: HEAD /swift/v1 HTTP/1.1, host: gateway.local:443
2014/10/08 13:37:34 [info] 18197#0: *48 client 10.5.5.222 closed
keepalive connection

I have wiresharked my connection, and there is no evidence that HTTP
traffic is going out, when I make the request via HTTPS, so thats why
I believe that the issue is on the gateway end.

NGINX Config:
server {
listen 80;
listen 443 ssl default;

server_name *.gateway.bcitestes.local gateway.bcitestes.local;
error_log logs/error_https.log debug;
client_max_body_size 10g;

# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;

ssl_certificate  /etc/pki/tls/certs/ca_rgw.crt;
ssl_certificate_key  /etc/pki/tls/private/ca_rgw.key;

ssl_session_timeout  5m;

ssl_protocols  SSLv2 SSLv3 TLSv1;
ssl_ciphers  HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers   on;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
fastcgi_param HTTPS on;

if ($request_method  = PUT ) {
rewrite ^ /PUT$request_uri;
 }
 include fastcgi_params;
 fastcgi_param HTTPS on;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

 location /PUT/ {
 internal;
 fastcgi_pass_header Authorization;
 fastcgi_pass_request_headers on;

 include fastcgi_params;
 fastcgi_param  CONTENT_LENGTH   $content_length;
 fastcgi_param HTTPS on;

 fastcgi_pass
unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

}

Ceph config:
[client.radosgw.gw]
host = GATEWAY
keyring = /etc/ceph/keyring.radosgw.gw
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/ceph/client.radosgw.gateway.log
rgw print continue = false
rgw dns name = gateway.bcitestes.local
rgw enable ops log = false
rgw enable usage log = true
rgw usage log tick interval = 30
rgw usage log flush threshold = 1024
rgw usage max shards = 32
rgw usage max user shards = 1
rgw cache lru size = 15000
rgw thread pool size = 2048

--

Thanks in advance,

Marco Garcês
#sysadmin
Maputo - Mozambique
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

I made some tests:
curl -D - -H X-Auth-Key: QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3 -H
X-Auth-User: frontend:swf0002 https://gateway.local/auth
HTTP/1.1 204
Server: Tengine/2.0.3
Date: Wed, 08 Oct 2014 14:04:18 GMT
Content-Type: application/json
Connection: keep-alive
X-Storage-Url: http://gateway.local:443/swift/v1
X-Storage-Token:
AUTH_rgwtk100066726f6e74656e643a73776630303032e697e4fb9734a3a2e2953654283e3a005ab9b8a2b1eb5025d053078d76b46f4957690240
X-Auth-Token: 
AUTH_rgwtk100066726f6e74656e643a73776630303032e697e4fb9734a3a2e2953654283e3a005ab9b8a2b1eb5025d053078d76b46f4957690240


What upsets me, is I'm getting response header X-Storage-Url all
wrong, it should be https!

Any clues are welcomed. Thanks!

Marco Garcês
#sysadmin
Maputo - Mozambique
[Phone] +258 84 4105579
[Skype] marcogarces


On Wed, Oct 8, 2014 at 1:53 PM, Marco Garcês ma...@garces.cc wrote:
 Hi there,

 I am using RadosGW over NGINX, with Swift API, and everything is
 working great, over HTTP, but with HTTPS, I keep getting errors, and
 I'm guessing is something on the gateway itself.

 Does anyone have a working HTTPS gateway with nginx? Can you provide
 it, so I can compare to mine?

 If I do a HTTP request, using Swift client from my machine, I get the
 response ok, but If I try it with HTTPS, I get:

 Account HEAD failed: http://gateway.local/swift/v1 400 Bad Request

 and on nginx side:

 2014/10/08 13:37:34 [info] 18198#0: *50 client sent plain HTTP request
 to HTTPS port while reading client request headers, client:
 10.5.5.222, server: *.gatew
 ay.local, request: HEAD /swift/v1 HTTP/1.1, host: gateway.local:443
 2014/10/08 13:37:34 [info] 18197#0: *48 client 10.5.5.222 closed
 keepalive connection

 I have wiresharked my connection, and there is no evidence that HTTP
 traffic is going out, when I make the request via HTTPS, so thats why
 I believe that the issue is on the gateway end.

 NGINX Config:
 server {
 listen 80;
 listen 443 ssl default;

 server_name *.gateway.bcitestes.local gateway.bcitestes.local;
 error_log logs/error_https.log debug;
 client_max_body_size 10g;

 # This is the important option that tengine has, but nginx does not
 fastcgi_request_buffering off;

 ssl_certificate  /etc/pki/tls/certs/ca_rgw.crt;
 ssl_certificate_key  /etc/pki/tls/private/ca_rgw.key;

 ssl_session_timeout  5m;

 ssl_protocols  SSLv2 SSLv3 TLSv1;
 ssl_ciphers  HIGH:!aNULL:!MD5;
 ssl_prefer_server_ciphers   on;
 location / {
 fastcgi_pass_header Authorization;
 fastcgi_pass_request_headers on;
 fastcgi_param HTTPS on;

 if ($request_method  = PUT ) {
 rewrite ^ /PUT$request_uri;
  }
  include fastcgi_params;
  fastcgi_param HTTPS on;

  fastcgi_pass
 unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
  }

  location /PUT/ {
  internal;
  fastcgi_pass_header Authorization;
  fastcgi_pass_request_headers on;

  include fastcgi_params;
  fastcgi_param  CONTENT_LENGTH   $content_length;
  fastcgi_param HTTPS on;

  fastcgi_pass
 unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
  }

 }

 Ceph config:
 [client.radosgw.gw]
 host = GATEWAY
 keyring = /etc/ceph/keyring.radosgw.gw
 rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
 log file = /var/log/ceph/client.radosgw.gateway.log
 rgw print continue = false
 rgw dns name = gateway.bcitestes.local
 rgw enable ops log = false
 rgw enable usage log = true
 rgw usage log tick interval = 30
 rgw usage log flush threshold = 1024
 rgw usage max shards = 32
 rgw usage max user shards = 1
 rgw cache lru size = 15000
 rgw thread pool size = 2048

 --

 Thanks in advance,

 Marco Garcês
 #sysadmin
 Maputo - Mozambique
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

Hi David,

I am indeed using Tengine 2.0.3, but I feel very strange that the
default config is returning X-Storage-Url in the headers, in http, not
https as the original request.
I will try your options, and perhaps downgrading to 1.5.*, and report back.

Thank you!

Marco Garcês
#sysadmin
Maputo - Mozambique


On Wed, Oct 8, 2014 at 4:26 PM, David Moreau Simard dmsim...@iweb.com wrote:
 Hi Marco,

 While I do not have a RadosGW implementation right now, I do have a 
 successful setup with tengine and Swift - it should be pretty similar.

 What version of tengine are you trying to use ?
 It dates back to a while.. but I remember having issues with the 2.0.x branch 
 of tengine. We package our own version of 1.5.x.
 In hindsight, the issues I got might've been because of the SPDY 
 implementation but I didn't put much thought into it at the time.

 On my end, the config is in fact very simple and looks a bit like this:

 server {
   listen ip:443;

   server_name swift.tld;

   access_log /var/log/nginx/swift_https_access.log;
   error_log /var/log/nginx/swift_https_error.log;

   ssl on;
   ssl_certificate /etc/nginx/ssl/swift.crt;
   ssl_certificate_key /etc/nginx/ssl/swift.key;

   chunkin on;

   error_page 502 503 504 = @errors;
   error_page 411 = @chunk_411_error;
   location @chunk_411_error {
   chunkin_resume;
   }

   proxy_cache swift;
   location / {
 proxy_pass http://swift;
 proxy_redirect off;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   }

   location @errors {
 proxy_pass http://127.0.0.1;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header Host 127.0.0.1;
   }
 }

 Regarding the HTTP thing, maybe you could set up a redirection and see what 
 happens - a bit like this:
 server {
   listen ip:80;

   server_name rgw.tld;

   access_log /var/log/nginx/rgw_http_access.log;
   error_log /var/log/nginx/rgw_http_error.log;

   error_page 502 503 504 = @errors;

   if ( $scheme = 'http' ) {
 rewrite ^ https://$server_name$request_uri? permanent;
   }

   location @errors {
 proxy_pass http://127.0.0.1;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header Host 127.0.0.1;
   }
 }
 --
 David Moreau Simard

 On Oct 8, 2014, at 7:53 AM, Marco Garcês ma...@garces.cc wrote:

 Hi there,

 I am using RadosGW over NGINX, with Swift API, and everything is
 working great, over HTTP, but with HTTPS, I keep getting errors, and
 I'm guessing is something on the gateway itself.

 Does anyone have a working HTTPS gateway with nginx? Can you provide
 it, so I can compare to mine?

 If I do a HTTP request, using Swift client from my machine, I get the
 response ok, but If I try it with HTTPS, I get:

 Account HEAD failed: http://gateway.local/swift/v1 400 Bad Request

 and on nginx side:

 2014/10/08 13:37:34 [info] 18198#0: *50 client sent plain HTTP request
 to HTTPS port while reading client request headers, client:
 10.5.5.222, server: *.gatew
 ay.local, request: HEAD /swift/v1 HTTP/1.1, host: gateway.local:443
 2014/10/08 13:37:34 [info] 18197#0: *48 client 10.5.5.222 closed
 keepalive connection

 I have wiresharked my connection, and there is no evidence that HTTP
 traffic is going out, when I make the request via HTTPS, so thats why
 I believe that the issue is on the gateway end.

 NGINX Config:
 server {
listen 80;
listen 443 ssl default;

server_name *.gateway.bcitestes.local gateway.bcitestes.local;
error_log logs/error_https.log debug;
client_max_body_size 10g;

# This is the important option that tengine has, but nginx does not
fastcgi_request_buffering off;

ssl_certificate  /etc/pki/tls/certs/ca_rgw.crt;
ssl_certificate_key  /etc/pki/tls/private/ca_rgw.key;

ssl_session_timeout  5m;

ssl_protocols  SSLv2 SSLv3 TLSv1;
ssl_ciphers  HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers   on;
location / {
fastcgi_pass_header Authorization;
fastcgi_pass_request_headers on;
fastcgi_param HTTPS on;

if ($request_method  = PUT ) {
rewrite ^ /PUT$request_uri;
 }
 include fastcgi_params;
 fastcgi_param HTTPS on;

 fastcgi_pass
 unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

 location /PUT/ {
 internal;
 fastcgi_pass_header Authorization;
 fastcgi_pass_request_headers on;

 include fastcgi_params;
 fastcgi_param  CONTENT_LENGTH   $content_length;
 fastcgi_param HTTPS on;

 fastcgi_pass
 unix:/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock;
 }

}

 Ceph config:
 [client.radosgw.gw]
 host = GATEWAY
 keyring = /etc/ceph/keyring.radosgw.gw
 rgw socket

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

Same thing:
curl -D - -H Server-Port-Secure: 443 -H X-Auth-Key:
QoakiyY0tg8jULacsJLsmAbyZHJbY5g/Rc/dOHK3 -H X-Auth-User:
frontend:swf0002 https://gateway.local/auth
HTTP/1.1 204
Server: Tengine/2.0.3
Date: Wed, 08 Oct 2014 15:04:27 GMT
Content-Type: application/json
Connection: keep-alive
X-Storage-Url: http://gateway.local:443/swift/v1
X-Storage-Token:
AUTH_rgwtk100066726f6e74656e643a737766303030323daad73c8234e91dfba33654a8ca962d64f0f2d492b4ec5b79aee87ac454bd38406d3bee
X-Auth-Token: 
AUTH_rgwtk100066726f6e74656e643a737766303030323daad73c8234e91dfba33654a8ca962d64f0f2d492b4ec5b79aee87ac454bd38406d3bee



On Wed, Oct 8, 2014 at 5:01 PM, Yehuda Sadeh yeh...@redhat.com wrote:
 Server-Port-Secure: 443




Marco Garcês
#sysadmin
Maputo - Mozambique
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW over HTTPS

[Marco Garcês]

:19:50.503255 7f911e98e700  0 WARNING:
RGWRados::log_usage(): user name empty (bucket=), skipping


On Wed, Oct 8, 2014 at 5:46 PM, Yehuda Sadeh yeh...@redhat.com wrote:
 debug rgw = 20


I

Marco Garcês
#sysadmin
Maputo - Mozambique
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] why no likely() and unlikely() used in Ceph's source code?

[Marco Garcês]

Perhaps this question belongs in ceph-dev ?


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces

On Mon, Sep 15, 2014 at 12:28 PM, Tim Zhang cofol1...@gmail.com wrote:

 Hey guys,
 After reading ceph source code, I find that there is a file named
 common/likely.h and it implements the function likely() and unlikey() which
 will optimize the prediction of code branch for cpu.
  But there isn't any place using these two functions, I am curious
 about why the developer of ceph not using these two functions to achieve
 more performance. Can anyone provide some hints?
 BR

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph on RHEL 7 with multiple OSD's

[Marco Garcês]

Actually in EL7, iptables does not come installed by default, they use
firewalld... just remove firewalld and install iptables, and you are back
in the game! Or learn firewalld, that will work to! :)


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces

On Tue, Sep 9, 2014 at 3:10 PM, Michal Kozanecki mkozane...@evertz.com
wrote:

 Network issue maybe? Have you checked your firewall settings? Iptables
 changed a bit in EL7 and might of broken any rules your normally try and
 use, try flushing the rules (iptables -F) and see if that fixes things, if
 you then you'll need to fix your firewall rules.

 I ran into a similar issue on EL7 where the OSD's appeared up and in, but
 were stuck in peering which was due to a few ports being blocked.

 Cheers

 -Original Message-
 From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
 BG
 Sent: September-09-14 6:05 AM
 To: ceph-users@lists.ceph.com
 Subject: Re: [ceph-users] Ceph on RHEL 7 with multiple OSD's

 Loic Dachary loic@... writes:

 
  Hi,
 
  It it looks like your osd.0 is down and you only have one osd left
  (osd.1) which would explain why the cluster cannot get to a healthy
  state. The size 2 in  pool 0 'data' replicated size 2 ... means
  the pool needs at least two OSDs up to function properly. Do you know
 why the osd.0 is not up ?
 
  Cheers
 

 I've been trying unsuccessfully to get this up and running since. I've
 added another OSD but still can't get to active + clean state. I'm not
 even sure if the problems I'm having are related to the OS version but I'm
 running out of ideas and unless somebody here can spot something obvious in
 the logs below I'm going to try rolling back to CentOS 6.

 $ echo HEALTH  ceph health  echo STATUS  ceph status  echo
 OSD_DUMP  ceph osd dump HEALTH HEALTH_WARN 129 pgs peering; 129 pgs
 stuck unclean STATUS
 cluster f68332e4-1081-47b8-9b22-e5f3dc1f4521
  health HEALTH_WARN 129 pgs peering; 129 pgs stuck unclean
  monmap e1: 1 mons at {hp09=10.119.16.14:6789/0}, election epoch 2,
 quorum
  0 hp09
  osdmap e43: 3 osds: 3 up, 3 in
   pgmap v61: 192 pgs, 3 pools, 0 bytes data, 0 objects
 15469 MB used, 368 GB / 383 GB avail
  129 peering
   63 active+clean
 OSD_DUMP
 epoch 43
 fsid f68332e4-1081-47b8-9b22-e5f3dc1f4521
 created 2014-09-09 10:42:35.490711
 modified 2014-09-09 10:47:25.077178
 flags
 pool 0 'data' replicated size 3 min_size 2 crush_ruleset 0 object_hash
 rjenkins pg_num 64 pgp_num 64 last_change 1 flags hashpspool
 crash_replay_interval 45 stripe_width 0 pool 1 'metadata' replicated size 3
 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64
 last_change 1 flags hashpspool stripe_width 0 pool 2 'rbd' replicated size
 3 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64
 last_change 1 flags hashpspool stripe_width 0 max_osd 3
 osd.0 up   in  weight 1 up_from 4 up_thru 42 down_at 0 last_clean_interval
 [0,0) 10.119.16.14:6800/24988 10.119.16.14:6801/24988
 10.119.16.14:6802/24988
 10.119.16.14:6803/24988 exists,up 63f3f351-eccc-4a98-8f18-e107bd33f82b
 osd.1 up   in  weight 1 up_from 38 up_thru 42 down_at 36
 last_clean_interval
 [7,37) 10.119.16.15:6800/22999 10.119.16.15:6801/4022999
 10.119.16.15:6802/4022999 10.119.16.15:6803/4022999 exists,up
 8e1c029d-ebfb-4a8d-b567-ee9cd9ebd876
 osd.2 up   in  weight 1 up_from 42 up_thru 42 down_at 40
 last_clean_interval
 [11,41) 10.119.16.16:6800/25605 10.119.16.16:6805/5025605
 10.119.16.16:6806/5025605 10.119.16.16:6807/5025605 exists,up
 5d398bba-59f5-41f8-9bd6-aed6a0204656

 Sample of warnings from monitor log:
 2014-09-09 10:51:10.636325 7f75037d0700  1 mon.hp09@0(leader).osd e72
 prepare_failure osd.1 10.119.16.15:6800/22999 from osd.2
 10.119.16.16:6800/25605 is reporting failure:1
 2014-09-09 10:51:10.636343 7f75037d0700  0 log [DBG] : osd.1
 10.119.16.15:6800/22999 reported failed by osd.2 10.119.16.16:6800/25605

 Sample of warnings from osd.2 log:
 2014-09-09 10:44:13.723714 7fb828c57700 -1 osd.2 18 heartbeat_check: no
 reply from osd.1 ever on either front or back, first ping sent 2014-09-09
 10:43:30.437170 (cutoff 2014-09-09 10:43:53.723713)
 2014-09-09 10:44:13.724883 7fb81f2f9700  0 log [WRN] : map e19 wrongly
 marked me down
 2014-09-09 10:44:13.726104 7fb81f2f9700  0 osd.2 19 crush map has features
 1107558400, adjusting msgr requires for mons
 2014-09-09 10:44:13.726741 7fb811edb700  0 -- 10.119.16.16:0/25605 
 10.119.16.15:6806/1022999 pipe(0x3171900 sd=34 :0 s=1 pgs=0 cs=0 l=1
 c=0x3ad8580).fault



 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing

Re: [ceph-users] docker + coreos + ceph

[Marco Garcês]

Amazing work, will test it as soon as I can!
Thanks


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces


On Wed, Sep 3, 2014 at 3:20 AM, David Moreau Simard dmsim...@iweb.com
wrote:

 Oh nasty typo in those release notes. RDB module :)

 Good thing nonetheless !
 --
 David Moreau Simard



 Le 2014-09-02, 8:57 PM, « Lorieri » lori...@gmail.com a écrit :

 it is added officially now
 
 https://coreos.com/releases/#423.0.0
 
 cheers,
 -lorieri
 
 On Mon, Aug 11, 2014 at 12:28 AM, Lorieri lori...@gmail.com wrote:
  Hi,
 
  I've playing with CoreOS and got it (dirty) running with Ceph.
  No big deal, but it can save some time.
 
  1 - An image of docker-registry that stores on radosgw:
 
  https://registry.hub.docker.com/u/lorieri/registry-ceph/
 
 
  2 - Steps to get the rbd running in the host machine:
 
   - find out in which kernel version your coreos runs
   - run a docker container (or the toolbox
 
 
 https://coreos.com/docs/cluster-management/debugging/install-debugging-to
 ols/)
   - clone the coreos kernel: https://github.com/coreos/linux
   - checkout to the branch of your kernel's version
  (https://github.com/coreos/linux/branches/active)
   - compile the ceph modules and copy to the coreos machine ( ceph.ko
  libceph.ko  libcrc32c.ko  rbd.ko )
   - load the modules
   - run the coreos toolbox (/usr/bin/toolbox), it is a fedora machine
   - run: yum ceph install
   - run: ldd /usr/bin/rbd
   - copy the libraries to the coreos machine in a directory (scp?)
   - copy the /usr/bin/rbd binary to the coreos machine
   - run it like this:
 
  LD_LIBRARY_PATH=/root/libs/ ./rbd map dockertest --keyfile key --id
  coreos -m ceph1
 
 
  the libraries I've put in /root/libs/ :
  libboost_system.so.1.54.0
  libboost_thread.so.1.54.0
  libcryptopp.so.6
  libgcc_s.so.1
  libleveldb.so.1
  liblzma.so.5
  librados.so.2
  librbd.so.1
  libselinux.so.1
  libsnappy.so.1
  libstdc++.so.6
 
  cheers,
  -lorieri
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW problems

[Marco Garcês]

I have noticed that when I make the request to HTTPS, the responde comes in
http form with port 443... Where is this happening, do you have any idea?

On Wed, Aug 20, 2014 at 1:30 PM, Marco Garcês ma...@garces.cc wrote:

 swift --insecure -V 1 -A https://gateway.bcitestes.local/auth -U
 testuser:swift -K MHA4vFaDy5XsJq+F5NuZLcBMCoJcuot44ASDuReY stat
 Account HEAD failed: http://gateway.bcitestes.local:443/swift/v1 400 Bad
 Request

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW problems

[Marco Garcês]

Hello,

Yehuda, I know I was using the correct fastcgi module, it was the one on
Ceph repositories; I had also disabled in apache, all other modules;

I tried to create a second swift user, using the provided instructions,
only to get the following:

# radosgw-admin user create --uid=marcogarces --display-name=Marco Garces
# radosgw-admin subuser create --uid=marcogarces
--subuser=marcogarces:swift --access=full
# radosgw-admin key create --subuser=marcogarces:swift --key-type=swift
--gen-secret
could not create key: unable to add access key, unable to store user info
2014-08-20 13:19:33.664945 7f925b130880  0 WARNING: can't store user info,
swift id () already mapped to another user (marcogarces)


So I have created another user, some other way:

# radosgw-admin user create --subuser=testuser:swift --display-name=Test
User One --key-type=swift --access=full
{ user_id: testuser,
  display_name: Test User One,
  email: ,
  suspended: 0,
  max_buckets: 1000,
  auid: 0,
  subusers: [],
  keys: [],
  swift_keys: [
{ user: testuser:swift,
  secret_key: MHA4vFaDy5XsJq+F5NuZLcBMCoJcuot44ASDuReY}],
  caps: [],
  op_mask: read, write, delete,
  default_placement: ,
  placement_tags: [],
  bucket_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  user_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  temp_url_keys: []}


Now, when I do, from the client:

swift -V 1 -A http://gateway.bcitestes.local/auth -U testuser:swift -K
MHA4vFaDy5XsJq+F5NuZLcBMCoJcuot44ASDuReY stat
   Account: v1
Containers: 0
   Objects: 0
 Bytes: 0
Server: Tengine/2.0.3
Connection: keep-alive
X-Account-Bytes-Used-Actual: 0
  Content-Type: text/plain; charset=utf-8


If I try using https, I still have errors:

swift --insecure -V 1 -A https://gateway.bcitestes.local/auth -U
testuser:swift -K MHA4vFaDy5XsJq+F5NuZLcBMCoJcuot44ASDuReY stat
Account HEAD failed: http://gateway.bcitestes.local:443/swift/v1 400 Bad
Request


And I could not validate this account using a Swift client (Cyberduck);
Also, there are no S3 credentials!
How can I have a user with both S3 and Swift credentials created, and valid
to use with http/https, and on all clients (command line and gui). The
first user works great with the S3 credentials, on all scenarios.

Thank you,
Marco Garcês

On Tue, Aug 19, 2014 at 7:59 PM, Yehuda Sadeh yeh...@inktank.com wrote:

 On Tue, Aug 19, 2014 at 5:32 AM, Marco Garcês ma...@garces.cc wrote:
 
  UPDATE:
 
  I have installed Tengine (nginx fork) and configured both HTTP and HTTPS
 to use radosgw socket.

 Looking back at this thread, and considering this solution it seems to
 me that you were running the wrong apache fastcgi module.

 
  I can login with S3, create buckets and upload objects.
 
  It's still not possible to use Swift credentials, can you help me on
 this part? What do I use when I login (url, username, password) ?
  Here is the info for the user:
 
  radosgw-admin user info --uid=mgarces
  { user_id: mgarces,
display_name: Marco Garces,
email: marco.gar...@bci.co.mz,
suspended: 0,
max_buckets: 1000,
auid: 0,
subusers: [
  { id: mgarces:swift,
permissions: full-control}],
keys: [
  { user: mgarces:swift,
access_key: AJW2BCBXHFJ1DPXT112O,
secret_key: },
  { user: mgarces,
access_key: S88Y6ZJRACZG49JFPY83,
secret_key: PlubMMjfQecJ5Py46e2kZz5VuUgHgsjLmYZDRdFg}],
swift_keys: [
  { user: mgarces:swift,
secret_key: TtKWhY67ujhjn36\/nhv44A2BVPw5wDi3Sp13YrMM}],
caps: [],
op_mask: read, write, delete,
default_placement: ,
placement_tags: [],
bucket_quota: { enabled: false,
max_size_kb: -1,
max_objects: -1},
user_quota: { enabled: false,
max_size_kb: -1,
max_objects: -1},
temp_url_keys: []}
 

 You might be hitting issue #8587 (aka #9155). Try creating a second
 swift user, see if it still happens.

 Yehuda

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] RadosGW problems

[Marco Garcês]

I have added the options as suggested, but no success yet!

Im also running radosgw manually (radosgw -c /etc/ceph/ceph.conf -n
client.radosgw.gw --rgw-frontends civetweb port=80) using civetweb, and I
still cant login with Swift, and S3 uploads are broken.
Somenone on #ceph mention that ceph-radosgw was broken on 0.80.5, can
someone confirm this?

Tanks you once again, hope to solve this soon.
Marco Garcês

On Mon, Aug 18, 2014 at 3:23 PM, Marco Garcês ma...@garces.cc wrote:

 Hi Kurt,

 I have pointed my DNS '*.gateway.testes.local' and 'gateway.testes.local,
 to the same IP (the radosgw server).

 I have added rgw_dns_name has you suggested to the config (it was comment
 out). I will try everything and give feedback.

 By the way, when I restart ceph-radosgw service, I get this in the logs
 (which previous I did not see anything):

 2014-08-18 15:19:44.812039 7fbf417fa700  1 handle_sigterm
 2014-08-18 15:19:44.812104 7fbf417fa700  1 handle_sigterm set alarm for 120
 2014-08-18 15:19:44.812235 7fbf5c495880 -1 shutting down
 2014-08-18 15:19:44.812305 7fbf40ff9700  0 ERROR: FCGX_Accept_r returned -4
 2014-08-18 15:19:44.812432 7fbf417fa700  1 handle_sigterm
 2014-08-18 15:19:44.857506 7fbf5c495880  1 final shutdown
 2014-08-18 15:19:45.010597 7fb318b96880  0 ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 3242
 2014-08-18 15:19:45.219582 7fb318b96880  0 framework: fastcgi
 2014-08-18 15:19:45.219599 7fb318b96880  0 starting handler: fastcgi
 2014-08-18 15:19:45.692248 7fb2fe6fb700  0 ERROR: can't read user header:
 ret=-2
 2014-08-18 15:19:45.692273 7fb2fe6fb700  0 ERROR: sync_user() failed,
 user=teste ret=-2

 The last 2 lines look suspicious...




 On Mon, Aug 18, 2014 at 2:58 PM, Bachelder, Kurt 
 kurt.bachel...@sierra-cedar.com wrote:

  Hi Marco,

  Is your DNS setup to use the wildcard (*.gateway.testes.local)?

  I noticed that you're using it in the server alias, but that you don't
 have an rgw_dns_name configured in your ceph.conf.  The rgw_dns_name
 should be set to gateway.testes.local if your dns is configured to use
 the wildcard naming with that subdomain.

  Also see that you're using SSL... which domain have you signed?
 *.gateway.testes.local?

  Since you can create a bucket, but not write to it, I'm wondering if
 there's an issue with the way your client is attempting to access the
 bucket... can you resolve bucket.gateway.testes.local from your client?

  Kurt


  Original message 
 From: Marco Garcês
 Date:08/18/2014 6:33 AM (GMT-05:00)
 To: Linux Chips
 Cc: Bachelder, Kurt , ceph-users@lists.ceph.com
 Subject: Re: [ceph-users] RadosGW problems

   Hi there,

  I have FastCgiWrapper Off in fastcgi.conf file; I also have SELinux in
 permissive state; 'ps aux | grep rados' shows me radosgw is running;

  The problems stays the same... I can login with S3 credentials, create
 buckets, but uploads write this in the logs:
 [Mon Aug 18 12:00:28.636378 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: comm with server /var/www/cgi-bin/s3gw.fcgi
 aborted: idle timeout (3
 0 sec)
 [Mon Aug 18 12:00:28.676825 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: incomplete headers (0 bytes) received from
 server /var/www/cgi-bin/s3
 gw.fcgi

  When I try Swift credentials, I cannot login at all.. I have tested
 both Cyberduck and Swift client on the command line, and I always get this
 on the logs:
 GET /v1.0 HTTP/1.1 404 78 - Cyberduck/4.5 (Mac OS X/10.9.3) (x86_64)
 GET /v1.0 HTTP/1.1 404 78 - python-swiftclient-2.2.0

  In S3 login, when I upload a file, I can see it almost at 100%
 complete, but then it fails with the above errors.

  A strange thing is... the /var/log/ceph/client.radosgw.gateway.log is
 not getting updated, I don't see any new logs in there.

  Thank you once again for your help, Marco Garcês




 On Mon, Aug 18, 2014 at 12:08 AM, Linux Chips linux.ch...@gmail.com
 wrote:

 On Mon 18 Aug 2014 12:45:33 AM AST, Bachelder, Kurt wrote:

 Hi Marco –

 In CentOS 6, you also had to edit /etc/httpd/conf.d/fastcgi.conf to
 turn OFF the fastcgi wrapper.  I haven’t tested in v7 yet, but I’d
 guess it’s required there too:

 # wrap all fastcgi script calls in suexec

 FastCgiWrapper Off

 Give that a try, if you haven’t already – restart httpd and
 ceph-radosgw afterward.

 Kurt

  *From:*ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On
 Behalf Of *Marco Garcês
 *Sent:* Friday, August 15, 2014 12:46 PM
 *To:* ceph-users@lists.ceph.com
 *Subject:* [ceph-users] RadosGW problems


 Hi there,

 I am using CentOS 7 with Ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), 3 OSD, 3 MON, 1 RadosGW
 (which also serves as ceph-deploy node)

 I followed all the instructions in the docs, regarding setting up a
 basic Ceph cluster, and then followed the one to setup RadosGW.

 I can't seem to use the Swift interface, and the S3 interface, times
 out after 30 seconds.

 [Fri Aug 15 18:25:33.290877 2014

Re: [ceph-users] RadosGW problems

[Marco Garcês]

UPDATE:

I have installed Tengine (nginx fork) and configured both HTTP and HTTPS to
use radosgw socket.

I can login with S3, create buckets and upload objects.

It's still not possible to use Swift credentials, can you help me on this
part? What do I use when I login (url, username, password) ?
Here is the info for the user:

radosgw-admin user info --uid=mgarces
{ user_id: mgarces,
  display_name: Marco Garces,
  email: marco.gar...@bci.co.mz,
  suspended: 0,
  max_buckets: 1000,
  auid: 0,
  subusers: [
{ id: mgarces:swift,
  permissions: full-control}],
  keys: [
{ user: mgarces:swift,
  access_key: AJW2BCBXHFJ1DPXT112O,
  secret_key: },
{ user: mgarces,
  access_key: S88Y6ZJRACZG49JFPY83,
  secret_key: PlubMMjfQecJ5Py46e2kZz5VuUgHgsjLmYZDRdFg}],
  swift_keys: [
{ user: mgarces:swift,
  secret_key: TtKWhY67ujhjn36\/nhv44A2BVPw5wDi3Sp13YrMM}],
  caps: [],
  op_mask: read, write, delete,
  default_placement: ,
  placement_tags: [],
  bucket_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  user_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  temp_url_keys: []}

Thank you in advance.
Marco Garcês


On Tue, Aug 19, 2014 at 10:59 AM, Marco Garcês ma...@garces.cc wrote:

 I have added the options as suggested, but no success yet!

 Im also running radosgw manually (radosgw -c /etc/ceph/ceph.conf -n
 client.radosgw.gw --rgw-frontends civetweb port=80) using civetweb, and
 I still cant login with Swift, and S3 uploads are broken.
 Somenone on #ceph mention that ceph-radosgw was broken on 0.80.5, can
 someone confirm this?

 Tanks you once again, hope to solve this soon.
 Marco Garcês

 On Mon, Aug 18, 2014 at 3:23 PM, Marco Garcês ma...@garces.cc wrote:

 Hi Kurt,

 I have pointed my DNS '*.gateway.testes.local' and 'gateway.testes.local,
 to the same IP (the radosgw server).

 I have added rgw_dns_name has you suggested to the config (it was comment
 out). I will try everything and give feedback.

 By the way, when I restart ceph-radosgw service, I get this in the logs
 (which previous I did not see anything):

 2014-08-18 15:19:44.812039 7fbf417fa700  1 handle_sigterm
 2014-08-18 15:19:44.812104 7fbf417fa700  1 handle_sigterm set alarm for
 120
 2014-08-18 15:19:44.812235 7fbf5c495880 -1 shutting down
 2014-08-18 15:19:44.812305 7fbf40ff9700  0 ERROR: FCGX_Accept_r returned
 -4
 2014-08-18 15:19:44.812432 7fbf417fa700  1 handle_sigterm
 2014-08-18 15:19:44.857506 7fbf5c495880  1 final shutdown
 2014-08-18 15:19:45.010597 7fb318b96880  0 ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 3242
 2014-08-18 15:19:45.219582 7fb318b96880  0 framework: fastcgi
 2014-08-18 15:19:45.219599 7fb318b96880  0 starting handler: fastcgi
 2014-08-18 15:19:45.692248 7fb2fe6fb700  0 ERROR: can't read user header:
 ret=-2
 2014-08-18 15:19:45.692273 7fb2fe6fb700  0 ERROR: sync_user() failed,
 user=teste ret=-2

 The last 2 lines look suspicious...




 On Mon, Aug 18, 2014 at 2:58 PM, Bachelder, Kurt 
 kurt.bachel...@sierra-cedar.com wrote:

  Hi Marco,

  Is your DNS setup to use the wildcard (*.gateway.testes.local)?

  I noticed that you're using it in the server alias, but that you don't
 have an rgw_dns_name configured in your ceph.conf.  The rgw_dns_name
 should be set to gateway.testes.local if your dns is configured to use
 the wildcard naming with that subdomain.

  Also see that you're using SSL... which domain have you signed?
 *.gateway.testes.local?

  Since you can create a bucket, but not write to it, I'm wondering if
 there's an issue with the way your client is attempting to access the
 bucket... can you resolve bucket.gateway.testes.local from your client?

  Kurt


  Original message 
 From: Marco Garcês
 Date:08/18/2014 6:33 AM (GMT-05:00)
 To: Linux Chips
 Cc: Bachelder, Kurt , ceph-users@lists.ceph.com
 Subject: Re: [ceph-users] RadosGW problems

   Hi there,

  I have FastCgiWrapper Off in fastcgi.conf file; I also have SELinux in
 permissive state; 'ps aux | grep rados' shows me radosgw is running;

  The problems stays the same... I can login with S3 credentials, create
 buckets, but uploads write this in the logs:
 [Mon Aug 18 12:00:28.636378 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: comm with server /var/www/cgi-bin/s3gw.fcgi
 aborted: idle timeout (3
 0 sec)
 [Mon Aug 18 12:00:28.676825 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: incomplete headers (0 bytes) received from
 server /var/www/cgi-bin/s3
 gw.fcgi

  When I try Swift credentials, I cannot login at all.. I have tested
 both Cyberduck and Swift client on the command line, and I always get this
 on the logs:
 GET /v1.0 HTTP/1.1 404 78 - Cyberduck/4.5 (Mac OS X/10.9.3)
 (x86_64)
 GET /v1.0 HTTP/1.1 404 78 - python-swiftclient-2.2.0

  In S3 login, when I upload a file, I can see it almost at 100%
 complete, but then it fails

Re: [ceph-users] RadosGW problems

[Marco Garcês]

Hi there,

I have FastCgiWrapper Off in fastcgi.conf file; I also have SELinux in
permissive state; 'ps aux | grep rados' shows me radosgw is running;

The problems stays the same... I can login with S3 credentials, create
buckets, but uploads write this in the logs:
[Mon Aug 18 12:00:28.636378 2014] [:error] [pid 11251] [client
10.5.1.1:49680] FastCGI: comm with server /var/www/cgi-bin/s3gw.fcgi
aborted: idle timeout (3
0 sec)
[Mon Aug 18 12:00:28.676825 2014] [:error] [pid 11251] [client
10.5.1.1:49680] FastCGI: incomplete headers (0 bytes) received from server
/var/www/cgi-bin/s3
gw.fcgi

When I try Swift credentials, I cannot login at all.. I have tested both
Cyberduck and Swift client on the command line, and I always get this on
the logs:
GET /v1.0 HTTP/1.1 404 78 - Cyberduck/4.5 (Mac OS X/10.9.3) (x86_64)
GET /v1.0 HTTP/1.1 404 78 - python-swiftclient-2.2.0

In S3 login, when I upload a file, I can see it almost at 100% complete,
but then it fails with the above errors.

A strange thing is... the /var/log/ceph/client.radosgw.gateway.log is not
getting updated, I don't see any new logs in there.

Thank you once again for your help, Marco Garcês


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces


On Mon, Aug 18, 2014 at 12:08 AM, Linux Chips linux.ch...@gmail.com wrote:

 On Mon 18 Aug 2014 12:45:33 AM AST, Bachelder, Kurt wrote:

 Hi Marco –

 In CentOS 6, you also had to edit /etc/httpd/conf.d/fastcgi.conf to
 turn OFF the fastcgi wrapper.  I haven’t tested in v7 yet, but I’d
 guess it’s required there too:

 # wrap all fastcgi script calls in suexec

 FastCgiWrapper Off

 Give that a try, if you haven’t already – restart httpd and
 ceph-radosgw afterward.

 Kurt

 *From:*ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On
 Behalf Of *Marco Garcês
 *Sent:* Friday, August 15, 2014 12:46 PM
 *To:* ceph-users@lists.ceph.com
 *Subject:* [ceph-users] RadosGW problems


 Hi there,

 I am using CentOS 7 with Ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), 3 OSD, 3 MON, 1 RadosGW
 (which also serves as ceph-deploy node)

 I followed all the instructions in the docs, regarding setting up a
 basic Ceph cluster, and then followed the one to setup RadosGW.

 I can't seem to use the Swift interface, and the S3 interface, times
 out after 30 seconds.

 [Fri Aug 15 18:25:33.290877 2014] [:error] [pid 6197] [client
 10.5.5.222:58051 http://10.5.5.222:58051] FastCGI: comm with server

 /var/www/cgi-bin/s3gw.fcgi aborted: idle timeout (30 sec)

 [Fri Aug 15 18:25:33.291781 2014] [:error] [pid 6197] [client
 10.5.5.222:58051 http://10.5.5.222:58051] FastCGI: incomplete

 headers (0 bytes) received from server /var/www/cgi-bin/s3gw.fcgi

 *My ceph.conf:*


 [global]

 fsid = 581bcd61-8760-4756-a7c8-e8275c0957ad

 mon_initial_members = CEPH01, CEPH02, CEPH03

 mon_host = 10.2.27.81,10.2.27.82,10.2.27.83

 public network = 10.2.27.0/25 http://10.2.27.0/25


 auth_cluster_required = cephx

 auth_service_required = cephx

 auth_client_required = cephx

 filestore_xattr_use_omap = true

 osd pool default size = 2

 osd pool default pg num = 333

 osd pool default pgp num = 333

 osd journal size = 1024

 [client.radosgw.gateway]

 host = GATEWAY

 keyring = /etc/ceph/ceph.client.radosgw.keyring

 rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock

 log file = /var/log/ceph/client.radosgw.gateway.log

 rgw print continue = false

 rgw enable ops log = true

 *My apache rgw.conf:*


 FastCgiExternalServer /var/www/cgi-bin/s3gw.fcgi -socket
 /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock

 VirtualHost *:443

 SSLEngine on

 SSLCertificateFile /etc/pki/tls/certs/ca_rgw.crt

 SSLCertificateKeyFile /etc/pki/tls/private/ca_rgw.key

 SetEnv SERVER_PORT_SECURE 443

 ServerName gateway.testes.local

 ServerAlias *.gateway.testes.local

 ServerAdmin marco.gar...@testes.co.mz
 mailto:marco.gar...@testes.co.mz


 DocumentRoot /var/www/cgi-bin

 RewriteEngine On

 #RewriteRule ^/(.*) /s3gw.fcgi?%{QUERY_STRING}
 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]

 RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*)
 /s3gw.fcgi?page=$1params=$2%{QUERY_STRING}
 [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]

 IfModule mod_fastcgi.c

 Directory /var/www

 Options +ExecCGI

 AllowOverride All

 SetHandler fastcgi-script

 Order allow,deny

 Allow from all

 AuthBasicAuthoritative Off

 /Directory

 /IfModule

 AllowEncodedSlashes On

 ErrorLog /var/log/httpd/error_rgw_ssl.log

 CustomLog /var/log/httpd/access_rgw_ssl.log combined

 ServerSignature Off

 /VirtualHost

 *My /var/www/cgi-bin/s3gw.fcgi *


 #!/bin/sh

 exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n

Re: [ceph-users] RadosGW problems

[Marco Garcês]

Hi Kurt,

I have pointed my DNS '*.gateway.testes.local' and 'gateway.testes.local,
to the same IP (the radosgw server).

I have added rgw_dns_name has you suggested to the config (it was comment
out). I will try everything and give feedback.

By the way, when I restart ceph-radosgw service, I get this in the logs
(which previous I did not see anything):

2014-08-18 15:19:44.812039 7fbf417fa700  1 handle_sigterm
2014-08-18 15:19:44.812104 7fbf417fa700  1 handle_sigterm set alarm for 120
2014-08-18 15:19:44.812235 7fbf5c495880 -1 shutting down
2014-08-18 15:19:44.812305 7fbf40ff9700  0 ERROR: FCGX_Accept_r returned -4
2014-08-18 15:19:44.812432 7fbf417fa700  1 handle_sigterm
2014-08-18 15:19:44.857506 7fbf5c495880  1 final shutdown
2014-08-18 15:19:45.010597 7fb318b96880  0 ceph version 0.80.5
(38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 3242
2014-08-18 15:19:45.219582 7fb318b96880  0 framework: fastcgi
2014-08-18 15:19:45.219599 7fb318b96880  0 starting handler: fastcgi
2014-08-18 15:19:45.692248 7fb2fe6fb700  0 ERROR: can't read user header:
ret=-2
2014-08-18 15:19:45.692273 7fb2fe6fb700  0 ERROR: sync_user() failed,
user=teste ret=-2

The last 2 lines look suspicious...


*Marco Garcês*
*#sysadmin*
Maputo - Mozambique
*[Phone]* +258 84 4105579
*[Skype]* marcogarces


On Mon, Aug 18, 2014 at 2:58 PM, Bachelder, Kurt 
kurt.bachel...@sierra-cedar.com wrote:

  Hi Marco,

  Is your DNS setup to use the wildcard (*.gateway.testes.local)?

  I noticed that you're using it in the server alias, but that you don't
 have an rgw_dns_name configured in your ceph.conf.  The rgw_dns_name
 should be set to gateway.testes.local if your dns is configured to use
 the wildcard naming with that subdomain.

  Also see that you're using SSL... which domain have you signed?
 *.gateway.testes.local?

  Since you can create a bucket, but not write to it, I'm wondering if
 there's an issue with the way your client is attempting to access the
 bucket... can you resolve bucket.gateway.testes.local from your client?

  Kurt


  Original message 
 From: Marco Garcês
 Date:08/18/2014 6:33 AM (GMT-05:00)
 To: Linux Chips
 Cc: Bachelder, Kurt , ceph-users@lists.ceph.com
 Subject: Re: [ceph-users] RadosGW problems

   Hi there,

  I have FastCgiWrapper Off in fastcgi.conf file; I also have SELinux in
 permissive state; 'ps aux | grep rados' shows me radosgw is running;

  The problems stays the same... I can login with S3 credentials, create
 buckets, but uploads write this in the logs:
 [Mon Aug 18 12:00:28.636378 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: comm with server /var/www/cgi-bin/s3gw.fcgi
 aborted: idle timeout (3
 0 sec)
 [Mon Aug 18 12:00:28.676825 2014] [:error] [pid 11251] [client
 10.5.1.1:49680] FastCGI: incomplete headers (0 bytes) received from
 server /var/www/cgi-bin/s3
 gw.fcgi

  When I try Swift credentials, I cannot login at all.. I have tested both
 Cyberduck and Swift client on the command line, and I always get this on
 the logs:
 GET /v1.0 HTTP/1.1 404 78 - Cyberduck/4.5 (Mac OS X/10.9.3) (x86_64)
 GET /v1.0 HTTP/1.1 404 78 - python-swiftclient-2.2.0

  In S3 login, when I upload a file, I can see it almost at 100% complete,
 but then it fails with the above errors.

  A strange thing is... the /var/log/ceph/client.radosgw.gateway.log is
 not getting updated, I don't see any new logs in there.

  Thank you once again for your help, Marco Garcês


  *Marco Garcês*
 *#sysadmin*
  Maputo - Mozambique
 *[Phone]* +258 84 4105579
 *[Skype]* marcogarces


 On Mon, Aug 18, 2014 at 12:08 AM, Linux Chips linux.ch...@gmail.com
 wrote:

 On Mon 18 Aug 2014 12:45:33 AM AST, Bachelder, Kurt wrote:

 Hi Marco –

 In CentOS 6, you also had to edit /etc/httpd/conf.d/fastcgi.conf to
 turn OFF the fastcgi wrapper.  I haven’t tested in v7 yet, but I’d
 guess it’s required there too:

 # wrap all fastcgi script calls in suexec

 FastCgiWrapper Off

 Give that a try, if you haven’t already – restart httpd and
 ceph-radosgw afterward.

 Kurt

  *From:*ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On
 Behalf Of *Marco Garcês
 *Sent:* Friday, August 15, 2014 12:46 PM
 *To:* ceph-users@lists.ceph.com
 *Subject:* [ceph-users] RadosGW problems


 Hi there,

 I am using CentOS 7 with Ceph version 0.80.5
 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), 3 OSD, 3 MON, 1 RadosGW
 (which also serves as ceph-deploy node)

 I followed all the instructions in the docs, regarding setting up a
 basic Ceph cluster, and then followed the one to setup RadosGW.

 I can't seem to use the Swift interface, and the S3 interface, times
 out after 30 seconds.

 [Fri Aug 15 18:25:33.290877 2014] [:error] [pid 6197] [client
  10.5.5.222:58051 http://10.5.5.222:58051] FastCGI: comm with server

 /var/www/cgi-bin/s3gw.fcgi aborted: idle timeout (30 sec)

 [Fri Aug 15 18:25:33.291781 2014] [:error] [pid 6197] [client
  10.5.5.222:58051 http://10.5.5.222:58051] FastCGI: incomplete

 headers (0 bytes

[ceph-users] RadosGW problems

[Marco Garcês]

Hi there,

I am using CentOS 7 with Ceph version 0.80.5
(38b73c67d375a2552d8ed67843c8a65c2c0feba6), 3 OSD, 3 MON, 1 RadosGW (which
also serves as ceph-deploy node)

I followed all the instructions in the docs, regarding setting up a basic
Ceph cluster, and then followed the one to setup RadosGW.
I can't seem to use the Swift interface, and the S3 interface, times out
after 30 seconds.

[Fri Aug 15 18:25:33.290877 2014] [:error] [pid 6197] [client
10.5.5.222:58051] FastCGI: comm with server /var/www/cgi-bin/s3gw.fcgi
aborted: idle timeout (30 sec)
[Fri Aug 15 18:25:33.291781 2014] [:error] [pid 6197] [client
10.5.5.222:58051] FastCGI: incomplete headers (0 bytes) received from
server /var/www/cgi-bin/s3gw.fcgi

*My ceph.conf:*
[global]
fsid = 581bcd61-8760-4756-a7c8-e8275c0957ad
mon_initial_members = CEPH01, CEPH02, CEPH03
mon_host = 10.2.27.81,10.2.27.82,10.2.27.83
public network = 10.2.27.0/25
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
osd pool default size = 2
osd pool default pg num = 333
osd pool default pgp num = 333
osd journal size = 1024

[client.radosgw.gateway]
host = GATEWAY
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/ceph/client.radosgw.gateway.log
rgw print continue = false
rgw enable ops log = true

*My apache rgw.conf:*
FastCgiExternalServer /var/www/cgi-bin/s3gw.fcgi -socket
/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
VirtualHost *:443
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/ca_rgw.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca_rgw.key
SetEnv SERVER_PORT_SECURE 443
ServerName gateway.testes.local
ServerAlias *.gateway.testes.local
ServerAdmin marco.gar...@testes.co.mz
DocumentRoot /var/www/cgi-bin
RewriteEngine On
#RewriteRule ^/(.*) /s3gw.fcgi?%{QUERY_STRING}
[E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*)
/s3gw.fcgi?page=$1params=$2%{QUERY_STRING}
[E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]

IfModule mod_fastcgi.c
Directory /var/www
Options +ExecCGI
AllowOverride All
SetHandler fastcgi-script
Order allow,deny
Allow from all
AuthBasicAuthoritative Off
/Directory
/IfModule

AllowEncodedSlashes On
ErrorLog /var/log/httpd/error_rgw_ssl.log
CustomLog /var/log/httpd/access_rgw_ssl.log combined
ServerSignature Off
/VirtualHost

*My /var/www/cgi-bin/s3gw.fcgi *
#!/bin/sh
exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway

*My Rados user:*
radosgw-admin user info --uid=johndoe
{ user_id: johndoe,
  display_name: John Doe,
  email: j...@example.com,
  suspended: 0,
  max_buckets: 1000,
  auid: 0,
  subusers: [
{ id: johndoe:swift,
  permissions: full-control}],
  keys: [
{ user: johndoe:swift,
  access_key: 265DJESOJGSK953EE4LE,
  secret_key: },
{ user: johndoe,
  access_key: U4AR5757MCON3AZYAB97,
  secret_key: 05rg47Oa+njo8uxTeX+urBPF0ZRPWvVq8nfrC5cN}],
  swift_keys: [
{ user: johndoe:swift,
  secret_key: Lags5xwX5aiPgkG\/QqA8HygKs6AQYO46dBXS0ZGS}],
  caps: [],
  op_mask: read, write, delete,
  default_placement: ,
  placement_tags: [],
  bucket_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  user_quota: { enabled: false,
  max_size_kb: -1,
  max_objects: -1},
  temp_url_keys: []}



I can reach https://gateway.testes.local, and I can login with S3, but cant
login with Swift (using Cyberduck). Also, I can create buckets with S3, but
if I upload a file, it times out with the error above. There is a necessity
to use both the S3 and Swift API.

Can you help me? Thank you in advance, regards, Marco Garcês
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com