Re: [ceph-users] Ceph MON can no longer join quorum
Hi Greg
I have seen this problem before in my cluster.
* What ceph version you are running
* Did you made any change recently in the cluster , that resulted in this
problem
You identified correct , the only problem is ceph-mon-2003 is listening to
incorrect port , it should listen on port 6789 ( like the other two monitors )
. How i resolved is cleanly removing the infected monitor node and adding it
back to cluster.
Regards
Karan
- Original Message -
From: Greg Poirier greg.poir...@opower.com
To: ceph-users@lists.ceph.com
Sent: Tuesday, 4 February, 2014 10:50:21 PM
Subject: [ceph-users] Ceph MON can no longer join quorum
I have a MON that at some point lost connectivity to the rest of the cluster
and now cannot rejoin.
Each time I restart it, it looks like it's attempting to create a new MON and
join the cluster, but the rest of the cluster rejects it, because the new one
isn't in the monmap.
I don't know why it suddenly decided it needed to be a new MON.
I am not really sure where to start.
root@ceph-mon-2003:/var/log/ceph# ceph -s
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2 pgs stuck
inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests are blocked
32 sec; 1 scrub errors; 1 mons down, quorum 0,1 ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at {ceph-mon-2001=
10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0
}, election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Notice ceph-mon-2003:6800
If I try to start ceph-mon-all, it will be listening on some other port...
root@ceph-mon-2003:/var/log/ceph# start ceph-mon-all
ceph-mon-all start/running
root@ceph-mon-2003:/var/log/ceph# ps -ef | grep ceph
root 6930 1 31 15:49 ? 00:00:00 /usr/bin/ceph-mon --cluster=ceph -i
ceph-mon-2003 -f
root 6931 1 3 15:49 ? 00:00:00 python /usr/sbin/ceph-create-keys --cluster=ceph
-i ceph-mon-2003
root@ceph-mon-2003:/var/log/ceph# ceph -s
2014-02-04 15:49:56.854866 7f9cf422d700 0 -- :/1007028 10.30.66.15:6789/0
pipe(0x7f9cf0021370 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cf00215d0).fault
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2 pgs stuck
inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests are blocked
32 sec; 1 scrub errors; 1 mons down, quorum 0,1 ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at {ceph-mon-2001=
10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0
}, election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Suggestions?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Frankfurt, before the Ceph day
Hi Loic, do you known if a ceph meetup is planned soon in France or Belgium ? I miss the Fosdem this year, and I'll be very happy to meet some ceph users/devs. Regards, Alexandre - Mail original - De: Loic Dachary l...@dachary.org À: ceph-users ceph-users@lists.ceph.com Envoyé: Mercredi 5 Février 2014 09:44:04 Objet: [ceph-users] Meetup in Frankfurt, before the Ceph day Hi Ceph, I'll be in Frankfurt for the Ceph day February 27th http://www.eventbrite.com/e/ceph-day-frankfurt-tickets-10173269523 and I will attend the meetup organized the evening before http://www.meetup.com/Ceph-Frankfurt/events/164620852/ Anyone interested to join ? Not sure where we should meet ... I've never been to Frankfurt before :-) Cheers -- Loïc Dachary, Artisan Logiciel Libre ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Frankfurt, before the Ceph day
Hi Alexandre, We have a meet up in Paris. Please see: http://www.meetup.com/Ceph-in-Paris/events/158942372/ Cheers. Sébastien Han Cloud Engineer Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 10, rue de la Victoire - 75009 Paris Web : www.enovance.com - Twitter : @enovance On 05 Feb 2014, at 13:51, Alexandre DERUMIER aderum...@odiso.com wrote: Hi Loic, do you known if a ceph meetup is planned soon in France or Belgium ? I miss the Fosdem this year, and I'll be very happy to meet some ceph users/devs. Regards, Alexandre - Mail original - De: Loic Dachary l...@dachary.org À: ceph-users ceph-users@lists.ceph.com Envoyé: Mercredi 5 Février 2014 09:44:04 Objet: [ceph-users] Meetup in Frankfurt, before the Ceph day Hi Ceph, I'll be in Frankfurt for the Ceph day February 27th http://www.eventbrite.com/e/ceph-day-frankfurt-tickets-10173269523 and I will attend the meetup organized the evening before http://www.meetup.com/Ceph-Frankfurt/events/164620852/ Anyone interested to join ? Not sure where we should meet ... I've never been to Frankfurt before :-) Cheers -- Loïc Dachary, Artisan Logiciel Libre ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com signature.asc Description: Message signed with OpenPGP using GPGMail ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] keyring generation
On Mon, Feb 3, 2014 at 11:28 AM, Kei.masumoto kei.masum...@gmail.com wrote:
Hi Alfredo,
Thanks for your reply!
I think I pasted all logs from ceph.log, but anyway, I re-executed
ceph-deploy mon create-initial again
Does that make sense? It seems like stack strace are added...
Those seem bad enough. There is a ticket open for these type of
tracebacks that should be gone with the
up coming release of ceph-deploy.
Your monitor does seem like in a good state. Have you checked the
monitor logs to see if they are complaining
about something?
I would also raise the log level in ceph.conf for the monitors specifically to:
debug mon = 10
--
[ceph_deploy.cli][INFO ] Invoked (1.3.4): /usr/bin/ceph-deploy mon
create-initial
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1
[ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ...
[ceph1][DEBUG ] connected to host: ceph1
[ceph1][DEBUG ] detect platform information from remote host
[ceph1][DEBUG ] detect machine type
[ceph_deploy.mon][INFO ] distro info: Ubuntu 13.04 raring
[ceph1][DEBUG ] determining if provided host has same hostname in remote
[ceph1][DEBUG ] get remote short hostname
[ceph1][DEBUG ] deploying mon to ceph1
[ceph1][DEBUG ] get remote short hostname
[ceph1][DEBUG ] remote hostname: ceph1
[ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph1][DEBUG ] create the mon path if it does not exist
[ceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph1/done
[ceph1][DEBUG ] create a done file to avoid re-doing the mon deployment
[ceph1][DEBUG ] create the init path if it does not exist
[ceph1][DEBUG ] locating the `service` executable...
[ceph1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph
id=ceph1
[ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
/var/run/ceph/ceph-mon.ceph1.asok mon_status
[ceph1][DEBUG ]
[ceph1][DEBUG ] status for monitor: mon.ceph1
[ceph1][DEBUG ] {
[ceph1][DEBUG ] election_epoch: 1,
[ceph1][DEBUG ] extra_probe_peers: [],
[ceph1][DEBUG ] monmap: {
[ceph1][DEBUG ] created: 0.00,
[ceph1][DEBUG ] epoch: 1,
[ceph1][DEBUG ] fsid: 26835656-6b29-455d-9d1f-545cad8f1e23,
[ceph1][DEBUG ] modified: 0.00,
[ceph1][DEBUG ] mons: [
[ceph1][DEBUG ] {
[ceph1][DEBUG ] addr: 192.168.111.11:6789/0,
[ceph1][DEBUG ] name: ceph1,
[ceph1][DEBUG ] rank: 0
[ceph1][DEBUG ] }
[ceph1][DEBUG ] ]
[ceph1][DEBUG ] },
[ceph1][DEBUG ] name: ceph1,
[ceph1][DEBUG ] outside_quorum: [],
[ceph1][DEBUG ] quorum: [
[ceph1][DEBUG ] 0
[ceph1][DEBUG ] ],
[ceph1][DEBUG ] rank: 0,
[ceph1][DEBUG ] state: leader,
[ceph1][DEBUG ] sync_provider: []
[ceph1][DEBUG ] }
[ceph1][DEBUG ]
[ceph1][INFO ] monitor: mon.ceph1 is running
[ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
/var/run/ceph/ceph-mon.ceph1.asok mon_status
[ceph_deploy.mon][INFO ] processing monitor mon.ceph1
[ceph1][DEBUG ] connected to host: ceph1
[ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
/var/run/ceph/ceph-mon.ceph1.asok mon_status
[ceph_deploy.mon][INFO ] mon.ceph1 monitor has reached quorum!
[ceph_deploy.mon][INFO ] all initial monitors are running and have formed
quorum
[ceph_deploy.mon][INFO ] Running gatherkeys...
gatherkeys.fetch_fileNamespace(cluster='ceph', dry_run=False,
func=function mon at 0xe14e60, mon=['ceph1'], overwrite_conf=False,
prog='ceph-deploy', quiet=False, subcommand='create-initial', username=None,
verbose=False) :: /etc/ceph/ceph.client.admin.keyring ::
ceph.client.admin.keyring :: ['ceph1']
[ceph_deploy.gatherkeys][DEBUG ] Checking ceph1 for
/etc/ceph/ceph.client.admin.keyring
[ceph1][DEBUG ] connected to host: ceph1
[ceph1][DEBUG ] detect platform information from remote host
[ceph1][DEBUG ] detect machine type
[ceph1][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find
/etc/ceph/ceph.client.admin.keyring on ['ceph1']
Traceback (most recent call last):
File string, line 1, in module
File string, line 6, in module
File
/usr/lib/python2.7/dist-packages/ceph_deploy/lib/remoto/lib/execnet/gateway_base.py,
line 1220, in serve
gatherkeys.fetch_fileNamespace(cluster='ceph', dry_run=False,
func=function mon at 0xe14e60, mon=['ceph1'], overwrite_conf=False,
prog='ceph-deploy', quiet=False, subcommand='create-initial', username=None,
verbose=False) :: /var/lib/ceph/mon/ceph-{hostname}/keyring ::
ceph.mon.keyring :: ['ceph1']
[ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
gatherkeys.fetch_file
Re: [ceph-users] Hostname not found error
Hi , I can actually ping via host1 and do ssh as well. Ceph-deploy worked when i used host1 instead of ip in that config file. What I noticed is, when I used ip format- 192.168.25.1 (last part only 1 digit) i got above error garbage found in that line. But when i used for other hosts, ip as 192.168.25.11 ,ceph-deploy worked. Thanks Sahana On Feb 5, 2014 8:22 PM, Alfredo Deza alfredo.d...@inktank.com wrote: On Wed, Feb 5, 2014 at 2:02 AM, Sahana shna...@gmail.com wrote: Hi I am trying to form ceph cluster using ceph-deploy. I generated ssh-keygen, copied key to all ceph nodes, then added user and hostname in ~/.ssh/config file : cat /home/ems/.ssh/config Host host1 Hostname aa.bbb.cc.d #ip User user For sample I ran purge command, I get this error: ceph-deploy purge host1 [ceph_deploy.cli][INFO ] Invoked (1.3.4): /usr/bin/ceph-deploy purge host1 [ceph_deploy.install][DEBUG ] Purging from cluster ceph hosts host1 [ceph_deploy.install][DEBUG ] Detecting platform for host host1... /home/ems/.ssh/config line 2: garbage at end of line; #ip. [ceph_deploy][ERROR ] RuntimeError: connecting to host: host1 resulted in errors: HostNotFound host1 The log output above does show that your ssh config file has 'garbage at the end of the line' You need to clean that config file so that it works correctly and make sure that 'host1' is resolvable, that for ceph-deploy means that you can ping or that you can actually ssh into it. When i tried with other ips like aa.bbb.cc.dd /hostname it worked. Example: 192.168.25.1 didnot work, but 192.168.25.11 worked. thanks Sahana ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph MON can no longer join quorum
Hi Karan,
I resolved it the same way you did. We had a network partition that caused
the MON to die, it appears.
I'm running 0.72.1
It would be nice if redeploying wasn't the solution, but if it's simply
cleaner to do so, then I will continue along that route.
I think what's more troubling is that when this occurred we lost all
connectivity to the Ceph cluster.
On Wed, Feb 5, 2014 at 1:11 AM, Karan Singh ksi...@csc.fi wrote:
Hi Greg
I have seen this problem before in my cluster.
- What ceph version you are running
- Did you made any change recently in the cluster , that resulted in
this problem
You identified correct , the only problem is ceph-mon-2003 is listening
to incorrect port , it should listen on port 6789 ( like the other two
monitors ) . How i resolved is cleanly removing the infected monitor node
and adding it back to cluster.
Regards
Karan
--
*From: *Greg Poirier greg.poir...@opower.com
*To: *ceph-users@lists.ceph.com
*Sent: *Tuesday, 4 February, 2014 10:50:21 PM
*Subject: *[ceph-users] Ceph MON can no longer join quorum
I have a MON that at some point lost connectivity to the rest of the
cluster and now cannot rejoin.
Each time I restart it, it looks like it's attempting to create a new MON
and join the cluster, but the rest of the cluster rejects it, because the
new one isn't in the monmap.
I don't know why it suddenly decided it needed to be a new MON.
I am not really sure where to start.
root@ceph-mon-2003:/var/log/ceph# ceph -s
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2
pgs stuck inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests
are blocked 32 sec; 1 scrub errors; 1 mons down, quorum 0,1
ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at {ceph-mon-2001=
10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0},
election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Notice ceph-mon-2003:6800
If I try to start ceph-mon-all, it will be listening on some other port...
root@ceph-mon-2003:/var/log/ceph# start ceph-mon-all
ceph-mon-all start/running
root@ceph-mon-2003:/var/log/ceph# ps -ef | grep ceph
root 6930 1 31 15:49 ?00:00:00 /usr/bin/ceph-mon
--cluster=ceph -i ceph-mon-2003 -f
root 6931 1 3 15:49 ?00:00:00 python
/usr/sbin/ceph-create-keys --cluster=ceph -i ceph-mon-2003
root@ceph-mon-2003:/var/log/ceph# ceph -s
2014-02-04 15:49:56.854866 7f9cf422d700 0 -- :/1007028
10.30.66.15:6789/0 pipe(0x7f9cf0021370 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f9cf00215d0).fault
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2
pgs stuck inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests
are blocked 32 sec; 1 scrub errors; 1 mons down, quorum 0,1
ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at {ceph-mon-2001=
10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0},
election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Suggestions?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Frankfurt, before the Ceph day
Hi, Anyone else from ceph community , willing to join. will also visit my first ceph day (-: See you all in Frankfurt! best regards Danny smime.p7s Description: S/MIME cryptographic signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Frankfurt, before the Ceph day
+1 It would be nice to meet you guys @ Ceph Day Frankfurt. Anyone else from ceph community , willing to join. Loic — Same here , first time visiting to Frankfurt. Regards Karan On 05 Feb 2014, at 10:44, Loic Dachary l...@dachary.org wrote: Hi Ceph, I'll be in Frankfurt for the Ceph day February 27th http://www.eventbrite.com/e/ceph-day-frankfurt-tickets-10173269523 and I will attend the meetup organized the evening before http://www.meetup.com/Ceph-Frankfurt/events/164620852/ Anyone interested to join ? Not sure where we should meet ... I've never been to Frankfurt before :-) Cheers -- Loïc Dachary, Artisan Logiciel Libre ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Frankfurt, before the Ceph day
Le 05/02/2014 14:10, Sebastien Han a écrit : We have a meet up in Paris. Please see: http://www.meetup.com/Ceph-in-Paris/events/158942372/ This is a great news ! thanks for input. Cheers Cédric ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph MON can no longer join quorum
Hi Greg
I have seen this problem before in my cluster.
What ceph version you are running
Did you made any change recently in the cluster , that resulted in this problem
You identified correct , the only problem is ceph-mon-2003 is listening to
incorrect port , it should listen on port 6789 ( like the other two monitors )
. How i resolved is cleanly removing the infected monitor node and adding it
back to cluster.
Regards
Karan
On 04 Feb 2014, at 22:50, Greg Poirier greg.poir...@opower.com wrote:
I have a MON that at some point lost connectivity to the rest of the cluster
and now cannot rejoin.
Each time I restart it, it looks like it's attempting to create a new MON and
join the cluster, but the rest of the cluster rejects it, because the new one
isn't in the monmap.
I don't know why it suddenly decided it needed to be a new MON.
I am not really sure where to start.
root@ceph-mon-2003:/var/log/ceph# ceph -s
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2
pgs stuck inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests are
blocked 32 sec; 1 scrub errors; 1 mons down, quorum 0,1
ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at
{ceph-mon-2001=10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0},
election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Notice ceph-mon-2003:6800
If I try to start ceph-mon-all, it will be listening on some other port...
root@ceph-mon-2003:/var/log/ceph# start ceph-mon-all
ceph-mon-all start/running
root@ceph-mon-2003:/var/log/ceph# ps -ef | grep ceph
root 6930 1 31 15:49 ?00:00:00 /usr/bin/ceph-mon
--cluster=ceph -i ceph-mon-2003 -f
root 6931 1 3 15:49 ?00:00:00 python
/usr/sbin/ceph-create-keys --cluster=ceph -i ceph-mon-2003
root@ceph-mon-2003:/var/log/ceph# ceph -s
2014-02-04 15:49:56.854866 7f9cf422d700 0 -- :/1007028 10.30.66.15:6789/0
pipe(0x7f9cf0021370 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cf00215d0).fault
cluster 4167d5f2-2b9e-4bde-a653-f24af68a45f8
health HEALTH_ERR 1 pgs inconsistent; 2 pgs peering; 126 pgs stale; 2
pgs stuck inactive; 126 pgs stuck stale; 2 pgs stuck unclean; 10 requests are
blocked 32 sec; 1 scrub errors; 1 mons down, quorum 0,1
ceph-mon-2001,ceph-mon-2002
monmap e2: 3 mons at
{ceph-mon-2001=10.30.66.13:6789/0,ceph-mon-2002=10.30.66.14:6789/0,ceph-mon-2003=10.30.66.15:6800/0},
election epoch 12964, quorum 0,1 ceph-mon-2001,ceph-mon-2002
Suggestions?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] RGW Replication
On 02/05/2014 01:23 PM, Craig Lewis wrote: On 2/4/14 20:02 , Josh Durgin wrote: From the log it looks like you're hitting the default maximum number of entries to be processed at once per shard. This was intended to prevent one really busy shard from blocking progress on syncing other shards, since the remainder will be synced the next time the shard is processed. Perhaps the default is too low though, or the idea should be scrapped altogether since you can sync other shards in parallel. For your particular usage, since you're updating the same few buckets, the max entries limit is hit constantly. You can increase it with max-entries: 100 in the config file or --max-entries 1000 on the command line. Josh This doesn't appear to have any effect: root@ceph1c:/etc/init.d# grep max-entries /etc/ceph/radosgw-agent.conf max-entries: 100 root@ceph1c:/etc/init.d# egrep 'has [0-9]+ entries after' /var/log/ceph/radosgw-agent.log | tail -1 2014-02-05T13:11:03.915 2743:INFO:radosgw_agent.worker:bucket instance live-2:us-west-1.35026898.2 has 1000 entries after 0206789.410535.3 Neither does --max-entries 1000: root@ceph1c:/etc/init.d# ps auxww | grep radosgw-agent | grep max-entries root 19710 6.0 0.0 74492 18708 pts/3S13:22 0:00 /usr/bin/python /usr/bin/radosgw-agent --incremental-sync-delay=10 --max-entries 1000 -c /etc/ceph/radosgw-agent.conf root@ceph1c:/etc/init.d# egrep 'has [0-9]+ entries after' /var/log/ceph/radosgw-agent.us-west-1.us-central-1.log | tail -1 2014-02-05T13:22:58.577 21626:INFO:radosgw_agent.worker:bucket instance live-2:us-west-1.35026898.2 has 1000 entries after 0207788.411542.2 I guess I'll look into that too, since I'll be in that area of the code. It seems to be a hardcoded limit on the server side to prevent a single osd operation from taking too long (see cls_log_list() in ceph.git src/cls/cls_log.cc). This should probably be fixed in radosgw, but you could work around it with a loop in radosgw-agent. Josh ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Paris
I'm on the waiting list too, but anyway, not sure to come yet. (I come from Lille, and I need to check with my boss to leave early) By the way, any chance to have some ceph conference or meetup at solutions linux expo at paris this year ? - Mail original - De: Loic Dachary l...@dachary.org À: Cedric Lemarchand ced...@yipikai.org, ceph-users@lists.ceph.com Envoyé: Mercredi 5 Février 2014 21:31:10 Objet: [ceph-users] Meetup in Paris Hi Cedric, Although it is fully booked at the moment, a few people notified me that they are unsure if they are going to attend or not. You can add yourself to the waiting list and I'll ask them next week to free a few slots. See you there :-) On 05/02/2014 18:46, Cedric Lemarchand wrote: Le 05/02/2014 14:10, Sebastien Han a écrit : We have a meet up in Paris. Please see: http://www.meetup.com/Ceph-in-Paris/events/158942372/ This is a great news ! thanks for input. Cheers Cédric ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Loïc Dachary, Artisan Logiciel Libre ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Meetup in Paris
Bonjour, Great idea : http://www.meetup.com/Ceph-in-Paris/events/164818312/ and I asked for a booth in the non profit area right now. A++ On 06/02/2014 06:13, Alexandre DERUMIER wrote: I'm on the waiting list too, but anyway, not sure to come yet. (I come from Lille, and I need to check with my boss to leave early) By the way, any chance to have some ceph conference or meetup at solutions linux expo at paris this year ? - Mail original - De: Loic Dachary l...@dachary.org À: Cedric Lemarchand ced...@yipikai.org, ceph-users@lists.ceph.com Envoyé: Mercredi 5 Février 2014 21:31:10 Objet: [ceph-users] Meetup in Paris Hi Cedric, Although it is fully booked at the moment, a few people notified me that they are unsure if they are going to attend or not. You can add yourself to the waiting list and I'll ask them next week to free a few slots. See you there :-) On 05/02/2014 18:46, Cedric Lemarchand wrote: Le 05/02/2014 14:10, Sebastien Han a écrit : We have a meet up in Paris. Please see: http://www.meetup.com/Ceph-in-Paris/events/158942372/ This is a great news ! thanks for input. Cheers Cédric ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
