Re: [ceph-users] Huge memory usage spike in OSD on hammer/giant

[池信泽]

Yeh, There is bug which would use huge memory. It be triggered when osd
down or add into cluster and do recovery/backfilling.

The patch https://github.com/ceph/ceph/pull/5656
https://github.com/ceph/ceph/pull/5451 merged into master would fix it, and
it would be backport.

I think ceph v0.93 or newer version maybe hit this bug.

2015-09-07 20:42 GMT+08:00 Shinobu Kinjo :

> How heavy network traffic was?
>
> Have you tried to capture that traffic between cluster and public network
> to see where such a bunch of traffic came from?
>
>  Shinobu
>
> - Original Message -
> From: "Jan Schermer" 
> To: "Mariusz Gronczewski" 
> Cc: ceph-users@lists.ceph.com
> Sent: Monday, September 7, 2015 9:17:04 PM
> Subject: Re: [ceph-users] Huge memory usage spike in OSD on hammer/giant
>
> Hmm, even network traffic went up.
> Nothing in logs on the mons which started 9/4 ~6 AM?
>
> Jan
>
> > On 07 Sep 2015, at 14:11, Mariusz Gronczewski <
> mariusz.gronczew...@efigence.com> wrote:
> >
> > On Mon, 7 Sep 2015 13:44:55 +0200, Jan Schermer  wrote:
> >
> >> Maybe some configuration change occured that now takes effect when you
> start the OSD?
> >> Not sure what could affect memory usage though - some ulimit values
> maybe (stack size), number of OSD threads (compare the number from this OSD
> to the rest of OSDs), fd cache size. Look in /proc and compare everything.
> >> Also look in "ceph osd tree" - didn't someone touch it while you were
> gone?
> >>
> >> Jan
> >>
> >
> >> number of OSD threads (compare the number from this OSD to the rest of
> > OSDs),
> >
> > it occured on all OSDs, and it looked like that
> > http://imgur.com/IIMIyRG
> >
> > sadly I was on vacation so I didnt manage to catch it before ;/ but I'm
> > sure there was no config change
> >
> >
> >>> On 07 Sep 2015, at 13:40, Mariusz Gronczewski <
> mariusz.gronczew...@efigence.com> wrote:
> >>>
> >>> On Mon, 7 Sep 2015 13:02:38 +0200, Jan Schermer 
> wrote:
> >>>
>  Apart from bug causing this, this could be caused by failure of other
> OSDs (even temporary) that starts backfills.
> 
>  1) something fails
>  2) some PGs move to this OSD
>  3) this OSD has to allocate memory for all the PGs
>  4) whatever fails gets back up
>  5) the memory is never released.
> 
>  A similiar scenario is possible if for example someone confuses "ceph
> osd crush reweight" with "ceph osd reweight" (yes, this happened to me :-)).
> 
>  Did you try just restarting the OSD before you upgraded it?
> >>>
> >>> stopped, upgraded, started. it helped a bit ( <3GB per OSD) but beside
> >>> that nothing changed. I've tried to wait till it stops eating CPU then
> >>> restart it but it still eats >2GB of memory which means I can't start
> >>> all 4 OSDs at same time ;/
> >>>
> >>> I've also added noin,nobackfill,norecover flags but that didnt help
> >>>
> >>> it is suprising for me because before all 4 OSDs total ate less than
> >>> 2GBs of memory so I though I have enough headroom, and we did restart
> >>> machines and removed/added os to test if recovery/rebalance goes fine
> >>>
> >>> it also does not have any external traffic at the moment
> >>>
> >>>
> > On 07 Sep 2015, at 12:58, Mariusz Gronczewski <
> mariusz.gronczew...@efigence.com> wrote:
> >
> > Hi,
> >
> > over a weekend (was on vacation so I didnt get exactly what happened)
> > our OSDs started eating in excess of 6GB of RAM (well RSS), which
> was a
> > problem considering that we had only 8GB of ram for 4 OSDs (about 700
> > pgs per osd and about 70GB space used. So spam of coredumps and OOMs
> > blocked the osds down to unusabiltity.
> >
> > I then upgraded one of OSDs to hammer which made it a bit better
> (~2GB
> > per osd) but still much higher usage than before.
> >
> > any ideas what would be a reason for that ? logs are mostly full on
> > OSDs trying to recover and timed out heartbeats
> >
> > --
> > Mariusz Gronczewski, Administrator
> >
> > Efigence S. A.
> > ul. Wołoska 9a, 02-583 Warszawa
> > T: [+48] 22 380 13 13
> > F: [+48] 22 380 13 14
> > E: mariusz.gronczew...@efigence.com
> > 
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> >>>
> >>>
> >>>
> >>> --
> >>> Mariusz Gronczewski, Administrator
> >>>
> >>> Efigence S. A.
> >>> ul. Wołoska 9a, 02-583 Warszawa
> >>> T: [+48] 22 380 13 13
> >>> F: [+48] 22 380 13 14
> >>> E: mariusz.gronczew...@efigence.com
> >>> 
> >>
> >
> >
> >
> > --
> > Mariusz Gronczewski, Administrator
> >
> > Efigence S. A.
> > ul. Wołoska 9a, 02-583 Warszawa
> > T: [+48] 22 380 13 13
> > F: [+48] 22 380 13 14
> > 

Re: [ceph-users] RAM usage only very slowly decreases after cluster recovery

[池信泽]

You could use 'ceph tell osd.* heap release' to release memory cached
by tcmalloc.

2015-08-28 12:51 GMT+08:00 Somnath Roy somnath@sandisk.com:
 Slow memory release could also be because of tcmalloc. Tcmalloc doesn't 
 release the memory the moment application issue a 'delete' but it cached it 
 inside for future use.
 If it is not a production cluster and you have spare time to reproduce this, 
 I would suggest to build Ceph code with jemalloc and see the behavior. It 
 should be releasing memory much faster than tcmalloc. Basically behavior of 
 jemalloc is similar to glibcmalloc.

 Thanks  Regards
 Somnath


 -Original Message-
 From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of 
 Haomai Wang
 Sent: Thursday, August 27, 2015 7:31 PM
 To: Chad William Seys
 Cc: ceph-users@lists.ceph.com
 Subject: Re: [ceph-users] RAM usage only very slowly decreases after cluster 
 recovery

 Yes, we already notice this, and have PR to fix partial of this I think 
 https://github.com/ceph/ceph/pull/5451/files

 On Fri, Aug 28, 2015 at 4:59 AM, Chad William Seys cws...@physics.wisc.edu 
 wrote:
 Hi all,

 It appears that OSD daemons only very slowly free RAM after an
 extended period of an unhealthy cluster (shuffling PGs around).

 Prior to a power outage (and recovery) around July 25th, the amount of
 RAM used was fairly constant, at most 10GB (out of 24GB).  You can see
 in the attached PNG osd6_stack2.png (Week 30) that the amount of
 used RAM on osd06.physics.wisc.edu was holding steady around 7GB.

 Around July 25th our Ceph cluster rebooted after a power outage.  Not
 all nodes booted successfully, so Ceph proceeded to shuffle PGs to
 attempt to return to health with the renaming nodes.  You can see in
 osd6_stack2.png two purplish spikes showing that the node used
 around 10GB swap space during the recovery period.

 Finally the cluster recovered around July 31st.  During that period
 some I had to take some osd daemons out of the pool b/c their nodes
 ran out of swap space and the daemons were killed by the out of memory
 (OOM) kernel feature.  (The recovery period was probably extended by
 me trying to add the daemons/drives back. If I recall correctly that
 is what was occurring during the second swap
 peak.)

 This RAM usage pattern is in generalthe same for all the nodes in the 
 cluster.

 Almost three weeks later, the amount of RAM used on the node is still
 decreasing, but it has not returned to pre-power outage levels. 15GB
 instead of 7GB.

 Why is Ceph using 2x more RAM than it used to in steady state?

 Thanks,
 Chad.

 (P.S.  It is really unfortunate that Ceph uses more RAM when
 recovering - can lead to cascading failure!)
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




 --
 Best Regards,

 Wheat
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

 

 PLEASE NOTE: The information contained in this electronic mail message is 
 intended only for the use of the designated recipient(s) named above. If the 
 reader of this message is not the intended recipient, you are hereby notified 
 that you have received this message in error and that any review, 
 dissemination, distribution, or copying of this message is strictly 
 prohibited. If you have received this communication in error, please notify 
 the sender by telephone or e-mail (as shown above) immediately and destroy 
 any and all copies of this message in your possession (whether hard copies or 
 electronically stored copies).

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] injectargs not working?

[池信泽]

hi, ceph osd set noscrub(or nodeep-scrub) would stop the scrub
forever. and ceph osd unset noscrub would continue to reschedule
scrub.

so maybe you could use this two command in crontab to schedule scrub manually.

2015-07-30 7:59 GMT+08:00 Quentin Hartman qhart...@direwolfdigital.com:
 well, that would certainly do it. I _always_ forget to twiddle the little
 thing on the web page that changes the version of the docs I'm looking at.

 So I guess then my question becomes, How do i prevent deep scrubs from
 happening in the middle of the day and ruining everything?

 QH


 On Wed, Jul 29, 2015 at 5:55 PM, Travis Rhoden trho...@gmail.com wrote:

 Hi Quentin,

 It may be the specific option you are trying to tweak.
 osd-scrub-begin-hour was first introduced in development release
 v0.93, which means it would be in 0.94.x (Hammer), but your cluster is
 0.87.1 (Giant).

 Cheers,

  - Travis

 On Wed, Jul 29, 2015 at 4:28 PM, Quentin Hartman
 qhart...@direwolfdigital.com wrote:
  I'm running a 0.87.1 cluster, and my ceph tell seems to not be
  working:
 
  # ceph tell osd.0 injectargs '--osd-scrub-begin-hour 1'
   failed to parse arguments: --osd-scrub-begin-hour,1
 
 
  I've also tried the daemon config set variant and it also fails:
 
  # ceph daemon osd.0 config set osd_scrub_begin_hour 1
  { error: error setting 'osd_scrub_begin_hour' to '1': (2) No such
  file or
  directory}
 
  I'm guessing I have something goofed in my admin socket client config:
 
  [client]
  rbd cache = true
  rbd cache writethrough until flush = true
  admin socket = /var/run/ceph/$cluster-$type.$id.asok
 
  but that seems to correlate with the structure that exists:
 
  # ls
  ceph-osd.24.asok  ceph-osd.25.asok  ceph-osd.26.asok
  # pwd
  /var/run/ceph
 
  I can show my configs all over the place, but changing them seems to
  always
  fail. It behaves the same if I'm working on a local daemon, or on my
  config
  node trying to make changes globally.
 
  Thanks in advance for any ideas
 
  QH
 
 
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 



 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ec pool history objects

[池信泽]

hi, all:

when I use ec poll, I see there are some object history for object
xx.

Such as: xx__head_610951D6__2_fe1_2, xx__head_610951D6__2_fe2_2
xx__head_610951D6__2__2

I think this object is used for roll_back when not all shards have
written object to filestore. Is it right?

If that, why not delete this history objects when all shards has
written object to filestore?

-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ec pool history objects

[池信泽]

hi, all:

when I use ec poll, I see there are some object history for object
xx.

[root@node3 2.1d6s2_head]# ll -R | grep xx
-rw-r--r--. 1 root root  65536 Jun 15 17:41 xx__head_610951D6__2_fe1_2
-rw-r--r--. 1 root root  65536 Jun 15 17:41 xx__head_610951D6__2_fe2_2
-rw-r--r--. 1 root root  65536 Jun 15 17:45
xx__head_610951D6__2__2

   I think this object is used for roll_back when not all shards have
written object to filestore. Is it right?

   If that, why not delete this history objects when all shards has written
object to filestore?

-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] deep scrubbing causes osd down

[池信泽]

Sorry, I am not sure whether it is look ok in your production environment.

Maybe you could use the command: ceph tell osd.0 injectargs
-osd_scrub_sleep 0.5 . This command would affect only one osd.

If it works fine for some days, you could set for all osd.

This is just a suggestion.

2015-04-13 14:34 GMT+08:00 Lindsay Mathieson lindsay.mathie...@gmail.com:

 On 13 April 2015 at 16:00, Christian Balzer ch...@gol.com wrote:

 However the vast majority of people with production clusters will be
 running something stable, mostly Firefly at this moment.

  Sorry, 0.87 is giant.
 
  BTW, you could also set osd_scrub_sleep to your cluster. ceph would
  sleep some time as you defined when it has scrub some objects.
  But I am not sure whether is could works good to you.
 
 Yeah, that bit is backported to Firefly and can definitely help, however
 the suggested initial value is too small for most people who have scrub
 issues, starting with 0.5 seconds and see how it goes seems to work
 better.



 Thanks xinze, Christian.

 Yah, I'm on 0.87 in production - I can wait for the next release :)

 In the meantime, from the prior msgs I've set this:

 [osd]
 osd_scrub_chunk_min = 1
 osd_scrub_chunk_max = 5
 osd_scrub_sleep = 0.5


 Do the values look ok? is the [osd] section the right spot?

 Thanks - Lindsay



 --
 Lindsay



-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] deep scrubbing causes osd down

[池信泽]

hi, Loic:

  Do you think it is patch https://github.com/ceph/ceph/pull/3318
worth of backport to firely and giant?



2015-04-13 14:00 GMT+08:00 Christian Balzer ch...@gol.com:

 On Mon, 13 Apr 2015 13:42:39 +0800 池信泽 wrote:

 I knew the scheduler was in the pipeline, good to see it made it in.

 However the vast majority of people with production clusters will be
 running something stable, mostly Firefly at this moment.

 Sorry, 0.87 is giant.

 BTW, you could also set osd_scrub_sleep to your cluster. ceph would
 sleep some time as you defined when it has scrub some objects.
 But I am not sure whether is could works good to you.

 Yeah, that bit is backported to Firefly and can definitely help, however
 the suggested initial value is too small for most people who have scrub
 issues, starting with 0.5 seconds and see how it goes seems to work better.

 Christian

 Thanks.

 2015-04-13 13:30 GMT+08:00 池信泽 xmdx...@gmail.com:
  hi, you could restrict scrub to certain times of day based on
  https://github.com/ceph/ceph/pull/3318.
 
  You could set osd_scrub_begin_hour and osd_scrub_begin_hour which are
  suitable for you. This feature is available since 0.93.
 
  But it has not been backport to 0.87 (hammer).
 
  2015-04-13 12:55 GMT+08:00 Lindsay Mathieson
  lindsay.mathie...@gmail.com:
 
  On 13 April 2015 at 11:02, Christian Balzer ch...@gol.com wrote:
 
  Yeah, that's a request/question that comes up frequently.
  And so far there's no option in Ceph to do that (AFAIK), it would be
  really nice along with scheduling options (don't scrub during peak
  hours), which have also been talked about.
 
 
 
 
  I was just about to post a question on that ... :)
 
  Just had devs and support bitching to me that all their VM's were
  running like dogs (they were). Ceph decided to run a deep scrub
  Monday afternoon.
 
  It boggles me that there are no options for controlling the schedule,
  considering how critically important the timing is. I'd be happy to
  run a deep scrub ever night at 1am. Anytime during the day is a
  disaster.
 
  So currently I have noscrub and nodeep-scrub set which is really less
  than ideal.
 
 
  --
  Lindsay
 
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 
 
 
 
  --
  Regards,
  xinze





 --
 Christian BalzerNetwork/Systems Engineer
 ch...@gol.com   Global OnLine Japan/Fusion Communications
 http://www.gol.com/



-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] deep scrubbing causes osd down

[池信泽]

Sorry, 0.87 is giant.

BTW, you could also set osd_scrub_sleep to your cluster. ceph would
sleep some time as you defined when it has scrub some objects.
But I am not sure whether is could works good to you.

Thanks.

2015-04-13 13:30 GMT+08:00 池信泽 xmdx...@gmail.com:
 hi, you could restrict scrub to certain times of day based on
 https://github.com/ceph/ceph/pull/3318.

 You could set osd_scrub_begin_hour and osd_scrub_begin_hour which are
 suitable for you. This feature is available since 0.93.

 But it has not been backport to 0.87 (hammer).

 2015-04-13 12:55 GMT+08:00 Lindsay Mathieson lindsay.mathie...@gmail.com:

 On 13 April 2015 at 11:02, Christian Balzer ch...@gol.com wrote:

 Yeah, that's a request/question that comes up frequently.
 And so far there's no option in Ceph to do that (AFAIK), it would be
 really nice along with scheduling options (don't scrub during peak hours),
 which have also been talked about.




 I was just about to post a question on that ... :)

 Just had devs and support bitching to me that all their VM's were running
 like dogs (they were). Ceph decided to run a deep scrub Monday afternoon.

 It boggles me that there are no options for controlling the schedule,
 considering how critically important the timing is. I'd be happy to run a
 deep scrub ever night at 1am. Anytime during the day is a disaster.

 So currently I have noscrub and nodeep-scrub set which is really less than
 ideal.


 --
 Lindsay

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




 --
 Regards,
 xinze



-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] more human readable log to track request or using mapreduce for data statistics

[池信泽]

hi，ceph:

Currently, the command ”ceph --admin-daemon
/var/run/ceph/ceph-osd.0.asok dump_historic_ops“ may return as below:

{ description: osd_op(client.4436.1:11617
rb.0.1153.6b8b4567.0192 [] 2.8eb4757c ondisk+write e92),
  received_at: 2015-03-25 19:41:47.146145,
  age: 2.186521,
  duration: 1.237882,
  type_data: [
commit sent; apply or cleanup,
{ client: client.4436,
  tid: 11617},
[
{ time: 2015-03-25 19:41:47.150803,
  event: event1},
{ time: 2015-03-25 19:41:47.150873,
  event: event2},
{ time: 2015-03-25 19:41:47.150895,
  event: event3},
{ time: 2015-03-25 19:41:48.384027,
  event: event4}]]}

I think this message is not so suitable for grep log or using
mapreduce for data statistics. Such as, I want to know
the write request avg latency for each rbd everyday. If we could
output the all latency in just one line, it would be very easy to
achieve it.

Such as, the output log maybe something like this:
2015-03-26 03:30:53.859759 osd=osd.0 pg=2.11 op=(client.4436.1:11617
rb.0.1153.6b8b4567.0192 [] 2.8eb4757c ondisk+write e92)
received_at=1427355253 age=2.186521 duration=1.237882 tid=11617
client=client.4436 event1=20ms event2=300ms event3=400ms event4=100ms.

The above:

duration means: the time between (reply_to_client_stamp -
request_received_stamp)
event1 means: the time between (the event1_stamp - request_received_stamp)
...
event4 means: the time between (the event4_stamp - request_received_stamp)

Now, If we output the every log as above. it would be every easy to
know the write request avg latency for each rbd everyday.
Or if I use grep it is more easy to find out which stage is the bottleneck.

-- 
Regards,
xinze
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] scubbing for a long time and not finished

[池信泽]

On 周二, 3月 17, 2015 at 10:01 上午, Xinze Chi xmdx...@gmail.com wrote:hi,all:



 I find a pg on my test cluster in doing scrubbing for a long time

and not finish. there are not some useful scrubbing log. scrubs_active

is 1, so inc_scrubs_pending return false. I think the reason is that

some scrub message is lost, so primary can not continue chunky_scrub ,

so it hang up at scrubbing.



 Could anyone give some suggestion?



 Thanks





[root@ceph0 ~]# date

Tue Mar 17 09:54:54 CST 2015

[root@ceph0 ~]# ceph pg dump | grep scrub

dumped all in format plain

pg_stat objects mip degr misp unf bytes log disklog state state_stamp

v reported up up_primary acting acting_primary last_scrub

scrub_stamplast_deep_scrub deep_scrub_stamp

1.97 30 0 0 0 0 117702656 31 31 active+clean+scrubbing 2015-03-16

14:50:02.110796 78'31 78:50 [9,6,1] 9 [9,6,1] 9 0'0 2015-03-15

14:49:33.661597 0'0 2015-03-13 14:48:53.341679



 The attachment is the log from primary, the scrubbing pg is 1.97s0.



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] perf counter reset

[池信泽]

Hi, cepher:

 How to reset the perf counter. Such as I want to reset the
journal_queue_ops 0. Is there a

comand to reset it?

 Thanks.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] why the erasure code pool not support random write?

[池信泽]

hi, cephers:

  When I look into the ceph source code, I found the erasure code pool
not support
the random write, it only support the append write. Why? Is that random
write of is erasure code high cost and the performance of the deep scrub is
very poor?

 Thanks.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] why the erasure code pool not support random write?

[池信泽]

Thanks.

   Another reason is the checksum in the attr of object used for deep scrub
in EC pools should be computed when modify the object. When supporting the
random write, We should caculate the whole object for checksum, even if
there is a bit modified. If only supporting append write, We can get the
checksum based on the previously checksum and the append date which is more
quickly.

   Am I right?

2014-10-21 0:36 GMT+08:00 Gregory Farnum g...@inktank.com:

 This is a common constraint in many erasure coding storage system. It
 arises because random writes turn into a read-modify-write cycle (in order
 to redo the parity calculations). So we simply disallow them in EC pools,
 which works fine for the target use cases right now.
 -Greg


 On Monday, October 20, 2014, 池信泽 xmdx...@gmail.com wrote:

 hi, cephers:

   When I look into the ceph source code, I found the erasure code
 pool not support
 the random write, it only support the append write. Why? Is that random
 write of is erasure code high cost and the performance of the deep scrub is
 very poor?

  Thanks.



 --
 Software Engineer #42 @ http://inktank.com | http://ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph data consistency

[池信泽]

hi, everyone:

  when I read the filestore.cc, I find the ceph use crc the check the data.
Why should check the data?

  In my knowledge,  the disk has error-correcting code
http://en.wikipedia.org/wiki/Error-correcting_code (ECC) for each sector.
Looking at wiki: http://en.wikipedia.org/wiki/Disk_sector, In disk drives,
each physical sector is made up of three basic parts, the sectorheader
http://en.wikipedia.org/wiki/Header_(computing), the data area and
the error-correcting
code http://en.wikipedia.org/wiki/Error-correcting_code (ECC).  So if
the data is not correct. the disk can recovery it or  return i/o error.

  Does anyone can explain it?

 Thanks
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com