On 08/09/2011 08:00 PM, Fyodor Ustinov wrote:
On 08/09/2011 07:26 PM, Fyodor Ustinov wrote:
Hi!
root@backup-ext:/etc/pure-ftpd/conf# ps v -C cfuse
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
916 ?Sl86:35 21025 2260 1514931 645632 63.1
/usr/bin/cfuse -m
On 08/06/2011 12:12 AM, Fyodor Ustinov wrote:
On 08/05/2011 11:17 PM, Sage Weil wrote:
On Fri, 5 Aug 2011, Fyodor Ustinov wrote:
On 08/05/2011 07:07 PM, Sage Weil wrote:
This is the problem. The readahead patches in the master branch of
git://ceph.newdream.net/git/ceph-client.git. They're
On 08/05/2011 07:07 PM, Sage Weil wrote:
This is not. I'll open a bug and try to track this one down. It looks
new.
In yours kernel version I do not see this trouble.
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to
On 08/05/2011 04:26 AM, Sage Weil wrote:
On Fri, 5 Aug 2011, Fyodor Ustinov wrote:
On 08/04/2011 10:53 PM, Sage Weil wrote:
The current patches are on top of v3.0, but you should be able to rebase
the readahead stuff on top of anything reasonably recent.
sage
As usual.
cluster - latest 0.32
Qian Wei hzqianwei at gmail.com writes:
Hi friends
Anyone has plan to develop a CEPH native client for Windows, or
give effective way to provide stroage to Windows user
Reshare via samba or nfs?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel
On 08/05/2011 07:07 PM, Sage Weil wrote:
This is the problem. The readahead patches in the master branch of
git://ceph.newdream.net/git/ceph-client.git. They're not upstream yet.
Sorry that wasn't clear!
On 08/05/2011 11:17 PM, Sage Weil wrote:
On Fri, 5 Aug 2011, Fyodor Ustinov wrote:
On 08/05/2011 07:07 PM, Sage Weil wrote:
This is the problem. The readahead patches in the master branch of
git://ceph.newdream.net/git/ceph-client.git. They're not upstream yet.
Sorry that wasn't clear!
http
Sage Weil sage at newdream.net writes:
Hi,
I've just pushed a wip-readahead branch to ceph-client.git that rewrites
ceph_readpages (used for readahead) to be fully asynchronous. This should
let us take full advantage of whatever the readahead window is. I'm still
doing some testing
On 08/04/2011 10:53 PM, Sage Weil wrote:
The current patches are on top of v3.0, but you should be able to rebase
the readahead stuff on top of anything reasonably recent.
sage
As usual.
cluster - latest 0.32 from your ubuntu rep.
client - latest git-pulled kernel.
dd file from cluster to
Hi!
I'm not sure that makes sense with errors on ext4 complain about errors
in ceph.fs, but maybe it will be interesting:
root@stb1:/mnt/dcvolia/amanda# rm -rf *
rm: cannot remove `state/servers/index/butan.dcv/_': Directory not empty
rm: cannot remove `state/servers/index/iprit.dcv/_':
On 07/31/2011 07:54 AM, Sage Weil wrote:
On Sat, 30 Jul 2011, Ted Ts'o wrote:
On Sat, Jul 30, 2011 at 10:21:13AM -0700, Sage Weil wrote:
We do use xattrs extensively, though; that was the last extN bug we
uncovered. That's where my money is.
Hmm, yes. That could very well be. How big are
On 07/31/2011 08:04 PM, Sage Weil wrote:
dout(10) setattrs cid / oid ' p-first ' len p-second.length()
= r dendl;
Well.
root@osd0:~# grep 1000483\.05d6 /var/log/ceph/osd.0.log
2011-07-31 23:06:49.172838 7f23c048c700 filestore(/osd.0) remove
fail. Epic fail.
Absolutely reproducible.
I have ceph cluster with this configuration:
8 physical servers
14 osd servers.
Each osd server have personal fs.
48T total size of ceph cluster.
17T used.
Now, step by step:
1. Stop ceph server osd0
/etc/init.d/ceph stop
2. Make fresh fs for osd
On 07/30/2011 06:34 PM, Theodore Tso wrote:
On Jul 30, 2011, at 5:38 AM, Fyodor Ustinov wrote:
fail. Epic fail
Inode 238551053, i_blocks is 24, should be 32. Fix? yes
Inode 238551054, i_blocks is 40, should be 32. Fix? yes
...
Fyodor, what kernel were you using? Was it a RHEL 6.x
kernel
Ted Ts'o tytso at mit.edu writes:
Right, sorry, I missed that. And just to be clear this wasn't an -rc
kernel but 3.0 final, right?
Yes.
Hmm, looking through recent commits which will shortly be merged into
3.1, this one leaps out, but I'm not sure it's the cause --- how full
was your
On 07/30/2011 08:21 PM, Sage Weil wrote:
Fyodor, if you set 'debug filestore = 10' you'll get a log of every
operation on the fs in the osd log. (Or close to it; there may be a few
that we missed, but to a first approximation at least it'll describe the
workload pretty well.)
Well, I do it.
On 07/28/2011 07:35 AM, Sage Weil wrote:
Oh, you mean the fsck errors? Got it. If you see it again (on another
disk/machine, with a reasonably fresh fs) the ext4 guys will be interested
in hearing about it. It's probably in the xattr code, which ceph uses
heavily but most people only set once
On 07/27/2011 01:24 AM, Gregory Farnum wrote:
Hmm, I'm afraid I can't see anything useful in these logs right now --
just confusion!
The only thing I can think of that might provide useful information
now is to turn off osd5, update its config file to have heavy
debugging information (debug osd
Christian Brunner chb at muc.de writes:
From what I read in Fyodors emails ext4 in 2.6.39 isn't much better.
So, what filesystem would you recommend?
Christian, IMHO ext3/ext4/btrfs unusable for ceph. I tested some time xfs, and
think to use it again. But I think that the ceph
On 07/26/2011 01:55 AM, Gregory Farnum wrote:
On Mon, Jul 25, 2011 at 2:16 PM, Fyodor Ustinovu...@ufm.su wrote:
On 07/25/2011 11:54 PM, Gregory Farnum wrote:
Hmmm, what are you doing with this PG? Have you been doing any snapshots?
I'm not use snapshot in ceph. Only as simple file system.
Sage Weil sage at newdream.net writes:
Ceph (by default) stripes files over 4MB objects. The default policy is
configurable on a per-directory or per-file basis.
Wow. Hot to?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message
On 07/19/2011 05:29 PM, Sage Weil wrote:
On Tue, 19 Jul 2011, Fyodor Ustinov wrote:
Hi!
2011-07-19 14:00:39.509391 log 2011-07-19 14:00:34.185718 osd5
10.5.51.145:6803/19737 1563 : [ERR] 0.f8 scrub stat mismatch, got 2074/2074
objects, 0/0 clones, 8624991435/8590119115 bytes, 8422853/8388798
Hi!
2011-07-19 14:00:39.509391 log 2011-07-19 14:00:34.185718 osd5
10.5.51.145:6803/19737 1563 : [ERR] 0.f8 scrub stat mismatch, got
2074/2074 objects, 0/0 clones, 8624991435/8590119115 bytes,
8422853/8388798 kb.
2011-07-19 14:00:39.509391 log 2011-07-19 14:00:34.185732 osd5
On 07/19/2011 05:29 PM, Sage Weil wrote:
ceph pg repair 0.f8
Thnx!
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 07/19/2011 05:31 PM, Sage Weil wrote:
Hi Fyodor,
Which commits are you running on the server and client side? There was a
bad protocol version number in master branch for several days (fixed now),
and some open intent stuff in ceph-client.git that wasn't quite right a
bit further back.
On 07/19/2011 06:49 PM, Sage Weil wrote:
Okay, I found the problem (file mode calculation when O_APPEND is
present). There is a fix pushed to the stable branch of ceph.git (for
mds) and to the for-linus branch of the kernel ceph-client.git. You need
both to behave.
Ok, I wait this. :)
What
Hi again.
I start to fear.
After deleting one of file I got:
2011-07-10 16:23:00.062754 log 2011-07-10 16:22:51.410099 mds0
10.5.51.230:6800/24957 5 : [ERR] loaded dup inode 100a5e9 [2,head]
v244 at
/dcvolia/amanda/state/servers/index/ticket.dcv/_/20110708000502_7.gz,
but inode
On 07/11/2011 12:12 AM, Sage Weil wrote:
Hmm, yeah the 'loaded dup inode' messages should never happen. They
appear to be just files, though, not directories, so you can safely ignore
them. The namespace repair will need to clean that up at some point. The
recursive rstat errors you saw are
On 07/11/2011 12:36 AM, Sage Weil wrote:
On Mon, 11 Jul 2011, Fyodor Ustinov wrote:
On 07/11/2011 12:12 AM, Sage Weil wrote:
Hmm, yeah the 'loaded dup inode' messages should never happen. They
appear to be just files, though, not directories, so you can safely ignore
them. The namespace
Hi!
ceph 0.30
the rate of re-reading the file 10G. 13 OSD servers (ie the file is
already in the cache in OSD servers).
cfuse: 118 MB/s
kenel (3.0.0-rc3): 84.0 MB/s
:(
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to
On 07/02/2011 11:45 PM, Sage Weil wrote:
On Sat, 2 Jul 2011, Fyodor Ustinov wrote:
Hi!
ceph 0.30
the rate of re-reading the file 10G. 13 OSD servers (ie the file is already in
the cache in OSD servers).
cfuse: 118 MB/s
kenel (3.0.0-rc3): 84.0 MB/s
:(
Yeah, this is still on the list. We
Hi!
mds - 0.30
I can not to reproduce, sorry.
mds/Locker.cc: In function 'void Locker::file_excl(ScatterLock*,
bool*)', in thread '0x7fefc6c68700'
mds/Locker.cc: 3982: FAILED assert(in-get_loner() = 0
in-mds_caps_wanted.empty())
ceph version 0.30
On 07/03/2011 01:03 AM, Sage Weil wrote:
Which commit were you running?
On mds server from here:
deb http://ceph.newdream.net/debian/ natty main
deb-src http://ceph.newdream.net/debian/ natty main
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message
On 06/20/2011 08:16 PM, Gregory Farnum wrote:
4. Can I recover the data without backup?
Possibly. If the inconsistency is just that one of the OSDs is storing the
object and another OSD says the object shouldn't exist, that can be recovered
from by working out which one is correct.
Have you
On 06/17/2011 07:27 PM, Gregory Farnum wrote:
4. Can I recover the data without backup?
Possibly. If the inconsistency is just that one of the OSDs is storing the
object and another OSD says the object shouldn't exist, that can be recovered
from by working out which one is correct.
Have you
Hi!
I have inconsistent chunk:
2011-06-17 08:11:33.844030pg v26069: 2376 pgs: 2375 active+clean, 1
active+clean+inconsistent; 285 GB data, 572 GB used, 24180 GB / 25494 GB
avail
Questions:
1. What does this mean?
2. How to learn what kind of file is damaged?
3. Delete damaged file
Hi!
Which configuration would you recommended for cluster with 50-80 million
files?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi!
kernel 2.6.39
ceph - 0.28.2
In sysctl.conf set
vm.min_free_kbytes=262144
Jun 2 03:08:17 amanda kernel: [35398.757055] libceph: msg_new can't
allocate 4096 bytes
Jun 2 03:08:17 amanda kernel: [35398.757088] libceph: msg_new can't
create type 0 front 4096
Jun 2 03:08:17 amanda kernel:
On 06/03/2011 06:39 PM, Tommi Virtanen wrote:
On Fri, Jun 03, 2011 at 01:40:44PM +0300, Fyodor Ustinov wrote:
Hi!
kernel 2.6.39
ceph - 0.28.2
In sysctl.conf set
vm.min_free_kbytes=262144
Jun 2 03:08:17 amanda kernel: [35398.757055] libceph: msg_new can't
allocate 4096 bytes
... so first
Hi.
I have question to developers.
ext3 and btrfs - unusable.
Guys, what kind of fs do you use at your stand?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at
ceph - 0.28.2 from repository
client - linux 2.6.39/64
All osd servers have 4G RAM. Journal on 1G tempfs. 2*1G ethernet
(etherchannel)
client server have 2G RAM. 2*1G ethernet (etherchannel)
In tests used dd if=/dev/zero of=file bs=8M count=3000 for write and
dd if=file of=/dev/null bs=8M
On 05/31/2011 01:50 AM, Fyodor Ustinov wrote:
Hi!
Fresh 0.28.2 cluster.
Why reading two times slower than the writing by dd, but rados show
different.
(Second question - why rados bench crash on read test?)
root@gate0:/mnt# dd if=/dev/zero of=aaa bs=1024000 count=1
1+0 records
On 05/30/2011 06:54 AM, Sage Weil wrote:
Hi Fyodor,
The problem is your rules:
On Sun, 29 May 2011, Fyodor Ustinov wrote:
# rules
rule data {
ruleset 0
type replicated
min_size 1
max_size 10
step take root
step choose firstn 0 type rack
Hi!
Hmm... As I see in sources - this release is out without patch for #1116 ?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi!
Fresh 0.28.2 cluster.
Why reading two times slower than the writing by dd, but rados show
different.
(Second question - why rados bench crash on read test?)
root@gate0:/mnt# dd if=/dev/zero of=aaa bs=1024000 count=1
1+0 records in
1+0 records out
1024000 bytes (10 GB)
Hi!
ceph is a very powerful thing.
it kills ext3:
[49531.320004] [811d3c4c] ext3_new_blocks+0x40c/0x6b0
[49531.320004] [811d3f11] ext3_new_block+0x21/0x30
[49531.320004] [811e798e] ext3_xattr_block_set+0x2ae/0x790
[49531.320004] [811e86ba]
On 05/27/2011 07:29 PM, Sage Weil wrote:
Which kernel version is this? I've only seen this on older (.26?)
kernels. If this is on the latest, we need to tell the extN guys about
it.
2.6.39
If you remember, about a month ago I wrote about a similar bug in 2.6.38
WBR,
Fyodor.
--
To
On 05/27/2011 06:16 PM, Gregory Farnum wrote:
This is an interesting one -- the invariant that assert is checking
isn't too complicated (that the object lives on the RecoveryWQ's
queue) and seems to hold everywhere the RecoveryWQ is called. And the
functions modifying the queue are always called
On 05/27/2011 06:18 PM, Gregory Farnum wrote:
You are really putting this stuff through the wringer -- nobody else
hits these issues so frequently!
I wait release of ceph bouncing on the chair with impatience. :)
And to help catch errors as many as possible before the release - it's a
minimum
On 05/27/2011 10:18 PM, Gregory Farnum wrote:
Can you check out the recoverywq_fix branch and see if that prevents
this issue? Or just apply the patch I've included below. :)
-Greg
Looks as though this patch has helped.
At least this osd has completd rebalancing.
Great! Thanks!
WBR,
Hi!
How to get information about status of each server in cluster?
#ceph osd stat
2011-05-26 15:07:05.103621 mon - [osd,stat]
2011-05-26 15:07:05.104201 mon0 - 'e413: 6 osds: 5 up, 5 in' (0)
I see - in cluster 6 osd servers and now up only 5. How do I know which
server is down?
More global
On 05/26/2011 10:41 PM, Gregory Farnum wrote:
On Thu, May 26, 2011 at 3:30 PM, Fyodor Ustinovu...@ufm.su wrote:
On 05/26/2011 08:37 PM, Gregory Farnum wrote:
That means something in the messaging layers got a bit busted, and you
should generally respond by filing bug reports.:) What version
On 05/24/2011 05:55 PM, Brian Chrisman wrote:
On Tue, May 24, 2011 at 3:05 AM, Fyodor Ustinovu...@ufm.su wrote:
Hi!
I need to implement this scheme of servers connections:
All servers of ceph cluster have 2 network addresses in different nets.
One net - for ceph interconnect. I.e. for
Hi!
It's me again (I feel like that I'm here soon banned :)
ceph have fuse client and kernel client.
Why are two of them?
Which is better?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More
Sage Weil sage at newdream.net writes:
Based on your requirements, it sounds like Ceph is a good fit. As for
status, here is the latest update:
- Ceph is not yet ready for production however, we are confident that we
are only a few months away
- We have not experienced any data loss
Hi!
Two questions.
1-st. As I understand it - any usable changes of CRUSH map on active
cluster can't destruct data? In the worst case (for example, I managed
to put two osd in one rack, and now I have two copies of the data in a
single rack) I get information about the degradation and this
Hi.
2011-05-24 00:17:45.490684 7f45415e1740 ceph version 0.28.commit:
071881d7e5599571e46bda17094bb4b48691e89a. process: cmds. pid: 4424
2011-05-24 00:17:45.492293 7f453ef81700 mds-1.0 ms_handle_connect on
77.120.112.193:6789/0
2011-05-24 00:17:49.497862 7f453ef81700 mds-1.0 handle_mds_map
On 05/24/2011 01:27 AM, Sage Weil wrote:
Hi Fyodor,
This looks like #1104. Will try to sort it out today, it should be a
simple one.
sage
May be need my debug mds = 20, debug ms = 1 log?
But it is zipped - 26M. Need?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line
Hi!
Sage, you receive my email about bt ?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi!
Let's say I have 6 osd. 5 of them are the same, 6 - has twice the amount
of disk size.
Should I somehow specifically describe it in CRUSH mapor seph will
understand without it?
WBR,
Fyodor.
P.S. I'd like to receive secret knowledge about the weight in crush
map. :)
--
To
Hi!
How should I respond to the message libceph: get_reply unknown tid
68724 from osd0 in syslog?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at
On 05/13/2011 10:11 PM, Sage Weil wrote:
Fyodor,
On Fri, 13 May 2011, Sage Weil wrote:
On Thu, 12 May 2011, Sage Weil wrote:
On Thu, 12 May 2011, Sage Weil wrote:
equation. I'm running iozone on ext3 now and not having any problems.
I take it back.. I just reproduced a similar error on
On Friday 13 May 2011 18:53:58 you wrote:
On Thu, 12 May 2011, Fyodor Ustinov wrote:
Hi!
Like previous, but ceph fs instead of rbd.
(i.e. iozone with 4G file).
[ 783.295035] ceph: loaded (mds proto 32)
[ 783.300122] libceph: client4125 fsid
ff352dfd-078c-e65f-a769-d25abb384d92
Hi!
Latest (git pulled) version of 2.6 kernel. Ceph - 0.27.1
Still troubles with rbd. Now with ocfs2 no messages in syslog, but iozone
still return error:
#df -h
FilesystemSize Used Avail Use% Mounted on
/dev/sda1 237G 15G 210G 7% /
none 2.0G
Hi!
Like previous, but ceph fs instead of rbd.
(i.e. iozone with 4G file).
[ 783.295035] ceph: loaded (mds proto 32)
[ 783.300122] libceph: client4125 fsid ff352dfd-078c-e65f-a769-d25abb384d92
[ 783.300642] libceph: mon0 77.120.112.193:6789 session established
[ 941.278185] libceph: msg_new
Additional - it's not just 2.6.29-rc6, it's torvalds/linux-2.6.git, i.e.
with latest patches, as I understand.
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel in
the body of a message to majord...@vger.kernel.org
More majordomo info at
Hi!
root@stb1:~# dd if=/dev/zero of=/dev/rbd0 bs=1024
^C13217+0 records in
13217+0 records out
13534208 bytes (135 GB) copied, 1135.75 s, 119 MB/s
root@stb1:~# dd if=/dev/rbd0 of=/dev/null bs=1024
^C930+0 records in
929+0 records out
951296 bytes (9.5 GB) copied, 394.773 s, 24.1
Hi!
Latest 64 bit kernel (2.6.39-rc6, but with latest ubuntu 2.6.38-8 - the
same).
Very similar to that rbd or libceph ruin the memory.
1. Make rbd.
rbd create tmt --size 102400
2. Attach and make disk
modprobe rbd
echo xx.xx.xx.xx name=admin rbd tmt /sys/bus/rbd/add
mkfs.ext4 -E
On 04/13/2011 07:22 PM, Sage Weil wrote:
On Wed, 13 Apr 2011, Henry Chang wrote:
Hi,
2011/4/13 Fyodor Ustinovu...@ufm.su:
Hi!
may be anyone seen this trouble?
I have seen this when running OSD with ext3 and never seen it after
switching to ext4.
Yeah. We've seen it on ext3 prior to 2.6.34
On 04/13/2011 10:32 PM, Fyodor Ustinov wrote:
May be md. I test now without md.
It's not md.
Excellent. It seems that ubuntu 11.04 (or, may be, kernel 2.6.38?) do
not have a working fs for ceph?
WBR,
Fyodor.
--
To unsubscribe from this list: send the line unsubscribe ceph-devel
70 matches
Mail list logo
