Re: [ceph-users] cephfs Kernel panic
Am 12.04.2016 um 12:09 schrieb Florian Haas:
> On Tue, Apr 12, 2016 at 11:53 AM, Simon Ferber
> wrote:
>> Thank you! That's it. I have installed the Kernel from the Jessie
>> backport. Now the crashes are gone.
>> How often do these things happen? It would be a worst case scenario, if
>> a system update breaks a productive system.
>
> For what it's worth, what you saw is kernel (i.e. client) side
> breakage. You didn't mess up your Ceph cluster, nor your CephFS
> metadata, nor any data. Also, anything you do in CephFS using a
> release before Jewel must be considered experimental, and while things
> will generally not break even on the client, you shouldn't be
> surprised if they do. Thirdly, my recommendation for any Ceph
> client-side kernel functionality (both rbd.ko and CephFS) would be to
> use nothing older than a 4.x kernel.
Thank you for clarification, Florian.
>
> A good update on the current state of CephFS is this tech talk, which
> John Spray did in February:
>
> https://www.youtube.com/watch?v=GbdHxL0vc9I
> slideshare.net/JohnSpray1/cephfs-update-february-2016
>
> Also, please don't ever do this:
>
> cluster 2a028d5e-5708-4fc4-9c0d-3495c1a3ef3d
> health HEALTH_OK
> monmap e2: 2 mons at
> {ollie2=129.217.207.207:6789/0,stan2=129.217.207.206:6789/0}
> election epoch 12, quorum 0,1 stan2,ollie2
> mdsmap e10: 1/1/1 up {0=ollie2=up:active}, 1 up:standby
> osdmap e72: 8 osds: 8 up, 8 in
> flags sortbitwise
> pgmap v137: 428 pgs, 4 pools, 2396 bytes data, 20 objects
> 281 MB used, 14856 GB / 14856 GB avail
> 428 active+clean
>
> 2 mons. Never, and I repeat never, run your Ceph cluster with 2 mons.
> You want to run 3.
Thus if there are two servers only (which used to use drdb) what would
be the best solution? Just grab another Linux server and install a ceph
cluster node without OSDs and a monitor only?
Best
Simon
>
> Cheers,
> Florian
>
--
Simon Ferber
Techniker
Technische Universität Dortmund
Fakultät Statistik
Vogelpothsweg 87
44227 Dortmund
Tel.: +49 231-755 3188
Fax: +49 231-755 5305
simon.fer...@tu-dortmund.de
www.tu-dortmund.de
Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie
ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der
für diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den
Absender und vernichten Sie diese Mail. Vielen Dank.
Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen
ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher
Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung
eines solchen Schriftstücks per Telefax erfolgen.
Important note: The information included in this e-mail is confidential.
It is solely intended for the recipient. If you are not the intended
recipient of this e-mail please contact the sender and delete this
message. Thank you.
Without prejudice of e-mail correspondence, our statements are only
legally binding when they are made in the conventional written form
(with personal signature) or when such documents are sent by fax.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] cephfs Kernel panic
Thank you! That's it. I have installed the Kernel from the Jessie
backport. Now the crashes are gone.
How often do these things happen? It would be a worst case scenario, if
a system update breaks a productive system.
Best
Simon
Am 11.04.2016 um 16:58 schrieb Ilya Dryomov:
> On Mon, Apr 11, 2016 at 4:37 PM, Simon Ferber
> wrote:
>> Hi,
>>
>> I try to setup an ceph cluster on Debian 8.4. Mainly I followed a
>> tutorial at
>> http://adminforge.de/raid/ceph/ceph-cluster-unter-debian-wheezy-installieren/
>>
>> As far as I can see, the first steps are just working fine. I have two
>> nodes with four OSD on both nodes.
>> This is the output of ceph -s
>>
>> cluster 2a028d5e-5708-4fc4-9c0d-3495c1a3ef3d
>> health HEALTH_OK
>> monmap e2: 2 mons at
>> {ollie2=129.217.207.207:6789/0,stan2=129.217.207.206:6789/0}
>> election epoch 12, quorum 0,1 stan2,ollie2
>> mdsmap e10: 1/1/1 up {0=ollie2=up:active}, 1 up:standby
>> osdmap e72: 8 osds: 8 up, 8 in
>> flags sortbitwise
>> pgmap v137: 428 pgs, 4 pools, 2396 bytes data, 20 objects
>> 281 MB used, 14856 GB / 14856 GB avail
>> 428 active+clean
>>
>> Then I tried to add cephfs following the manual at
>> http://docs.ceph.com/docs/hammer/cephfs/createfs/ which seem to do it's
>> magic:
>> root@stan2:~# ceph fs ls
>> name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ]
>>
>> However, as soon as I try to mount the cephfs with mount.ceph
>> 129.217.207.206:6789:/ /mnt/ -v -o
>> name=cephfs,secretfile=/etc/ceph/client.cephfs the server which tries to
>> mount crashes and has to be cold started again. To be able to use
>> mount.ceph I had to install ceph-fs-common - if that does matter...
>>
>> Here is the kernel.log. Can you give me hints? I am pretty stuck on this
>> for the last few days.
>>
>> Apr 11 16:25:02 stan2 kernel: [ 171.086381] Key type ceph registered
>> Apr 11 16:25:02 stan2 kernel: [ 171.086649] libceph: loaded (mon/osd
>> proto 15/24)
>> Apr 11 16:25:02 stan2 kernel: [ 171.090582] FS-Cache: Netfs 'ceph'
>> registered for caching
>> Apr 11 16:25:02 stan2 kernel: [ 171.090596] ceph: loaded (mds proto 32)
>> Apr 11 16:25:02 stan2 kernel: [ 171.096727] libceph: client34164 fsid
>> 2a028d5e-5708-4fc4-9c0d-3495c1a3ef3d
>> Apr 11 16:25:02 stan2 kernel: [ 171.133832] libceph: mon0
>> 129.217.207.206:6789 session established
>> Apr 11 16:25:02 stan2 kernel: [ 171.161199] [ cut here
>> ]
>> Apr 11 16:25:02 stan2 kernel: [ 171.161239] kernel BUG at
>> /build/linux-lqALYs/linux-3.16.7-ckt25/fs/ceph/mds_client.c:1846!
>> Apr 11 16:25:02 stan2 kernel: [ 171.161294] invalid opcode: [#1] SMP
>> Apr 11 16:25:02 stan2 kernel: [ 171.161328] Modules linked in: cbc ceph
>> libceph xfs libcrc32c crc32c_generic binfmt_misc mptctl mptbase nfsd
>> auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc nls_utf8
>> nls_cp437 vfat fat x86_pkg_temp_thermal intel_powerclamp intel_rapl
>> coretemp kvm_intel kvm crc32_pclmul cryptd iTCO_wdt iTCO_vendor_support
>> efi_pstore efivars pcspkr joydev evdev ast i2c_i801 ttm drm_kms_helper
>> drm lpc_ich mfd_core mei_me mei shpchp ioatdma tpm_tis wmi tpm ipmi_si
>> ipmi_msghandler processor thermal_sys acpi_power_meter button acpi_pad
>> fuse autofs4 ext4 crc16 mbcache jbd2 dm_mod raid1 md_mod hid_generic sg
>> usbhid hid sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
>> crct10dif_common crc32c_intel ahci libahci ehci_pci mpt3sas igb
>> raid_class i2c_algo_bit xhci_hcd libata ehci_hcd scsi_transport_sas
>> i2c_core dca usbcore ptp usb_common scsi_mod pps_core
>> Apr 11 16:25:02 stan2 kernel: [ 171.162046] CPU: 0 PID: 3513 Comm:
>> kworker/0:9 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.7-ckt25-2
>> Apr 11 16:25:02 stan2 kernel: [ 171.162104] Hardware name: Supermicro
>> SYS-6028R-WTR/X10DRW-i, BIOS 1.0c 01/07/2015
>> Apr 11 16:25:02 stan2 kernel: [ 171.162158] Workqueue: ceph-msgr
>> con_work [libceph]
>> Apr 11 16:25:02 stan2 kernel: [ 171.162194] task: 88103f2e8ae0 ti:
>> 88103bfbc000 task.ti: 88103bfbc000
>> Apr 11 16:25:02 stan2 kernel: [ 171.162243] RIP:
>> 0010:[] []
>> __prepare_send_request+0x801/0x810 [ceph]
>> Apr 11 16:25:02 stan2 kernel: [ 171.162312] RSP: 0018:88103bfbfba8
>> EFLAGS: 00010283
>> Apr 11 16:25:02 stan2 kernel: [ 171.162347] RAX: 88103f88ad42 RBX:
>> 88103f7f7400 RCX:
>> Apr 11 16:25:02 stan2 kernel: [ 171.162394] RDX: 0
[ceph-users] cephfs Kernel panic
Hi,
I try to setup an ceph cluster on Debian 8.4. Mainly I followed a
tutorial at
http://adminforge.de/raid/ceph/ceph-cluster-unter-debian-wheezy-installieren/
As far as I can see, the first steps are just working fine. I have two
nodes with four OSD on both nodes.
This is the output of ceph -s
cluster 2a028d5e-5708-4fc4-9c0d-3495c1a3ef3d
health HEALTH_OK
monmap e2: 2 mons at
{ollie2=129.217.207.207:6789/0,stan2=129.217.207.206:6789/0}
election epoch 12, quorum 0,1 stan2,ollie2
mdsmap e10: 1/1/1 up {0=ollie2=up:active}, 1 up:standby
osdmap e72: 8 osds: 8 up, 8 in
flags sortbitwise
pgmap v137: 428 pgs, 4 pools, 2396 bytes data, 20 objects
281 MB used, 14856 GB / 14856 GB avail
428 active+clean
Then I tried to add cephfs following the manual at
http://docs.ceph.com/docs/hammer/cephfs/createfs/ which seem to do it's
magic:
root@stan2:~# ceph fs ls
name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ]
However, as soon as I try to mount the cephfs with mount.ceph
129.217.207.206:6789:/ /mnt/ -v -o
name=cephfs,secretfile=/etc/ceph/client.cephfs the server which tries to
mount crashes and has to be cold started again. To be able to use
mount.ceph I had to install ceph-fs-common - if that does matter...
Here is the kernel.log. Can you give me hints? I am pretty stuck on this
for the last few days.
Apr 11 16:25:02 stan2 kernel: [ 171.086381] Key type ceph registered
Apr 11 16:25:02 stan2 kernel: [ 171.086649] libceph: loaded (mon/osd
proto 15/24)
Apr 11 16:25:02 stan2 kernel: [ 171.090582] FS-Cache: Netfs 'ceph'
registered for caching
Apr 11 16:25:02 stan2 kernel: [ 171.090596] ceph: loaded (mds proto 32)
Apr 11 16:25:02 stan2 kernel: [ 171.096727] libceph: client34164 fsid
2a028d5e-5708-4fc4-9c0d-3495c1a3ef3d
Apr 11 16:25:02 stan2 kernel: [ 171.133832] libceph: mon0
129.217.207.206:6789 session established
Apr 11 16:25:02 stan2 kernel: [ 171.161199] [ cut here
]
Apr 11 16:25:02 stan2 kernel: [ 171.161239] kernel BUG at
/build/linux-lqALYs/linux-3.16.7-ckt25/fs/ceph/mds_client.c:1846!
Apr 11 16:25:02 stan2 kernel: [ 171.161294] invalid opcode: [#1] SMP
Apr 11 16:25:02 stan2 kernel: [ 171.161328] Modules linked in: cbc ceph
libceph xfs libcrc32c crc32c_generic binfmt_misc mptctl mptbase nfsd
auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc nls_utf8
nls_cp437 vfat fat x86_pkg_temp_thermal intel_powerclamp intel_rapl
coretemp kvm_intel kvm crc32_pclmul cryptd iTCO_wdt iTCO_vendor_support
efi_pstore efivars pcspkr joydev evdev ast i2c_i801 ttm drm_kms_helper
drm lpc_ich mfd_core mei_me mei shpchp ioatdma tpm_tis wmi tpm ipmi_si
ipmi_msghandler processor thermal_sys acpi_power_meter button acpi_pad
fuse autofs4 ext4 crc16 mbcache jbd2 dm_mod raid1 md_mod hid_generic sg
usbhid hid sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel ahci libahci ehci_pci mpt3sas igb
raid_class i2c_algo_bit xhci_hcd libata ehci_hcd scsi_transport_sas
i2c_core dca usbcore ptp usb_common scsi_mod pps_core
Apr 11 16:25:02 stan2 kernel: [ 171.162046] CPU: 0 PID: 3513 Comm:
kworker/0:9 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.7-ckt25-2
Apr 11 16:25:02 stan2 kernel: [ 171.162104] Hardware name: Supermicro
SYS-6028R-WTR/X10DRW-i, BIOS 1.0c 01/07/2015
Apr 11 16:25:02 stan2 kernel: [ 171.162158] Workqueue: ceph-msgr
con_work [libceph]
Apr 11 16:25:02 stan2 kernel: [ 171.162194] task: 88103f2e8ae0 ti:
88103bfbc000 task.ti: 88103bfbc000
Apr 11 16:25:02 stan2 kernel: [ 171.162243] RIP:
0010:[] []
__prepare_send_request+0x801/0x810 [ceph]
Apr 11 16:25:02 stan2 kernel: [ 171.162312] RSP: 0018:88103bfbfba8
EFLAGS: 00010283
Apr 11 16:25:02 stan2 kernel: [ 171.162347] RAX: 88103f88ad42 RBX:
88103f7f7400 RCX:
Apr 11 16:25:02 stan2 kernel: [ 171.162394] RDX: 164c5ec6 RSI:
RDI: 88103f88ad32
Apr 11 16:25:02 stan2 kernel: [ 171.162440] RBP: 88103f7f95e0 R08:
R09:
Apr 11 16:25:02 stan2 kernel: [ 171.162485] R10: R11:
002c R12: 88103f7f7c00
Apr 11 16:25:02 stan2 kernel: [ 171.162531] R13: 88103f88acc0 R14:
R15: 88103f88ad3a
Apr 11 16:25:02 stan2 kernel: [ 171.162578] FS: ()
GS:88107fc0() knlGS:
Apr 11 16:25:02 stan2 kernel: [ 171.162629] CS: 0010 DS: ES:
CR0: 80050033
Apr 11 16:25:02 stan2 kernel: [ 171.162668] CR2: 7fa73ca0a000 CR3:
01a13000 CR4: 001407f0
Apr 11 16:25:02 stan2 kernel: [ 171.162713] Stack:
Apr 11 16:25:02 stan2 kernel: [ 171.162730] 88103bfbfbd4
88103ef39540 0001
Apr 11 16:25:02 stan2 kernel: [ 171.162787]
88103ef39540
Apr 11 16:25:02 stan2 kernel: [ 171.162845] 0001
fff
