[ceph-users] installing multi osd and monitor of ceph in single VM
Hi Ceph users, I am new in ceph. I've been succeed installing ceph in 4 VM using Quick installation guide in ceph documentation. And I've also done to compile ceph from source code, build and install in single vm. What I want to do next is that run ceph multiple nodes in a cluster but only inside a single machine. I need this because I will learn the ceph code and will modify some codes, recompile and redeploy on the node/VM. On my study, I've also to be able to run/kill particular node. does somebody know how to configure single vm to run multiple osd and monitor of ceph? Advises and comments are very appreciate. thanks -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] installing multi osd and monitor of ceph in single VM
I've seen the Ansible before but not in detail for that. I also have tried to run quick guide for development. It did not work on my VM that I already install ceph inside it. the error is : agung@arrasyid:~/ceph/ceph/src$ ./vstart.sh -d -n -x ** going verbose ** [./fetch_config /tmp/fetched.ceph.conf.3818] ./init-ceph: failed to fetch config with './fetch_config /tmp/fetched.ceph.conf.3818' Do I need to use a vanilla ceph to make vstart.sh work? When I learn a cloud system, usually I compile the source code, run in pseudo-distributed, modify the code and add prints somewhere, recompile and re-run the system. Might this method work for exploring ceph? On Wed, Aug 10, 2016 at 9:14 AM, Brad Hubbard wrote: > On Wed, Aug 10, 2016 at 12:26 AM, agung Laksono > wrote: > > > > Hi Ceph users, > > > > I am new in ceph. I've been succeed installing ceph in 4 VM using Quick > > installation guide in ceph documentation. > > > > And I've also done to compile > > ceph from source code, build and install in single vm. > > > > What I want to do next is that run ceph multiple nodes in a cluster > > but only inside a single machine. I need this because I will > > learn the ceph code and will modify some codes, recompile and > > redeploy on the node/VM. On my study, I've also to be able to run/kill > > particular node. > > > > does somebody know how to configure single vm to run multiple osd and > > monitor of ceph? > > > > Advises and comments are very appreciate. thanks > > Hi, > > Did you see this? > > http://docs.ceph.com/docs/hammer/dev/quick_guide/#running-a-development- > deployment > > Also take a look at the AIO (all in one) options in ceph-ansible. > > HTH, > Brad > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] installing multi osd and monitor of ceph in single VM
Thank you Brad, I am able to run ceph with 3 MONs, 3 OSDs and 1 MDS now. However, I still not get the workflow of the ceph using this step. I might need print in somewhere, inject a crash by kill one node, etc. Does this also possible using this method? On Thu, Aug 11, 2016 at 4:17 AM, Brad Hubbard wrote: > On Thu, Aug 11, 2016 at 12:45 AM, agung Laksono > wrote: > > I've seen the Ansible before but not in detail for that. > > I also have tried to run quick guide for development. > > It did not work on my VM that I already install ceph inside it. > > > > the error is : > > > > agung@arrasyid:~/ceph/ceph/src$ ./vstart.sh -d -n -x > > ** going verbose ** > > [./fetch_config /tmp/fetched.ceph.conf.3818] > > ./init-ceph: failed to fetch config with './fetch_config > > /tmp/fetched.ceph.conf.3818' > > > > > > Do I need to use a vanilla ceph to make vstart.sh work? > > > > When I learn a cloud system, usually I compile > > the source code, run in pseudo-distributed, modify the code > > and add prints somewhere, recompile and re-run the system. > > Might this method work for exploring ceph? > > It should, sure. > > Try this. > > 1) Clone a fresh copy of the repo. > 2) ./do_cmake.sh > 3) cd build > 4) make > 5) OSD=3 MON=3 MDS=1 ../src/vstart.sh -n -x -l > 6) bin/ceph -s > > That should give you a working cluster with 3 MONs, 3 OSDs and 1 MDS. > > -- > Cheers, > Brad > > > > > > > On Wed, Aug 10, 2016 at 9:14 AM, Brad Hubbard > wrote: > >> > >> On Wed, Aug 10, 2016 at 12:26 AM, agung Laksono > > >> wrote: > >> > > >> > Hi Ceph users, > >> > > >> > I am new in ceph. I've been succeed installing ceph in 4 VM using > Quick > >> > installation guide in ceph documentation. > >> > > >> > And I've also done to compile > >> > ceph from source code, build and install in single vm. > >> > > >> > What I want to do next is that run ceph multiple nodes in a cluster > >> > but only inside a single machine. I need this because I will > >> > learn the ceph code and will modify some codes, recompile and > >> > redeploy on the node/VM. On my study, I've also to be able to run/kill > >> > particular node. > >> > > >> > does somebody know how to configure single vm to run multiple osd and > >> > monitor of ceph? > >> > > >> > Advises and comments are very appreciate. thanks > >> > >> Hi, > >> > >> Did you see this? > >> > >> > >> http://docs.ceph.com/docs/hammer/dev/quick_guide/# > running-a-development-deployment > >> > >> Also take a look at the AIO (all in one) options in ceph-ansible. > >> > >> HTH, > >> Brad > > > > > > > > > > -- > > Cheers, > > > > Agung Laksono > > > > > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] build and Compile ceph in development mode takes an hour
Hi Ceph User, When I make change inside ceph codes in the development mode, I found that recompiling takes around an hour because I have to remove a build folder and all the contest and then reproduce it. Is there a way to make the compiling process be faster? something like only compile a particular code that I change. Thanks before -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] build and Compile ceph in development mode takes an hour
HI Brad, After exploring Ceph, I found sometimes when I run *make* command without removing the build folder, it will end with this error: /home/agung/project/samc/ceph/build/bin/init-ceph: ceph conf ./ceph.conf not found; system is not configured. rm -f core* ip 127.0.0.1 port 6789 NOTE: hostname resolves to loopback; remote hosts will not be able to connect. either adjust /etc/hosts, or edit this script to use your machine's real IP. /home/agung/project/samc/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mon. /home/agung/project/samc/ceph/build/keyring --cap mon allow * creating /home/agung/project/samc/ceph/build/keyring /home/agung/project/samc/ceph/build/bin/ceph-authtool --gen-key --name=client.admin --set-uid=0 --cap mon allow * --cap osd allow * --cap mds allow * /home/agung/project/samc/ceph/build/keyring /home/agung/project/samc/ceph/build/bin/monmaptool --create --clobber --add a 127.0.0.1:6789 --add b 127.0.0.1:6790 --add c 127.0.0.1:6791 --print /tmp/ceph_monmap.8374 /home/agung/project/samc/ceph/build/bin/monmaptool: monmap file /tmp/ceph_monmap.8374 /home/agung/project/samc/ceph/build/bin/monmaptool: generated fsid 9a08986c-b051-48e1-9002-f49af0cb9efd epoch 0 fsid 9a08986c-b051-48e1-9002-f49af0cb9efd last_changed 2016-08-31 20:54:42.473678 created 2016-08-31 20:54:42.473678 0: 127.0.0.1:6789/0 mon.a 1: 127.0.0.1:6790/0 mon.b 2: 127.0.0.1:6791/0 mon.c /home/agung/project/samc/ceph/build/bin/monmaptool: writing epoch 0 to /tmp/ceph_monmap.8374 (3 monitors) rm -rf -- /home/agung/project/samc/ceph/build/dev/mon.a mkdir -p /home/agung/project/samc/ceph/build/dev/mon.a /home/agung/project/samc/ceph/build/bin/ceph-mon --mkfs -c /home/agung/project/samc/ceph/build/ceph.conf -i a --monmap=/tmp/ceph_monmap.8374 --keyring=/home/agung/project/samc/ceph/build/keyring Segmentation fault (core dumped) I have tried to remove the ceph.conf and clean up the temp/ folder before run OSD=3 MON=3 MDS=1 ../src/vstart.sh -n -x -l. But still did not work. probably you have any suggestion? On Thu, Aug 18, 2016 at 5:42 AM, Brad Hubbard wrote: > On Thu, Aug 18, 2016 at 1:12 AM, agung Laksono > wrote: > > Hi Ceph User, > > > > When I make change inside ceph codes in the development mode, > > I found that recompiling takes around an hour because I have to remove > > a build folder and all the contest and then reproduce it. > > > > Is there a way to make the compiling process be faster? something like > only > > compile a particular code that I change. > > Sure, just use the same build directory and run "make" again after you > make code > changes and it should only re-compile the binaries that are effected > by your code > changes. > > You can use "make -jX" if you aren't already where 'X' is usually > number of CPUs / 2 > which may speed up the build. > > HTH, > Brad > > > > > Thanks before > > > > > > -- > > Cheers, > > > > Agung Laksono > > > > > > ___ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] how run multiple node in single machine in previous version of ceph
The latest version of ceph, it's easy to develop a cluster with multiple nodes in an only single machine by using vstart.sh. How if I want to develop a cluster with multiple nodes on a single machine for an old version of ceph like infernalis or an older one? Any answer is very appreciated. thanks -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] how to trigger ms_Handle_reset on monitor
Hi Ceph user, I am learning bugs in ceph and am trying to reproduce http://tracker.ceph.com/issues/15113 on my local machine. On the description, I may see the scenario: 1. ___*_*__ SafeTimer::timer_thread(), with mon_lock: 2. ___*_*__ elector: in win_election(), it resend_routed_requests(), and collects the routed requests 3. msgr: in ms_handle_reset(), it reset the session 4. msgr: it waits for the lock 5. ___*_*__ elector: in win_election(), it handle_command(), but the session is reset, hence it panics. 6. msgr: remove session, and erase related requests from Monitor::routed_requests. I could understand the whole flow of the bug except ms_handle_reset(). What's a thing that triggers this ms_handle_reset()? I could not see this function is called in any part of the monitor class. I've tried to reproduce this on a cluster with 3 MONs and 4 OSDs and put the printf whenever this function's called. However, It seems arbitrary. Perhaps anyone can help me to explain this? thanks -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Fwd: if ceph tracker says the source was QA, what version that I have to checkout?
Hi All ceph users, I am interesting to learn ceph bug. Some bugs that I learned pointing particular ceph version or tags for their buggy part. However I found some bugs put Q/A as the source. For instance http://tracker.ceph.com/issues/9301 when I checked out on giant and firefly version, I saw the significant different between the version and the revision that changed by the developer. My question is how to have a buggy version( version when the patch has not been implemented)? Thanks -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Fwd: if ceph tracker says the source was QA, what version that I have to checkout?
thank you for the answer Brad, so the thing that I have to do is that: - see the commit version - checkout to the commit version - see the log ( git log -p -1) - ultimately, checkout the older one. Am I correct? On Mon, Oct 10, 2016 at 6:28 AM, Brad Hubbard wrote: > On Mon, Oct 10, 2016 at 9:21 AM, agung Laksono > wrote: > > > > Hi All ceph users, > > > > I am interesting to learn ceph bug. > > > > Some bugs that I learned pointing particular ceph version or tags for > their > > buggy part. > > However I found some bugs put Q/A as the source. > > For instance http://tracker.ceph.com/issues/9301 > > > > when I checked out on giant and firefly version, I saw the significant > > different between > > the version and the revision that changed by the developer. > > > > My question is how to have a buggy version( version when the patch has > not > > been implemented)? > > For this particular commit we can ask git. > > http://tracker.ceph.com/projects/ceph/repository/revisions/ > c421b55e8e15ef04ca8aeb47f7d090375eaa8573 > > $ git describe c421b55e8e15ef04ca8aeb47f7d090375eaa8573 > v0.85-884-gc421b55 > > So this was the 884th commit after the v0.85 tag and you need to check > out a copy of the source before that. > > > > > Thanks > > > > -- > > Cheers, > > > > Agung Laksono > > > > _______ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > -- > Cheers, > Brad > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Fwd: if ceph tracker says the source was QA, what version that I have to checkout?
Yes you're right. Sometimes it tested not in a single step back then. but it also possible to be tested some version steps in the previous commits. thanks for helping! On Mon, Oct 10, 2016 at 7:18 AM, Brad Hubbard wrote: > > > On Mon, Oct 10, 2016 at 9:51 AM, agung Laksono > wrote: > > thank you for the answer Brad, > > > > so the thing that I have to do is that: > > - see the commit version > > - checkout to the commit version > > - see the log ( git log -p -1) > > - ultimately, checkout the older one. > > > > Am I correct? > > Yes, that's pretty much it. The following should be your starting point. > > $ git checkout c421b55e8e15ef04ca8aeb47f7d090375eaa8573~1 > > It's not guaranteed that every commit will build and test successfully so > you > may need to move further back from that point or even use the previous tag > (in > this case v0.85). > > > > > > > On Mon, Oct 10, 2016 at 6:28 AM, Brad Hubbard > wrote: > >> > >> On Mon, Oct 10, 2016 at 9:21 AM, agung Laksono > >> wrote: > >> > > >> > Hi All ceph users, > >> > > >> > I am interesting to learn ceph bug. > >> > > >> > Some bugs that I learned pointing particular ceph version or tags for > >> > their > >> > buggy part. > >> > However I found some bugs put Q/A as the source. > >> > For instance http://tracker.ceph.com/issues/9301 > >> > > >> > when I checked out on giant and firefly version, I saw the significant > >> > different between > >> > the version and the revision that changed by the developer. > >> > > >> > My question is how to have a buggy version( version when the patch has > >> > not > >> > been implemented)? > >> > >> For this particular commit we can ask git. > >> > >> > >> http://tracker.ceph.com/projects/ceph/repository/revisions/ > c421b55e8e15ef04ca8aeb47f7d090375eaa8573 > >> > >> $ git describe c421b55e8e15ef04ca8aeb47f7d090375eaa8573 > >> v0.85-884-gc421b55 > >> > >> So this was the 884th commit after the v0.85 tag and you need to check > >> out a copy of the source before that. > >> > >> > > >> > Thanks > >> > > >> > -- > >> > Cheers, > >> > > >> > Agung Laksono > >> > > >> > ___ > >> > ceph-users mailing list > >> > ceph-users@lists.ceph.com > >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> > > >> > >> > >> > >> -- > >> Cheers, > >> Brad > > > > > > > > > > -- > > Cheers, > > > > Agung Laksono > > > > > > -- > Cheers, > Brad > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] How do I restart node that I've killed in development mode
Hi Ceph Users, I deploy development cluster using vstart with 3 MONs and 3 OSDs. On my experiment, Kill one of the monitor nodes by its pid. like this: $ kill -SIGSEGV 27557 After a new monitor leader is chosen, I would like to re-run the monitor that I've killed in the previous step. How do I do this? Thanks -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] How do I restart node that I've killed in development mode
Thanks Huang jun. I will try it. On Wed, Oct 12, 2016 at 2:06 PM, huang jun wrote: > ./init-ceph start mon.a > > 2016-10-12 14:54 GMT+08:00 agung Laksono : > > Hi Ceph Users, > > > > I deploy development cluster using vstart with 3 MONs and 3 OSDs. > > On my experiment, Kill one of the monitor nodes by its pid. like this: > > > > $ kill -SIGSEGV 27557 > > > > After a new monitor leader is chosen, I would like to re-run the monitor > > that I've killed in the previous step. How do I do this? > > > > > > Thanks > > > > -- > > Cheers, > > > > Agung Laksono > > > > > > ___ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > -- > Thank you! > HuangJun > -- Cheers, Agung Laksono ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
