[Gluster-devel] Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t
Hi Aravinda/Kotresh, With [1], I consistently see cores generated with the test './tests/geo-rep/georep-basic-dr-tarssh.t' in release-3.7 branch. From the cores, looks like we are trying to dereference a freed changelog_rpc_clnt_t(crpc) object in changelog_rpc_notify(). Strangely this was not reported in master branch. I tried debugging but couldn't find any possible suspects. I request you to take a look and let me know if [1] caused any regression. Thanks, Soumya [1] http://review.gluster.org/#/c/13507/ ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] tests/basic/tier/tier.t failure in NetBSD
Thanks Nithya. BTW there is a http://review.gluster.org/#/c/13535 has a Centos failure. with ./tests/bugs/distribute/bug-860663.t failing. ~Joe - Original Message - > From: "Nithya Balachandran" > To: "Atin Mukherjee" > Cc: "Gluster Devel" > Sent: Monday, February 29, 2016 12:24:51 PM > Subject: Re: [Gluster-devel] tests/basic/tier/tier.t failure in NetBSD > > Sorry - I pressed send before attaching the log messages. > > - > > > From: "Atin Mukherjee" > > > To: "Gluster Devel" > > > Sent: Friday, 26 February, 2016 11:24:31 AM > > > Subject: [Gluster-devel] tests/basic/tier/tier.t failure in NetBSD > > > > > > The above test failed here [1] > > > > > > [1] > > > https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/14590/consoleFull > > > > > > > This looks like glusterd crashed. From the glusterd log: > > > > > The test that failed is > > TEST $CLI volume tier $V0 detach start > > From the glusterd log: > > > - Original Message -[2016-02-26 04:45:04.6N]:++ > > G_LOG:./tests/basic/tier/tier.t: TEST: 198 0 detach_start patchy > > ++ > [2016-02-26 04:45:05.650278] I [MSGID: 106568] > [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is > stopped > [2016-02-26 04:45:05.652782] I [MSGID: 106540] > [glusterd-utils.c:4304:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered MOUNTV3 successfully > [2016-02-26 04:45:05.653252] I [MSGID: 106540] > [glusterd-utils.c:4313:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered MOUNTV1 successfully > [2016-02-26 04:45:05.653709] I [MSGID: 106540] > [glusterd-utils.c:4322:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered NFSV3 successfully > [2016-02-26 04:45:05.654164] I [MSGID: 106540] > [glusterd-utils.c:4331:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered NLM v4 successfully > [2016-02-26 04:45:05.654619] I [MSGID: 106540] > [glusterd-utils.c:4340:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered NLM v1 successfully > [2016-02-26 04:45:05.655081] I [MSGID: 106540] > [glusterd-utils.c:4349:glusterd_nfs_pmap_deregister] 0-glusterd: > De-registered ACL v3 successfully > [2016-02-26 04:45:05.716672] I [MSGID: 106567] > [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting nfs > service > [2016-02-26 04:45:05.729253] W [socket.c:3006:socket_connect] 0-nfs: Ignore > failed connection attempt on , (No such file or directory) > [2016-02-26 04:45:05.729459] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-glustershd: setting frame-timeout to 600 > [2016-02-26 04:45:05.761918] I [MSGID: 106568] > [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping > glustershd daemon running in pid: 20834 > [2016-02-26 04:45:06.770262] I [MSGID: 106568] > [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service > is stopped > [2016-02-26 04:45:06.770400] I [MSGID: 106567] > [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting > glustershd service > [2016-02-26 04:45:06.782561] W [socket.c:3006:socket_connect] 0-glustershd: > Ignore failed connection attempt on , (No such file or directory) > [2016-02-26 04:45:06.782784] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-quotad: setting frame-timeout to 600 > [2016-02-26 04:45:06.783060] I [MSGID: 106132] > [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already > stopped > [2016-02-26 04:45:06.783384] I [MSGID: 106568] > [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is > stopped > [2016-02-26 04:45:06.783485] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-bitd: setting frame-timeout to 600 > [2016-02-26 04:45:06.783649] I [MSGID: 106132] > [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already > stopped > [2016-02-26 04:45:06.783955] I [MSGID: 106568] > [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is > stopped > [2016-02-26 04:45:06.784039] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-scrub: setting frame-timeout to 600 > [2016-02-26 04:45:06.784164] I [MSGID: 106132] > [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already > stopped > [2016-02-26 04:45:06.784468] I [MSGID: 106568] > [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is > stopped > [2016-02-26 04:45:06.784644] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2016-02-26 04:45:06.789894] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2016-02-26 04:45:06.790102] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2016-02-26 04:45:06.791583] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2016-02-26 04:45:06.791866] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2016-02-26 04:45:06.792148] I [rpc-clnt.c:971:rpc_clnt_connection_init] > 0-management: set
Re: [Gluster-devel] Change in glusterfs[master]: tests: fix bug-860663.t
Hi Jeff, I would suggest add the author of this script to the review panel. Regards, Joe - Original Message - > From: "Jeff Darcy (Code Review)" > To: "Joseph Fernandes" > Sent: Monday, February 29, 2016 3:42:45 PM > Subject: Change in glusterfs[master]: tests: fix bug-860663.t > > Hello Joseph Fernandes, > > I'd like you to do a code review. Please visit > > http://review.gluster.org/13544 > > to review the following change. > > Change subject: tests: fix bug-860663.t > .. > > tests: fix bug-860663.t > > Three changes: > > * Removed the second round of file creation, which wasn't really >testing anything useful and was causing spurious failures. Under the >conditions we've set up, the rational expectation would be for the >file-creation helper program to succeed, but the test expected it to >fail. > > * Removed Yet Another Unnecessary Sleep. > > * Reduced the number of files from 10K to 1K. That's more than >sufficient to test what we're trying to test, and saves significant >time. > > There's still a bit of a mystery about how this test *ever* passed. If > I want mystery I'll read a detective novel. The more important thing is > that the line in question was irrelevant to the purpose of the test > (which had to do with not allowing a fix-layout while bricks were down) > and was far too heavyweight to be included as a "while we're here might > as well" kind of addition. > > Change-Id: If1c623853745ab42ce7d058d1009bbe1dcc1e985 > Signed-off-by: Jeff Darcy > --- > M tests/bugs/distribute/bug-860663.t > 1 file changed, 5 insertions(+), 6 deletions(-) > > > git pull ssh://git.gluster.org/glusterfs refs/changes/44/13544/1 > -- > To view, visit http://review.gluster.org/13544 > To unsubscribe, visit http://review.gluster.org/settings > > Gerrit-MessageType: newchange > Gerrit-Change-Id: If1c623853745ab42ce7d058d1009bbe1dcc1e985 > Gerrit-PatchSet: 1 > Gerrit-Project: glusterfs > Gerrit-Branch: master > Gerrit-Owner: Jeff Darcy > Gerrit-Reviewer: Joseph Fernandes > Gerrit-Reviewer: N Balachandran > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Change in glusterfs[master]: tests: fix bug-860663.t
Oops the authors are not actively contributing to gluster now. Sorry about that. ~Joe - Original Message - > From: "Joseph Fernandes" > To: jda...@redhat.com > Cc: "Gluster Devel" > Sent: Monday, February 29, 2016 3:44:57 PM > Subject: Re: [Gluster-devel] Change in glusterfs[master]: tests: fix > bug-860663.t > > Hi Jeff, > > I would suggest add the author of this script to the review panel. > > Regards, > Joe > > - Original Message - > > From: "Jeff Darcy (Code Review)" > > To: "Joseph Fernandes" > > Sent: Monday, February 29, 2016 3:42:45 PM > > Subject: Change in glusterfs[master]: tests: fix bug-860663.t > > > > Hello Joseph Fernandes, > > > > I'd like you to do a code review. Please visit > > > > http://review.gluster.org/13544 > > > > to review the following change. > > > > Change subject: tests: fix bug-860663.t > > .. > > > > tests: fix bug-860663.t > > > > Three changes: > > > > * Removed the second round of file creation, which wasn't really > >testing anything useful and was causing spurious failures. Under the > >conditions we've set up, the rational expectation would be for the > >file-creation helper program to succeed, but the test expected it to > >fail. > > > > * Removed Yet Another Unnecessary Sleep. > > > > * Reduced the number of files from 10K to 1K. That's more than > >sufficient to test what we're trying to test, and saves significant > >time. > > > > There's still a bit of a mystery about how this test *ever* passed. If > > I want mystery I'll read a detective novel. The more important thing is > > that the line in question was irrelevant to the purpose of the test > > (which had to do with not allowing a fix-layout while bricks were down) > > and was far too heavyweight to be included as a "while we're here might > > as well" kind of addition. > > > > Change-Id: If1c623853745ab42ce7d058d1009bbe1dcc1e985 > > Signed-off-by: Jeff Darcy > > --- > > M tests/bugs/distribute/bug-860663.t > > 1 file changed, 5 insertions(+), 6 deletions(-) > > > > > > git pull ssh://git.gluster.org/glusterfs refs/changes/44/13544/1 > > -- > > To view, visit http://review.gluster.org/13544 > > To unsubscribe, visit http://review.gluster.org/settings > > > > Gerrit-MessageType: newchange > > Gerrit-Change-Id: If1c623853745ab42ce7d058d1009bbe1dcc1e985 > > Gerrit-PatchSet: 1 > > Gerrit-Project: glusterfs > > Gerrit-Branch: master > > Gerrit-Owner: Jeff Darcy > > Gerrit-Reviewer: Joseph Fernandes > > Gerrit-Reviewer: N Balachandran > > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t
Hi Soumya, I just tested that it is reproducible only with your patch both in master and 3.76 branch. The geo-rep test cases are marked bad in master. So it's not hit in master. rpc is introduced in changelog xlator to communicate to applications via libgfchangelog. Venky/Me will check why is the crash happening and will update. Thanks and Regards, Kotresh H R - Original Message - > From: "Soumya Koduri" > To: avish...@redhat.com, "kotresh" > Cc: "Gluster Devel" > Sent: Monday, February 29, 2016 2:10:51 PM > Subject: Cores generated with ./tests/geo-rep/georep-basic-dr-tarssh.t > > Hi Aravinda/Kotresh, > > With [1], I consistently see cores generated with the test > './tests/geo-rep/georep-basic-dr-tarssh.t' in release-3.7 branch. From > the cores, looks like we are trying to dereference a freed > changelog_rpc_clnt_t(crpc) object in changelog_rpc_notify(). Strangely > this was not reported in master branch. > > I tried debugging but couldn't find any possible suspects. I request you > to take a look and let me know if [1] caused any regression. > > Thanks, > Soumya > > [1] http://review.gluster.org/#/c/13507/ > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Gap in protocol client-server handshake
Hi, Currently on a successful connection between protocol server and client, the protocol client initiates a CHILD_UP event in the client stack. At this point in time, only the connection between server and client is established, and there is no guarantee that the server side stack is ready to serve requests. It works fine now, as most server side translators are not dependent on any other factors, before being able to serve requests today and hence they are up by the time the client stack translators receive the CHILD_UP (initiated by client handshake). The gap here is exposed when certain server side translators like NSR-Server for example, have a couple of protocol clients as their child(connecting them to other bricks), and they can't really serve requests till a quorum of their children are up. Hence these translators *should* defer sending CHILD_UP till they have enough children up, and the same CHILD_UP event needs to be propagated to the client stack translators. I have sent a patch(http://review.gluster.org/#/c/13549/) addressing this, where we maintain a child_up variable in both the protocol client and protocol server translators. The protocol server updates this value based on the CHILD_UP and CHILD_DOWN events it receives from the translators below it. On receiving such an event it forwards that event to the client. The protocol client on receiving such an event forwards it up the client stack, thereby letting the client translators correctly know that the server is up and ready to serve. The clients connecting later(long after a server has initialized and processed it's CHILD_UP events), receive a child_up status as part of the handshake, and based on the status of the server's child_up, either propagate a CHILD_UP event or defer it. Please have a look at the patch, and kindly state if it you have any concerns or you foresee any scenarios of interest which we might have missed. Regards, Avra ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Gap in protocol client-server handshake
- Original Message - > From: "Avra Sengupta" > To: "Gluster Devel" > Sent: Monday, February 29, 2016 5:20:53 PM > Subject: [Gluster-devel] Gap in protocol client-server handshake > > Hi, > > Currently on a successful connection between protocol server and client, > the protocol client initiates a CHILD_UP event in the client stack. At > this point in time, only the connection between server and client is > established, and there is no guarantee that the server side stack is > ready to serve requests. It works fine now, as most server side > translators are not dependent on any other factors, before being able to > serve requests today and hence they are up by the time the client stack > translators receive the CHILD_UP (initiated by client handshake). > > The gap here is exposed when certain server side translators like > NSR-Server for example, have a couple of protocol clients as their > child(connecting them to other bricks), and they can't really serve > requests till a quorum of their children are up. Hence these translators > *should* defer sending CHILD_UP till they have enough children up, and > the same CHILD_UP event needs to be propagated to the client stack > translators. Yes. We have seen this problem (mostly in the form of crashes of brick process). > > I have sent a patch(http://review.gluster.org/#/c/13549/) addressing > this, where we maintain a child_up variable in both the protocol client > and protocol server translators. The protocol server updates this value > based on the CHILD_UP and CHILD_DOWN events it receives from the > translators below it. On receiving such an event it forwards that event > to the client. The protocol client on receiving such an event forwards > it up the client stack, thereby letting the client translators correctly > know that the server is up and ready to serve. > > The clients connecting later(long after a server has initialized and > processed it's CHILD_UP events), receive a child_up status as part of > the handshake, and based on the status of the server's child_up, either > propagate a CHILD_UP event or defer it. > > Please have a look at the patch, and kindly state if it you have any > concerns or you foresee any scenarios of interest which we might have > missed. Thanks for the patch. I'll review it. > > Regards, > Avra > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Two questions about the performance of ls in glusterfs
hi, I have two questions about the performance of ls in glusterfs,and I know that the community has noticed the poor performance of ls and made something to solve the problem,like readdir-ahead.Here are my questions. 1,The preload (a readdir request not initiated by application, but instead triggered by readdir-ahead in an attempt to pre-emptively fill the read-ahead buffer) is in process,a readdir request from application waits for its completion. In the code, when the preload is in progress,it locks the ctx. For applications, the readdir-ahead's request processing is synchronous, is it possible that the preload handles request asynchronously by reducing lock granularity? In other words, the client fetches the dentries from the preload,while the preload gets the dentries from the servers. 2, when does it can be a lager buffer,like io-cahce,which cached the data or dentries read before? As you know, ls is so slow.___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] [master] FAILED tests/bugs/distribute/bug-860663.t
== Running tests in file ./tests/bugs/distribute/bug-860663.t [11:02:28] ./tests/bugs/distribute/bug-860663.t .. not ok 13 Failed 1/14 subtests [11:02:28] Test Summary Report --- ./tests/bugs/distribute/bug-860663.t (Wstat: 0 Tests: 14 Failed: 1) Failed test: 13 Files=1, Tests=14, 71 wallclock secs ( 0.03 usr 0.00 sys + 2.14 cusr 2.22 csys = 4.39 CPU) Result: FAIL End of test ./tests/bugs/distribute/bug-860663.t == Run complete 1 test(s) failed ./tests/bugs/distribute/bug-860663.t 0 test(s) generated core Please advise. -- Milind ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] [master] FAILED tests/bugs/distribute/bug-860663.t
Jeff has posted a patch [1] for this [1] http://review.gluster.org/13544 ~Atin On 02/29/2016 06:10 PM, Milind Changire wrote: > > == > Running tests in file ./tests/bugs/distribute/bug-860663.t > [11:02:28] ./tests/bugs/distribute/bug-860663.t .. > not ok 13 > Failed 1/14 subtests > [11:02:28] > > Test Summary Report > --- > ./tests/bugs/distribute/bug-860663.t (Wstat: 0 Tests: 14 Failed: 1) > Failed test: 13 > Files=1, Tests=14, 71 wallclock secs ( 0.03 usr 0.00 sys + 2.14 cusr 2.22 > csys = 4.39 CPU) > Result: FAIL > End of test ./tests/bugs/distribute/bug-860663.t > == > > > Run complete > 1 test(s) failed > ./tests/bugs/distribute/bug-860663.t > 0 test(s) generated core > > > > Please advise. > -- > Milind > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Introducing storhaug (formerly Storage-HA)
Hey folks, I'd like to introduce you all to storhaug, a Pacemaker-based HA solution for clustered storage platforms: https://github.com/linux-ha-storage/storhaug While still fairly new, it should be ready for people to download and play with if they so desire. A few important notes: - Currently only targeting CentOS 6 and CentOS 7 - As this has yet to receive widespread testing, only recommended for VMs - There is a fair amount of pre-configuration that needs to take place. - Currently the RPM installation does most of that for you - ...but that means you should take care about where you install it ;) Naturally, collaboration is fully welcomed. I expect things to fall over or catch fire in spectacular ways... and if they don't that can only mean people aren't really testing it yet. I especially want to hear about the installation and ease-of-use (or lack thereof) of the software, as those are the hardest part of the forest to see when I've been staring at the trees for so long. :) Expect more regular, front-facing activity in the coming days. Cheers! --Jose ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Introducing storhaug (formerly Storage-HA)
Hey folks, I'd like to introduce you all to storhaug, a Pacemaker-based HA solution for clustered storage platforms: https://github.com/linux-ha-storage/storhaug While still fairly new, it should be ready for people to download and play with if they so desire. A few important notes: - Currently only targeting CentOS 6 and CentOS 7 - As this has yet to receive widespread testing, only recommended for VMs - There is a fair amount of pre-configuration that needs to take place. - Currently the RPM installation does most of that for you - ...but that means you should take care about where you install it ;) Naturally, collaboration is fully welcomed. I expect things to fall over or catch fire in spectacular ways... and if they don't that can only mean people aren't really testing it yet. I especially want to hear about the installation and ease-of-use (or lack thereof) of the software, as those are the hardest part of the forest to see when I've been staring at the trees for so long. :) Expect more regular, front-facing activity in the coming days. Cheers! --Jose ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Bugs with incorrect status
1008839 (mainline) POST: Certain blocked entry lock info not retained after the lock is granted [master] Ie37837 features/locks : Certain blocked entry lock info not retained after the lock is granted (ABANDONED) ** ata...@redhat.com: Bug 1008839 is in POST, but all changes have been abandoned ** 1062437 (mainline) POST: stripe does not work with empty xlator [master] I778699 stripe: fix FIRST_CHILD checks to be more general (ABANDONED) ** jda...@redhat.com: Bug 1062437 is in POST, but all changes have been abandoned ** 1074947 (mainline) ON_QA: add option to bulld rpm without server [master] Iaa1498 build: add option to bulld rpm without server (NEW) ** b...@gluster.org: Bug 1074947 should be in POST, change Iaa1498 under review ** 1089642 (mainline) POST: Quotad doesn't load io-stats xlator, which implies none of the logging options have any effect on it. [master] Iccc033 glusterd: add io-stats to all quotad's sub-graphs (ABANDONED) ** spa...@redhat.com: Bug 1089642 is in POST, but all changes have been abandoned ** 1092414 (mainline) ASSIGNED: Disable NFS by default [master] If25502 [WIP] glusterd: make Gluster/NFS an optional component (NEW) ** nde...@redhat.com: Bug 1092414 should be in POST, change If25502 under review ** 1093768 (3.5.0) POST: Comment typo in gf-history.changelog.c ** kschi...@redhat.com: No change posted, but bug 1093768 is in POST ** 1094478 (3.5.0) POST: Bad macro in changelog-misc.h ** kschi...@redhat.com: No change posted, but bug 1094478 is in POST ** 1099294 (3.5.0) POST: Incorrect error message in /features/changelog/lib/src/gf-history-changelog.c ** kschi...@redhat.com: No change posted, but bug 1099294 is in POST ** 1099460 (3.5.0) NEW: file locks are not released within an acceptable time when a fuse-client uncleanly disconnects [release-3.5] I5e5f54 socket: use TCP_USER_TIMEOUT to detect client failures quicker (NEW) ** nde...@redhat.com: Bug 1099460 should be in POST, change I5e5f54 under review ** 1099683 (3.5.0) POST: Silent error from call to realpath in features/changelog/lib/src/gf-history-changelog.c ** vshan...@redhat.com: No change posted, but bug 1099683 is in POST ** 110 (mainline) ASSIGNED: [RFE] Add regression tests for the component geo-replication [master] Ie27848 tests/geo-rep: Automated configuration for geo-rep regression. (NEW) [master] I9c9ae8 geo-rep: Regression tests improvements (ABANDONED) [master] I433dd8 Geo-rep: Adding regression tests for geo-rep (MERGED) [master] Ife8201 Geo-rep: Adding regression tests for geo-rep (ABANDONED) ** khire...@redhat.com: Bug 110 should be in POST, change Ie27848 under review ** 020 (3.5.0) POST: Unused code changelog_entry_length ** kschi...@redhat.com: No change posted, but bug 020 is in POST ** 031 (3.5.0) POST: CHANGELOG_FILL_HTIME_DIR macro fills buffer without size limits ** kschi...@redhat.com: No change posted, but bug 031 is in POST ** 1114415 (mainline) MODIFIED: There is no way to monitor if the healing is successful when the brick is erased ** pkara...@redhat.com: No change posted, but bug 1114415 is in MODIFIED ** 1116714 (3.5.0) POST: indices/xattrop directory contains stale entries [release-3.5] I470cf8 afr : Added xdata flags to indicate probable existence of stale index. (ABANDONED) ** ata...@redhat.com: Bug 1116714 is in POST, but all changes have been abandoned ** 1117886 (mainline) MODIFIED: Gluster not resolving hosts with IPv6 only lookups [master] Icbaa3c Reinstate ipv6 support (NEW) [master] Iebc96e glusterd: Bug fixes for IPv6 support (MERGED) ** nithind1...@yahoo.in: Bug 1117886 should be in POST, change Icbaa3c under review ** 1122120 (3.5.1) MODIFIED: Bricks crashing after disable and re-enabled quota on a volume ** b...@gluster.org: No change posted, but bug 1122120 is in MODIFIED ** 1131846 (mainline) POST: remove-brick - once you stop remove-brick using stop command, status says ' failed: remove-brick not started.' ** gg...@redhat.com: No change posted, but bug 1131846 is in POST ** 1132074 (mainline) POST: Document steps to perform for replace-brick [master] Ic7292b doc: Steps for Replacing brick in gluster volume (ABANDONED) ** pkara...@redhat.com: Bug 1132074 is in POST, but all changes have been abandoned ** 1134305 (mainline) POST: rpc actor failed to complete successfully messages in Glusterd [master] I094516 protocol/client: Add explicit states for connection sequence (ABANDONED) ** pkara...@redhat.com: Bug 1134305 is in POST, but all changes have been abandoned ** 1142423 (mainline) MODIFIED: [DHT-REBALANCE]-DataLoss: The data appended to a file during its migration will be lost once the migration is done [master] I044a83 cluster/dht Use additional dst_info in inode_ctx (NEW) [master] I5c8810 cluster/dht: Fix stale subvol cache for files under migration (NEW) [master] I05a2e9 cluster/dht: fix incorrect dst subvol info in inode_c
[Gluster-devel] Understanding ePoll, an interesting tutorial
I was trying to understand what epoll is from Gluster mailing lists. Unfortunately, could not find anything useful for a newbie like me. :). Found this tutorial extremely helpful for likes of me. https://banu.com/blog/2/how-to-use-epoll-a-complete-example-in-c/ - Ajil ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Posix lock migration design
On Mon, Feb 29, 2016 at 12:52 PM, Susant Palai wrote:
> Hi Raghavendra,
>I have a question on the design.
>
>Currently in case of a client disconnection, pl_flush cleans up the
> locks associated with the fd created from that client.
> From the design, rebalance will migrate the locks to the new destination.
> Now in case client gets disconnected from the
> destination brick, how it is supposed to clean up the locks as
> rebalance/brick have no idea whether the client has opened
> an fd on destination and what the fd is.
>
>So the question is how to associate the client fd with locks on
> destination.
>
We don't use fds to cleanup the locks during flush. We use lk-owner which
doesn't change across migration. Note that lk-owner for posix-locks is
filled by the vfs/kernel where we've glusterfs mount.
pthread_mutex_lock (&pl_inode->mutex);
{
__delete_locks_of_owner (pl_inode, frame->root->client,
&frame->root->lk_owner);
}
pthread_mutex_unlock (&pl_inode->mutex);
> Thanks,
> Susant
>
> - Original Message -
> From: "Susant Palai"
> To: "Gluster Devel"
> Sent: Friday, 29 January, 2016 3:15:14 PM
> Subject: [Gluster-devel] Posix lock migration design
>
> Hi,
>Here, [1]
>
> https://docs.google.com/document/d/17SZAKxx5mhM-cY5hdE4qRq9icmFqy3LBaTdewofOXYc/edit?usp=sharing
> is a google document about proposal for "POSIX_LOCK_MIGRATION". Problem
> statement and design are explained in the document it self.
>
> Requesting the devel list to go through the document and
> comment/analyze/suggest, to take the thoughts forward (either on the
> google doc itself or here on the devel list).
>
>
> Thanks,
> Susant
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
--
Raghavendra G
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Introducing file based snapshots in gluster
Hello Gluster, Introducing a new file based snapshot feature in gluster which is based on reflinks feature which will be out from xfs in a couple of months (downstream) what is a reflink ? You might have surely used softlinks and hardlinks everyday! Reflink supports transparent copy on write, unlike soft/hardlinks which if useful for snapshotting, basically reflink points to same data blocks that are used by actual file (blocks are common to real file and a reflink file hence space efficient), they use different inode numbers hence they can have different permissions to access same data blocks, although they may look similar to hardlinks but are more space efficient and can handle all operations that can be performed on a regular file, unlike hardlinks that are limited to unlink(). which filesystem support reflink ? I think its Btrfs who put it for the first time, now xfs trying hard to make them available, in the future we can see them in ext4 as well You can get a feel of reflinks by following tutorial https://pkalever.wordpress.com/2016/01/22/xfs-reflinks-tutorial/ POC in gluster: https://asciinema.org/a/be50ukifcwk8tqhvo0ndtdqdd?speed=2 How we are doing it ? Currently we don't have a specific system-call that gives handle to reflinks, so I decided to go with ioctl call with XFS_IOC_CLONE command. In POC I have used setxattr/getxattr to create/delete/list the snapshot. Restore feature will use setxattr as well. We can have a fop although Fuse does't understand it, we will manage with a setxattr at Fuse mount point and again from client side it will be a fop till the posix xlator then as a ioctl to the underlying filesystem. Planing to expose APIs for create, delete, list and restore. Are these snapshots Internal or external? We will have a separate file each time we create a snapshot, obviously the snapshot file will have a different inode number and will be a readonly, all these files are maintained in the ".fsnap/ " directory which is maintained by the parent directory where the snapshot-ted/actual file resides, therefore they will not be visible to user (even with ls -a option, just like USS). *** We can always restore to any snapshot available in the list and the best part is we can delete any snapshot between snapshot1 and snapshotN because all of them are independent *** It is applications duty to ensure the consistency of the file before it tries to create a snapshot, say in case of VM file snapshot it is the hyper-visor that should freeze the IO and then request for the snapshot Integration with gluster: (Initial state, need more investigation) Quota: Since the snapshot files resides in ".fsnap/" directory which is maintained by the same directory where the actual file exist, it falls in the same users quota :) DHT: As said the snapshot files will resides in the same directory where the actual file resides may be in a ".fsnap/" directory Re-balancing: Simplest solution could be, copy the actual file as whole copy then for snapshotfiles rsync only delta's and recreate snapshots history by repeating snapshot sequence after each snapshotfile rsync. AFR: Mostly will be same as write fop (inodelk's and quorum's). There could be no way to recover or recreate a snapshot on node (brick to be precise) which was down while taking snapshot and comes back later in time. Disperse: Mostly take the inodelk and snapshot the file, on each of the bricks should work. Sharding: Assume we have a file split into 4 shards. If the fop for take snapshot is sent to all the subvols having the shards, it would be sufficient. All shards will have the snapshot for the state of the shard. List of snap fop should be sent only to the main subvol where shard 0 resides. Delete of a snap should be similar to create. Restore would be a little difficult because metadata of the file needs to be updated in shard xlator. Also in case of sharding, the bricks have gfid based flat filesystem. Hence the snaps created will also be in the shard directory, hence quota is not straight forward and needs additional work in this case. How can we make it better ? Discussion page: http://pad.engineering.redhat.com/kclYd9TPjr Thanks to "Pranith Kumar Karampuri", "Raghavendra Talur", "Rajesh Joseph", "Poornima Gurusiddaiah" and "Kotresh Hiremath Ravishankar" for all initial discussions. -Prasanna ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Posix lock migration design
On Tue, 2016-03-01 at 11:40 +0530, Raghavendra G wrote:
>
>
> On Mon, Feb 29, 2016 at 12:52 PM, Susant Palai
> wrote:
> > Hi Raghavendra,
> > I have a question on the design.
> >
> > Currently in case of a client disconnection, pl_flush cleans up
> > the locks associated with the fd created from that client.
> > From the design, rebalance will migrate the locks to the new
> > destination. Now in case client gets disconnected from the
> > destination brick, how it is supposed to clean up the locks as
> > rebalance/brick have no idea whether the client has opened
> > an fd on destination and what the fd is.
> So the question is how to associate the client fd with locks on
> destination.
> We don't use fds to cleanup the locks during flush. We use lk-owner
> which doesn't change across migration. Note that lk-owner for posix-
> locks is filled by the vfs/kernel where we've glusterfs mount.
A small note:
Since we don't set lk_owner for gfapi-based glusterfs clients, in those
scenarios frame->root->lk_owner is being calculated out of bit-wise
shift operations done on pid of the client.
. . .
if (call_frame->root->lk_owner.len)
au.lk_owner.lk_owner_val = call_frame->root->lk_owner.data;
else {
owner[0] = (char)(au.pid & 0xff);
owner[1] = (char)((au.pid >> 8) & 0xff);
owner[2] = (char)((au.pid >> 16) & 0xff);
owner[3] = (char)((au.pid >> 24) & 0xff);
au.lk_owner.lk_owner_val = owner;
au.lk_owner.lk_owner_len = 4;
}
. . .
To address this issue, we have http://review.gluster.org/#/c/12876/ whi
ch will expose a public api to set lk_owner for gfapi_based clients.
>
> pthread_mutex_lock (&pl_inode->mutex);
> {
> __delete_locks_of_owner (pl_inode, frame->root->client,
> &frame->root->lk_owner);
> }
> pthread_mutex_unlock (&pl_inode->mutex);
>
> > >
> >
> > Thanks,
> >
> > Susant
> >
> >
> > - Original Message -
> >
> > From: "Susant Palai"
> >
> > To: "Gluster Devel"
> >
> > Sent: Friday, 29 January, 2016 3:15:14 PM
> >
> > Subject: [Gluster-devel] Posix lock migration design
> >
> >
> > Hi,
> >
> > Here, [1]
> >
https://docs.google.com/document/d/17SZAKxx5mhM-cY5hdE4qRq9icmFqy3LBaTdewofOXYc/edit?usp=sharing
> >
> > is a google document about proposal for "POSIX_LOCK_MIGRATION". Problem
> > statement and design are explained in the document it self.
> >
> >
> > Requesting the devel list to go through the document and
> > comment/analyze/suggest, to take the thoughts forward (either on the
> >
> > google doc itself or here on the devel list).
> >
> >
> >
> > Thanks,
> >
> > Susant
> >
> > ___
> >
> > Gluster-devel mailing list
> >
Gluster-devel@gluster.org
> >
http://www.gluster.org/mailman/listinfo/gluster-devel
> >
> > ___
> >
> > Gluster-devel mailing list
> >
Gluster-devel@gluster.org
> >
http://www.gluster.org/mailman/listinfo/gluster-devel
> >
>
> > --
> Raghavendra G>
> ___
> Gluster-devel mailing list
>
Gluster-devel@gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-devel___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Introducing file based snapshots in gluster
On Tue, Mar 1, 2016 at 12:37 PM, Prasanna Kumar Kalever wrote: > Hello Gluster, > > > Introducing a new file based snapshot feature in gluster which is based on > reflinks feature which will be out from xfs in a couple of months > (downstream) > > > what is a reflink ? > > You might have surely used softlinks and hardlinks everyday! > > Reflink supports transparent copy on write, unlike soft/hardlinks which if > useful for snapshotting, basically reflink points to same data blocks that > are used by actual file (blocks are common to real file and a reflink file > hence space efficient), they use different inode numbers hence they can have > different permissions to access same data blocks, although they may look > similar to hardlinks but are more space efficient and can handle all > operations that can be performed on a regular file, unlike hardlinks that > are limited to unlink(). > > which filesystem support reflink ? > I think its Btrfs who put it for the first time, now xfs trying hard to > make them available, in the future we can see them in ext4 as well > > You can get a feel of reflinks by following tutorial > https://pkalever.wordpress.com/2016/01/22/xfs-reflinks-tutorial/ > > > POC in gluster: https://asciinema.org/a/be50ukifcwk8tqhvo0ndtdqdd?speed=2 > > > How we are doing it ? > Currently we don't have a specific system-call that gives handle to > reflinks, so I decided to go with ioctl call with XFS_IOC_CLONE command. > > In POC I have used setxattr/getxattr to create/delete/list the snapshot. > Restore feature will use setxattr as well. > > We can have a fop although Fuse does't understand it, we will manage with a > setxattr at Fuse mount point and again from client side it will be a fop till > the posix xlator then as a ioctl to the underlying filesystem. Planing to > expose APIs for create, delete, list and restore. > > Are these snapshots Internal or external? > We will have a separate file each time we create a snapshot, obviously the > snapshot file will have a different inode number and will be a readonly, all > these files are maintained in the ".fsnap/ " directory which is maintained > by the parent directory where the snapshot-ted/actual file resides, > therefore they will not be visible to user (even with ls -a option, just like > USS). > > *** We can always restore to any snapshot available in the list and the best > part is we can delete any snapshot between snapshot1 and snapshotN because > all of them are independent *** > > It is applications duty to ensure the consistency of the file before it > tries to create a snapshot, say in case of VM file snapshot it is the > hyper-visor that should freeze the IO and then request for the snapshot > > > > Integration with gluster: (Initial state, need more investigation) > > Quota: > Since the snapshot files resides in ".fsnap/" directory which is maintained > by the same directory where the actual file exist, it falls in the same > users quota :) > > DHT: > As said the snapshot files will resides in the same directory where the > actual file resides may be in a ".fsnap/" directory > > Re-balancing: > Simplest solution could be, copy the actual file as whole copy then for > snapshotfiles rsync only delta's and recreate snapshots history by repeating > snapshot sequence after each snapshotfile rsync. > > AFR: > Mostly will be same as write fop (inodelk's and quorum's). There could be no > way to recover or recreate a snapshot on node (brick to be precise) which > was down while taking snapshot and comes back later in time. > > Disperse: > Mostly take the inodelk and snapshot the file, on each of the bricks should > work. > > Sharding: > Assume we have a file split into 4 shards. If the fop for take snapshot is > sent to all the subvols having the shards, it would be sufficient. All shards > will have the snapshot for the state of the shard. > List of snap fop should be sent only to the main subvol where shard 0 resides. > Delete of a snap should be similar to create. > Restore would be a little difficult because metadata of the file needs to be > updated in shard xlator. > > Also in case of sharding, the bricks have gfid based flat filesystem. Hence > the snaps created will also be in the shard directory, hence quota is not > straight forward and needs additional work in this case. > > > How can we make it better ? > Discussion page: http://pad.engineering.redhat.com/kclYd9TPjr This link is not accessible externally. Could you move the contents to a public location? > > > Thanks to "Pranith Kumar Karampuri", "Raghavendra Talur", "Rajesh Joseph", > "Poornima Gurusiddaiah" and "Kotresh Hiremath Ravishankar" > for all initial discussions. > > > -Prasanna > > > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel ___
