[Gluster-users] glusterfs replica volume self heal lots of small file very very slow！how to improve？ why slow？
Hi all： I do the following test： I create a glusterfs replica volume (replica count is 2 ) with two server node(server A and server B)，use XFS as the underlying filesystem， then mount the volume in client node， then， I shut down the network of server A node， in client node， I copy a dir（which has a lot of small files）， the dir size is 2.9GByte， when copy finish， I unmount the volume from the client， then I start the network of server A node， now， glusterfs self-heal-daemon start heal dir from server B to server A， in the end， I find the self-heal-daemon heal the dir use 40 minutes， It's too slow！ why？ I find out related options with self-heal， as follow： cluster.self-heal-window-size cluster.self-heal-readdir-size cluster.background-self-heal-count then I config : cluster.self-heal-window-size is 1024(max value) cluster.self-heal-readdir-size is 131072(max value) and then do the same test case, find this times heal the dir use 35 minutes, The effective is not obvious, I want to ask, If there are better ways to improve replica volume self heal lots of small file performance?? thanks！ justgluste...@gmail.com ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] [Gluster-devel] Transparent encryption in GlusterFS: Implications on manageability
Hi all, Unfortunately it is impossible to validate non-trusted volfiles using existing glusterfs options. Semantic and format of values passed by the --xlator-option don't allow to "deliver" trusted values without compromises with security. So I have added a new --secure-xlator-option, Please, review: review.gluster.org/8657 Thanks, Edward. On Wed, 13 Aug 2014 12:26:29 -0700 Anand Avati wrote: > +1 for all the points. > > > On Wed, Aug 13, 2014 at 11:22 AM, Jeff Darcy > wrote: > > > > I.1 Generating the master volume key > > > > > > > > > Master volume key should be generated by user on the trusted > > > machine. Recommendations on master key generation provided at > > > section 6.2 of the manpages [1]. Generating of master volume key > > > is in user's competence. > > > > That was fine for an initial implementation, but it's still the > > single largest obstacle to adoption of this feature. Looking > > forward, we need to provide full CLI support for generating keys in > > the necessary format, specifying their location, etc. > > > > >I.2 Location of the master volume key when mounting a > > >volume > > > > > > > > > At mount time the crypt translator searches for a master volume > > > key on the client machine at the location specified by the > > > respective translator option. If there is no any key at the > > > specified location, or the key at specified location is in > > > improper format, then mount will fail. Otherwise, the crypt > > > translator loads the key to its private memory data structures. > > > > > > Location of the master volume key can be specified at volume > > > creation time (see option "master-key", section 6.7 of the man > > > pages [1]). However, this option can be overridden by user at > > > mount time to specify another location, see section 7 of manpages > > > [1], steps 6, 7, 8. > > > > Again, we need to improve on this. We should support this as a > > volume or mount option in its own right, not rely on the generic > > --xlator-option mechanism. Adding options to mount.glusterfs isn't > > hard. Alternatively, we could make this look like a volume option > > settable once through the CLI, even though the path is stored > > locally on the client. Or we could provide a separate > > special-purpose command/script, which again only needs to be run > > once. It would even be acceptable to treat the path to the key > > file (not its contents!) as a true volume option, stored on the > > servers. Any of these would be better than requiring the user to > > understand our volfile format and construction so that they can add > > the necessary option by hand. > > > > >II. Check graph of translators on your client > > > machine after mount! > > > > > > > > > During mount your client machine receives configuration info from > > > the non-trusted server. In particular, this info contains the > > > graph of translators, which can be subjected to tampering, so > > > that encryption won't be invoked for your volume at all. So it is > > > highly important to verify this graph. After successful mount > > > make sure that the graph of translators contains the crypt > > > translator with proper options (see FAQ#1, section 11 of the > > > manpages [1]). > > > > It is important to verify the graph, but not by poking through log > > files and not without more information about what to look for. So > > we got a volfile that includes the crypt translator, with some > > options. The *code* should ensure that the master-key option has > > the value from the command line or local config, and not some > > other. If we have to add special support for this in > > otherwise-generic graph initialization code, that's fine. > > ___ > > Gluster-devel mailing list > > gluster-de...@gluster.org > > http://supercolony.gluster.org/mailman/listinfo/gluster-devel > > ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] How do I temporarily take a brick out of service and then put it back later?
On 09/17/2014 05:25 PM, Eliezer Croitoru wrote: Get into glustefs irc channel at freenode.net and try to get live help. The actual channel name is "gluster" and I do not remember the page help related issues. Eliezer ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] How do I temporarily take a brick out of service and then put it back later?
Before you rush into posting and reposting issue on the mailing list try first to: Get into glustefs irc channel at freenode.net and try to get live help. And it will not help you to spam the list You need to post the relevant information: glusterfs versions of server and client OS version df output etc.. I can still do not understand the basics about your setup.. Eliezer On 09/16/2014 09:41 PM, Greg Scott wrote: This seems like such an innocent question. I have a firewall system controlling tunnels all over the USA. It’s an HA setup with two nodes. And I use Gluster to keep all the configs and logs replicated. It’s an active/standby system and it’s been in place for something like 3 years. The standby had a catastrophic hardware failure a while ago and it looks like it needs a new motherboard. We have people rebuilding the hardware. The standby hard drive seems fine. But now the primary system repeatedly stalls its I/Os, sometimes to directories that aren’t even part of Glusterfs. And the problem is getting worse day by day, hour by hour. Before they barbecue me, how do I tell Gluster to temporarily take the failed node offline while the motherboard is replaced, then put it back in service and copy everything over to it? I don’t want to completely remove the brick because when the hardware is repaired and we start it up again, I want it to join back up and have everything replicate over to it. So for now – what can I do on the surviving node to tell it not to try to replicate until further notice, and then how to I tell it to go back to normal when I get the standby system back online? Thanks -Greg Scott ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Reminder: Weekly GlusterFS Community Meeting in 45 minutes
On 17/09/2014, at 12:14 PM, Justin Clift wrote: > Reminder!!! > > The weekly Gluster Community meeting is in 45 minutes, in > #gluster-meeting on IRC. > > This is a completely public meeting, everyone is encouraged > to attend and be a part of it. :) Short meeting today. ;) Meeting Minutes: http://meetbot.fedoraproject.org/gluster-meeting/2014-09-17/gluster-meeting.2014-09-17-12.02.html Full logs: http://meetbot.fedoraproject.org/gluster-meeting/2014-09-17/gluster-meeting.2014-09-17-12.02.log.html Regards and best wishes, Justin Clift -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Mirroring in GlusterFS
Hello. I'm wondering, how is the actual mirroring done in Gluster? Is client responsible for creating two copies of data? If yes, does it perform two writes in parallel (to two bricks) or in sequential manner? Or, maybe, it is server's responsibility - to make a copy from each write request? Thank you for your opinions. ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Reminder: Weekly GlusterFS Community Meeting in 45 minutes
Reminder!!! The weekly Gluster Community meeting is in 45 minutes, in #gluster-meeting on IRC. This is a completely public meeting, everyone is encouraged to attend and be a part of it. :) To add Agenda items *** Add new items under the "Other items to discuss" point on the Etherpad: https://public.pad.fsfe.org/p/gluster-community-meetings And be at the meeting to explain what they're about. :) Regards and best wishes, Justin Clift -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Use geo-replication without passwordless ssh login
On 09/15/2014 08:48 PM, Bo Yu wrote: Hi, I wonder if it is possible to configure Gluster geo-replication in a manner that it does not require passwordless ssh login, since in our system passwordless ssh is not allowed. Or, is it possible to configure passwordless ssh for Gluster only, not for every user or programm. Discussion is going on to add support for any user with sudo access can create geo-rep session. Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1136296 -- regards Aravinda http://aravindavk.in Thanks. Bo ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
