On Wed, Jun 22, 2016 at 5:50 AM, Sachin Pandit <span...@commvault.com> wrote:
> Hey Pranith, I am good, I hope you are doing good too. > > Please find the comments inline. > > > > *From:* Pranith Kumar Karampuri [mailto:pkara...@redhat.com] > *Sent:* Tuesday, June 21, 2016 5:58 AM > *To:* Sachin Pandit <span...@commvault.com> > *Cc:* gluster-devel@gluster.org > *Subject:* Re: [Gluster-devel] Reduce memcpy in glfs read and write > > > > Hey!! > > Hope you are doing good. I took a look at the bt. So when flush > comes write-behind has to flush all the writes down. I see the following > frame hung in iob_unref: > Thread 7 (Thread 0x7fa601a30700 (LWP 16218)): > #0 0x00007fa60cc55225 in pthread_spin_lock () from > /lib64/libpthread.so.0 <<---- Does it always hang there? > > --------------------------------- > > >>It does always hang here. > > --------------------------------- > #1 0x00007fa60e1f373e in iobref_unref (iobref=0x19dc7e0) at iobuf.c:907 > #2 0x00007fa60e246fb2 in args_wipe (args=0x19e70ec) at > default-args.c:1593 > #3 0x00007fa60e1ea534 in call_stub_wipe_args (stub=0x19e709c) at > call-stub.c:2466 > #4 0x00007fa60e1ea5de in call_stub_destroy (stub=0x19e709c) at > call-stub.c:2482 > > Is this on top of master branch? It seems like we missed an unlock of the > spin-lock or the iobref has junk value which gives the feeling that it is > in locked state (May be double free?). Do you have any extra patches you > have in your repo which make changes in iobuf? > > ---------------------------------- > > >>I have implemented a method to reduce memcpy in libgfapi (My patch is on > top of master branch), by making use of buffer from iobuf pool and passing > the buffer to application. However, I have not made any changes in iobuf > core feature. I don’t think double free is happening anywhere in the code > (I did check this using logs) > > > > Method that I have implemented: > > 1) Application asks for a buffer of specific size, and the buffer is > allocated from the iobuf pool. > > 2) Buffer is passed on to application, and the application writes > the data into that buffer. > > 3) Buffer with data in it is passed from application to libgfapi and > the underlying translators (no memcpy in glfs_write) > > > > I have couple of questions, and observations: > > > > Observations: > > ------------------ > > 1) For every write if I get a fresh buffer then I don’t see any > problem. All the writes are going through. > > 2) If I try to make use of buffer for consecutive writes, then I am > seeing the hang in flush. > > > > Question1: Is it fine if I reuse the buffer for consecutive writes?? > > Question2: Is it always ensured that the data is written to the file when > I get a response from syncop_writev. > Will it be possible to share the patch on master and a test program which can recreate this issue? > > > Thanks, > > Sachin Pandit. > > > > ---------------------------------- > > > > On Tue, Jun 21, 2016 at 4:07 AM, Sachin Pandit <span...@commvault.com> > wrote: > > Hi all, > > > > I bid adieu to you all with the hope of crossing path again, and the time > has come rather quickly. It feels great to work on GlusterFS again. > > > > Currently we are trying to write data backed up by Commvault Simpana to > glusterfs volume (Disperse volume). To improve the performance, I have > implemented the proposal put forward my Rafi K C [1]. I have some > questions regarding libgfapi and iobuf pool. > > > > To reduce an extra level of copy in glfs read and write, I have > implemented few APIs to request a buffer (similar to the one represented in > [1]) from iobuf pool which can be used by the application to write data > to. With this implementation, when I try to reuse the buffer for > consecutive writes, I could see a hang in syncop_flush of glfs_close (BT of > the hang can be found in [2]). I wanted to know if reusing the buffer is > recommended. If not, do we need to request buffer for each writes? > > > > Setup : Distributed-Disperse ( 4 * (2+1)). Bricks scattered over 3 nodes. > > > > [1] > http://www.gluster.org/pipermail/gluster-devel/2015-February/043966.html > > [2] Attached file - bt.txt > > > > Thanks & Regards, > > Sachin Pandit. > > > > ***************************Legal Disclaimer*************************** > > "This communication may contain confidential and privileged material for the > > sole use of the intended recipient. Any unauthorized review, use or > distribution > > by others is strictly prohibited. If you have received the message by mistake, > > please advise the sender by reply email and delete the message. Thank you." > > ********************************************************************** > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > > > > > -- > > Pranith > > ***************************Legal Disclaimer*************************** > "This communication may contain confidential and privileged material for the > sole use of the intended recipient. Any unauthorized review, use or > distribution > by others is strictly prohibited. If you have received the message by mistake, > please advise the sender by reply email and delete the message. Thank you." > ********************************************************************** > > -- Pranith
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel