[Gluster-devel] spurios failures in tests/encryption/crypt.t
hi, crypt.t is failing regression builds once in a while and most of the times it is because of the failures just after the remount in the script. TEST rm -f $M0/testfile-symlink TEST rm -f $M0/testfile-link Both of these are failing with ENOTCONN. I got a chance to look at the logs. According to the brick logs, this is what I see: [2014-05-17 05:43:43.363979] E [posix.c:2272:posix_open] 0-patchy-posix: open on /d/backends/patchy1/testfile-symlink: Transport endpoint is not connected This is the very first time I saw posix failing with ENOTCONN. Do we have these bricks on some other network mounts? I wonder why it fails with ENOTCONN. I also see that it happens right after a call_bail on the mount. Pranith ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Changes to Regression script
- Original Message - From: Vijay Bellur vbel...@redhat.com To: gluster-infra gluster-in...@gluster.org Cc: gluster-devel@gluster.org Sent: Tuesday, May 13, 2014 4:13:02 PM Subject: [Gluster-devel] Changes to Regression script Hi All, Me and Kaushal have effected the following changes on regression.sh in build.gluster.org: 1. If a regression run results in a core and all tests pass, that particular run will be flagged as a failure. Previously a core that would cause test failures only would get marked as a failure. 2. Cores from a particular test run are now archived and are available at /d/archived_builds/. This will also prevent manual intervention for managing cores. 3. Logs from failed regression runs are now archived and are available at /d/logs/glusterfs-timestamp.tgz Do let us know if you have any comments on these changes. This is already proving to be useful :-). I was able to debug one of the spurious failures for crypt.t. But the only problem is I was not able copy out the logs. Had to take avati's help to get the log files. Will it be possible to give access to these files so that anyone can download them? Pranith Thanks, Vijay ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Changes to Regression script
On 05/17/2014 02:10 PM, Pranith Kumar Karampuri wrote: - Original Message - From: Vijay Bellur vbel...@redhat.com To: gluster-infra gluster-in...@gluster.org Cc: gluster-devel@gluster.org Sent: Tuesday, May 13, 2014 4:13:02 PM Subject: [Gluster-devel] Changes to Regression script Hi All, Me and Kaushal have effected the following changes on regression.sh in build.gluster.org: 1. If a regression run results in a core and all tests pass, that particular run will be flagged as a failure. Previously a core that would cause test failures only would get marked as a failure. 2. Cores from a particular test run are now archived and are available at /d/archived_builds/. This will also prevent manual intervention for managing cores. 3. Logs from failed regression runs are now archived and are available at /d/logs/glusterfs-timestamp.tgz Do let us know if you have any comments on these changes. This is already proving to be useful :-). I was able to debug one of the spurious failures for crypt.t. But the only problem is I was not able copy out the logs. Had to take avati's help to get the log files. Will it be possible to give access to these files so that anyone can download them? Good to know! You can access the .tgz files from: http://build.gluster.org:443/logs/ -Vijay ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Changes to Regression script
- Original Message - From: Vijay Bellur vbel...@redhat.com To: Pranith Kumar Karampuri pkara...@redhat.com Cc: gluster-infra gluster-in...@gluster.org, gluster-devel@gluster.org Sent: Saturday, May 17, 2014 2:52:03 PM Subject: Re: [Gluster-devel] Changes to Regression script On 05/17/2014 02:10 PM, Pranith Kumar Karampuri wrote: - Original Message - From: Vijay Bellur vbel...@redhat.com To: gluster-infra gluster-in...@gluster.org Cc: gluster-devel@gluster.org Sent: Tuesday, May 13, 2014 4:13:02 PM Subject: [Gluster-devel] Changes to Regression script Hi All, Me and Kaushal have effected the following changes on regression.sh in build.gluster.org: 1. If a regression run results in a core and all tests pass, that particular run will be flagged as a failure. Previously a core that would cause test failures only would get marked as a failure. 2. Cores from a particular test run are now archived and are available at /d/archived_builds/. This will also prevent manual intervention for managing cores. 3. Logs from failed regression runs are now archived and are available at /d/logs/glusterfs-timestamp.tgz Do let us know if you have any comments on these changes. This is already proving to be useful :-). I was able to debug one of the spurious failures for crypt.t. But the only problem is I was not able copy out the logs. Had to take avati's help to get the log files. Will it be possible to give access to these files so that anyone can download them? Good to know! You can access the .tgz files from: http://build.gluster.org:443/logs/ Awesome!! Pranith -Vijay ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] New project on the Forge - gstatus
Avati that's awesome! I didn't know that the meta xlator could do that already. On Sat, May 17, 2014 at 9:30 AM, Anand Avati av...@gluster.org wrote: KP, Vipul, It will be awesome to get io-stats like instrumentation on the client side. Here are some further thoughts on how to implement that. If you have a recent git HEAD build, I would suggest that you explore the latency stats on the client side exposed through meta at $MNT/.meta/graphs/active/$xlator/profile. You can enable latency measurement with echo 1 $MNT/.meta/measure_latency. I would suggest extending these stats with the extra ones io-stats has, and make glusterfsiostats expose these stats. If you can compare libglusterfs/src/latency.c:gf_latency_begin(), gf_latency_end() and gf_latency_udpate() with the macros in io-stats.c UPDATE_PROFILE_STATS() and START_FOP_LATENCY(), you will quickly realize how a lot of logic is duplicated between io-stats and latency.c. If you can enhance latency.c and make it capture the remaining stats what io-stats is capturing, the benefits of this approach would be: - stats are already getting captured at all xlator levels, and not just at the position where io-stats is inserted - file like interface makes the stats more easily inspectable and consumable, and updated on the fly - conforms with the way rest of the internals are exposed through $MNT/.meta In order to this, you might want to look into: - latency.c as of today captures fop count, mean latency, total time, whereas io-stats measures these along with min-time, max-time and block-size histogram. - extend gf_proc_dump_latency_info() to dump the new stats - either prettify that output like 'volume profile info' output, or JSONify it like xlators/meta/src/frames-file.c - add support for cumulative vs interval stats (store an extra copy of this-latencies[]) etc.. Thanks! On Fri, Apr 25, 2014 at 9:09 PM, Krishnan Parthasarathi kpart...@redhat.com wrote: [Resending due to gluster-devel mailing list issue] Apologies for the late reply. glusterd uses its socket connection with brick processes (where io-stats xlator is loaded) to gather information from io-stats via an RPC request. This facility is restricted to brick processes as it stands today. Some background ... io-stats xlator is loaded, both in GlusterFS mounts and brick processes. So, we have the capabilities to monitor I/O statistics on both sides. To collect I/O statistics at the server side, we have # gluster volume profile VOLNAME [start | info | stop] AND #gluster volume top VOLNAME info [and other options] We don't have a usable way of gathering I/O statistics (not monitoring, though the counters could be enhanced) at the client-side, ie. for a given mount point. This is the gap glusterfsiostat aims to fill. We need to remember that the machines hosting GlusterFS mounts may not have glusterd installed on them. We are considering rrdtool as a possible statistics database because it seems like a natural choice for storing time-series data. rrdtool is capable of answering high-level statistical queries on statistics that were logged in it by io-stats xlator over and above printing running counters periodically. Hope this gives some more clarity on what we are thinking. thanks, Krish - Original Message - Probably me not understanding. the comment iostats making data available to glusterd over RPC - is what I latched on to. I wondered whether this meant that a socket could be opened that way to get at the iostats data flow. Cheers, PC - Original Message - From: Vipul Nayyar nayyar_vi...@yahoo.com To: Paul Cuzner pcuz...@redhat.com, Krishnan Parthasarathi kpart...@redhat.com Cc: Vijay Bellur vbel...@redhat.com, gluster-devel gluster-de...@nongnu.org Sent: Thursday, 20 February, 2014 5:06:27 AM Subject: Re: [Gluster-devel] New project on the Forge - gstatus Hi Paul, I'm really not sure, if this can be done in python(at least comfortably). Maybe we can tread on the same path as Justin's glusterflow in python. But I don't think, all the io-stats counters will be available with the way how Justin's used Jeff Darcy's previous work to build his tool. I can be wrong. My knowledge is a bit incomplete and based on very less experience as a user and an amateur Gluster developer. Please do correct me, if I can be. Regards Vipul Nayyar ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org
[Gluster-devel] corrupted hash table
This is master branch, client side:
Program terminated with signal 11, Segmentation fault.
#0 uuid_unpack (in=0xffc0 Address 0xffc0 out of bounds,
uu=0xbf7fd7b0) at ../../contrib/uuid/unpack.c:43
warning: Source file is more recent than executable.
43 tmp = *ptr++;
(gdb) print tmp
Cannot access memory at address 0xffc0
(gdb) bt
#0 uuid_unpack (in=0xffc0 Address 0xffc0 out of bounds,
uu=0xbf7fd7b0) at ../../contrib/uuid/unpack.c:43
#1 0xbb788f63 in uuid_compare (
uu1=0xffc0 Address 0xffc0 out of bounds,
uu2=0xb811f938 k\350_6) at ../../contrib/uuid/compare.c:46
#2 0xbb769993 in __inode_find (table=0xbb213368, gfid=0xb811f938 k\350_6)
at inode.c:763
#3 0xbb769cdc in __inode_link (inode=0x5a70b768, parent=optimized out,
name=0x5a7e3148 conf24746.file, iatt=0xb811f930) at inode.c:831
#4 0xbb769f3f in inode_link (inode=0x5a70b768, parent=0x5af47728,
name=0x5a7e3148 conf24746.file, iatt=0xb811f930) at inode.c:892
#5 0xbb36bdaa in fuse_create_cbk (frame=0xba417c44, cookie=0xbb28cb98,
this=0xb9cbe018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
buf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00, xdata=0x0)
at fuse-bridge.c:1888
#6 0xb92a30a0 in io_stats_create_cbk (frame=0xbb28cb98, cookie=0xbb287418,
this=0xb9df2018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
buf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00, xdata=0x0)
at io-stats.c:1260
#7 0xb92afd80 in mdc_create_cbk (frame=0xbb287418, cookie=0xbb28d7d8,
this=0xb9df1018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
buf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00, xdata=0x0)
at md-cache.c:1404
#8 0xb92c790f in ioc_create_cbk (frame=0xbb28d7d8, cookie=0xbb28b008,
this=0xb9dee018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
buf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00, xdata=0x0)
at io-cache.c:701
#9 0xbb3079ba in ra_create_cbk (frame=0xbb28b008, cookie=0xbb287f08,
this=0xb9dec018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
buf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00, xdata=0x0)
at read-ahead.c:173
#10 0xb92f66b9 in dht_create_cbk (frame=0xbb287f08, cookie=0xbb28f988,
this=0xb9cff018, op_ret=0, op_errno=0, fd=0xb799e808, inode=0x5a70b768,
stbuf=0xb811f930, preparent=0xb811f998, postparent=0xb811fa00,
xdata=0x5c491028) at dht-common.c:3942
#11 0xb932fa22 in afr_create_unwind (frame=0xba40439c, this=0xb9cfd018)
at afr-dir-write.c:397
#12 0xb9330a02 in __afr_dir_write_cbk (frame=0xba40439c, cookie=0x2,
this=0xb9cfd018, op_ret=0, op_errno=0, buf=0xbf7fdfe4,
preparent=0xbf7fdf7c, postparent=0xbf7fdf14, preparent2=0x0,
postparent2=0x0, xdata=0x5cb61ea8) at afr-dir-write.c:244
#13 0xb939a401 in client3_3_create_cbk (req=0xb805f028, iov=0xb805f048,
count=1, myframe=0xbb28e4f8) at client-rpc-fops.c:2211
#14 0xbb7daecf in rpc_clnt_handle_reply (clnt=0xb9cd93b8, pollin=0x5a7dbe38)
at rpc-clnt.c:767
#15 0xbb7db7a4 in rpc_clnt_notify (trans=0xb80a7018, mydata=0xb9cd93d8,
---Type return to continue, or q return to quit---
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x5a7dbe38) at rpc-clnt.c:895
#16 0xbb7d7d9c in rpc_transport_notify (this=0xb80a7018,
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x5a7dbe38) at rpc-transport.c:512
#17 0xbb3214ab in socket_event_poll_in (this=0xb80a7018) at socket.c:2120
#18 0xbb3246fc in socket_event_handler (fd=16, idx=4, data=0xb80a7018,
poll_in=1, poll_out=0, poll_err=0) at socket.c:2233
#19 0xbb7a4c9a in event_dispatch_poll_handler (i=4, ufds=0xbb285118,
event_pool=0xbb242098) at event-poll.c:357
#20 event_dispatch_poll (event_pool=0xbb242098) at event-poll.c:436
#21 0xbb77a160 in event_dispatch (event_pool=0xbb242098) at event.c:113
#22 0x08050567 in main (argc=4, argv=0xbf7fe880) at glusterfsd.c:2023
(gdb) frame 2
#2 0xbb769993 in __inode_find (table=0xbb213368, gfid=0xb811f938 k\350_6)
at inode.c:763
763 if (uuid_compare (tmp-gfid, gfid) == 0) {
(gdb) list
758 return table-root;
759
760 hash = hash_gfid (gfid, 65536);
761
762 list_for_each_entry (tmp, table-inode_hash[hash], hash) {
763 if (uuid_compare (tmp-gfid, gfid) == 0) {
764 inode = tmp;
765 break;
766 }
767 }
--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
