Got time to test subdir-mount.t failing in brick-mux scenario. I noticed some issues, where I need further help from glusterd team.
subdir-mount.t expects 'hook' script to run after add-brick to make sure the required subdirectories are healed and are present in new bricks. This is important as subdir mount expects the subdirs to exist for successful mount. But in case of brick-mux setup, I see that in some cases (6/10), hook script (add-brick/post-hook/S13-create-subdir-mount.sh) started getting executed after 20second of finishing the add-brick command. Due to this, the mount which we execute after add-brick failed. My question is, what is making post hook script to run so late ?? I can recreate the issues locally on my laptop too. On Sat, Jun 1, 2019 at 4:55 PM Atin Mukherjee <amukh...@redhat.com> wrote: > subdir-mount.t has started failing in brick mux regression nightly. This > needs to be fixed. > > Raghavendra - did we manage to get any further clue on uss.t failure? > > ---------- Forwarded message --------- > From: <jenk...@build.gluster.org> > Date: Fri, 31 May 2019 at 23:34 > Subject: [Gluster-Maintainers] Build failed in Jenkins: > regression-test-with-multiplex #1359 > To: <maintain...@gluster.org>, <rkavu...@redhat.com>, <ama...@redhat.com>, > <amukh...@redhat.com>, <khire...@redhat.com> > > > See < > https://build.gluster.org/job/regression-test-with-multiplex/1359/display/redirect?page=changes > > > > Changes: > > [atin] glusterd: add an op-version check > > [atin] glusterd/svc: glusterd_svcs_stop should call individual wrapper > function > > [atin] glusterd/svc: Stop stale process using the glusterd_proc_stop > > [Amar Tumballi] lcov: more coverage to shard, old-protocol, sdfs > > [Kotresh H R] tests/geo-rep: Add EC volume test case > > [Amar Tumballi] glusterfsd/cleanup: Protect graph object under a lock > > [Mohammed Rafi KC] glusterd/shd: Optimize the glustershd manager to send > reconfigure > > [Kotresh H R] tests/geo-rep: Add tests to cover glusterd geo-rep > > [atin] glusterd: Optimize code to copy dictionary in handshake code path > > ------------------------------------------ > [...truncated 3.18 MB...] > ./tests/basic/afr/stale-file-lookup.t - 9 second > ./tests/basic/afr/granular-esh/replace-brick.t - 9 second > ./tests/basic/afr/granular-esh/add-brick.t - 9 second > ./tests/basic/afr/gfid-mismatch.t - 9 second > ./tests/performance/open-behind.t - 8 second > ./tests/features/ssl-authz.t - 8 second > ./tests/features/readdir-ahead.t - 8 second > ./tests/bugs/upcall/bug-1458127.t - 8 second > ./tests/bugs/transport/bug-873367.t - 8 second > ./tests/bugs/replicate/bug-1498570-client-iot-graph-check.t - 8 second > ./tests/bugs/replicate/bug-1132102.t - 8 second > ./tests/bugs/quota/bug-1250582-volume-reset-should-not-remove-quota-quota-deem-statfs.t > - 8 second > ./tests/bugs/quota/bug-1104692.t - 8 second > ./tests/bugs/posix/bug-1360679.t - 8 second > ./tests/bugs/posix/bug-1122028.t - 8 second > ./tests/bugs/nfs/bug-1157223-symlink-mounting.t - 8 second > ./tests/bugs/glusterfs/bug-861015-log.t - 8 second > ./tests/bugs/glusterd/sync-post-glusterd-restart.t - 8 second > ./tests/bugs/glusterd/bug-1696046.t - 8 second > ./tests/bugs/fuse/bug-983477.t - 8 second > ./tests/bugs/ec/bug-1227869.t - 8 second > ./tests/bugs/distribute/bug-1088231.t - 8 second > ./tests/bugs/distribute/bug-1086228.t - 8 second > ./tests/bugs/cli/bug-1087487.t - 8 second > ./tests/bugs/cli/bug-1022905.t - 8 second > ./tests/bugs/bug-1258069.t - 8 second > ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t > - 8 second > ./tests/basic/xlator-pass-through-sanity.t - 8 second > ./tests/basic/quota-nfs.t - 8 second > ./tests/basic/glusterd/arbiter-volume.t - 8 second > ./tests/basic/ctime/ctime-noatime.t - 8 second > ./tests/line-coverage/cli-peer-and-volume-operations.t - 7 second > ./tests/gfid2path/get-gfid-to-path.t - 7 second > ./tests/bugs/upcall/bug-1369430.t - 7 second > ./tests/bugs/snapshot/bug-1260848.t - 7 second > ./tests/bugs/shard/shard-inode-refcount-test.t - 7 second > ./tests/bugs/shard/bug-1258334.t - 7 second > ./tests/bugs/replicate/bug-767585-gfid.t - 7 second > ./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t - 7 second > ./tests/bugs/replicate/bug-1250170-fsync.t - 7 second > ./tests/bugs/posix/bug-1175711.t - 7 second > ./tests/bugs/nfs/bug-915280.t - 7 second > ./tests/bugs/md-cache/setxattr-prepoststat.t - 7 second > ./tests/bugs/md-cache/bug-1211863_unlink.t - 7 second > ./tests/bugs/glusterfs/bug-848251.t - 7 second > ./tests/bugs/distribute/bug-1122443.t - 7 second > ./tests/bugs/changelog/bug-1208470.t - 7 second > ./tests/bugs/bug-1702299.t - 7 second > ./tests/bugs/bug-1371806_2.t - 7 second > ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t - 7 > second > ./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t - 7 second > ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t - > 7 second > ./tests/bitrot/br-stub.t - 7 second > ./tests/basic/glusterd/arbiter-volume-probe.t - 7 second > ./tests/basic/gfapi/libgfapi-fini-hang.t - 7 second > ./tests/basic/fencing/fencing-crash-conistency.t - 7 second > ./tests/basic/distribute/file-create.t - 7 second > ./tests/basic/afr/tarissue.t - 7 second > ./tests/basic/afr/gfid-heal.t - 7 second > ./tests/bugs/snapshot/bug-1178079.t - 6 second > ./tests/bugs/snapshot/bug-1064768.t - 6 second > ./tests/bugs/shard/bug-1342298.t - 6 second > ./tests/bugs/shard/bug-1259651.t - 6 second > ./tests/bugs/replicate/bug-1686568-send-truncate-on-arbiter-from-shd.t - > 6 second > ./tests/bugs/replicate/bug-1626994-info-split-brain.t - 6 second > ./tests/bugs/replicate/bug-1325792.t - 6 second > ./tests/bugs/replicate/bug-1101647.t - 6 second > ./tests/bugs/quota/bug-1243798.t - 6 second > ./tests/bugs/protocol/bug-1321578.t - 6 second > ./tests/bugs/nfs/bug-877885.t - 6 second > ./tests/bugs/nfs/bug-1143880-fix-gNFSd-auth-crash.t - 6 second > ./tests/bugs/md-cache/bug-1476324.t - 6 second > ./tests/bugs/md-cache/afr-stale-read.t - 6 second > ./tests/bugs/io-cache/bug-858242.t - 6 second > ./tests/bugs/glusterfs/bug-893378.t - 6 second > ./tests/bugs/glusterfs/bug-856455.t - 6 second > ./tests/bugs/glusterd/quorum-value-check.t - 6 second > ./tests/bugs/ec/bug-1179050.t - 6 second > ./tests/bugs/distribute/bug-912564.t - 6 second > ./tests/bugs/distribute/bug-884597.t - 6 second > ./tests/bugs/distribute/bug-1368012.t - 6 second > ./tests/bugs/core/bug-986429.t - 6 second > ./tests/bugs/core/bug-1699025-brick-mux-detach-brick-fd-issue.t - 6 > second > ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t - 6 second > ./tests/bugs/bug-1371806_1.t - 6 second > ./tests/bugs/bitrot/bug-1229134-bitd-not-support-vol-set.t - 6 second > ./tests/bugs/bitrot/bug-1210684-scrub-pause-resume-error-handling.t - 6 > second > ./tests/bitrot/bug-1221914.t - 6 second > ./tests/basic/trace.t - 6 second > ./tests/basic/playground/template-xlator-sanity.t - 6 second > ./tests/basic/ec/nfs.t - 6 second > ./tests/basic/ec/ec-read-policy.t - 6 second > ./tests/basic/ec/ec-anonymous-fd.t - 6 second > ./tests/basic/distribute/non-root-unlink-stale-linkto.t - 6 second > ./tests/basic/changelog/changelog-rename.t - 6 second > ./tests/basic/afr/heal-info.t - 6 second > ./tests/basic/afr/afr-read-hash-mode.t - 6 second > ./tests/gfid2path/gfid2path_nfs.t - 5 second > ./tests/bugs/upcall/bug-1422776.t - 5 second > ./tests/bugs/replicate/bug-886998.t - 5 second > ./tests/bugs/replicate/bug-1365455.t - 5 second > ./tests/bugs/readdir-ahead/bug-1670253-consistent-metadata.t - 5 second > ./tests/bugs/posix/bug-gfid-path.t - 5 second > ./tests/bugs/posix/bug-765380.t - 5 second > ./tests/bugs/nfs/bug-847622.t - 5 second > ./tests/bugs/nfs/bug-1116503.t - 5 second > ./tests/bugs/io-stats/bug-1598548.t - 5 second > ./tests/bugs/glusterfs-server/bug-877992.t - 5 second > ./tests/bugs/glusterfs-server/bug-873549.t - 5 second > ./tests/bugs/glusterfs/bug-895235.t - 5 second > ./tests/bugs/fuse/bug-1126048.t - 5 second > ./tests/bugs/distribute/bug-907072.t - 5 second > ./tests/bugs/core/bug-913544.t - 5 second > ./tests/bugs/core/bug-908146.t - 5 second > ./tests/bugs/access-control/bug-1051896.t - 5 second > ./tests/basic/ec/ec-internal-xattrs.t - 5 second > ./tests/basic/ec/ec-fallocate.t - 5 second > ./tests/basic/distribute/bug-1265677-use-readdirp.t - 5 second > ./tests/basic/afr/arbiter-remove-brick.t - 5 second > ./tests/performance/quick-read.t - 4 second > ./tests/gfid2path/block-mount-access.t - 4 second > ./tests/features/delay-gen.t - 4 second > ./tests/bugs/upcall/bug-upcall-stat.t - 4 second > ./tests/bugs/upcall/bug-1394131.t - 4 second > ./tests/bugs/unclassified/bug-1034085.t - 4 second > ./tests/bugs/snapshot/bug-1111041.t - 4 second > ./tests/bugs/shard/bug-1272986.t - 4 second > ./tests/bugs/shard/bug-1256580.t - 4 second > ./tests/bugs/shard/bug-1250855.t - 4 second > ./tests/bugs/shard/bug-1245547.t - 4 second > ./tests/bugs/rpc/bug-954057.t - 4 second > ./tests/bugs/replicate/bug-976800.t - 4 second > ./tests/bugs/replicate/bug-880898.t - 4 second > ./tests/bugs/replicate/bug-1480525.t - 4 second > ./tests/bugs/read-only/bug-1134822-read-only-default-in-graph.t - 4 > second > ./tests/bugs/readdir-ahead/bug-1446516.t - 4 second > ./tests/bugs/readdir-ahead/bug-1439640.t - 4 second > ./tests/bugs/readdir-ahead/bug-1390050.t - 4 second > ./tests/bugs/quota/bug-1287996.t - 4 second > ./tests/bugs/quick-read/bug-846240.t - 4 second > ./tests/bugs/posix/disallow-gfid-volumeid-removexattr.t - 4 second > ./tests/bugs/posix/bug-1619720.t - 4 second > ./tests/bugs/nl-cache/bug-1451588.t - 4 second > ./tests/bugs/nfs/zero-atime.t - 4 second > ./tests/bugs/nfs/subdir-trailing-slash.t - 4 second > ./tests/bugs/nfs/socket-as-fifo.t - 4 second > ./tests/bugs/nfs/showmount-many-clients.t - 4 second > ./tests/bugs/nfs/bug-1210338.t - 4 second > ./tests/bugs/nfs/bug-1166862.t - 4 second > ./tests/bugs/nfs/bug-1161092-nfs-acls.t - 4 second > ./tests/bugs/md-cache/bug-1632503.t - 4 second > ./tests/bugs/glusterfs-server/bug-864222.t - 4 second > ./tests/bugs/glusterfs/bug-1482528.t - 4 second > ./tests/bugs/glusterd/bug-948729/bug-948729-mode-script.t - 4 second > ./tests/bugs/glusterd/bug-948729/bug-948729-force.t - 4 second > ./tests/bugs/glusterd/bug-1482906-peer-file-blank-line.t - 4 second > ./tests/bugs/glusterd/bug-1091935-brick-order-check-from-cli-to-glusterd.t > - 4 second > ./tests/bugs/geo-replication/bug-1296496.t - 4 second > ./tests/bugs/fuse/bug-1336818.t - 4 second > ./tests/bugs/fuse/bug-1283103.t - 4 second > ./tests/bugs/core/io-stats-1322825.t - 4 second > ./tests/bugs/core/bug-834465.t - 4 second > ./tests/bugs/core/bug-1135514-allow-setxattr-with-null-value.t - 4 second > ./tests/bugs/core/949327.t - 4 second > ./tests/bugs/cli/bug-977246.t - 4 second > ./tests/bugs/cli/bug-961307.t - 4 second > ./tests/bugs/cli/bug-1004218.t - 4 second > ./tests/bugs/bug-1138841.t - 4 second > ./tests/bugs/access-control/bug-1387241.t - 4 second > ./tests/bitrot/bug-internal-xattrs-check-1243391.t - 4 second > ./tests/basic/quota-rename.t - 4 second > ./tests/basic/hardlink-limit.t - 4 second > ./tests/basic/ec/dht-rename.t - 4 second > ./tests/basic/distribute/lookup.t - 4 second > ./tests/line-coverage/meta-max-coverage.t - 3 second > ./tests/gfid2path/gfid2path_fuse.t - 3 second > ./tests/bugs/unclassified/bug-991622.t - 3 second > ./tests/bugs/trace/bug-797171.t - 3 second > ./tests/bugs/glusterfs-server/bug-861542.t - 3 second > ./tests/bugs/glusterfs/bug-869724.t - 3 second > ./tests/bugs/glusterfs/bug-860297.t - 3 second > ./tests/bugs/glusterfs/bug-844688.t - 3 second > ./tests/bugs/glusterd/bug-948729/bug-948729.t - 3 second > ./tests/bugs/distribute/bug-1204140.t - 3 second > ./tests/bugs/core/bug-924075.t - 3 second > ./tests/bugs/core/bug-845213.t - 3 second > ./tests/bugs/core/bug-1421721-mpx-toggle.t - 3 second > ./tests/bugs/core/bug-1119582.t - 3 second > ./tests/bugs/core/bug-1117951.t - 3 second > ./tests/bugs/cli/bug-983317-volume-get.t - 3 second > ./tests/bugs/cli/bug-867252.t - 3 second > ./tests/basic/glusterd/check-cloudsync-ancestry.t - 3 second > ./tests/basic/fops-sanity.t - 3 second > ./tests/basic/fencing/test-fence-option.t - 3 second > ./tests/basic/distribute/debug-xattrs.t - 3 second > ./tests/basic/afr/ta-check-locks.t - 3 second > ./tests/line-coverage/volfile-with-all-graph-syntax.t - 2 second > ./tests/line-coverage/some-features-in-libglusterfs.t - 2 second > ./tests/bugs/shard/bug-1261773.t - 2 second > ./tests/bugs/replicate/bug-884328.t - 2 second > ./tests/bugs/readdir-ahead/bug-1512437.t - 2 second > ./tests/bugs/nfs/bug-970070.t - 2 second > ./tests/bugs/nfs/bug-1302948.t - 2 second > ./tests/bugs/logging/bug-823081.t - 2 second > ./tests/bugs/glusterfs-server/bug-889996.t - 2 second > ./tests/bugs/glusterfs/bug-892730.t - 2 second > ./tests/bugs/glusterfs/bug-811493.t - 2 second > ./tests/bugs/glusterd/bug-1085330-and-bug-916549.t - 2 second > ./tests/bugs/distribute/bug-924265.t - 2 second > ./tests/bugs/core/log-bug-1362520.t - 2 second > ./tests/bugs/core/bug-903336.t - 2 second > ./tests/bugs/core/bug-1111557.t - 2 second > ./tests/bugs/cli/bug-969193.t - 2 second > ./tests/bugs/cli/bug-949298.t - 2 second > ./tests/bugs/cli/bug-921215.t - 2 second > ./tests/bugs/cli/bug-1378842-volume-get-all.t - 2 second > ./tests/basic/peer-parsing.t - 2 second > ./tests/basic/md-cache/bug-1418249.t - 2 second > ./tests/basic/afr/arbiter-cli.t - 2 second > ./tests/bugs/replicate/ta-inode-refresh-read.t - 1 second > ./tests/bugs/glusterfs/bug-853690.t - 1 second > ./tests/bugs/cli/bug-764638.t - 1 second > ./tests/bugs/cli/bug-1047378.t - 1 second > ./tests/basic/netgroup_parsing.t - 1 second > ./tests/basic/gfapi/sink.t - 1 second > ./tests/basic/exports_parsing.t - 1 second > ./tests/basic/posixonly.t - 0 second > ./tests/basic/glusterfsd-args.t - 0 second > > > 2 test(s) failed > ./tests/basic/uss.t > ./tests/features/subdir-mount.t > > 0 test(s) generated core > > > 5 test(s) needed retry > ./tests/basic/afr/split-brain-favorite-child-policy.t > ./tests/basic/ec/self-heal.t > ./tests/basic/uss.t > ./tests/basic/volfile-sanity.t > ./tests/features/subdir-mount.t > > Result is 1 > > tar: Removing leading `/' from member names > kernel.core_pattern = /%e-%p.core > Build step 'Execute shell' marked build as failure > _______________________________________________ > maintainers mailing list > maintain...@gluster.org > https://lists.gluster.org/mailman/listinfo/maintainers > > > -- > - Atin (atinm) > _______________________________________________ > maintainers mailing list > maintain...@gluster.org > https://lists.gluster.org/mailman/listinfo/maintainers > -- Amar Tumballi (amarts)
_______________________________________________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/836554017 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/486278655 Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
