Re: [Gluster-devel] trash.t failure with brick multiplexing [Was Re: Build failed in Jenkins: regression-test-with-multiplex #574]

2018-01-02 Thread Milind Changire 
On Tue, Jan 2, 2018 at 5:32 PM, Milind Changire  wrote:

>
> On Tue, Jan 2, 2018 at 10:44 AM, Atin Mukherjee 
> wrote:
>
>>
>>
>> On Thu, Dec 21, 2017 at 7:27 PM, Atin Mukherjee 
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec 20, 2017 at 11:58 AM, Atin Mukherjee 
>>> wrote:
>>>
 ./tests/bugs/glusterd/bug-1230121-replica_subvol_count_correct_cal.t

>>>
>>> Unfortunately the above is passing in my setup. I'll be checking the
>>> logs to see if I can figure out the issue.
>>>
>>> ./tests/features/trash.t

>>>
>>> Rebalance fails here consistently when brick mux is enabled with the
>>> following message:
>>>
>>> [2017-12-21 13:55:31.881268] I [MSGID: 109081]
>>> [dht-common.c:5538:dht_setxattr] 0-patchy-dht: fixing the layout of /
>>> [2017-12-21 13:55:31.881289] W [MSGID: 109016]
>>> [dht-selfheal.c:1930:dht_fix_layout_of_directory] 0-patchy-dht: Layout
>>> fix failed: 1 subvolume(s) are down. Skipping fix layout. path:/
>>> gfid:----0001
>>> [2017-12-21 13:55:31.881525] E [MSGID: 109026]
>>> [dht-rebalance.c::gf_defrag_start_crawl] 0-patchy-dht: fix layout
>>> on / failed [Transport endpoint is not connected]
>>>
>>> When I revert the commit 56e5fda "rpc: merge ssl infra with epoll infra"
>>> the test passes through.
>>> I tend to believe that the other failures especially ssl-ciphers.t
>>> 
>>> from fstat.gluster.log could be due to the same patch.
>>>
>>> @Milind - Need your attention here.
>>>
>>
>> Since the test failures are constant, I propose to revert the commit
>> 56e5fda from mainline till these issues are looked at.
>>
>
> After setting the SSL CRL path to NULL, if the volume is restarted, the
> test tests/features/ssl-ciphers.t passes in multiplex mode.
> SSL socket options aren't handled in the reconfigure() entry-point for
> the socket transport.
> I'll post a patch soon to fix the test case.
>

Or maybe not.
Need to take a closer look here.


>
>>
>>>
 The above two are new failures since day before yesterday. Job link is
 at https://build.gluster.org/job/regression-test-with-multiplex
 /574/consoleFull .



 -- Forwarded message --
 From: 
 Date: Wed, Dec 20, 2017 at 12:24 AM
 Subject: Build failed in Jenkins: regression-test-with-multiplex #574
 To: maintain...@gluster.org, amukh...@redhat.com, j...@pl.atyp.us,
 jaher...@redhat.com, jda...@fb.com, kdhan...@redhat.com,
 rgowd...@redhat.com, khire...@redhat.com, ama...@redhat.com,
 nbala...@redhat.com, nig...@redhat.com, srang...@redhat.com


 See 

 Changes:

 [Jeff Darcy] protocol/server: add dump_metrics method

 [Jeff Darcy] snapshot: Fix several coverity issues in
 glusterd-snapshot-utils.c

 [Kotresh H R] feature/bitrot: remove internal xattrs from lookup cbk

 --
 [...truncated 784.50 KB...]
 ./tests/bugs/nfs/bug-904065.t  -  9 second
 ./tests/bugs/nfs/bug-1157223-symlink-mounting.t  -  9 second
 ./tests/bugs/md-cache/bug-1211863.t  -  9 second
 ./tests/bugs/glusterd/bug-949930.t  -  9 second
 ./tests/bugs/glusterd/bug-1420637-volume-sync-fix.t  -  9 second
 ./tests/bugs/glusterd/bug-1121584-brick-existing-validation-
 for-remove-brick-status-stop.t  -  9 second
 ./tests/bugs/glusterd/bug-1104642.t  -  9 second
 ./tests/bugs/distribute/bug-961615.t  -  9 second
 ./tests/bugs/distribute/bug-1247563.t  -  9 second
 ./tests/bugs/distribute/bug-1086228.t  -  9 second
 ./tests/bugs/cli/bug-1087487.t  -  9 second
 ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
 -  9 second
 ./tests/basic/tier/ctr-rename-overwrite.t  -  9 second
 ./tests/basic/stats-dump.t  -  9 second
 ./tests/basic/quota_aux_mount.t  -  9 second
 ./tests/basic/inode-quota-enforcing.t  -  9 second
 ./tests/basic/fop-sampling.t  -  9 second
 ./tests/gfid2path/get-gfid-to-path.t  -  8 second
 ./tests/bugs/upcall/bug-1227204.t  -  8 second
 ./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t  -  8
 second
 ./tests/bugs/glusterfs/bug-902610.t  -  8 second
 ./tests/bugs/glusterd/bug-889630.t  -  8 second
 ./tests/bugs/glusterd/bug-859927.t  -  8 second
 ./tests/bugs/glusterd/bug-1323287-real_path-handshake-test.t  -  8
 second
 ./tests/bugs/glusterd/bug-1213295-snapd-svc-uninitialized.t  -  8
 second
 ./tests/bugs/glusterd/bug-1109741-auth-mgmt-handshake.t  -  8 second
 ./tests/bugs/glusterd/bug-1046308.t  -  8 second
 ./tests/bugs/ec/bug-1179050.t  -  8 second
 ./tests/bugs/distribute/bug-1122443.t  -  8 second

Re: [Gluster-devel] trash.t failure with brick multiplexing [Was Re: Build failed in Jenkins: regression-test-with-multiplex #574]

2018-01-02 Thread Milind Changire 
On Tue, Jan 2, 2018 at 10:44 AM, Atin Mukherjee  wrote:

>
>
> On Thu, Dec 21, 2017 at 7:27 PM, Atin Mukherjee 
> wrote:
>
>>
>>
>> On Wed, Dec 20, 2017 at 11:58 AM, Atin Mukherjee 
>> wrote:
>>
>>> ./tests/bugs/glusterd/bug-1230121-replica_subvol_count_correct_cal.t
>>>
>>
>> Unfortunately the above is passing in my setup. I'll be checking the logs
>> to see if I can figure out the issue.
>>
>> ./tests/features/trash.t
>>>
>>
>> Rebalance fails here consistently when brick mux is enabled with the
>> following message:
>>
>> [2017-12-21 13:55:31.881268] I [MSGID: 109081]
>> [dht-common.c:5538:dht_setxattr] 0-patchy-dht: fixing the layout of /
>> [2017-12-21 13:55:31.881289] W [MSGID: 109016]
>> [dht-selfheal.c:1930:dht_fix_layout_of_directory] 0-patchy-dht: Layout
>> fix failed: 1 subvolume(s) are down. Skipping fix layout. path:/
>> gfid:----0001
>> [2017-12-21 13:55:31.881525] E [MSGID: 109026]
>> [dht-rebalance.c::gf_defrag_start_crawl] 0-patchy-dht: fix layout on
>> / failed [Transport endpoint is not connected]
>>
>> When I revert the commit 56e5fda "rpc: merge ssl infra with epoll infra"
>> the test passes through.
>> I tend to believe that the other failures especially ssl-ciphers.t
>> 
>> from fstat.gluster.log could be due to the same patch.
>>
>> @Milind - Need your attention here.
>>
>
> Since the test failures are constant, I propose to revert the commit
> 56e5fda from mainline till these issues are looked at.
>

After setting the SSL CRL path to NULL, if the volume is restarted, the
test tests/features/ssl-ciphers.t passes in multiplex mode.
SSL socket options aren't handled in the reconfigure() entry-point for the
socket transport.
I'll post a patch soon to fix the test case.


>
>>
>>> The above two are new failures since day before yesterday. Job link is
>>> at https://build.gluster.org/job/regression-test-with-multiplex
>>> /574/consoleFull .
>>>
>>>
>>>
>>> -- Forwarded message --
>>> From: 
>>> Date: Wed, Dec 20, 2017 at 12:24 AM
>>> Subject: Build failed in Jenkins: regression-test-with-multiplex #574
>>> To: maintain...@gluster.org, amukh...@redhat.com, j...@pl.atyp.us,
>>> jaher...@redhat.com, jda...@fb.com, kdhan...@redhat.com,
>>> rgowd...@redhat.com, khire...@redhat.com, ama...@redhat.com,
>>> nbala...@redhat.com, nig...@redhat.com, srang...@redhat.com
>>>
>>>
>>> See >> x/574/display/redirect?page=changes>
>>>
>>> Changes:
>>>
>>> [Jeff Darcy] protocol/server: add dump_metrics method
>>>
>>> [Jeff Darcy] snapshot: Fix several coverity issues in
>>> glusterd-snapshot-utils.c
>>>
>>> [Kotresh H R] feature/bitrot: remove internal xattrs from lookup cbk
>>>
>>> --
>>> [...truncated 784.50 KB...]
>>> ./tests/bugs/nfs/bug-904065.t  -  9 second
>>> ./tests/bugs/nfs/bug-1157223-symlink-mounting.t  -  9 second
>>> ./tests/bugs/md-cache/bug-1211863.t  -  9 second
>>> ./tests/bugs/glusterd/bug-949930.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1420637-volume-sync-fix.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1121584-brick-existing-validation-
>>> for-remove-brick-status-stop.t  -  9 second
>>> ./tests/bugs/glusterd/bug-1104642.t  -  9 second
>>> ./tests/bugs/distribute/bug-961615.t  -  9 second
>>> ./tests/bugs/distribute/bug-1247563.t  -  9 second
>>> ./tests/bugs/distribute/bug-1086228.t  -  9 second
>>> ./tests/bugs/cli/bug-1087487.t  -  9 second
>>> ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
>>> -  9 second
>>> ./tests/basic/tier/ctr-rename-overwrite.t  -  9 second
>>> ./tests/basic/stats-dump.t  -  9 second
>>> ./tests/basic/quota_aux_mount.t  -  9 second
>>> ./tests/basic/inode-quota-enforcing.t  -  9 second
>>> ./tests/basic/fop-sampling.t  -  9 second
>>> ./tests/gfid2path/get-gfid-to-path.t  -  8 second
>>> ./tests/bugs/upcall/bug-1227204.t  -  8 second
>>> ./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t  -  8
>>> second
>>> ./tests/bugs/glusterfs/bug-902610.t  -  8 second
>>> ./tests/bugs/glusterd/bug-889630.t  -  8 second
>>> ./tests/bugs/glusterd/bug-859927.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1323287-real_path-handshake-test.t  -  8
>>> second
>>> ./tests/bugs/glusterd/bug-1213295-snapd-svc-uninitialized.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1109741-auth-mgmt-handshake.t  -  8 second
>>> ./tests/bugs/glusterd/bug-1046308.t  -  8 second
>>> ./tests/bugs/ec/bug-1179050.t  -  8 second
>>> ./tests/bugs/distribute/bug-1122443.t  -  8 second
>>> ./tests/bugs/distribute/bug-1088231.t  -  8 second
>>> ./tests/bugs/changelog/bug-1208470.t  -  8 second
>>> ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t  -
>>> 8 second
>>> ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t

Re: [Gluster-devel] trash.t failure with brick multiplexing [Was Re: Build failed in Jenkins: regression-test-with-multiplex #574]

2018-01-01 Thread Atin Mukherjee 
On Thu, Dec 21, 2017 at 7:27 PM, Atin Mukherjee  wrote:

>
>
> On Wed, Dec 20, 2017 at 11:58 AM, Atin Mukherjee 
> wrote:
>
>> ./tests/bugs/glusterd/bug-1230121-replica_subvol_count_correct_cal.t
>>
>
> Unfortunately the above is passing in my setup. I'll be checking the logs
> to see if I can figure out the issue.
>
> ./tests/features/trash.t
>>
>
> Rebalance fails here consistently when brick mux is enabled with the
> following message:
>
> [2017-12-21 13:55:31.881268] I [MSGID: 109081] 
> [dht-common.c:5538:dht_setxattr]
> 0-patchy-dht: fixing the layout of /
> [2017-12-21 13:55:31.881289] W [MSGID: 109016]
> [dht-selfheal.c:1930:dht_fix_layout_of_directory] 0-patchy-dht: Layout
> fix failed: 1 subvolume(s) are down. Skipping fix layout. path:/
> gfid:----0001
> [2017-12-21 13:55:31.881525] E [MSGID: 109026] 
> [dht-rebalance.c::gf_defrag_start_crawl]
> 0-patchy-dht: fix layout on / failed [Transport endpoint is not connected]
>
> When I revert the commit 56e5fda "rpc: merge ssl infra with epoll infra"
> the test passes through.
> I tend to believe that the other failures especially ssl-ciphers.t
> 
> from fstat.gluster.log could be due to the same patch.
>
> @Milind - Need your attention here.
>

Since the test failures are constant, I propose to revert the commit
56e5fda from mainline till these issues are looked at.


>
>> The above two are new failures since day before yesterday. Job link is at
>> https://build.gluster.org/job/regression-test-with-multiplex
>> /574/consoleFull .
>>
>>
>>
>> -- Forwarded message --
>> From: 
>> Date: Wed, Dec 20, 2017 at 12:24 AM
>> Subject: Build failed in Jenkins: regression-test-with-multiplex #574
>> To: maintain...@gluster.org, amukh...@redhat.com, j...@pl.atyp.us,
>> jaher...@redhat.com, jda...@fb.com, kdhan...@redhat.com,
>> rgowd...@redhat.com, khire...@redhat.com, ama...@redhat.com,
>> nbala...@redhat.com, nig...@redhat.com, srang...@redhat.com
>>
>>
>> See > x/574/display/redirect?page=changes>
>>
>> Changes:
>>
>> [Jeff Darcy] protocol/server: add dump_metrics method
>>
>> [Jeff Darcy] snapshot: Fix several coverity issues in
>> glusterd-snapshot-utils.c
>>
>> [Kotresh H R] feature/bitrot: remove internal xattrs from lookup cbk
>>
>> --
>> [...truncated 784.50 KB...]
>> ./tests/bugs/nfs/bug-904065.t  -  9 second
>> ./tests/bugs/nfs/bug-1157223-symlink-mounting.t  -  9 second
>> ./tests/bugs/md-cache/bug-1211863.t  -  9 second
>> ./tests/bugs/glusterd/bug-949930.t  -  9 second
>> ./tests/bugs/glusterd/bug-1420637-volume-sync-fix.t  -  9 second
>> ./tests/bugs/glusterd/bug-1121584-brick-existing-validation-
>> for-remove-brick-status-stop.t  -  9 second
>> ./tests/bugs/glusterd/bug-1104642.t  -  9 second
>> ./tests/bugs/distribute/bug-961615.t  -  9 second
>> ./tests/bugs/distribute/bug-1247563.t  -  9 second
>> ./tests/bugs/distribute/bug-1086228.t  -  9 second
>> ./tests/bugs/cli/bug-1087487.t  -  9 second
>> ./tests/bugs/bitrot/1209752-volume-status-should-show-bitrot-scrub-info.t
>> -  9 second
>> ./tests/basic/tier/ctr-rename-overwrite.t  -  9 second
>> ./tests/basic/stats-dump.t  -  9 second
>> ./tests/basic/quota_aux_mount.t  -  9 second
>> ./tests/basic/inode-quota-enforcing.t  -  9 second
>> ./tests/basic/fop-sampling.t  -  9 second
>> ./tests/gfid2path/get-gfid-to-path.t  -  8 second
>> ./tests/bugs/upcall/bug-1227204.t  -  8 second
>> ./tests/bugs/replicate/bug-1448804-check-quorum-type-values.t  -  8
>> second
>> ./tests/bugs/glusterfs/bug-902610.t  -  8 second
>> ./tests/bugs/glusterd/bug-889630.t  -  8 second
>> ./tests/bugs/glusterd/bug-859927.t  -  8 second
>> ./tests/bugs/glusterd/bug-1323287-real_path-handshake-test.t  -  8 second
>> ./tests/bugs/glusterd/bug-1213295-snapd-svc-uninitialized.t  -  8 second
>> ./tests/bugs/glusterd/bug-1109741-auth-mgmt-handshake.t  -  8 second
>> ./tests/bugs/glusterd/bug-1046308.t  -  8 second
>> ./tests/bugs/ec/bug-1179050.t  -  8 second
>> ./tests/bugs/distribute/bug-1122443.t  -  8 second
>> ./tests/bugs/distribute/bug-1088231.t  -  8 second
>> ./tests/bugs/changelog/bug-1208470.t  -  8 second
>> ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t  -  8
>> second
>> ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t
>> -  8 second
>> ./tests/bitrot/br-stub.t  -  8 second
>> ./tests/basic/volume-status.t  -  8 second
>> ./tests/basic/quota-nfs.t  -  8 second
>> ./tests/basic/glusterd/arbiter-volume-probe.t  -  8 second
>> ./tests/basic/ec/ec-anonymous-fd.t  -  8 second
>> ./tests/gfid2path/block-mount-access.t  -  7 second
>> ./tests/features/ssl-authz.t  -  7 second
>> ./tests/bugs/upcall/bug-1458127.t  -  7 second
>> ./tests/bugs/tier/bug-1205545-CTR-and-trash-integration.t  -