Re: mod_perl Silent Failure, Very Mysterious

[Jie Gao]

Hi Will

I can't put my finger on it, but B::Hooks::OP was last released on 11 Sept 
2011, quite a while ago, and there could be a problem with recent versions of 
perl.


Regards,

Jie 

* William N. Braswell, Jr.  wrote:

> Date: Tue, 22 Nov 2016 23:14:11 -0600
> From: "William N. Braswell, Jr." 
> To: Jie Gao 
> CC: modperl@perl.apache.org
> Subject: Re: mod_perl Silent Failure, Very Mysterious
> 
> Hi Jie,
> If I temporarily disable B::Hooks::OP::Check, as shown below, then it
> tells me it was included from Devel::Declare, which was in turn
> (indireclty) included from Method::Signatures::Simple.
> I am not the author of either Catalyst or ShinyCMS, so unfortunately I
> don't know any more in-depth details off the top of my head.
> Does that answer your question?
> Thanks,~ Will
> wbraswell@cloud-comp0-00:~$ pm_location.sh
> B::Hooks::OP::Check/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm
> wbraswell@cloud-comp0-00:~$ mv
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm.DISABLED
> wbraswell@cloud-comp0-00:~/public_html/cloud-comp0-00.autoparallel.com-latest$
> perl -e 'use ShinyCMS;'Can't locate B/Hooks/OP/Check.pm in @INC (you
> may need to install the B::Hooks::OP::Check module) (@INC contains
> ) at
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare.pm
> line 18.BEGIN failed--compilation aborted at
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare.pm
> line 18.Compilation failed in require at
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/Context/Simple.pm
> line 5.BEGIN failed--compilation aborted at
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/Context/Simple.pm
> line 5.Compilation failed in require at /usr/share/perl/5.22/base.pm
> line 97. ...propagated at /usr/share/perl/5.22/base.pm line 106.BEGIN
> failed--compilation aborted at
> /home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/MethodInstaller/Simple.pm
> line 3.Compilation failed in require at /usr/share/perl/5.22/base.pm
> line 97. ...propagated at /usr/share/perl/5.22/base.pm line 106.BEGIN
> failed--compilation aborted at
> /home/wbraswell/perl5/lib/perl5/Method/Signatures/Simple.pm line
> 19.Compilation failed in require at blib/lib/ShinyCMS.pm line 21.BEGIN
> failed--compilation aborted at blib/lib/ShinyCMS.pm line
> 21.Compilation failed in require at -e line 1.BEGIN
> failed--compilation aborted at -e line 1.
> On 11/22/2016 at 11:03 PM, "Jie Gao"  wrote:Hi William
> 
> Where excatly is B::Hooks::OP::Check required / its Check.xs called?
> Regards,
> Jie 
> 
> * William N. Braswell, Jr.  wrote:
> 
> > Date: Tue, 22 Nov 2016 06:04:36 -0600
> > From: "William N. Braswell, Jr." 
> > To: Jie Gao 
> > CC: modperl@perl.apache.org
> > Subject: RE: mod_perl Silent Failure, Very Mysterious
> > 
> > Hello Again Jie,
> > I have run the modperl_testreport.pl script per your direction, and
> > have attached the output file.
> > Also, I have posted this as a mod_perl issue:
> > 
> > https://rt.cpan.org/Ticket/Display.html?id=118900
> > I am attempting to run an already-bug-free Catalyst application
> under
> > mod_perl, which should work just fine but instead gives a segfault
> in
> > the Check.xs file.
> > The name of the Catalyst application is ShinyCMS, it works just fine
> > using the PSGI stand-alone testing web server as well as fastcgi,
> and
> > mod_perl is the only option which causes a segfault.  I must have
> > mod_perl support for my specific use of ShinyCMS, so I really really
> > need to fix this segfault and get Shiny working under mod_perl. 
> Yes,
> > Shiny is specifically designed to work with mod_perl, as are most
> > Catalyst apps.  In fact, my simple ">" print statements in
> ShinyCMS.pm
> > show that Shiny is totally loading and returning control back to
> > Apache (or mod_perl?) before the actual segfault occurs.
> > My system info:
> > $ apache2 -vServer version: Apache/2.4.18 (Ubuntu)Server built:  
> > 2016-07-14T12:32:26
> > $ perl -vThis is perl 5, version 22, subversion 1 (v5.22.1) built
> for
> > x86_64-linux-gnu-thread-multi(with 58 registered patches, see perl
> -V
> > for more detail)
> > $ pm_version.pl mod_perl2$mod_perl2::VERSION = 2.09
> > $ pm_version.pl B::Hooks::OP::Check$B::Hooks::OP::Check::VERSION =
> > 0.19
> > $ cat /etc/issueUbuntu 16.04.1 LTS
> > I have spent the last week trying all of the following and more, but
> > the segfault persists:
> > 1.  manually-compiled perl v5.22.1 & mod_perl2 v2.09 w/out
> > threads2.  mod_perl2 back to v2.053.  B::Hooks::OP::Check back
> to
> > v0.17 (maybe earlier)4.  move all possible 'use' statements to be
> > called as early as possible in Shiny_dependencies.pm
> 

Re: mod_perl Silent Failure, Very Mysterious

[William N. Braswell, Jr.]

Hi Jie,
If I temporarily disable B::Hooks::OP::Check, as shown below, then it
tells me it was included from Devel::Declare, which was in turn
(indireclty) included from Method::Signatures::Simple.
I am not the author of either Catalyst or ShinyCMS, so unfortunately I
don't know any more in-depth details off the top of my head.
Does that answer your question?
Thanks,~ Will
wbraswell@cloud-comp0-00:~$ pm_location.sh
B::Hooks::OP::Check/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm
wbraswell@cloud-comp0-00:~$ mv
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/B/Hooks/OP/Check.pm.DISABLED
wbraswell@cloud-comp0-00:~/public_html/cloud-comp0-00.autoparallel.com-latest$
perl -e 'use ShinyCMS;'Can't locate B/Hooks/OP/Check.pm in @INC (you
may need to install the B::Hooks::OP::Check module) (@INC contains
) at
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare.pm
line 18.BEGIN failed--compilation aborted at
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare.pm
line 18.Compilation failed in require at
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/Context/Simple.pm
line 5.BEGIN failed--compilation aborted at
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/Context/Simple.pm
line 5.Compilation failed in require at /usr/share/perl/5.22/base.pm
line 97. ...propagated at /usr/share/perl/5.22/base.pm line 106.BEGIN
failed--compilation aborted at
/home/wbraswell/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Devel/Declare/MethodInstaller/Simple.pm
line 3.Compilation failed in require at /usr/share/perl/5.22/base.pm
line 97. ...propagated at /usr/share/perl/5.22/base.pm line 106.BEGIN
failed--compilation aborted at
/home/wbraswell/perl5/lib/perl5/Method/Signatures/Simple.pm line
19.Compilation failed in require at blib/lib/ShinyCMS.pm line 21.BEGIN
failed--compilation aborted at blib/lib/ShinyCMS.pm line
21.Compilation failed in require at -e line 1.BEGIN
failed--compilation aborted at -e line 1.
On 11/22/2016 at 11:03 PM, "Jie Gao"  wrote:Hi William

Where excatly is B::Hooks::OP::Check required / its Check.xs called?
Regards,
Jie 

* William N. Braswell, Jr.  wrote:

> Date: Tue, 22 Nov 2016 06:04:36 -0600
> From: "William N. Braswell, Jr." 
> To: Jie Gao 
> CC: modperl@perl.apache.org
> Subject: RE: mod_perl Silent Failure, Very Mysterious
> 
> Hello Again Jie,
> I have run the modperl_testreport.pl script per your direction, and
> have attached the output file.
> Also, I have posted this as a mod_perl issue:
> 
> https://rt.cpan.org/Ticket/Display.html?id=118900
> I am attempting to run an already-bug-free Catalyst application
under
> mod_perl, which should work just fine but instead gives a segfault
in
> the Check.xs file.
> The name of the Catalyst application is ShinyCMS, it works just fine
> using the PSGI stand-alone testing web server as well as fastcgi,
and
> mod_perl is the only option which causes a segfault.  I must have
> mod_perl support for my specific use of ShinyCMS, so I really really
> need to fix this segfault and get Shiny working under mod_perl. 
Yes,
> Shiny is specifically designed to work with mod_perl, as are most
> Catalyst apps.  In fact, my simple ">" print statements in
ShinyCMS.pm
> show that Shiny is totally loading and returning control back to
> Apache (or mod_perl?) before the actual segfault occurs.
> My system info:
> $ apache2 -vServer version: Apache/2.4.18 (Ubuntu)Server built:  
> 2016-07-14T12:32:26
> $ perl -vThis is perl 5, version 22, subversion 1 (v5.22.1) built
for
> x86_64-linux-gnu-thread-multi(with 58 registered patches, see perl
-V
> for more detail)
> $ pm_version.pl mod_perl2$mod_perl2::VERSION = 2.09
> $ pm_version.pl B::Hooks::OP::Check$B::Hooks::OP::Check::VERSION =
> 0.19
> $ cat /etc/issueUbuntu 16.04.1 LTS
> I have spent the last week trying all of the following and more, but
> the segfault persists:
> 1.  manually-compiled perl v5.22.1 & mod_perl2 v2.09 w/out
> threads2.  mod_perl2 back to v2.053.  B::Hooks::OP::Check back
to
> v0.17 (maybe earlier)4.  move all possible 'use' statements to be
> called as early as possible in Shiny_dependencies.pm
> I also tried Apache v2.2 & mod_perl2 v2.08 & Perl v5.18.2 in
> Ubuntu v14.04, it did not segfault but instead it served up totally
> blank pages with 0 bytes of data.  I have not dug deeper because I
use
> Ubuntu v16.04 in production and I figure it is harder to debug w/out
> an actual segfault to start from.
> I have set up SSH access on a cloud server so you can see and debug
> the segfault directly, all 3 of the usernames below are set as
sudoers
> so please wield your root powers with caution:
> HOSTNAME: cloud-comp0-00.autoparallel.comHOST IP: 
> 64.137.255.53USERNAME: shay OR fmoyer OR jgao OR
modperl_teamPASSWORD:
> check your CPAN e-mail address
> Pertinent 

Re: mod_perl Silent Failure, Very Mysterious

[Jie Gao]

Hi William

Where excatly is B::Hooks::OP::Check required / its Check.xs called?


Regards,


Jie 

* William N. Braswell, Jr.  wrote:

> Date: Tue, 22 Nov 2016 06:04:36 -0600
> From: "William N. Braswell, Jr." 
> To: Jie Gao 
> CC: modperl@perl.apache.org
> Subject: RE: mod_perl Silent Failure, Very Mysterious
> 
> Hello Again Jie,
> I have run the modperl_testreport.pl script per your direction, and
> have attached the output file.
> Also, I have posted this as a mod_perl issue:
> 
> https://rt.cpan.org/Ticket/Display.html?id=118900
> I am attempting to run an already-bug-free Catalyst application under
> mod_perl, which should work just fine but instead gives a segfault in
> the Check.xs file.
> The name of the Catalyst application is ShinyCMS, it works just fine
> using the PSGI stand-alone testing web server as well as fastcgi, and
> mod_perl is the only option which causes a segfault.  I must have
> mod_perl support for my specific use of ShinyCMS, so I really really
> need to fix this segfault and get Shiny working under mod_perl.  Yes,
> Shiny is specifically designed to work with mod_perl, as are most
> Catalyst apps.  In fact, my simple ">" print statements in ShinyCMS.pm
> show that Shiny is totally loading and returning control back to
> Apache (or mod_perl?) before the actual segfault occurs.
> My system info:
> $ apache2 -vServer version: Apache/2.4.18 (Ubuntu)Server built:  
> 2016-07-14T12:32:26
> $ perl -vThis is perl 5, version 22, subversion 1 (v5.22.1) built for
> x86_64-linux-gnu-thread-multi(with 58 registered patches, see perl -V
> for more detail)
> $ pm_version.pl mod_perl2$mod_perl2::VERSION = 2.09
> $ pm_version.pl B::Hooks::OP::Check$B::Hooks::OP::Check::VERSION =
> 0.19
> $ cat /etc/issueUbuntu 16.04.1 LTS
> I have spent the last week trying all of the following and more, but
> the segfault persists:
> 1.  manually-compiled perl v5.22.1 & mod_perl2 v2.09 w/out
> threads2.  mod_perl2 back to v2.053.  B::Hooks::OP::Check back to
> v0.17 (maybe earlier)4.  move all possible 'use' statements to be
> called as early as possible in Shiny_dependencies.pm
> I also tried Apache v2.2 & mod_perl2 v2.08 & Perl v5.18.2 in
> Ubuntu v14.04, it did not segfault but instead it served up totally
> blank pages with 0 bytes of data.  I have not dug deeper because I use
> Ubuntu v16.04 in production and I figure it is harder to debug w/out
> an actual segfault to start from.
> I have set up SSH access on a cloud server so you can see and debug
> the segfault directly, all 3 of the usernames below are set as sudoers
> so please wield your root powers with caution:
> HOSTNAME: cloud-comp0-00.autoparallel.comHOST IP: 
> 64.137.255.53USERNAME: shay OR fmoyer OR jgao OR modperl_teamPASSWORD:
> check your CPAN e-mail address
> Pertinent paths:
> /home/wbraswell/public_html/cloud-comp0-00.autoparallel.com-latest/etc/apache2/sites-available/cloud-comp0-00.autoparallel.com.conf
> Here is an example gdb showing Check.xs as the likely culprit,
> starting with `sudo` and then `source` to load Apache env vars:
> [[[ BEGIN GDB SESSION ]]]
> zefram@cloud-comp0-00:/$ sudo -i...root@cloud-comp0-00:~# source
> /etc/apache2/envvarsroot@cloud-comp0-00:~# gdb
> /usr/sbin/apache2...Reading symbols from /usr/sbin/apache2...(no
> debugging symbols found)...done.
> (gdb) run -k start -XStarting program: /usr/sbin/apache2 -k start
> -X[Thread debugging using libthread_db enabled]Using host libthread_db
> library "/lib/x86_64-linux-gnu/libthread_db.so.1".>: top of
> ShinyCMS.pm>: in ShinyCMS.pm, about to use Moose>: in ShinyCMS.pm,
> about to use Catalyst::Runtime>: in ShinyCMS.pm, about to use
> Catalyst>: in ShinyCMS.pm, about to use CatalystX::RoleApplicator>: in
> ShinyCMS.pm, about to use Method::Signatures::Simple>: in ShinyCMS.pm,
> about to call config()>: in ShinyCMS.pm, have __PACKAGE__ = ShinyCMS>:
> in ShinyCMS.pm, about to call setup()...>: in ShinyCMS.pm, returned
> from setup()>: in ShinyCMS.pm, about to return 1
> Program received signal SIGSEGV, Segmentation fault.0x7fffebabcb10
> in ?? ()(gdb) bt#0  0x7fffebabcb10 in ?? ()#1  0x73e5302b
> in Perl_newUNOP (my_perl=my_perl@entry=0x577ddde0,
> type=type@entry=17, flags=, flags@entry=8192, first=0x562141b8) at
> op.c:4811#2  0x73e54a1d in Perl_newCVREF
> (my_perl=my_perl@entry=0x577ddde0, flags=flags@entry=8192, o=) at
> op.c:9367#3  0x73e8b686 in Perl_yylex
> (my_perl=my_perl@entry=0x577ddde0) at toke.c:6693#4 
> 0x73e97228 in Perl_yyparse
> (my_perl=my_perl@entry=0x577ddde0, gramtype=gramtype@entry=258) at
> perly.c:322...#44 0x5558709f in main ()
> (gdb) startThe program being debugged has been started already.Start
> it from the beginning? (y or n) yTemporary breakpoint 1 at
> 0x555867b0Starting program: /usr/sbin/apache2 -k start -X[Thread
> debugging using libthread_db 

RE: mod_perl Silent Failure, Very Mysterious

[William N. Braswell, Jr.]

Hello Again Jie,
I have run the modperl_testreport.pl script per your direction, and
have attached the output file.
Also, I have posted this as a mod_perl issue:

https://rt.cpan.org/Ticket/Display.html?id=118900
I am attempting to run an already-bug-free Catalyst application under
mod_perl, which should work just fine but instead gives a segfault in
the Check.xs file.
The name of the Catalyst application is ShinyCMS, it works just fine
using the PSGI stand-alone testing web server as well as fastcgi, and
mod_perl is the only option which causes a segfault.  I must have
mod_perl support for my specific use of ShinyCMS, so I really really
need to fix this segfault and get Shiny working under mod_perl.  Yes,
Shiny is specifically designed to work with mod_perl, as are most
Catalyst apps.  In fact, my simple ">" print statements in ShinyCMS.pm
show that Shiny is totally loading and returning control back to
Apache (or mod_perl?) before the actual segfault occurs.
My system info:
$ apache2 -vServer version: Apache/2.4.18 (Ubuntu)Server built:  
2016-07-14T12:32:26
$ perl -vThis is perl 5, version 22, subversion 1 (v5.22.1) built for
x86_64-linux-gnu-thread-multi(with 58 registered patches, see perl -V
for more detail)
$ pm_version.pl mod_perl2$mod_perl2::VERSION = 2.09
$ pm_version.pl B::Hooks::OP::Check$B::Hooks::OP::Check::VERSION =
0.19
$ cat /etc/issueUbuntu 16.04.1 LTS
I have spent the last week trying all of the following and more, but
the segfault persists:
1.  manually-compiled perl v5.22.1 & mod_perl2 v2.09 w/out
threads2.  mod_perl2 back to v2.053.  B::Hooks::OP::Check back to
v0.17 (maybe earlier)4.  move all possible 'use' statements to be
called as early as possible in Shiny_dependencies.pm
I also tried Apache v2.2 & mod_perl2 v2.08 & Perl v5.18.2 in
Ubuntu v14.04, it did not segfault but instead it served up totally
blank pages with 0 bytes of data.  I have not dug deeper because I use
Ubuntu v16.04 in production and I figure it is harder to debug w/out
an actual segfault to start from.
I have set up SSH access on a cloud server so you can see and debug
the segfault directly, all 3 of the usernames below are set as sudoers
so please wield your root powers with caution:
HOSTNAME: cloud-comp0-00.autoparallel.comHOST IP: 
64.137.255.53USERNAME: shay OR fmoyer OR jgao OR modperl_teamPASSWORD:
check your CPAN e-mail address
Pertinent paths:
/home/wbraswell/public_html/cloud-comp0-00.autoparallel.com-latest/etc/apache2/sites-available/cloud-comp0-00.autoparallel.com.conf
Here is an example gdb showing Check.xs as the likely culprit,
starting with `sudo` and then `source` to load Apache env vars:
[[[ BEGIN GDB SESSION ]]]
zefram@cloud-comp0-00:/$ sudo -i...root@cloud-comp0-00:~# source
/etc/apache2/envvarsroot@cloud-comp0-00:~# gdb
/usr/sbin/apache2...Reading symbols from /usr/sbin/apache2...(no
debugging symbols found)...done.
(gdb) run -k start -XStarting program: /usr/sbin/apache2 -k start
-X[Thread debugging using libthread_db enabled]Using host libthread_db
library "/lib/x86_64-linux-gnu/libthread_db.so.1".>: top of
ShinyCMS.pm>: in ShinyCMS.pm, about to use Moose>: in ShinyCMS.pm,
about to use Catalyst::Runtime>: in ShinyCMS.pm, about to use
Catalyst>: in ShinyCMS.pm, about to use CatalystX::RoleApplicator>: in
ShinyCMS.pm, about to use Method::Signatures::Simple>: in ShinyCMS.pm,
about to call config()>: in ShinyCMS.pm, have __PACKAGE__ = ShinyCMS>:
in ShinyCMS.pm, about to call setup()...>: in ShinyCMS.pm, returned
from setup()>: in ShinyCMS.pm, about to return 1
Program received signal SIGSEGV, Segmentation fault.0x7fffebabcb10
in ?? ()(gdb) bt#0  0x7fffebabcb10 in ?? ()#1  0x73e5302b
in Perl_newUNOP (my_perl=my_perl@entry=0x577ddde0,
type=type@entry=17, flags=, flags@entry=8192, first=0x562141b8) at
op.c:4811#2  0x73e54a1d in Perl_newCVREF
(my_perl=my_perl@entry=0x577ddde0, flags=flags@entry=8192, o=) at
op.c:9367#3  0x73e8b686 in Perl_yylex
(my_perl=my_perl@entry=0x577ddde0) at toke.c:6693#4 
0x73e97228 in Perl_yyparse
(my_perl=my_perl@entry=0x577ddde0, gramtype=gramtype@entry=258) at
perly.c:322...#44 0x5558709f in main ()
(gdb) startThe program being debugged has been started already.Start
it from the beginning? (y or n) yTemporary breakpoint 1 at
0x555867b0Starting program: /usr/sbin/apache2 -k start -X[Thread
debugging using libthread_db enabled]Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Temporary breakpoint 1, 0x555867b0 in main ()
(gdb) break perl_parseFunction "perl_parse" not defined.Make
breakpoint pending on future shared library load? (y or [n])
yBreakpoint 2 (perl_parse) pending.
(gdb) continueContinuing.
Breakpoint 2, perl_parse (my_perl=0x5581ae50,
xsinit=0x7420a280, argc=2, argv=0x77f88070, env=0x0) at
perl.c:14731473perl.c: No such file or directory.
(gdb) watch -l PL_check[17]Hardware watchpoint 3: -location
PL_check[17]
(gdb)