Hi Emeric,
I have followed all instructions and I got to the point where HAProxy
starts and does the job using QAT (backend healthchecks work and I
frontend can provide content over HTTPS). The problems starts when
HAProxy gets reloaded. With our current configuration on reload old
HAProxy processes do not exit, so after reload you end up with 2
generations of HAProxy processes: before reload and after reload. I
tried to find out what are conditions in which HAProxy processes get
"stuck" and I was not able to replicate it consistently. In one case it
was related to amount of backend servers with 'ssl' on their line, but
trying to add 'ssl' to some other servers in other place had no effect.
Interestingly in some cases for example with simple configuration (1
frontend + 1 backend) HAProxy produced errors on reload (see attachment)
- in those cases processes rarely got "stuck" even though errors were
present.
/dev/qat_adf_ctl is group writable for the group HAProxy runs on. Any
help to get this fixed / resolved would be welcome.
Regards,
Marcin Deranek
On 3/13/19 12:04 PM, Emeric Brun wrote:
Hi Marcin,
On 3/11/19 4:27 PM, Marcin Deranek wrote:
On 3/11/19 11:51 AM, Emeric Brun wrote:
Mode async is enabled on both sides, server and frontend side.
But on server side, haproxy is using session resuming, so there is a new key
computation (full handshake with RSA/DSA computation) only every 5 minutes
(openssl default value).
You can force to recompute each time setting "no-ssl-reuse" on server line, but
it will add a heavy load for ssl computation on the server.
Indeed, setting no-ssl-reuse makes use of QAT for healthchecks.
Looks like finally we are ready for QAT testing.
Thank you Emeric.
Regards,
Marcin Deranek
I've just re-check and i think you should also enable the 'PKEY_CRYPTO' algo to
the engine
ssl-engine qat algo RSA,DSA,EC,DH,PKEY_CRYPTO
It will enable rhe offloading of the TLS1-PRF you can see there:
# /opt/booking-openssl/bin/openssl engine -c qat
(qat) Reference implementation of QAT crypto engine
[RSA, DSA, DH, AES-128-CBC-HMAC-SHA1, AES-128-CBC-HMAC-SHA256,
AES-256-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA256, TLS1-PRF]
R,
Emeric
2019-04-09T14:22:45.523342+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #1 (61249) forked
2019-04-09T14:22:45.523368+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #2 (61250) forked
2019-04-09T14:22:45.523393+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #3 (61251) forked
2019-04-09T14:22:45.523418+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #4 (61252) forked
2019-04-09T14:22:45.523444+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #5 (61253) forked
2019-04-09T14:22:45.523469+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #6 (61255) forked
2019-04-09T14:22:45.523493+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #7 (61258) forked
2019-04-09T14:22:45.523518+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #8 (61259) forked
2019-04-09T14:22:45.523548+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #9 (61261) forked
2019-04-09T14:22:45.523596+02:00 externallb hapee-lb[60816]: [error]
cpaCyStopInstance() - : Can not get instance info
2019-04-09T14:22:45.523623+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523649+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523674+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523699+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523724+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523749+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523774+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523799+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523823+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523848+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523874+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523899+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.523941+02:00 externallb hapee-lb[60816]: [error]
cpaCyStopInstance() - : Can not get instance info
2019-04-09T14:22:45.523968+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.523993+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524018+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524043+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524068+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524092+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524118+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524143+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524168+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524193+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524218+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524243+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524273+02:00 externallb hapee-lb[60816]: [NOTICE]
098/142244 (60816) : New worker #10 (61263) forked
2019-04-09T14:22:45.524299+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #10 (60834) exited with code 0 (Exit)
2019-04-09T14:22:45.524344+02:00 externallb hapee-lb[60816]: [error]
cpaCyStopInstance() - : Can not get instance info
2019-04-09T14:22:45.524371+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524396+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524421+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524446+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524471+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524496+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524521+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524551+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524577+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524602+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524627+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524652+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524694+02:00 externallb hapee-lb[60816]: [error]
cpaCyStopInstance() - : Can not get instance info
2019-04-09T14:22:45.524721+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524746+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524771+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524796+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524821+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524846+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524871+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524896+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524921+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524946+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.524971+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.524996+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.525026+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525052+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525077+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525102+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 0
2019-04-09T14:22:45.525127+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525151+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 1
2019-04-09T14:22:45.525176+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525201+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 2
2019-04-09T14:22:45.525226+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525252+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525277+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525301+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525326+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525351+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525376+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525401+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525426+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525456+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525482+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525507+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525532+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 0
2019-04-09T14:22:45.525562+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525587+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 1
2019-04-09T14:22:45.525612+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525637+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 2
2019-04-09T14:22:45.525662+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525687+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525712+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525737+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525762+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525788+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525812+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.525837+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.525862+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.525893+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #8 (60832) exited with code 0 (Exit)
2019-04-09T14:22:45.525919+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #9 (60833) exited with code 0 (Exit)
2019-04-09T14:22:45.525944+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #3 (60827) exited with code 0 (Exit)
2019-04-09T14:22:45.525974+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526000+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526025+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526050+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 0
2019-04-09T14:22:45.526075+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526100+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 1
2019-04-09T14:22:45.526125+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526150+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 2
2019-04-09T14:22:45.526175+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526200+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526225+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526250+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526275+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526299+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526324+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526350+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526375+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526405+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526431+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526456+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526481+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 0
2019-04-09T14:22:45.526506+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526531+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 1
2019-04-09T14:22:45.526560+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526585+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 2
2019-04-09T14:22:45.526610+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526635+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526660+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526685+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526710+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526734+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526759+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.526784+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.526809+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.526840+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #7 (60831) exited with code 0 (Exit)
2019-04-09T14:22:45.526866+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #4 (60828) exited with code 0 (Exit)
2019-04-09T14:22:45.526911+02:00 externallb hapee-lb[60816]: [error]
cpaCyStopInstance() - : Can not get instance info
2019-04-09T14:22:45.526938+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.526963+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.526988+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.527013+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.527038+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.527063+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.527088+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.527113+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.527138+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.527163+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.527188+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_GetEnabledServices() - : Failed to get enabled services from ADF
2019-04-09T14:22:45.527213+02:00 externallb hapee-lb[60816]: [error]
SalCtrl_ServiceEventHandler() - : Failed to get enabled services
2019-04-09T14:22:45.527243+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #5 (60829) exited with code 0 (Exit)
2019-04-09T14:22:45.527269+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #1 (60825) exited with code 0 (Exit)
2019-04-09T14:22:45.527299+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527325+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527351+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527376+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 0
2019-04-09T14:22:45.527401+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527426+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 1
2019-04-09T14:22:45.527451+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527476+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to stop subservice SAL for dev 2
2019-04-09T14:22:45.527501+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527525+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.527555+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.527581+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527606+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.527631+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.527656+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
open_dev: Error: Failed to open device /dev/qat_adf_ctl
2019-04-09T14:22:45.527680+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
do_shutdown: Failed to shutdown subservice SAL
2019-04-09T14:22:45.527705+02:00 externallb hapee-lb[60816]: ADF_UIO_PROXY err:
icp_adf_subsystemUnregister: Failed to shutdown subservice SAL.
2019-04-09T14:22:45.527735+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #2 (60826) exited with code 0 (Exit)
2019-04-09T14:22:45.538836+02:00 externallb hapee-lb[60816]: [WARNING]
098/142245 (60816) : Former worker #6 (60830) exited with code 0 (Exit)
