Hi Anders, Here is the text that I have put into the README file for this patch. Please give it a look.
--- Return try-again on write requests if file system is unresponsive =================================================================== When underlying file system is unresponsive to pbe write request, all IMM requests that need their changes to be persistent such as creating an IMM object or creating an IMM class likely gets SA_AIS_ERR_TIMEOUT. And in case of putting pbe database on a network file system, there is a high possibility the network file system server is unresponsive for a quite long time that will result in timeout to all write requests at IMM library side. With this patch, IMM introduces two new administrative operations which are used to inform IMM in advance that the file system is being unavailable/unresponsive or not. If IMM is informed that the file system is not being available for write, IMM will return SA_AIS_ERR_TRY_AGAIN *earlier* by local IMMND to the caller [...] -- In my opinion, the idea of this ticket is quite similar to what IMM has been done in the case *PBE mode is enabled but the PBE implementer is not yet attached/PBE process is not started yet* Please note that, the try-again is only returned to callers when IMM has been informed that the underlying file system is currently not able for write. And once the file system is back, IMM must be updated that status and then IMM will operate normally. Regards, Vu > -----Original Message----- > From: and...@acm.org <anders.bjornerst...@telia.com> > Sent: Wednesday, April 10, 2019 7:54 AM > To: gary....@dektech.com.au; hans.nordeb...@ericsson.com; > lennart.l...@ericsson.com; vu.m.ngu...@dektech.com.au > Cc: opensaf-devel@lists.sourceforge.net > Subject: Re: [devel] [PATCH 1/1] imm: return try-again on write requests > while fs is unavailable [#3019] > > Hi again > > I have an additional objection/comment to this ticket. > It concerns the semantics of the returncode ERR_TRY_AGAIN. > > The meaning of this error code is that the operation could NOT be performed > and the server has NOT executed the request. > > That is the server currently rejects the requests but invites the client to > try- > again. > > Crucial here is that if the client descides NOT to retry thre request, then > the > client *knows* that the request was not (will not) be done!! > This is the original and only sensible semantics of ERR_TRY_AGAIN. > > In essence a client that gets TRY_AGAIN is in control of whether or not the > operation shall be retried and may decide NOT to. > > Do you see how this is glaringly missmatched in this proposed ticket ? > This ticket proposes to return TRY_AGAIN for a case where the operation > may *still* be on-going. > THis violates the semantics of TRY_AGAIN. > > ERR_TIMEOUT is verry different. > It simply tells the user that his client side timer expired and it is unknown > at > the client what happened with the request in the server. > It may have succeeded, it may have failed, or it may still be stuck in > processing. > > Receiving ERR_TIMEOUT is a nuisance since the client can not know what > happened. > > But the solution must be to provide tools for the client to find out what > happened. > You need something like a new SaAisCcbDidItCommit API to get arround this > problem at the API level > > But it gets complicated since the CCB Handles are a handle to a chain of > actual ccb-ids (actual transactions). > The ccb-ids are not vissible/exposed over the IMM SAF API. > It is truly a flawed api in its design. > > I dont see any really good solution. > > THe work arround that was in place when I departed was the hidden logic on > the ccb-client side that > tried to recover the result in case of timeout by actually probing the imm ccb > commit log in the database. > But that *hidden* client side retry mechanism had to have a meta timeout > since we could not block the actual client (ccb api user) forever. > And it was (is?) ugly since it overrides the normal/original/general client > side > timeout concept on handles > > /Anders > > > > > > >----Ursprungligt meddelande---- > >Från : anders.bjornerst...@telia.com > >Datum : 2019-04-09 - 10:40 (CEST) > >Till : hans.nordeb...@ericsson.com, vu.m.ngu...@dektech.com.au, > lennart.l...@ericsson.com, gary....@dektech.com.au > >Kopia : opensaf-devel@lists.sourceforge.net > >Ämne : Re: [devel] [PATCH 1/1] imm: return try-again on write requests > while fs is unavailable [#3019] > > > >Hi Vu, > > > >Well I think all these three cases could cause problems. > >The key issues here are the perceived reliability of the IMM service from the > perspective of the user. > > > >If the user of th IMM API *ever* gets the "impression" that a CCB has failed > to commit when in fact it has committed; > >or gets the impression that a CCB has committed when in fact it has not, > then the IMM service is in fact unreliable in > >its core functionality. > > > >So to begin with I would declare that a result of the user being uncertain of > the outcome (commit/abort) of a CCB that has terminated, > >is not a catastrophe. Partivcularly if compared with the user getting the > "impression" of the incorrect result!!!!!!! > >The service and the API must be constructed so that this can never ever ever > ever ever happen. > > > >It is better for the API to provide a result such as ERR_TIMEOUT (which > communicates uncertainty about the outcome) when/if > >the API/library side in fact does not know the outcome. > > > >Case 1 is not very well defined by you because you say the first apply has > "returned", but I have to assume it has NOT returned to the user/library. > >Otherwise why would the user/library retry the apply? I think what you > mean here is that it has been applied/commited in the server but the user > timed-out. > >So case 1 I have to assume is that the CCB commits, but the user/library has > timed out beffore receiving the reply? > >As I remeber it when I designed this I had quite a bit of library code for > >ccb > handling of precisely the case of ERR_TIMEOUT on ccb-apply. > >REmeber that ERR_TIMEOUT is generated locally in the library by a timer in > the library expiring. > >The logic there did NOT attempt to redo the apply, OF COURSE NOT. > >Insteead it sent a separate message(type) probing the server for CCB > *outcome*. > >Is that logic still there? and if so what is the problem. If it has been > >removed > then I would like to know why? > >That code essentially already does what you I believe you are trying to do > here. > >So logic that is already there (if it has not been removed) does in fact do > (internally in the library) some attempts at PROBING the result outcome. > >But it (of course) does so by a separate probing message on outcomme and > NOT by a renewed attempt to apply the same ccb-id!! > > > >There is actually NO POINT in retrying an apply of a CCB when receiving > ERR_TIMEOUT in the client, because you must assume > >that FEVS is reliable and FEVS messages that have reached the server side do > not get lost. Even if the local server crashes, > >the CCB is processed in paralell at all IMMND servers so the CCB outcome > should be retrivable by the server from the sqlite file. > >Conclusion: ERR_TIMEOUT on a ccb-apply truly means that the server is still > processing and that it is worse than a waste of time > >to attempt redos of the apply! It is worse than a waste of time because > from the server side it will look lile the library is faulty, > >trying to appply a ccb that is already being processed or has already been > committed or aborted. > > > > > >Case2 > >Case2 should already be handled by library code > >As you know, CCB handles are used to generate a chain of CCBs. > >That is, a CCB handle is linked to one CCB at a time, but it is not one CCB > period but a potential chain of CCB(ids). > >A successsfull commit of one CCB results in the handle IMPLICITLY opening > (becoming ready) for the next CCB. > >This implicit chaining "feature" of CCBs under a CCB handle is a dubious > feature precisely because it risks confusing > >a user. There is no explicit "begin transaction" handshake between the user > and the service except for the first CCB > >generated by that handle, initiqqlly and after each apply. > >The IMM standard is uniquely sloppy on this in my opinion. > >I know of no other database standard that makes the start of a transaction > "implicit". > >This precisley because you never ever want a user/human- > client/programmer to get confused about the transaction boundary context. > > > >Case3 should be impossible. > >I assume here you are talking about not jjust another thread, but a separate > CCB-handle. > >One CCB-handle should not be used by more than one thread. > >I belive we have explicit warnings rules about this in the user guide. > >The main issue is not thread safety, but that the user/programmer or their > "program" simply gets confused/fooled by > >concurrently operating one ccb-handle over several thereads. I dont see any > good reason for anyone to use one > >ccb-handle in more than one thread. If they do then they better make sure > through their own thread synchronizations > >that the handle is never used concurrently. > > > >In summary. > >I dont know what problem this ticket is trying to fix and the fix it proposes > seems both dangerous and misguided. > > > >If I missed something then dont hesitate to point it out. > >Maybe a lot has happened in the implementation or even the SAF standard > since I left. > >I have not tried to follow it. > > > >I only keep an eye on the IMM ticket process to catch possible dangers as > this case seems to be. > > > >Finally one more thing. > >The main reason that this problem is occuring is the unavailability of the > >file > system. > >As long as the file system is unavailabale the user will have a problem in > obtaining the result of a commit request that > >has gone "to the file system". It goes via the sqlite library of course. > > > > > >/Anders > > > >>----Ursprungligt meddelande---- > >>Från : vu.m.ngu...@dektech.com.au > >>Datum : 2019-04-09 - 04:55 () > >>Till : anders.bjornerst...@telia.com, gary....@dektech.com.au, > lennart.l...@ericsson.com, hans.nordeb...@ericsson.com > >>Kopia : opensaf-devel@lists.sourceforge.net > >>Ämne : Re: [devel] [PATCH 1/1] imm: return try-again on write requests > while fs is unavailable [#3019] > >> > >>Hi AndersBj, > >> > >>Thanks for your comments. > >> > >>However, I need your help to provide more info on your concern so that I > can fully understand it. Which use case below describe exactly your concern > about "additional Apply requests"? > >> > >>Here are some use cases that I can think about the "additional apply > requests". > >> > >>1) "Additional Apply requests" are invoked after the first apply has > >>returned > with SA_AIS_OK (the transaction has been committed). > >>The additional apply should get FAILED_OPERATION as the CCB has been > applied. > >> > >>2) "Additional Apply requests" are invoked after the first apply has > >>returned > due to timeout even the transaction commit is in progress. > >>The additional apply request should get BAD_HANDLE as the ccb has been > cleaned up as the result of timeout from the first one. > >> > >>3) "Additional Apply requests" are invoked concurrently in another thread > while the first one is on-going. > >>The additional apply request should get TRY_AGAIN as in legacy code the > first call is blocking on waiting for reply > >>even such code is *not allowed* though - user should make sure not using > same handle concurrently from different threads. > >> > >>4) Other cases > >> > >>Regards, Vu > >> > >>> -----Original Message----- > >>> From: and...@acm.org <anders.bjornerst...@telia.com> > >>> Sent: Monday, April 8, 2019 6:15 PM > >>> To: gary....@dektech.com.au; vu.m.ngu...@dektech.com.au; > >>> lennart.l...@ericsson.com; hans.nordeb...@ericsson.com > >>> Cc: opensaf-devel@lists.sourceforge.net > >>> Subject: Re: [devel] [PATCH 1/1] imm: return try-again on write requests > >>> while fs is unavailable [#3019] > >>> > >>> Hi, > >>> > >>> I think this ticket (#3019) is erroneous and dangerous. > >>> If you currently have an apply in progress (a commit of a transaction) > that > >>> means you DONT KNOW if the operation will/has succeeded. > >>> What you DO know is that the wrtite/commit most likely will succeed or > fail > >>> at some time. > >>> If you during the uncertainty generate additional Apply requests on the > same > >>> CCB, then you can be pretty sure that those requests > >>> do not "by pass" the blocked request. Instead you can be pretty sure that > the > >>> redundant apply request(s) will return an error code > >>> which most likely will cause the user to conclude that the CCB faile to > apply, > >>> when in fact this is uncertain and likely to be false. > >>> > >>> A disaster in other words. > >>> > >>> /AndersBj > >>> > >>> > >>> >----Ursprungligt meddelande---- > >>> >Från : vu.m.ngu...@dektech.com.au > >>> >Datum : 2019-04-08 - 03:05 () > >>> >Till : hans.nordeb...@ericsson.com, lennart.l...@ericsson.com, > >>> gary....@dektech.com.au > >>> >Kopia : opensaf-devel@lists.sourceforge.net > >>> >Ämne : Re: [devel] [PATCH 1/1] imm: return try-again on write requests > >>> while fs is unavailable [#3019] > >>> > > >>> >Hi all, > >>> > > >>> >Have you had time to review the patch? I will push this ticket by this > >>> >Wednesday if there is no comment/feedback. > >>> > > >>> >Regards, Vu > >>> > > >>> >> -----Original Message----- > >>> >> From: Vu Minh Nguyen <vu.m.ngu...@dektech.com.au> > >>> >> Sent: Tuesday, March 26, 2019 1:19 PM > >>> >> To: hans.nordeb...@ericsson.com; lennart.l...@ericsson.com; > >>> >> gary....@dektech.com.au > >>> >> Cc: opensaf-devel@lists.sourceforge.net; Vu Minh Nguyen > >>> >> <vu.m.ngu...@dektech.com.au> > >>> >> Subject: [PATCH 1/1] imm: return try-again on write requests while fs > is > >>> >> unavailable [#3019] > >>> >> > >>> >> When underlying file system is unresponsive to pbe write request, all > IMM > >>> >> requests that required to be persistent such as creating an IMM object > or > >>> >> creating an IMM class likely gets SA_AIS_ERR_TIMEOUT. > >>> >> > >>> >> This changeset introduces two administrative operations which are > >>> provided > >>> >> to > >>> >> let user inform IMM if the file system is unavailable or not. If the > >>> >> file > >>> >> system is not available, IMM will return SA_AIS_ERR_TRY_AGAIN > earlier to > >>> >> the > >>> >> caller instead of SA_AIS_ERR_TIMEOUT. > >>> >> > >>> >> Besides, a new IMM attribute, saImmFileSystemStatus, is added to > >>> >> SaImmMngt class; the value shows the current status of the file > system > >>> >> according to IMM view. > >>> >> --- > >>> >> src/imm/README | 49 ++++++- > >>> >> .../management/test_saImmOmClassCreate_2.c | 136 > >>> >> ++++++++++++++++++ > >>> >> src/imm/common/immsv_api.h | 5 +- > >>> >> src/imm/config/immsv_classes.xml | 7 + > >>> >> src/imm/immnd/ImmModel.cc | 91 ++++++++++-- > >>> >> src/imm/immnd/ImmModel.h | 1 + > >>> >> src/imm/immnd/immnd_evt.c | 11 +- > >>> >> src/imm/immnd/immnd_init.h | 1 + > >>> >> 8 files changed, 288 insertions(+), 13 deletions(-) > >>> >> > >>> >> diff --git a/src/imm/README b/src/imm/README > >>> >> index 132ee0ac0..8e91b534c 100644 > >>> >> --- a/src/imm/README > >>> >> +++ b/src/imm/README > >>> >> @@ -2969,7 +2969,7 @@ attribute in the object: > >>> >> > >>> >> The following is the shell command: > >>> >> > >>> >> - immadm -o 1 -p opensafImmNostdFlags:SA_UINT32_T:1024 \ > >>> >> + immadm -o 1 -p opensafImmNostdFlags:SA_UINT32_T:512 \ > >>> >> opensafImm=opensafImm,safApp=safImmService > >>> >> > >>> >> This will set bit 10 of the 'opensafImmNostdFlags' runtime attribute > >>> >inside > >>> >> the immsv. > >>> >> @@ -3027,7 +3027,6 @@ expires. > >>> >> To be possible to use this new feature, bit 10 must be set in > >>> >> opensafImmNostdFlags attribute in IMM object. > >>> >> > >>> >> - > >>> >> Provide an admin-operation for re-generating backend database > from > >>> one in > >>> >> RAM > >>> >> > >>> >> > >>> > ============================================================== > >>> >> =============== > >>> >> https://sourceforge.net/p/opensaf/tickets/2940/ > >>> >> @@ -3046,6 +3045,52 @@ back-end database from one in memory > to > >>> keep > >>> >> them both consistent. > >>> >> > >>> >> immadm -o 303 safRdn=immManagement,safApp=safImmService > >>> >> > >>> >> +Return try-again on write requests while file system is unavailable > >>> >> > >>> > +============================================================= > >>> >> ====== > >>> >> +https://sourceforge.net/p/opensaf/tickets/3019 > >>> >> + > >>> >> +When underlying file system is unresponsive to pbe write request, all > >>> IMM > >>> >> +requests that required to be persistent such as creating an IMM > object or > >>> >> +creating an IMM class likely gets SA_AIS_ERR_TIMEOUT. > >>> >> + > >>> >> +Since OpenSAF version 5.19.06, IMM introduces two new > administrative > >>> >> operations > >>> >> +which are used to inform IMM if the file system is unavailable or not. > If > >>> >the > >>> >> +file system is not available for write, IMM will return > >>> >> SA_AIS_ERR_TRY_AGAIN > >>> >> +earlier to the caller instead of SA_AIS_ERR_TIMEOUT. > >>> >> + > >>> >> +Operation ID 400 is used to inform the file system is unavailable: > >>> >> + immadm -o 400 safRdn=immManagement,safApp=safImmService > >>> >> + > >>> >> +and operation ID 401 is used to inform the file system is back: > >>> >> + immadm -o 401 safRdn=immManagement,safApp=safImmService > >>> >> + > >>> >> +Besides, a new runtime IMM attribute, saImmFileSystemStatus, is > added > >>> to > >>> >> +SaImmMngt class; the value shows the current status of the file > system: > >>> >> +saImmFileSystemStatus = 0 means the file system is unavailable and > >>> >> +saImmFileSystemStatus = 1 means the file system is fine for write. > >>> >> + > >>> >> +Fetching the value of that attribute, other services or applications > that > >>> >> +have activities to read/write data from/to the file system may > benefit as > >>> >> well. > >>> >> + > >>> >> +Note that, to use this new feature, bit 11 must be set in > >>> >> opensafImmNostdFlags. > >>> >> + > >>> >> +The following is the shell command to set the 11st bit: > >>> >> + immadm -o 1 -p opensafImmNostdFlags:SA_UINT32_T:1024 \ > >>> >> + opensafImm=opensafImm,safApp=safImmService > >>> >> + > >>> >> +In summary: > >>> >> + Bit 1 controls schema (imm class) changes allowed or not (normally > >>> >off/0). > >>> >> + Bit 2 controls OpenSAF4.1 protocols allowed or not (normally on/1). > >>> >> + Bit 3 controls OpenSAF4.3 protocols allowed or not (normally on/1). > >>> >> + Bit 4 controls 2PBE oneSafe2PBE, see 2PBE feature in OpenSAF4.4 > above > >>> >> (normally off/0). > >>> >> + Bit 5 controls OpenSAF4.5 protocols allowed or not (normally on/1). > >>> >> + Bit 6 controls OpenSAF4.6 protocols allowed or not (normally on/1). > >>> >> + Bit 7 controls OpenSAF4.7 protocols allowed or not (normally on/1). > >>> >> + Bit 8 controls OpenSAF5.0 protocols allowed or not (normally on/1). > >>> >> + Bit 9 controls OpenSAF5.1 protocols allowed or not (normally on/1). > >>> >> + Bit 10 controls OpenSAF5.17.11 protocols allowed or not (normally > >>> on/1). > >>> >> + Bit 11 controls OpenSAF5.19.06 protocols allowed or not (normally > >>> on/1). > >>> >> + > >>> >> ---------------------------------------- > >>> >> DEPENDENCIES > >>> >> ============ > >>> >> diff --git > a/src/imm/apitest/management/test_saImmOmClassCreate_2.c > >>> >> b/src/imm/apitest/management/test_saImmOmClassCreate_2.c > >>> >> index f43884e1b..9e12f4f61 100644 > >>> >> --- a/src/imm/apitest/management/test_saImmOmClassCreate_2.c > >>> >> +++ b/src/imm/apitest/management/test_saImmOmClassCreate_2.c > >>> >> @@ -16,6 +16,7 @@ > >>> >> */ > >>> >> > >>> >> #include "imm/apitest/immtest.h" > >>> >> +#include "imm/common/immsv_api.h" > >>> >> > >>> >> void saImmOmClassCreate_2_01(void) > >>> >> { > >>> >> @@ -709,6 +710,7 @@ void > >>> >> saImmOmClassCreate_SchemaChange_2_02(void) > >>> >> /* > >>> >> * [CONFIG_CLASS] Add default value to default-removed > attribute > >>> >> */ > >>> >> + > >>> >> int schemaChangeEnabled = enableSchemaChange(); > >>> >> safassert(immutil_saImmOmInitialize(&immOmHandle, > >>> >> &immOmCallbacks, &immVersion), > >>> >> SA_AIS_OK); > >>> >> @@ -1534,6 +1536,133 @@ void > >>> >> saImmOmClassCreate_SchemaChange_2_18(void) > >>> >> disableSchemaChange(); > >>> >> } > >>> >> > >>> >> +/* Here are test cases to verify: > >>> >> + 1) SA_AIS_ERR_TRY_AGAIN is returned on IMM class creation > request if > >>> >> file > >>> >> + system is unavailable which is informed to IMM via admop id 400, > and > >>> >also > >>> >> + ensure saImmFileSystemStatus attribute of SaImmMgnt class must > be > >>> >> zero. > >>> >> + Note that, the test is only valid if PBE is enabled, otherwise we > >>> >expect > >>> >> + getting SA_AIS_OK instead. > >>> >> + > >>> >> + 2) SA_AIS_OK is returned on IMM class creation request if the file > >>> >system > >>> >> + is back which is informed to IMM via admop id 401, and also ensure > >>> >that > >>> >> + saImmFileSystemStatus attribute of SaImmMgnt class must be > equal to > >>> >> one. > >>> >> +*/ > >>> >> +const char mgmt_object[] = > >>> >> "safRdn=immManagement,safApp=safImmService"; > >>> >> +static void updateFileSystemStatus(uint64_t operation_id) > >>> >> +{ > >>> >> + SaImmHandleT myOmHandle; > >>> >> + SaImmAdminOwnerHandleT ownerHandle; > >>> >> + const SaImmAdminOwnerNameT adminOwnerName = > "imm_test"; > >>> >> + SaNameT objectName; > >>> >> + const SaNameT *objectNames[] = {&objectName, NULL}; > >>> >> + const SaImmAdminOperationParamsT_2 *params[] = {NULL}; > >>> >> + SaAisErrorT operation_return_value; > >>> >> + > >>> >> + objectName.length = strlen(mgmt_object); > >>> >> + strncpy((char *)objectName.value, mgmt_object, > >>> >> + objectName.length); > >>> >> + > >>> >> + safassert(immutil_saImmOmInitialize(&myOmHandle, NULL, > >>> >> &immVersion), > >>> >> + SA_AIS_OK); > >>> >> + > safassert(immutil_saImmOmAdminOwnerInitialize(myOmHandle, > >>> >> + adminOwnerName, > >>> >> + SA_TRUE, > >>> >&ownerHandle), > >>> >> + SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmAdminOwnerSet(ownerHandle, > >>> >> objectNames, > >>> >> + SA_IMM_ONE), > >>> >> + SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmAdminOperationInvoke_2( > >>> >> + ownerHandle, &objectName, 0, operation_id, > >>> >> + params, &operation_return_value, > >>> >> SA_TIME_ONE_MINUTE), > >>> >> + SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmFinalize(myOmHandle), > SA_AIS_OK); > >>> >> +} > >>> >> + > >>> >> +uint32_t fetchAttributeValue(const char* attribute_name) > >>> >> +{ > >>> >> + SaImmHandleT myOmHandle; > >>> >> + SaImmAccessorHandleT accessorHandle; > >>> >> + SaImmAttrValuesT_2 **attributes; > >>> >> + SaImmAttrNameT attributeNames[] = { > >>> >> + (SaImmAttrNameT)attribute_name, > >>> >> + NULL > >>> >> + }; > >>> >> + SaNameT objectName; > >>> >> + objectName.length = strlen(mgmt_object); > >>> >> + strncpy((char *)objectName.value, mgmt_object, > >>> >> + objectName.length); > >>> >> + > >>> >> + safassert(immutil_saImmOmInitialize(&myOmHandle, NULL, > >>> >> + &immVersion), > >>> >> + SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmAccessorInitialize(myOmHandle, > >>> >> + &accessorHandle), > >>> >> + SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmAccessorGet_2(accessorHandle, > >>> >> &objectName, > >>> >> + attributeNames, > >>> >> &attributes), > >>> >> + SA_AIS_OK); > >>> >> + > >>> >> + uint32_t attribute_value = *(uint32_t > >>> >*)(attributes[0]->attrValues[0]); > >>> >> + safassert(immutil_saImmOmFinalize(myOmHandle), > SA_AIS_OK); > >>> >> + return attribute_value; > >>> >> +} > >>> >> + > >>> >> +bool pbeEnabled(void) > >>> >> +{ > >>> >> + return fetchAttributeValue("saImmRepositoryInit") == 1; > >>> >> +} > >>> >> + > >>> >> +void saImmOmClassCreate_with_fs_unavailable(void) > >>> >> +{ > >>> >> + const SaImmClassNameT className = (SaImmClassNameT) > >>> >> "TestClass"; > >>> >> + SaImmAttrDefinitionT_2 attr1 = { > >>> >> + "rdn", SA_IMM_ATTR_SANAMET, > >>> >> + SA_IMM_ATTR_RUNTIME | SA_IMM_ATTR_RDN | > >>> >> SA_IMM_ATTR_CACHED, > >>> >> + NULL}; > >>> >> + const SaImmAttrDefinitionT_2 *attrDefinitions[] = {&attr1, > NULL}; > >>> >> + > >>> >> + updateFileSystemStatus(SA_IMM_ADMIN_FS_UNAVAILABLE); > >>> >> + safassert(fetchAttributeValue("saImmFileSystemStatus"), 0); > >>> >> + > >>> >> + safassert(immutil_saImmOmInitialize(&immOmHandle, NULL, > >>> >> &immVersion), > >>> >> + SA_AIS_OK); > >>> >> + rc = saImmOmClassCreate_2(immOmHandle, className, > >>> >> SA_IMM_CLASS_RUNTIME, > >>> >> + attrDefinitions); > >>> >> + if (pbeEnabled()) { > >>> >> + test_validate(rc, SA_AIS_ERR_TRY_AGAIN); > >>> >> + } else { > >>> >> + test_validate(rc, SA_AIS_OK); > >>> >> + > safassert(immutil_saImmOmClassDelete(immOmHandle, > >>> >> className), > >>> >> + SA_AIS_OK); > >>> >> + } > >>> >> + safassert(immutil_saImmOmFinalize(immOmHandle), > SA_AIS_OK); > >>> >> + updateFileSystemStatus(SA_IMM_ADMIN_FS_AVAILABLE); > >>> >> + safassert(fetchAttributeValue("saImmFileSystemStatus"), 1); > >>> >> +} > >>> >> + > >>> >> +void saImmOmClassCreate_with_fs_available(void) > >>> >> +{ > >>> >> + const SaImmClassNameT className = (SaImmClassNameT) > >>> >> "TestClass"; > >>> >> + SaImmAttrDefinitionT_2 attr1 = { > >>> >> + "rdn", SA_IMM_ATTR_SANAMET, > >>> >> + SA_IMM_ATTR_RUNTIME | SA_IMM_ATTR_RDN | > >>> >> SA_IMM_ATTR_CACHED, > >>> >> + NULL}; > >>> >> + const SaImmAttrDefinitionT_2 *attrDefinitions[] = {&attr1, > NULL}; > >>> >> + > >>> >> + updateFileSystemStatus(SA_IMM_ADMIN_FS_AVAILABLE); > >>> >> + safassert(fetchAttributeValue("saImmFileSystemStatus"), 1); > >>> >> + > >>> >> + safassert(immutil_saImmOmInitialize(&immOmHandle, > >>> >> &immOmCallbacks, > >>> >> + &immVersion), > >>> >> + SA_AIS_OK); > >>> >> + rc = immutil_saImmOmClassCreate_2(immOmHandle, > className, > >>> >> + SA_IMM_CLASS_RUNTIME, > >>> >> + attrDefinitions); > >>> >> + test_validate(rc, SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmClassDelete(immOmHandle, > className), > >>> >> SA_AIS_OK); > >>> >> + safassert(immutil_saImmOmFinalize(immOmHandle), > SA_AIS_OK); > >>> >> +} > >>> >> + > >>> >> extern void saImmOmClassDescriptionGet_2_01(void); > >>> >> extern void saImmOmClassDescriptionGet_2_02(void); > >>> >> extern void saImmOmClassDescriptionGet_2_03(void); > >>> >> @@ -1724,4 +1853,11 @@ __attribute__((constructor)) static void > >>> >> saImmOmInitialize_constructor(void) > >>> >> test_case_add( > >>> >> 2, saImmOmClassCreate_SchemaChange_2_18, > >>> >> "SchemaChange - SA_AIS_OK, Remove STRONG_DEFAULT > flag from > >>> >> an attribute"); > >>> >> + test_case_add( > >>> >> + 2, saImmOmClassCreate_with_fs_unavailable, > >>> >> + "FileSystemUnavailable - SA_AIS_ERR_TRY_AGAIN, > create > >>> >> class while FS is unavailable"); > >>> >> + test_case_add( > >>> >> + 2, saImmOmClassCreate_with_fs_available, > >>> >> + "FileSystemAvailable - SA_AIS_OK, create class while FS > is > >>> >> available"); > >>> >> + > >>> >> } > >>> >> diff --git a/src/imm/common/immsv_api.h > >>> b/src/imm/common/immsv_api.h > >>> >> index e6d613705..3ab3892d1 100644 > >>> >> --- a/src/imm/common/immsv_api.h > >>> >> +++ b/src/imm/common/immsv_api.h > >>> >> @@ -151,6 +151,7 @@ typedef enum { > >>> >> #define OPENSAF_IMM_FLAG_PRT50_ALLOW 0x00000080 > >>> >> #define OPENSAF_IMM_FLAG_PRT51_ALLOW 0x00000100 > >>> >> #define OPENSAF_IMM_FLAG_PRT51710_ALLOW 0x00000200 > >>> >> +#define OPENSAF_IMM_FLAG_PRT51906_ALLOW 0x00000800 > >>> >> > >>> >> #define OPENSAF_IMM_SERVICE_NAME "safImmService" > >>> >> > >>> >> @@ -159,7 +160,9 @@ typedef enum { > >>> >> SA_IMM_ADMIN_INIT_FROM_FILE = 100, /* Non standard, force > PBE > >>> >> disable. */ > >>> >> SA_IMM_ADMIN_ABORT_CCBS = 202, /* Non standard, abort non > >>> critical > >>> >> CCBs. */ > >>> >> /* Non standard, regenerate pbe database from RAM */ > >>> >> - SA_IMM_ADMIN_REGENERATE_PBE_DB = 303 > >>> >> + SA_IMM_ADMIN_REGENERATE_PBE_DB = 303, > >>> >> + SA_IMM_ADMIN_FS_UNAVAILABLE = 400, > >>> >> + SA_IMM_ADMIN_FS_AVAILABLE = 401, > >>> >> } SaImmMngtAdminOperationT; > >>> >> > >>> >> /* > >>> >> diff --git a/src/imm/config/immsv_classes.xml > >>> >> b/src/imm/config/immsv_classes.xml > >>> >> index 6c20fecda..c6f448e82 100644 > >>> >> --- a/src/imm/config/immsv_classes.xml > >>> >> +++ b/src/imm/config/immsv_classes.xml > >>> >> @@ -46,5 +46,12 @@ > >>> >> <category>SA_CONFIG</category> > >>> >> <flag>SA_WRITABLE</flag> > >>> >> </attr> > >>> >> + <attr> > >>> >> + <name>saImmFileSystemStatus</name> > >>> >> + <type>SA_UINT32_T</type> > >>> >> + <category>SA_RUNTIME</category> > >>> >> + <category>SA_RUNTIME</category> > >>> >> + <flag>SA_CACHED</flag> > >>> >> + </attr> > >>> >> </class> > >>> >> </imm:IMM-contents> > >>> >> diff --git a/src/imm/immnd/ImmModel.cc > >>> b/src/imm/immnd/ImmModel.cc > >>> >> index 4bc3b8522..2cf2649cf 100644 > >>> >> --- a/src/imm/immnd/ImmModel.cc > >>> >> +++ b/src/imm/immnd/ImmModel.cc > >>> >> @@ -594,7 +594,7 @@ static const std::string immManagementDn( > >>> >> "safRdn=immManagement,safApp=safImmService"); > >>> >> static const std::string saImmRepositoryInit("saImmRepositoryInit"); > >>> >> static const std::string saImmOiTimeout("saImmOiTimeout"); > >>> >> - > >>> >> +static const std::string > >>> saImmFileSystemStatus("saImmFileSystemStatus"); > >>> >> static SaImmRepositoryInitModeT immInitMode = > >>> >> SA_IMM_INIT_FROM_FILE; > >>> >> static bool sRegenerateDb = false; > >>> >> > >>> >> @@ -609,6 +609,8 @@ static bool sAbortNonCriticalCcbs = > >>> >> static SaUint32T sTerminatedCcbcount = 0; /* Terminated ccbs count. > >>> >> calculated > >>> >> at cleanTheBasement for > >>> >every > >>> >> second*/ > >>> >> +// Show the status of underlying file system. > >>> >> +static bool sFileSystemAvailable = true; > >>> >> > >>> >> struct AttrFlagIncludes { > >>> >> explicit AttrFlagIncludes(SaImmAttrFlagsT attrFlag) : > >>> >> mFlag(attrFlag) > >>> >{} > >>> >> @@ -1125,6 +1127,10 @@ bool > >>> >> immModel_protocol51710Allowed(IMMND_CB* cb) { > >>> >> return ImmModel::instance(&cb->immModel)- > >protocol51710Allowed(); > >>> >> } > >>> >> > >>> >> +bool immModel_protocol51906Allowed(IMMND_CB* cb) { > >>> >> + return ImmModel::instance(&cb->immModel)- > >>> >protocol51906Allowed(); > >>> >> +} > >>> >> + > >>> >> OsafImmAccessControlModeT > >>> immModel_accessControlMode(IMMND_CB* > >>> >> cb) { > >>> >> return ImmModel::instance(&cb->immModel)- > >accessControlMode(); > >>> >> } > >>> >> @@ -2360,6 +2366,7 @@ bool ImmModel::immNotWritable() { > >>> >> bool ImmModel::immNotPbeWritable(bool isPrtoClient) { > >>> >> SaUint32T dummyCon; > >>> >> unsigned int dummyNode; > >>> >> + > >>> >> /* Not writable => Not persitent writable. */ > >>> >> if (immNotWritable() || is_sync_aborting()) { > >>> >> return true; > >>> >> @@ -2379,7 +2386,7 @@ bool ImmModel::immNotPbeWritable(bool > >>> >> isPrtoClient) { > >>> >> /* immInitMode == SA_IMM_KEEP_REPOSITORY */ > >>> >> /* Check if PBE OI is available and making progress. */ > >>> >> > >>> >> - if (!getPbeOi(&dummyCon, &dummyNode)) { > >>> >> + if (!getPbeOi(&dummyCon, &dummyNode) || > !sFileSystemAvailable) { > >>> >> /* Pbe SHOULD be present but is NOT */ > >>> >> return true; > >>> >> } > >>> >> @@ -4023,6 +4030,27 @@ bool ImmModel::protocol51710Allowed() > { > >>> >> return noStdFlags & OPENSAF_IMM_FLAG_PRT51710_ALLOW; > >>> >> } > >>> >> > >>> >> +bool ImmModel::protocol51906Allowed() { > >>> >> + /* Assume that all nodes are running the same version when loading > */ > >>> >> + if (sImmNodeState == IMM_NODE_LOADING) { > >>> >> + return true; > >>> >> + } > >>> >> + ObjectMap::iterator oi = sObjectMap.find(immObjectDn); > >>> >> + if (oi == sObjectMap.end()) { > >>> >> + return false; > >>> >> + } > >>> >> + > >>> >> + ObjectInfo* immObject = oi->second; > >>> >> + ImmAttrValueMap::iterator avi = > >>> >> + immObject->mAttrValueMap.find(immAttrNostFlags); > >>> >> + osafassert(avi != immObject->mAttrValueMap.end()); > >>> >> + osafassert(!(avi->second->isMultiValued())); > >>> >> + ImmAttrValue* valuep = avi->second; > >>> >> + unsigned int noStdFlags = valuep->getValue_int(); > >>> >> + > >>> >> + return noStdFlags & OPENSAF_IMM_FLAG_PRT51906_ALLOW; > >>> >> +} > >>> >> + > >>> >> bool ImmModel::protocol41Allowed() { > >>> >> // TRACE_ENTER(); > >>> >> ObjectMap::iterator oi = sObjectMap.find(immObjectDn); > >>> >> @@ -5085,6 +5113,8 @@ SaAisErrorT > >>> >> ImmModel::adminOwnerDelete(SaUint32T ownerId, bool hard, > >>> >> flags when cluster is started/loaded or > >>> >> restarted/reloaded. > >>> >> */ > >>> >> ObjectMap::iterator oi = sObjectMap.find(immObjectDn); > >>> >> + auto oi2 = sObjectMap.find(immManagementDn); > >>> >> + > >>> >> if (oi == sObjectMap.end()) { > >>> >> LOG_ER("Failed to find object %s - loading failed", > >>> >> immObjectDn.c_str()); > >>> >> @@ -5093,6 +5123,16 @@ SaAisErrorT > >>> >> ImmModel::adminOwnerDelete(SaUint32T ownerId, bool hard, > >>> >> err = SA_AIS_ERR_NOT_READY; > >>> >> goto done; > >>> >> } > >>> >> + > >>> >> + if (oi2 == sObjectMap.end()) { > >>> >> + LOG_ER("Failed to find object %s - loading failed", > >>> >> + immManagementDn.c_str()); > >>> >> + /* Return special error up to immnd_evt so that NackToNid > can > >>> >be > >>> >> + sent before aborting. */ > >>> >> + err = SA_AIS_ERR_NOT_READY; > >>> >> + goto done; > >>> >> + } > >>> >> + > >>> >> ObjectInfo* immObject = oi->second; > >>> >> ImmAttrValueMap::iterator avi = > >>> >> immObject->mAttrValueMap.find(immAttrNostFlags); > >>> >> @@ -5108,6 +5148,9 @@ SaAisErrorT > >>> >> ImmModel::adminOwnerDelete(SaUint32T ownerId, bool hard, > >>> >> immObject->mAttrValueMap.find(immMaxCcbs); > >>> >> ImmAttrValueMap::iterator avi5 = > >>> >> immObject->mAttrValueMap.find(immMinApplierTimeout); > >>> >> + auto mgnt_object = oi2->second; > >>> >> + auto fs_attr_it = > >>> >> + mgnt_object->mAttrValueMap.find(saImmFileSystemStatus); > >>> >> > >>> >> ImmAttrValue* valuep = (ImmAttrValue*)avi->second; > >>> >> unsigned int noStdFlags = valuep->getValue_int(); > >>> >> @@ -5134,6 +5177,14 @@ SaAisErrorT > >>> >> ImmModel::adminOwnerDelete(SaUint32T ownerId, bool hard, > >>> >> } else { > >>> >> noStdFlags |= OPENSAF_IMM_FLAG_PRT51710_ALLOW; > >>> >> } > >>> >> + > >>> >> + if (fs_attr_it == immObject->mAttrValueMap.end()) { > >>> >> + LOG_NO("protocol51906 is not set for > opensafImmNostdFlags > >>> >> because " > >>> >> + "the new OpenSAF 5.19.06 attribute is not added to > >>> >> " > >>> >> + "SaImmMngt class"); > >>> >> + } else { > >>> >> + noStdFlags |= OPENSAF_IMM_FLAG_PRT51906_ALLOW; > >>> >> + } > >>> >> valuep->setValue_int(noStdFlags); > >>> >> LOG_NO("%s changed to: 0x%x", immAttrNostFlags.c_str(), > >>> >> noStdFlags); > >>> >> /* END Temporary code. */ > >>> >> @@ -12272,6 +12323,11 @@ SaAisErrorT > ImmModel::accessorGet(const > >>> >> ImmsvOmSearchInit* req, > >>> >> // The attribute is cached and the OI is transiently > >>> >> detached > >>> >> op.addAttrValue(*j->second); > >>> >> checkAttribute = true; > >>> >> + } else if ((k->second->mFlags & SA_IMM_ATTR_CACHED) && > >>> >> + (objectName == immManagementDn) && > >>> >> + !(j->second->empty())) { > >>> >> + op.addAttrValue(*j->second); > >>> >> + checkAttribute = true; > >>> >> } else { > >>> >> checkAttribute = false; > >>> >> } > >>> >> @@ -12844,6 +12900,11 @@ SaAisErrorT > >>> >> ImmModel::searchInitialize(ImmsvOmSearchInit* req, > >>> >> // detached > >>> >> op.addAttrValue(*j->second); > >>> >> checkAttribute = true; > >>> >> + } else if ((k->second->mFlags & SA_IMM_ATTR_CACHED) > && > >>> >> + (objectName == immManagementDn) && > >>> >> + !(j->second->empty())) { > >>> >> + op.addAttrValue(*j->second); > >>> >> + checkAttribute = true; > >>> >> } else { > >>> >> checkAttribute = false; > >>> >> } > >>> >> @@ -13798,8 +13859,11 @@ SaAisErrorT > >>> >> ImmModel::admoImmMngtObject(const > >>> ImmsvOmAdminOperationInvoke* > >>> >> req, > >>> >> TRACE_ENTER(); > >>> >> ObjectMap::iterator oi = sObjectMap.find(immManagementDn); > >>> >> if (oi == sObjectMap.end()) { > >>> >> - err = SA_AIS_ERR_NOT_EXIST; > >>> >> - goto done; > >>> >> + return SA_AIS_ERR_NOT_EXIST; > >>> >> + } > >>> >> + > >>> >> + if (req->params != NULL) { > >>> >> + return SA_AIS_ERR_INVALID_PARAM; > >>> >> } > >>> >> > >>> >> immObject = oi->second; > >>> >> @@ -13808,10 +13872,10 @@ SaAisErrorT > >>> >> ImmModel::admoImmMngtObject(const > >>> ImmsvOmAdminOperationInvoke* > >>> >> req, > >>> >> osafassert(!(avi->second->isMultiValued())); > >>> >> valuep = (ImmAttrValue*)avi->second; > >>> >> > >>> >> - if (req->params != NULL) { > >>> >> - err = SA_AIS_ERR_INVALID_PARAM; > >>> >> - goto done; > >>> >> - } > >>> >> + auto fs_attr_iter = immObject- > >>> >> >mAttrValueMap.find(saImmFileSystemStatus); > >>> >> + osafassert(fs_attr_iter != immObject->mAttrValueMap.end()); > >>> >> + auto fs_attr_value = fs_attr_iter->second; > >>> >> + > >>> >> > >>> >> if (req->operationId == SA_IMM_ADMIN_EXPORT) { /* Standard */ > >>> >> err = SA_AIS_ERR_NOT_SUPPORTED; > >>> >> @@ -13833,13 +13897,22 @@ SaAisErrorT > >>> >> ImmModel::admoImmMngtObject(const > >>> ImmsvOmAdminOperationInvoke* > >>> >> req, > >>> >> } else if (req->operationId == > SA_IMM_ADMIN_REGENERATE_PBE_DB) { > >>> >> LOG_NO("Re-generate the pbe database from one in memory."); > >>> >> sRegenerateDb = true; > >>> >> + } else if (req->operationId == SA_IMM_ADMIN_FS_AVAILABLE) { > >>> >> + LOG_NO("Received: immadm -o %u > >>> >> safRdn=immManagement,safApp=safImmService", > >>> >> + SA_IMM_ADMIN_FS_AVAILABLE); > >>> >> + sFileSystemAvailable = true; > >>> >> + fs_attr_value->setValue_int(1); > >>> >> + } else if (req->operationId == SA_IMM_ADMIN_FS_UNAVAILABLE) { > >>> >> + LOG_NO("Received: immadm -o %u > >>> >> safRdn=immManagement,safApp=safImmService", > >>> >> + SA_IMM_ADMIN_FS_UNAVAILABLE); > >>> >> + sFileSystemAvailable = false; > >>> >> + fs_attr_value->setValue_int(0); > >>> >> } else { > >>> >> LOG_NO("Invalid operation ID %llu, for operation on %s", > >>> >> (SaUint64T)req->operationId, immManagementDn.c_str()); > >>> >> err = SA_AIS_ERR_INVALID_PARAM; > >>> >> } > >>> >> > >>> >> -done: > >>> >> TRACE_LEAVE(); > >>> >> return err; > >>> >> } > >>> >> diff --git a/src/imm/immnd/ImmModel.h > b/src/imm/immnd/ImmModel.h > >>> >> index 65e20d305..0b54bfa1d 100644 > >>> >> --- a/src/imm/immnd/ImmModel.h > >>> >> +++ b/src/imm/immnd/ImmModel.h > >>> >> @@ -114,6 +114,7 @@ class ImmModel { > >>> >> bool protocol50Allowed(); > >>> >> bool protocol51Allowed(); > >>> >> bool protocol51710Allowed(); > >>> >> + bool protocol51906Allowed(); > >>> >> bool oneSafe2PBEAllowed(); > >>> >> bool purgeSyncRequest(SaUint32T clientId); > >>> >> bool verifySchemaChange(const std::string& className, ClassInfo* > >>> >> oldClass, > >>> >> diff --git a/src/imm/immnd/immnd_evt.c > b/src/imm/immnd/immnd_evt.c > >>> >> index 0415ebedf..adae1e15a 100644 > >>> >> --- a/src/imm/immnd/immnd_evt.c > >>> >> +++ b/src/imm/immnd/immnd_evt.c > >>> >> @@ -3822,8 +3822,17 @@ static SaAisErrorT > >>> >> immnd_fevs_local_checks(IMMND_CB *cb, IMMSV_FEVS *fevsReq, > >>> >> > >>> >> case IMMND_EVT_A2ND_IMM_ADMOP: > >>> >> case IMMND_EVT_A2ND_IMM_ADMOP_ASYNC: > >>> >> - /* No restrictions at cluster level. */ > >>> >> + { > >>> >> + IMMSV_OM_ADMIN_OP_INVOKE *adm = > >>> >> &frwrd_evt.info.immnd.info.admOpReq; > >>> >> + bool fs_op = (adm->operationId == > >>> >> SA_IMM_ADMIN_FS_AVAILABLE || > >>> >> + adm->operationId == > >>> >> SA_IMM_ADMIN_FS_UNAVAILABLE); > >>> >> + if (!immModel_protocol51906Allowed(cb) && fs_op > && > >>> >> + strcmp(adm->objectName.buf, > >>> >> + > "safRdn=immManagement,safApp=safImmService") == > >>> >> 0) { > >>> >> + error = SA_AIS_ERR_TRY_AGAIN; > >>> >> + } > >>> >> break; > >>> >> + } > >>> >> > >>> >> case IMMND_EVT_A2ND_CLASS_CREATE: > >>> >> if (fevsReq->sender_count != 0x1) { > >>> >> diff --git a/src/imm/immnd/immnd_init.h > b/src/imm/immnd/immnd_init.h > >>> >> index 58ed48329..9a3f70072 100644 > >>> >> --- a/src/imm/immnd/immnd_init.h > >>> >> +++ b/src/imm/immnd/immnd_init.h > >>> >> @@ -345,6 +345,7 @@ bool > immModel_protocol45Allowed(IMMND_CB > >>> >> *cb); > >>> >> bool immModel_protocol46Allowed(IMMND_CB *cb); > >>> >> bool immModel_protocol47Allowed(IMMND_CB *cb); > >>> >> bool immModel_protocol50Allowed(IMMND_CB *cb); > >>> >> +bool immModel_protocol51906Allowed(IMMND_CB *cb); > >>> >> bool immModel_oneSafe2PBEAllowed(IMMND_CB *cb); > >>> >> OsafImmAccessControlModeT > >>> immModel_accessControlMode(IMMND_CB > >>> >> *cb); > >>> >> const char *immModel_authorizedGroup(IMMND_CB *cb); > >>> >> -- > >>> >> 2.19.2 > >>> > > >>> > > >>> > > >>> > > >>> >_______________________________________________ > >>> >Opensaf-devel mailing list > >>> >Opensaf-devel@lists.sourceforge.net > >>> >https://lists.sourceforge.net/lists/listinfo/opensaf-devel > >>> > > >> > >> > >> > >>_______________________________________________ > >>Opensaf-devel mailing list > >>Opensaf-devel@lists.sourceforge.net > >>https://lists.sourceforge.net/lists/listinfo/opensaf-devel > >> > > _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
