Continuous query not transactional ?
Hello team, This is in continuation to these posts on the ignite users http://apache-ignite-users.70518.x6.nabble.com/Continuous-query-not-transactional-td34270.html and http://apache-ignite-users.70518.x6.nabble.com/Lag-before-records-are-visible-after-transaction-commit-tp33787p33861.html Does this mean we could get dirty reads as updates in continuous query ? i.e. for eg if the code is as below: 1. Start transaction 2. update records of cache a 3. update records of cache b 4. update records for cache c 5. commit if update of cache a succeeds , but update of cache b fails, will the local listener for continuous query for 'cache a' get an update ? regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
IGNITE-12781 - ache_put event contains the node id instead of the remote_client subject id
Posting this here, as guided in the ignite users group.
Hi Team,
1. I noticed that this issue (
https://issues.apache.org/jira/browse/IGNITE-12781) is not resolved in
2.8.1.
Could you guide how can we get audit information if a cache record
modification is done on dbeaver and the cache_put event contains the node id
instead of the remote_client subject id ?
Please note this is a blocker issue for us to use Apache Ignite , since we
use dbeaver to update records sometimes.
/*Could we kindly ask this to be included in the next
release.*/
2. Even if the cache_put event did contain the remote_client user id , how
are we supposed to fetch it from the auditstoragespi ?
The below link mentions
http://apache-ignite-users.70518.x6.nabble.com/JDBC-thin-client-incorrect-security-context-td31354.html
public class EventStorageSpi extends IgniteSpiAdapter implements
EventStorageSpi {
@LoggerResource
private IgniteLogger log;
@Override
public Collection localEvents(IgnitePredicate p)
{
return null;
}
@Override
public void record(Event evt) throws IgniteSpiException {
if (evt.type() == EVT_MANAGEMENT_TASK_STARTED) {
TaskEvent taskEvent = (TaskEvent) evt;
SecuritySubject subj = taskEvent.subjectId() != null
?
getSpiContext().authenticatedSubject(taskEvent.subjectId())
: null;
log.info("Management task started: [" +
"name=" + taskEvent.taskName() + ", " +
"eventNode=" + taskEvent.node() + ", " +
"timestamp=" + taskEvent.timestamp() + ", " +
"info=" + taskEvent.message() + ", " +
"subjectId=" + taskEvent.subjectId() + ", " +
"secureSubject=" + subj +
"]");
}
}
@Override
public void spiStart(@Nullable String igniteInstanceName) throws
IgniteSpiException {
/* No-op. */
}
@Override
public void spiStop() throws IgniteSpiException {
/* No-op. */
}
}
IgniteSpiContext exposes authenticatedSubject which according to some
discussions gets the subject *only for node* . (
http://apache-ignite-developers.2346864.n4.nabble.com/Security-Subject-of-thin-client-on-remote-nodes-td46029.html#a46412
)
/*securityContext(uuid ) was added to the GridSecurityProcessor to get the
securitycontext of the thin client. However this is not exposed via the
IgniteSpiContext.*/
regards,
Veena.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Server Node comes down with : (err) Failed to notify listener: GridDhtTxPrepareFuture Error
>>As a temporary solution you can stop relying on peer class loading for continuous queries and provide the code of remote filters to the classpath of server nodes. Yes.. I was thinking of a solution on similar lines. Thank you for the reply. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Server Node comes down with : (err) Failed to notify listener: GridDhtTxPrepareFuture Error
Raised this jira : https://issues.apache.org/jira/browse/IGNITE-12784 Observed in 2.7.6. Unable to easily test in 2.8.0 because of other issues. One of them being - http://apache-ignite-users.70518.x6.nabble.com/2-8-0-JDBC-Thin-Client-Unable-to-load-the-tables-via-DBeaver-td31681.html Please note this happens -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Security Subject of thin client on remote nodes
HI , Created this jira : https://issues.apache.org/jira/browse/IGNITE-12781 regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Server Node comes down with : (err) Failed to notify listener: GridDhtTxPrepareFuture Error
Hi , Did anyone get a chance to look at this ? Summary of the issue I am facing. : We have a 3 node server cluster A 4th node joins as a client with a continuous query on a Table A( Transaction_mode = transactional ). Now If I bring the client down and issue an update to the Table A within failureDetectionTimeout 3 , I get the following error and */this error brings the server down/*: "(err) Failed to notify listener: GridDhtTxPrepareFuture Error" === Basically the server , tries to update the record on the Table A, and tries to notify Client since it had registered a continuous query for Table A. But since the Client Node has been brought down, it undeploys the remotefilterfactory lambda. Hence the server is no longer able to complete the transaction . */This also brings the server down./ * regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Security Subject of thin client on remote nodes
>>2. Method GridSecurityProcessor#authenticatedSubject returns authenticated *node subject* (from JavaDoc). I dont think that is true. Nothing stops an implementer to return a Remote Client Security Subject. The primary issue is how to link the event which contains the remote node uuid ( but was originated by a remote client ) , to the originator remote client. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Server Node comes down with : (err) Failed to notify listener: GridDhtTxPrepareFuture Error
Hi Team, We have a 3 node server cluster A 4th node joins as a client with a continuous query on a Table A( Transaction_mode = transactional ). Now If I bring the client down and issue an update to the Table A within failureDetectionTimeout 3 , I get the following error and */this error brings the server down/*: "(err) Failed to notify listener: GridDhtTxPrepareFuture Error" === Basically the server , tries to update the record on the Table A, and tries to notify Client since it had registered a continuous query for Table A. But since the Client Node has been brought down, and it undeploys the remotefilterfactory lambda, and the server is no longer able to complete the transaction . */This also brings the server down./ * How can I resolve this issue ? === Please find the complete stack trace for this error : [12:14:12] (err) Failed to notify listener: GridDhtTxPrepareFuture [futId=0a69e79c071-93faf34d-a776-4166-9f3b-4b5a0f54b8f9, err=null, replied=1, mapped=1, req=GridNearTxPrepareRequest [futId=4250e79c071-51438f4f-c061-45f7-b34e-57c90f2055e9, miniId=1, topVer=AffinityTopologyVersion [topVer=7, minorTopVer=0], subjId=da486d0b-36a1-43d4-b05b-47d126fd880e, taskNameHash=0, flags=[implicitSingle], super=GridDistributedTxPrepareRequest [threadId=382, concurrency=OPTIMISTIC, isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=195408427, order=1583928843624, nodeOrder=1], timeout=1000, reads=null, writes=[IgniteTxEntry [key=ABCKEY [idHash=1413504800, hash=-1419375634, VALUETYPE=somevaluetype, NAME=TEST4375234], cacheId=-1512899836, txKey=IgniteTxKey [key=ABCKEY [idHash=1413504800, hash=-1419375634, VALUETYPE=somevaluetype, NAME=TEST4375234], cacheId=-1512899836], val=[op=CREATE, val=ABC [idHash=108633195, hash=-965148880, ACTIVE=true, MODIFICATIONDATE=2020-02-03 18:29:03.501, VALUETYPE=null, SCHEMAREF=null, VALUE=DEV, MACHINENAME=null, COMMENT=null, NAME=null, APPLICATIONNAME=null, SCHEMANAME=null, KEYNAME=ENVIRONMENT, USERNAME=null, INTERNALVERSION=null, MODIFICATIONTYPE=null]], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null, filters=[o.a.i.i.processors.cache.CacheEntrySerializablePredicate@388c822f], filtersPassed=false, filtersSet=false, entry=GridDhtCacheEntry [rdrs=[], part=136, super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=ABCKEY [idHash=1413504800, hash=-1419375634, VALUETYPE=somevaluetype, NAME=TEST4375234], val=null, ver=GridCacheVersion [topVer=195408427, order=1583928843625, nodeOrder=4], hash=-1419375634, extras=GridCacheObsoleteEntryExtras [obsoleteVer=GridCacheVersion [topVer=2147483647, order=0, nodeOrder=0]], flags=2]]], prepared=1, locked=false, nodeId=null, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, serReadVer=null, xidVer=null]], dhtVers=null, txSize=0, plc=2, txState=IgniteTxImplicitSingleStateImpl [init=true, recovery=false], flags=onePhase|last, super=GridDistributedBaseMessage [ver=GridCacheVersion [topVer=195408427, order=1583928843624, nodeOrder=1], committedVers=null, rolledbackVers=null, cnt=0, super=GridCacheIdMessage [cacheId=0, trackable=true, nearMiniId=1, last=true, retVal=false, ret=GridCacheReturn [v=null, cacheObj=null, success=true, invokeRes=false, loc=false, cacheId=0], lockKeys=[], forceKeysFut=null, locksReady=true, invoke=false, timeoutObj=PrepareTimeoutObject [timeout=1000], xid=GridCacheVersion [topVer=195408427, order=1583928843625, nodeOrder=4], innerFuts=[[node=da486d0b-36a1-43d4-b05b-47d126fd880e, loc=false, done=true]], super=GridCompoundFuture [rdc=o.a.i.i.processors.cache.distributed.dht.GridDhtTxPrepareFuture$1@73415bf, initFlag=1, lsnrCalls=1, done=true, cancelled=false, err=null, futs=[true]]]java.lang.NoClassDefFoundError: com/companyname/abc/configstore/helper/ContinuousQueryHelper at com.companyname.abc.configstore.helper.ContinuousQueryHelper$ConfigStoreTableRemoteFilterFactory$1.evaluate(ContinuousQueryHelper.java:293) at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.filter(CacheContinuousQueryHandler.java:833) at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$2.onEntryUpdated(CacheContinuousQueryHandler.java:422) at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:426) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1584) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:741) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796) at org.apache.ignite.internal.processo
Re: Security Subject of thin client on remote nodes
The biggest blocker is the one I mention here : http://apache-ignite-developers.2346864.n4.nabble.com/JDBC-thin-client-incorrect-security-context-td45929.html -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Security Subject of thin client on remote nodes
Hi Denis, Thank you for working on the security issues w.r.to thin client logins . We plan to use the thin clients extensively in our project , it will really help if the blockers around it are resolved, regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Security Subject of thin client on remote nodes
HI Denis,
Ref :
https://cwiki.apache.org/confluence/display/IGNITE/IEP-41%3A+Security+Context+of+thin+client+on+remote+nodes
How is this method
public interface GridSecurityProcessor extends GridProcessor {
// Other methods.
/**
* Gets security context.
*
* @param subjId Security subject id.
* @return Security context or null if not found.
*/
public default SecurityContext securityContext(UUID subjId){
return null; //for backward compatibility
}
}
Different from :
/**
* Gets authenticated node subject.
*
* @param subjId Subject ID.
* @return Security subject.
* @throws IgniteCheckedException If error occurred.
*/
public SecuritySubject authenticatedSubject(UUID subjId) throws
IgniteCheckedException;
securityContext(UUID subjId) method will give us SecurityContext,
authenticatedSubject will give us SecuritySubject. We do the
securitysubject's permission check on authorize method. What is the usecase
for which we would want to use securityContext(UUID subjId) instead of
authenticatedSubject or authorize?
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Unable to get the security context
HI Denis, Which version of Apache Ignite are the changes you mention in the comment( security context always not null ) above available with ? In 2.7.6 I do get security context as null in authorize method. regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: JDBC thin client incorrect security context
Is there anything I can do to associate the cacheevent with the logged in jdbc client(DBEAVER user) (Can I do something in any of the methods of the Security Plugin ) . -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: JDBC thin client incorrect security context
Another thing I noticed is that during a update through jdbc client, there could be 2 kinds of threads acting on it. The 'client-connector' thread and the 'sys-stripe'. If the client-connector thread completes the transaction and calls the record, it is able to send a 'cache_put' event with the uuid of the person logged in to through the jdbc thin client . Whereas if the 'sys-stripe' thread completes the transaction, it sends the 'cache_put' event with the node uuid. Is there anyway I can get the session id of the associated jdbc session in the SecurityPlugin and the Audit plugin so that I can link the right UUID ( i.e. the UUID of the logged in jdbc thin client user ). -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
JDBC thin client incorrect security context
Hi , Hi , We have built a security and audit plugin for security of our ignite cluster. We are unable to get the right audit information i.e. we are unable to get the right subject for users logged in through dbeaver ( jdbc thin client. ). This is because the subjectid associated with the "CACHE_PUT" event when an update is triggered by the jdbc thin client, contains the uuid of the node that executed the update rather than the logged in jdbc thin client user. If this is a limitation with the current version of ignite, is there any workaround to get this information ? This was discussed in the 'Ignite users' group http://apache-ignite-users.70518.x6.nabble.com/JDBC-thin-client-incorrect-security-context-td31354.html And I was advised to continue my conversation here. Andrei mentions that the uuid associated with the event should be that of the jdbc client( in our case the logged in dbeaver user ). But I notice that the event always contains the uuid of the node where the query gets executed . I do manage to get the right jdbc client from the authenticationcontext in the authorize and form the right securitycontext. But this doesnt reflect in the generated cacheevent. Kindly guide. regards, Veena. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
