Hi, My business case: A user can add multiple IPs to subscription to scan the assets. So the subscription can have N number of IPs.
User can add IPs in following format: 10.10.10.10, 10.10.10.12-10.10.10.100, 10.10.10.102, 10.10.10.105-10.10.11.255 So the Subscription is top level entity. Then there is a Asset group. A user can create multiple asset group under given subscription. There can be few thousands asset groups under one subscription. Asset group is a container for IPs, i.e. a user can add IPs to asset groups which are present in his subscription and user can launch a scan using asset groups. Now when a user removes IPs from subscription, it needs to be removed from the subscription as well the all asset groups. Now this must happen in a single transaction. This is the simplest case I have. What I meant by data processing? - The add and remove operations are performed on these IPs using tree range set. After performing add/remove operation I identify the exact Ids or rows which needs to be added/updated and removed from cache. I call it as finding delta. Performing all mentioned operations is data processing for me. I have created Callable/Runnable tasks to perform these operations. My affinity key is subscription id. I submit IgniteRunnable task to remove IPs from subscription. In normal case a subscription can have 100K to 1 Million individual IPs/IP ranges. An asset group can have 100K to multi million individual IPs and IP ranges. A subscription can have 1000 to 5000 asset groups. In side this Ignite task I perform following steps. a) Start ignite transaction. b) Remove IPs from subscription in current thread. c) Then create simple Callable/Runnable tasks to remove IPs (Data processing) from asset groups in parallel fashion and in this task I am also doing the cache updates. [This cache updates does not seem to be working now as it is happening new threads.] d) Commit the transaction. *Now my work around to solve this problem:* *I**n step C instead of updating the cache in callable task, return the delta (data needs to be updated in cache) to the thread from which ignite transaction is started and do the cache updates. * 1) Do you think you this approach is ok or do you have any better suggestion? As you said, Transaction is always attached to a single thread. If you do txStart in one thread, and then a cache operation in any other thread, this operation will not be enlisted. 2) Is there any plan to support multithreaded cache updates inside ignite transaction? Thanks, Prasad On Fri, Jun 8, 2018 at 5:06 AM vkulichenko <valentin.kuliche...@gmail.com> wrote: > Prasad, > > What is your business use case and what do you mean by processing? Please > describe in more detail. Are you sure you need to use the transaction here > in the first place? > > -Val > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
