Re: Question on the absolute time at the grid level

[evariste galois]

 Hello,
Basically, there is a modulle providing notifications upon certain expiration 
moments. Clients can manage timer objects which do expire at some point in the 
future. Different clients will use different nodes in the Ignite cluster so 
there will be errors if the clock is not aligned at the cluster level. I 
understood the suggested solution would be to enforce this for each of the 
machines nodes reside on.
Best Regards
On Wednesday, October 2, 2019, 7:16:22 AM GMT+8, Denis Magda 
 wrote:  
 
 Hello, 
Could you please share more details on a business/technical task you're trying 
to solve? We might come up with some solution.
-Denis

On Sun, Sep 29, 2019 at 6:42 PM evariste galois  
wrote:

Hello,
The context of this question is we would be using a grid composed out 
ofmultiple nodes and it is very important the absolute time is the same on 
allthe nodes.
How is the time considered at the grid level? The time returned by 
issuing/SELECT CURRENT_TIMESTAMP()/ would be the same on all the nodes or 
differentJDBC connections to different hosts would yield different results?
It may be possible nodes are residing on machines and there is an offset 
ofseveral minutes between machine system time values
What would Apache Ignite consider as CURRENT_TIMESTAMP() in the 
situationsabove? And is there any way to control the time at the grid level?

Thank you


  

Re: Question on the absolute time at the grid level

[Denis Magda]

Hello,

Could you please share more details on a business/technical task you're
trying to solve? We might come up with some solution.

-
Denis


On Sun, Sep 29, 2019 at 6:42 PM evariste galois <
evariste_galois...@yahoo.com> wrote:

> Hello,
>
> The context of this question is we would be using a grid composed out of
> multiple nodes and it is very important the absolute time is the same on
> all
> the nodes.
>
> How is the time considered at the grid level? The time returned by issuing
> /SELECT CURRENT_TIMESTAMP()/ would be the same on all the nodes or
> different
> JDBC connections to different hosts would yield different results?
>
> It may be possible nodes are residing on machines and there is an offset of
> several minutes between machine system time values
>
> What would Apache Ignite consider as CURRENT_TIMESTAMP() in the situations
> above? And is there any way to control the time at the grid level?
>
>
> Thank you
>
>
>

Re: Changing password of USER "ignite"

[Mahesh Renduchintala]

-1-
Step a)
Login to GridGain
enter user name "ignite" and password "ignite" to connect to your cluster.

Step b)
Go to the notebook tab, execute the SQL
   The SQL used was  in note book was  - ALTER USER "ignite" WITH PASSWORD 
'abc123'

Gridgain pops up
Error: Operation not allowed: authorized context is empty.

-2-

Sql = "ALTER USER "ignite" WITH PASSWORD 'abc123'"

results = superCache.query(sql).getAll();

We get a print
Error: Operation not allowed: authorized context is empty.
Very easy to produce.

regards
mahesh

Re: Confusion with Events

[KJQ]

Thank you for the quick response - it is much appreciated.  We are still
working through our Ignite integration, so far it has been nice, but it is
definitely a learning curve.

So, we have an environment that is deployed as services in a K8S cluster. 
Each service uses Caffeine as the in-memory cache (i know Ignite has a
near-cache but cannot make that change now).  Caffeine is also being tested
inside of Reactive Flux/Mono calls making it harder to change quickly.  Each
service, a deployed pod, is also an Ignite "server" node as well.  We use
Ignite, partitioned, as the primary cache (and some distributed compute with
Drools).

Because all of the nodes use Caffeine, and becoming just a near-cache when
Ignite is included, we would like Ignite to raise an "expired" event to all
the nodes and evict that item from Caffeine (before Ignite, Caffeine was
used as the only in-memory cache per service) - we want to cleanup the local
caches on all the nodes.  Each Ignite cache configuration has an expiration
policy setup.

1) I tried using the `addCacheEntryListenerConfiguration` with each Ignite
cache thinking this was a better choice because (i thought) it was backed by
the continuous query and would not require me to explicitly use events. 
But, it looked like that only fired on the node where the operation happened
(i.e. locally).  Maybe I could broadcast the event from within this listener
to the other nodes?

2) My next attempt is using the "remoteListen()."  If I understand you
correctly, I do not need a "local listener" but the "remote listener" should
broadcast a message when it is triggered?

*Couple of things I noticed in my test below:*
- If i take out the PUT it seems I never see any callback notifications
- Without the local event listener I do not see any expiration messages
(possibly because of where the data is being cached in the test - local vs.
remote node)

Basically, I would like to do this:
1) R/W to Ignite cache with an "expiration" policy
2) When Ignite decides to "expire" an item raise an event to all Ignite
nodes
3) From the event, evict the cache item locally.

This is what I have right now for testing:

ig.events(
ig.cluster().forServers())
.remoteListen(
  new IgniteBiPredicate()
  {
@Override
public boolean apply(UUID uuid, CacheEvent evt)
{
log.debug("Received local event {} {} {}  // {} //
{} ",
  evt.cacheName(),
  evt.name(),
  evt.key(),
  evt.node().id(),
  evt.eventNode().id());
return true; // Continue listening.
  },
new IgnitePredicate()
{
@Override
public boolean apply(final CacheEvent evt)
{
log.debug("Received remote event {} {} {}  / {} {} ",
  evt.cacheName(),
  evt.name(),
  evt.key(),
  evt.node().id(),
  evt.eventNode().id());

//Broadcast the callback
ig.compute().broadcastAsync(new IgniteCallable()
{
@Override
public Object call() throws Exception
{
log.debug("Received callback {} {} {}  / {} {} ",
 evt.cacheName(),
 evt.name(),
 evt.key(),
 evt.node().id(),
 evt.eventNode().id());
return null;
}
});

return true;
}
},
EVT_CACHE_OBJECT_PUT,
EVT_CACHE_OBJECT_EXPIRED)






-
KJQ
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Changing password of USER "ignite"

[Anton Kurbanov]

Hi Mahesh,

How do you execute this query, what kind of client do you use to execute
the query? The syntax is correct, but it seems that there is some issue
with authorization context. What steps must be done to reproduce that?

Regards,
Anton

пн, 30 сент. 2019 г. в 20:36, Mahesh Renduchintala <
mahesh.renduchint...@aline-consulting.com>:

> We followed all that, ignite user name and ignite password for the DB work
> fine
> Now we want change the password to something else.  This is when we get
>
> Error: Operation not allowed: authorized context is empty.
>
> The SQL used was  ALTER USER "ignite" WITH PASSWORD 'abc123'
>
>

RE: How exactly does one start multiple Ignite Clusters on a given YARN (Hadoop) cluster?

[Seshan, Manoj N. (TR Tech, Content & Ops)]

Thanks. A couple of follow-up questions:


  1.  There doesn’t seem to be a way to specify the Port Range, without 
specifying IP addresses? Could you help more specifically?
  2.  How would we support multiple clusters with the Zookeeper Discovery 
method?
  3.  Could we use different MultiCast Groups, or a different Multicast Port to 
setup the multiple clusters?

Rgds

Manoj Seshan - Senior Architect
Platform Content Technology, Bangalore
[cid:image001.gif@01C95541.6801BF70]
Voice: +91-98806 72987  +91-80-67492572

From: Ilya Kasnacheev 
Sent: Monday, September 30, 2019 4:16 PM
To: user@ignite.apache.org
Subject: Re: How exactly does one start multiple Ignite Clusters on a given 
YARN (Hadoop) cluster?

Hello!

You can start two clusters if they have different Discovery settings, e.g., one 
cluster is active on ports 47500..47520, other on 48500..47520.

You should configure clients accordingly. You can also change connector 
configurations for thin clients' connecting.

Regards,
--
Ilya Kasnacheev


пн, 30 сент. 2019 г. в 13:31, Seshan, Manoj N. (TR Tech, Content & Ops) 
mailto:manoj.ses...@thomsonreuters.com>>:
Please help, on how we can achieve this with the Ignite Yarn client


  1.  How exactly do we specify Cluster1 & Cluster2? Would we use the 
“igniteInstanceName” property & Spring Configuration




2)  How would the Ignite Client specify which Cluster (Cluster1 or Cluster2) to 
work with?

Rgds



Manoj Seshan - Senior Architect
Platform Content Technology, Bangalore
[cid:image001.gif@01C95541.6801BF70]
Voice: +91-98806 72987  +91-80-67492572

Re: Confusion with Events

[Ivan Pavlukhin]

Hi KJQ,

A following comes to my mind:
1. Use 
IgniteEvents#remoteListen(org.apache.ignite.lang.IgniteBiPredicate,
org.apache.ignite.lang.IgnitePredicate, int...) with null first
predicate (locLsnr) (because notifying caller seems not needed).
2. Use IgniteCompute#broadcastAsync(org.apache.ignite.lang.IgniteCallable)
in remote event handler (rmtFilter) to notify all nodes about events.

But here can be following caveats:
1. Such broadcasts can lead to poor performance, events buffering
before broadcast might help.
2. Event listeners can be also triggered for backup partitions
updates, perhaps backup notifications should be filtered to avoid
duplicate broadcasts.

Also more details about your use case can help to develop a good
solution. Currently use case is not fully clear for me.

вт, 1 окт. 2019 г. в 03:38, KJQ :
>
> I have some questions regarding Cache Listeners/Events.
>
> We have a system that used a lot of "Caffeine" based caches spread across
> multiple services (in K8S).  Basically "near-caches" (without a backing
> store).  We are now trying to fit Ignite behind those usages.
>
> *What we are trying to do is when Ignite /expires/ an entry receive the
> event on all the nodes and evict it in from Caffeine*.
>
> Are one of these approaches below correct? And/or how can I accomplish this?
> Is there a better/easier way?
>
> 1) I tried registering a CacheListener with each cache configuration but
> that seemed to only fire where the cache event was fired:
>
> config.addCacheEntryListenerConfiguration(new
> IgniteExpiredListener<>(cacheManagerProvider));
>
> 2) I am experimenting with cache events as well like this below.
>
> ig.events(
> ig.cluster().forServers())
> .remoteListen(
> new IgniteBiPredicate()
> {
> @Override
> public boolean apply(UUID uuid, CacheEvent evt)
> {
> log.debug("Received local event "
>   + evt.name()
>   + ", key="
>   + evt.key()
>   + ", at="
>   + evt.node().consistentId().toString()
>   + ", "
>   +
> evt.eventNode().consistentId().toString() );
> cm.getCache(evt.cacheName()).evict(evt.key());
> return true; // Continue listening.
> }
> },
> new IgnitePredicate()
> {
> @Override
> public boolean apply(final CacheEvent evt)
> {
> log.debug("Received remote event "
>   + evt.name()
>   + ", key="
>   + evt.key()
>   + ", at="
>   + evt.node().consistentId().toString()
>   + ", "
>   +
> evt.eventNode().consistentId().toString() );
> return true;
> }
> },
> EVTS_CACHE);
>
>
>
>
> -
> KJQ
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/



-- 
Best regards,
Ivan Pavlukhin

Re: Map Reduce over cache items, where values are sequences

[Evgenii Zhuravlev]

For example, query with partition will look like :

QueryCursor> qry =
cache.query(new ScanQuery().setPartition(part));


вт, 1 окт. 2019 г. в 15:25, Evgenii Zhuravlev :

> Hi,
>
> To parallelise everything properly, I would recommend starting an
> affinityCallable per partition(1024 by default). Inside this compute job,
> you can collect information for the certain partition only using
> ScanQuery(or SQLQuery)
>
>
> пт, 27 сент. 2019 г. в 18:09, Stas Girkin :
>
>> Hello everyone,
>>
>> I would like to use MapReduce over cache items representing events
>> happened in a process to calculate certain statistics. Could you be so kind
>> to help me how can I do that with apache ignite?
>>
>> I have tens of millions of processes that happened in the past. The
>> processes look like a sequence of events [event1, event2, event3, ...
>> eventN], where number of events per process could vary (50-100). Every
>> event has certain sets of attributes like timestamp, event type, set of
>> metrics. I put these data to a cache as process_id => [e1, e2, e3, e4,
>> ...]. What I would like to get is to get a histogram how often event of a
>> certain type happens in all the processes or processes that have certain
>> condition. What I managed to do is to broadcast a callable that lands on
>> ignite nodes and can access local cache items and counts what I want and
>> returns it back to the caller in K chunks which I have to aggregate on the
>> client.
>>
>> Ignite localIgnite = Ignition.localIgnite();
>> IgniteCache localCache =
>> localIgnite.cache("processes");
>> MyHistogram hist = new MyHistogram()
>> for (Cache.Entry e : localCache.localEntries()) {
>> hist.process(e.getValue());
>> }
>> return hist;
>>
>> The problem with the approach is it utilizes only a single core on the
>> ignite node, while I have 64. How could I do something similar in more
>> efficient manner?
>>
>> thank you in advance.
>>
>

Re: Map Reduce over cache items, where values are sequences

[Evgenii Zhuravlev]

Hi,

To parallelise everything properly, I would recommend starting an
affinityCallable per partition(1024 by default). Inside this compute job,
you can collect information for the certain partition only using
ScanQuery(or SQLQuery)


пт, 27 сент. 2019 г. в 18:09, Stas Girkin :

> Hello everyone,
>
> I would like to use MapReduce over cache items representing events
> happened in a process to calculate certain statistics. Could you be so kind
> to help me how can I do that with apache ignite?
>
> I have tens of millions of processes that happened in the past. The
> processes look like a sequence of events [event1, event2, event3, ...
> eventN], where number of events per process could vary (50-100). Every
> event has certain sets of attributes like timestamp, event type, set of
> metrics. I put these data to a cache as process_id => [e1, e2, e3, e4,
> ...]. What I would like to get is to get a histogram how often event of a
> certain type happens in all the processes or processes that have certain
> condition. What I managed to do is to broadcast a callable that lands on
> ignite nodes and can access local cache items and counts what I want and
> returns it back to the caller in K chunks which I have to aggregate on the
> client.
>
> Ignite localIgnite = Ignition.localIgnite();
> IgniteCache localCache = localIgnite.cache("processes");
> MyHistogram hist = new MyHistogram()
> for (Cache.Entry e : localCache.localEntries()) {
> hist.process(e.getValue());
> }
> return hist;
>
> The problem with the approach is it utilizes only a single core on the
> ignite node, while I have 64. How could I do something similar in more
> efficient manner?
>
> thank you in advance.
>

Re: .net thin client taking time to establish connection to server node

[siva]

Hi Pavel,


*Maybe something else is affecting request time in staging?*
chances of others services trying to connect (from front end requests)

*Do you make requests from your local machine to staging environment?*
yes,i have made a request to the staging environment ignite server from
local machine console app to test.

*How fast is the connection between thin client and server?
*

from local machine to local ignite server its taking microseconds and
milliseconds upto 10ms max

but from local machine to  staging ignite server(cloud) its taking more
time.
i have attached the screen shot


*Code
*

This github code describes similer to the client applications trying  to the
ignite server 

https://github.com/cvakarna/ignitethinclient-reproducer/tree/master/IgniteConnectionTest

  

This github link similer to the server code
https://github.com/cvakarna/ignitereproducer/tree/master/IgniteReproducerServer

  



 





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Authorizing thin clients

[Kurt Semba]

Hi all,

is there a way to define which SQL statements a thin client is allowed to 
execute (some users don’t need the rights to drop / create /alter tables, 
etc.)? Or which SQL tables that client is allowed to query?

Thanks
Kurt

Re: Getting javax.cache.CacheException after upgrading to Ignite 2.7 (IGNITE-10884 )

[Ivan Pavlukhin]

Hi Prasad,

The ticket is resolved with fix version 2.8, which was not released
yet. Unfortunately we overlooked to include it into 2.7.6 scope. Sorry
for that.

пт, 27 сент. 2019 г. в 16:49, Prasad Bhalerao :
>
> Hi,
>
> this issue still exists in ignite 2.7.6 version but the ticket is marked as 
> Fixed. I am still able to reproduce the issue using following reproducer.
>
> Because this issue I am not able to upgrade to latest version from 2.6.0 
> version.
>
> GitHub project: https://github.com/prasadbhalerao1983/IgniteTestPrj.git
>
> Please run IgniteTransactionTester class to check the issue.
>
> Thanks,
> Prasad
>
> On Thu, Jan 10, 2019 at 7:22 PM Ilya Kasnacheev  
> wrote:
>>
>> Hello!
>>
>> I have filed a blocker ticket about it: 
>> https://issues.apache.org/jira/browse/IGNITE-10884
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 17:43, Prasad Bhalerao :
>>>
>>>
>>> Hi Ilya,
>>>
>>> I have created a reproducer for this issue and uploaded it to GitHub.
>>>
>>> GitHub project: https://github.com/prasadbhalerao1983/IgniteTestPrj.git
>>>
>>> Please run IgniteTransactionTester class to check the issue.
>>>
>>>
>>> Exception:
>>>
>>> Exception in thread "main" javax.cache.CacheException: Only pessimistic 
>>> repeatable read transactions are supported at the moment.
>>>  at 
>>> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:697)
>>>  at 
>>> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:636)
>>>  at 
>>> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:388)
>>>  at 
>>> IgniteTransactionTester.testTransactionException(IgniteTransactionTester.java:53)
>>>  at IgniteTransactionTester.main(IgniteTransactionTester.java:38)
>>> Caused by: class 
>>> org.apache.ignite.internal.processors.query.IgniteSQLException: Only 
>>> pessimistic repeatable read transactions are supported at the moment.
>>>  at 
>>> org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.tx(MvccUtils.java:690)
>>>  at 
>>> org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.tx(MvccUtils.java:671)
>>>  at 
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.runQueryTwoStep(IgniteH2Indexing.java:1793)
>>>  at 
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunDistributedQuery(IgniteH2Indexing.java:2610)
>>>  at 
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunPrepared(IgniteH2Indexing.java:2315)
>>>  at 
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:2209)
>>>  at 
>>> org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2135)
>>>  at 
>>> org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2130)
>>>  at 
>>> org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
>>>  at 
>>> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2707)
>>>  at 
>>> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2144)
>>>  at 
>>> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:685)
>>>
>>> Thanks,
>>>
>>> Prasad
>>>
>>>
>>>
>>> On Wed, Jan 9, 2019 at 6:22 PM Ilya Kasnacheev  
>>> wrote:

 Hello!

 It was discussed recently: 
 http://apache-ignite-users.70518.x6.nabble.com/Migrate-from-2-6-to-2-7-td25738.html

 I don't think you will be able to use SQL from transactions in Ignite 2.7. 
 While this looks like a regression, you will have to work around it for 
 now.

 Do you have a small reproducer for this issue? I could file a ticket if 
 you had. You can try to do it yourself, too.

 Regards,
 --
 Ilya Kasnacheev


 ср, 9 янв. 2019 г. в 15:33, Prasad Bhalerao :
>
> Hi,
>
> My cache configuration is as follows. I am using TRANSACTIONAL and not  
> TRANSACTIONAL_SNAPSHOT.
>
>
>
> private CacheConfiguration ipContainerIPV4CacheCfg() {
>
>   CacheConfiguration ipContainerIpV4CacheCfg = new 
> CacheConfiguration<>(CacheName.IP_CONTAINER_IPV4_CACHE.name());
>   
> ipContainerIpV4CacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
>   ipContainerIpV4CacheCfg.setWriteThrough(ENABLE_WRITE_THROUGH);
>   ipContainerIpV4CacheCfg.setReadThrough(false);
>   ipContainerIpV4CacheCfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
>   
> ipContainerIpV4CacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>   ipContainerIpV4CacheCfg.setBackups(1);
>   Factory storeFactory = 
> FactoryBuilder.factoryOf(IpContainerIpV4CacheStore.class);
>   ipContainerIpV4CacheCfg.setCacheStoreFactory(storeFactory);
>