Fast Load Large Datasets

2020-09-24 Thread vtchernyi 
Igniters,My tutorial post about loading big tables into Apache Ignite has finally arrived [1]. Many thanks to @Denis Magda and @Ksenia Romanova for their valuable help.[1] https://www.gridgain.com/resources/blog/how-fast-load-large-datasets-apache-ignite-using-key-value-apiVladimir

Re: Issue in Alter Table - Drop Column functionality

2020-09-24 Thread Shravya Nethula 
Thank you for the information Ilya.


Regards,

Shravya Nethula,

BigData Developer,

[cid:9d5782fd-7700-4644-9354-38435191aa05]

Hyderabad.


From: Ilya Kasnacheev 
Sent: Thursday, September 24, 2020 5:01 PM
To: user@ignite.apache.org 
Subject: Re: Issue in Alter Table - Drop Column functionality

Hello!

I think it will be in 2.10 or 2.9.1. Maybe early next year?

Regards,
--
Ilya Kasnacheev


чт, 24 сент. 2020 г. в 14:04, Shravya Nethula 
mailto:shravya.neth...@aline-consulting.com>>:
Hi Ilya,

Thank you for the confirmation and the workaround is working as expected. Much 
appreciated.

Is there any rough release date or release version for this issue?



Regards,

Shravya Nethula,

BigData Developer,

[cid:174bfe19e70c0299585]

Hyderabad.


From: Ilya Kasnacheev 
mailto:ilya.kasnach...@gmail.com>>
Sent: Thursday, September 24, 2020 3:53 PM
To: user@ignite.apache.org 
mailto:user@ignite.apache.org>>
Subject: Re: Issue in Alter Table - Drop Column functionality

Hello!

Thank you for this report, now I can indeed confirm I can see the issue! I have 
filed https://issues.apache.org/jira/browse/IGNITE-13482 on your behalf.

As a workaround you can do SELECT * FROM tbl LIMIT 0; before issuing ALTER 
TABLE on that node.

Regards,
--
Ilya Kasnacheev


чт, 24 сент. 2020 г. в 10:41, Shravya Nethula 
mailto:shravya.neth...@aline-consulting.com>>:
Hi Ilya,

Yes, the code that you sent is working fine.

But can you please try the below code, by executing CREATE TABLE and ALTER 
TABLE in separate programs. In this scenario, its not working.
public class ClientNode {

public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY KEY, 
name VARCHAR, age INT);"));
foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) VALUES 
(1, 'Vassisualy', 45);"));
//foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
(age);"));

//System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll());

} }

public class ClientNode {
public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

//foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
KEY, name VARCHAR, age INT);"));
//foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
VALUES (1, 'Vassisualy', 45);"));
foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN (age);"));

System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll())

} }

This second program is giving the following error:
Exception in thread "main" javax.cache.CacheException
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424)
at net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28)
Caused by: class 
org.apache.ignite.internal.processors.query.IgniteSQLException: null
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888)
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
at

Re: OutOfMemoryException with Persistence and Eviction Enabled

2020-09-24 Thread Mitchell Rathbun (BLOOMBERG/ 731 LEX) 
I tried doubling it from 200 MB to 400 MB. My initial test worked, but as I 
increased the number of fields that I was writing per entry, the same issue 
occurred again. So it seems to just increase the capacity of what can be 
written, not actually prevent the exception from occurring. I guess my main 
question is why is it possible for Ignite to get an OOME when persistence and 
eviction are enabled? It seems like if there are a lot of writes, performance 
should degrade as the in memory cache evicts members of the cache, but no 
exceptions should occur. The error is always "Failed to find a page for 
eviction", which doesn't really make sense when eviction is enabled. What are 
the internal structures that Ignite holds in Off-heap memory?

Also, why isn't this an issue when using IgniteDataStreamer if the issue has to 
do with space for internal structures? Wouldn't Ignite need the same internal 
structures for either case?

From: user@ignite.apache.org At: 09/24/20 11:42:21To:  user@ignite.apache.org
Subject: Re: OutOfMemoryException with Persistence and Eviction Enabled

  
Hi,
  
  Did you try to increase the DataRegion size a little bit? It 
looks like 190 MB isn't enough for some internal structures that Ignite 
stores in OFF-HEAP except the data.
  
  I suggest you increase the data region size to for example 512 MB 
- 1024 MB and take a look at how it will work.
  
  If you still will see the issue then I guess we should create the 
ticket:
  
  1)Collect the logs
  2)Provide the java code example
  3)Provide the configuration of the nodes
  
  After that, we can take a look more deeply into it and if it's an 
issue then file JIRA.
  
  BR,
  Andrei 
9/23/2020 7:36 PM, Mitchell Rathbun   (BLOOMBERG/ 731 LEX) пишет:
 
  
Here is the exception:
Sep 22, 2020 7:58:22 PM java.util.logging.LogManager$RootLogger log
SEVERE: Critical system error detected. Will be handled accordingly to 
configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, 
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext 
[type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: Out of 
memory in data region [name=customformulacalcrts, initSize=190.7 MiB, 
maxSize=190.7 MiB, persistenceEnabled=true] Try the following:
  ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)
  ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
  ^-- Enable eviction or expiration policies]]
class org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Out of memory 
in data region [name=customformulacalcrts, initSize=190.7 MiB, maxSize=190.7 
MiB, persistenceEnabled=true] Try the following:
  ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)
  ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
  ^-- Enable eviction or expiration policies
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.allocatePage(PageMemoryImpl.java:607)
  at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.allocateDataPage(AbstractFreeList.java:464)
 at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.insertDataRow(AbstractFreeList.java:491)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl.insertDataRow(CacheFreeListImpl.java:59)
   at 
org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl.insertDataRow(CacheFreeListImpl.java:35)
   at 
org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:103)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1691)
   at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.createRow(GridCacheOffheapManager.java:1910)
   at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$UpdateClosure.call(GridCacheMapEntry.java:5701)
 at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$UpdateClosure.call(GridCacheMapEntry.java:5643)
 at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3719)
  at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5900(BPlusTree.java:3613)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1895)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1872)
at

Re: [DISCUSSION] Renaming Ignite's product category

2020-09-24 Thread Denis Magda 
Adam,

Like your way of thinking. It makes sense to me.

For me, a computing platform comprises two essential components - a storage
engine and compute APIs (not only compute grid, but also streaming, SQL,
etc.). Under this definition, Ignite fits the "compute platform" category
for sure, but the problem (and the reason why I started this discussion
thread) is that the "in-memory computing platform" is not a ubiquitous
product category. At the same time, we all know that Ignite is used to
speed things up by storing and executing in memory. And if you need to make
software faster, you usually search for something scalable or in-memory
(in-memory caches, in-memory, NoSQL, distributed databases). Basically,
most of the folks search for "caches" and "in-memory database" which is
related to Ignite and only the tip of the iceberg searches for "in-memory
computing platforms" and "data grids".

Btw, how has your story started with Ignite? How did you find out about it?

-
Denis


On Wed, Sep 23, 2020 at 1:24 PM Carbone, Adam 
wrote:

> Good Evening Denis,
>
> I’m sure there are more out there as well,  I would consider platforms
> that the entire applications but that all the execution of code happens
> exclusively on the platform, and most of the applications are written to
> run in, not connected to the platform.
>
> Hmmm by this criteria k8s could possible fit the bill…
>
> Spark might even be considered a compute grid as well but it is not a
> generic compute framework and people don’t usually run there whole
> applications inside.
>
> What is the vision for the platform? That might help in this discussion,
> set your category with the direction you are heading, and what you are
> trying to achieve.
>
> Regards
>
>
>
> Adam Carbone | Director of Innovation – Intelligent Platform Team |
> Bottomline Technologies
> Office: 603-501-6446 | Mobile: 603-570-8418
> www.bottomline.com
>
>
>
>
>
>
>
> *From: *Denis Magda 
> *Date: *Wednesday, September 23, 2020 at 4:14 PM
> *To: *dev , "Carbone, Adam" <
> adam.carb...@bottomline.com>
> *Cc: *"user@ignite.apache.org" 
> *Subject: *Re: [DISCUSSION] Renaming Ignite's product category
>
>
>
> Adam,
>
>
>
> You defined GigaSpaces as a true in-memory computing platform. What is the
> true platform for you?
>
>
>
>
> -
>
> Denis
>
>
>
>
>
> On Fri, Sep 18, 2020 at 7:02 AM Carbone, Adam 
> wrote:
>
> So when I came across Ignite It was described as an In Memory Data Grid
>
> So one way to look at this is who do you fashion as Ignite competing
> against?
>
> Are competing against Redis, Aerospike - In Memory Databases
>
> Or are you more competing with
>
> Gigaspaces - True In memory Compute platform
>
> And then you have like of
>
> Hazelcast that started as a Distributed Hash and have gained some
> features...
>
> On thing that I think is a differentiator that isn't being highlighted but
> Is  unique feature to Ignited, and the primary reason we ended up here; The
> integration with spark and it's distributed/shared Datasets/Dataframes.
>
> I don't know for me the In Memory Data Grid I think fits what Ignite is...
>
> Regards
>
> ~Adam
>
> Adam Carbone | Director of Innovation – Intelligent Platform Team |
> Bottomline Technologies
> Office: 603-501-6446 | Mobile: 603-570-8418
> www.bottomline.com
>
>
>
> On 9/17/20, 11:45 AM, "Glenn Wiebe"  wrote:
>
> I agree with Stephen about "database" devaluing what Ignite can do
> (though
> it probably hits the majority of existing use cases). I tend to go with
> "massively distributed storage and compute platform"
>
> I know, I didn't take sides, I just have both.
>
> Cheers,
>   Glenn
>
> On Thu., Sep. 17, 2020, 7:04 a.m. Stephen Darlington, <
> stephen.darling...@gridgain.com> wrote:
>
> > I think this is a great question. Explaining what Ignite does is
> always a
> > challenge, so having a useful “tag line” would be very valuable.
> >
> > I’m not sure what the answer is but I think calling it a “database”
> > devalues all the compute facilities. "Computing platform” may be too
> vague
> > but it at least says that we do more than “just” store data.
> >
> > On 17 Sep 2020, at 06:29, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > My vote is for the "distributed memory-first database". It clearly
> states
> > that Ignite is a database (which is true at this point), while still
> > emphasizing the in-memory computing power endorsed by the platform.
> >
> > The "in-memory computing platform" is an ambiguous term and doesn't
> really
> > reflect what Ignite is, especially in its current state.
> >
> > -Val
> >
> > On Wed, Sep 16, 2020 at 3:53 PM Denis Magda 
> wrote:
> >
> >> Igniters,
> >>
> >> Throughout the history of our project, we could see how the
> addition of
> >> certain features required us to reassess the project's name and
> category.
> >>
> >> Before Ignite

Re: IgniteCache.size() is hanging

2020-09-24 Thread Andrei Aleksandrov 

Hi,

Highly likely some of the nodes go offline and try to connect again. 
Probably you had some network issues. I think I will see this and other 
information in the logs. Can you provide them?


BR,
Andrei

9/24/2020 6:54 PM, Alan Ward пишет:
The only log I see is from one of the server nodes, which is spewing 
at a very high rate:


[grid-nio-worker-tcp-comm-...][TcpCommunicationSpi] Accepted incoming 
communication connection [locAddr=/:47100, rmtAddr=:


Note that each time the log is printed, i see a different value for 
.


Also note  that I only see these logs when i try to run 
ignitevisorcmd's "cache" command. When I run the java application that 
calls IgniteCache.size(), I don't see any such logs. But in both 
cases, the result is that the operation is just hanging.


The cluster is active and I am able to insert data (albeit at a pretty 
slow rate), so it's not like things are completely non-functional. 
It's really confusing :\


On Thu, Sep 24, 2020 at 11:04 AM aealexsandrov 
mailto:aealexsand...@gmail.com>> wrote:


Hi,

Can you please provide the full server logs?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: IgniteCache.size() is hanging

2020-09-24 Thread Alan Ward 
The only log I see is from one of the server nodes, which is spewing at a
very high rate:

[grid-nio-worker-tcp-comm-...][TcpCommunicationSpi] Accepted incoming
communication connection [locAddr=/:47100, rmtAddr=:

Note that each time the log is printed, i see a different value for .

Also note  that I only see these logs when i try to run ignitevisorcmd's
"cache" command. When I run the java application that calls
IgniteCache.size(), I don't see any such logs. But in both cases, the
result is that the operation is just hanging.

The cluster is active and I am able to insert data (albeit at a pretty slow
rate), so it's not like things are completely non-functional. It's really
confusing :\

On Thu, Sep 24, 2020 at 11:04 AM aealexsandrov 
wrote:

> Hi,
>
> Can you please provide the full server logs?
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: OutOfMemoryException with Persistence and Eviction Enabled

2020-09-24 Thread Andrei Aleksandrov 

Hi,

Did you try to increase the DataRegion size a little bit? It looks like 
190 MB isn't enough for some internal structures that Ignite stores in 
OFF-HEAP except the data.


I suggest you increase the data region size to for example 512 MB - 1024 
MB and take a look at how it will work.


If you still will see the issue then I guess we should create the ticket:

1)Collect the logs
2)Provide the java code example
3)Provide the configuration of the nodes

After that, we can take a look more deeply into it and if it's an issue 
then file JIRA.


BR,
Andrei

9/23/2020 7:36 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX) пишет:

Here is the exception:
Sep 22, 2020 7:58:22 PM java.util.logging.LogManager$RootLogger log
SEVERE: Critical system error detected. Will be handled accordingly to 
configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, 
timeout=0, super=AbstractFailureHandler 
[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, 
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext 
[type=CRITICAL_ERROR, err=class 
o.a.i.i.mem.IgniteOutOfMemoryException: Out of memory in data region 
[name=customformulacalcrts, initSize=190.7 MiB, maxSize=190.7 MiB, 
persistenceEnabled=true] Try the following:
^-- Increase maximum off-heap memory size 
(DataRegionConfiguration.maxSize)

^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
^-- Enable eviction or expiration policies]]
class org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Out 
of memory in data region [name=customformulacalcrts, initSize=190.7 
MiB, maxSize=190.7 MiB, persistenceEnabled=true] Try the following:
^-- Increase maximum off-heap memory size 
(DataRegionConfiguration.maxSize)

^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)
^-- Enable eviction or expiration policies
at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.allocatePage(PageMemoryImpl.java:607)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.allocateDataPage(AbstractFreeList.java:464)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.insertDataRow(AbstractFreeList.java:491)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl.insertDataRow(CacheFreeListImpl.java:59)
at 
org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl.insertDataRow(CacheFreeListImpl.java:35)
at 
org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:103)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1691)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.createRow(GridCacheOffheapManager.java:1910)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$UpdateClosure.call(GridCacheMapEntry.java:5701)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$UpdateClosure.call(GridCacheMapEntry.java:5643)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3719)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5900(BPlusTree.java:3613)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1895)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1872)
at 
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1779)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1638)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1621)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1935)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:428)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:4248)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:4226)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdateLocal(GridCacheMapEntry.java:2106)
at 
org.apache.ignite.internal.processors.cache.local.atomic.GridLocalAtomicCache.updateAllInternal(GridLocalAtomicCache.java:929)
at 
org.apache.ignite.internal.processors.cache.local.atomic.GridLocalAtomicCache.access$100(GridLocalAtomicCache.java:86)
at 
org.apache.ignite.internal.processors.cache.local.atomic.GridLocalAtomicCache$6.call(GridLocalAtomicCache.java:776)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6817)
at

Re: IgniteCache.size() is hanging

2020-09-24 Thread aealexsandrov 
Hi,

Can you please provide the full server logs?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

IgniteCache.size() is hanging

2020-09-24 Thread Alan Ward 
[image: Selected post]
I'm running a 5 node Ignite cluster, version 2.8.1, with persistence
enabled
and a small number of partitioned caches, ranging between a few thousand
records to one cache with over 1 billion records. No SQL use.

When I run a Java client app and connect to the cluster (with clientMode =
true), I connect fine and can retrieve the names of all caches on the
cluster quickly. However, attempting to get the size of a cache via
ignite.getOrCreateCache("existingCacheName").size() just hangs. This
happens
regardless of which cache I try to get the size of.

Sometimes I see a suspicious warning after a minute or so: WARNING: Node
FAILED:
TcpDiscoveryNode[...] - it appears to be referencing my client node. I
don't know why the node failed, what to do about it, or why it seems to
happen so frequently. There are no
relevant logs coming from any of the ignite server nodes, nor the java
app/client.

There are also many times when I do not get a Node FAILED warning, but
still the size() operation just hangs with no other information.

Thanks for your help!

Alan

Re: Issue in Alter Table - Drop Column functionality

2020-09-24 Thread Ilya Kasnacheev 
Hello!

I think it will be in 2.10 or 2.9.1. Maybe early next year?

Regards,
-- 
Ilya Kasnacheev


чт, 24 сент. 2020 г. в 14:04, Shravya Nethula <
shravya.neth...@aline-consulting.com>:

> Hi Ilya,
>
> Thank you for the confirmation and the workaround is working as expected.
> Much appreciated.
>
> Is there any rough release date or release version for this issue?
>
>
> Regards,
>
> Shravya Nethula,
>
> BigData Developer,
>
>
> Hyderabad.
>
> --
> *From:* Ilya Kasnacheev 
> *Sent:* Thursday, September 24, 2020 3:53 PM
> *To:* user@ignite.apache.org 
> *Subject:* Re: Issue in Alter Table - Drop Column functionality
>
> Hello!
>
> Thank you for this report, now I can indeed confirm I can see the issue! I
> have filed https://issues.apache.org/jira/browse/IGNITE-13482 on your
> behalf.
>
> As a workaround you can do SELECT * FROM tbl LIMIT 0; before issuing ALTER
> TABLE on that node.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 24 сент. 2020 г. в 10:41, Shravya Nethula <
> shravya.neth...@aline-consulting.com>:
>
> Hi Ilya,
>
> Yes, the code that you sent is working fine.
>
> But can you please try the below code, by executing CREATE TABLE and ALTER
> TABLE in separate programs. In this scenario, its not working.
> public class ClientNode {
>
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite ignite = Ignition.start(igniteCfg);
> IgniteCache foo = ignite.getOrCreateCache("foo");
>
> *foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
> KEY, name VARCHAR, age INT);"));
> foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
> VALUES (1, 'Vassisualy', 45);"));*
> //foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
> (age);"));
>
> //System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
> person;")).getAll());
>
> } }
>
> public class ClientNode {
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite ignite = Ignition.start(igniteCfg);
> IgniteCache foo = ignite.getOrCreateCache("foo");
>
> //foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
> KEY, name VARCHAR, age INT);"));
> //foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
> VALUES (1, 'Vassisualy', 45);"));
> *foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
> (age);"));*
>
> *System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
> person;")).getAll())*
>
> } }
>
> *This second program is giving the following error:*
> *Exception in thread "main" javax.cache.CacheException*
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424)
> at
> net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28)
> *Caused by: class
> org.apache.ignite.internal.processors.query.IgniteSQLException: null*
> at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888)
> at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
> at
> org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802)
> ... 3 more
> *Caused by: java.lang.NullPointerException*
> *at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:834)*
> ... 15 more
>
>
>
>
>

Re: Issue in Alter Table - Drop Column functionality

2020-09-24 Thread Shravya Nethula 
Hi Ilya,

Thank you for the confirmation and the workaround is working as expected. Much 
appreciated.

Is there any rough release date or release version for this issue?



Regards,

Shravya Nethula,

BigData Developer,

[cid:357d40f2-e612-4cef-b899-fb17a36896bb]

Hyderabad.


From: Ilya Kasnacheev 
Sent: Thursday, September 24, 2020 3:53 PM
To: user@ignite.apache.org 
Subject: Re: Issue in Alter Table - Drop Column functionality

Hello!

Thank you for this report, now I can indeed confirm I can see the issue! I have 
filed https://issues.apache.org/jira/browse/IGNITE-13482 on your behalf.

As a workaround you can do SELECT * FROM tbl LIMIT 0; before issuing ALTER 
TABLE on that node.

Regards,
--
Ilya Kasnacheev


чт, 24 сент. 2020 г. в 10:41, Shravya Nethula 
mailto:shravya.neth...@aline-consulting.com>>:
Hi Ilya,

Yes, the code that you sent is working fine.

But can you please try the below code, by executing CREATE TABLE and ALTER 
TABLE in separate programs. In this scenario, its not working.
public class ClientNode {

public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY KEY, 
name VARCHAR, age INT);"));
foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) VALUES 
(1, 'Vassisualy', 45);"));
//foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
(age);"));

//System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll());

} }

public class ClientNode {
public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

//foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
KEY, name VARCHAR, age INT);"));
//foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
VALUES (1, 'Vassisualy', 45);"));
foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN (age);"));

System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll())

} }

This second program is giving the following error:
Exception in thread "main" javax.cache.CacheException
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424)
at net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28)
Caused by: class 
org.apache.ignite.internal.processors.query.IgniteSQLException: null
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888)
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802)
... 3 more
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:834)
... 15 more





Regards,

Shravya Nethula,

BigData Developer,

[cid:174bfa1c3c42d587d5b4]

Hyderabad.


From: Ilya Kasnacheev 
mailto:ilya.kasnach...@gmail.com>>
Sent: Monday, September 21, 2020 3:04 PM
To: user@ignite.apache.org 
mailto:user@ignite.apache.org>>
Subject: Re: Issue in Alter Table - Drop Column functionality

Re: Issue in Alter Table - Drop Column functionality

2020-09-24 Thread Ilya Kasnacheev 
Hello!

Thank you for this report, now I can indeed confirm I can see the issue! I
have filed https://issues.apache.org/jira/browse/IGNITE-13482 on your
behalf.

As a workaround you can do SELECT * FROM tbl LIMIT 0; before issuing ALTER
TABLE on that node.

Regards,
-- 
Ilya Kasnacheev


чт, 24 сент. 2020 г. в 10:41, Shravya Nethula <
shravya.neth...@aline-consulting.com>:

> Hi Ilya,
>
> Yes, the code that you sent is working fine.
>
> But can you please try the below code, by executing CREATE TABLE and ALTER
> TABLE in separate programs. In this scenario, its not working.
> public class ClientNode {
>
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite ignite = Ignition.start(igniteCfg);
> IgniteCache foo = ignite.getOrCreateCache("foo");
>
> *foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
> KEY, name VARCHAR, age INT);"));
> foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
> VALUES (1, 'Vassisualy', 45);"));*
> //foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
> (age);"));
>
> //System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
> person;")).getAll());
>
> } }
>
> public class ClientNode {
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite ignite = Ignition.start(igniteCfg);
> IgniteCache foo = ignite.getOrCreateCache("foo");
>
> //foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
> KEY, name VARCHAR, age INT);"));
> //foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
> VALUES (1, 'Vassisualy', 45);"));
> *foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
> (age);"));*
>
> *System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
> person;")).getAll())*
>
> } }
>
> *This second program is giving the following error:*
> *Exception in thread "main" javax.cache.CacheException*
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424)
> at
> net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28)
> *Caused by: class
> org.apache.ignite.internal.processors.query.IgniteSQLException: null*
> at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888)
> at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
> at
> org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802)
> ... 3 more
> *Caused by: java.lang.NullPointerException*
> *at
> org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:834)*
> ... 15 more
>
>
>
>
> Regards,
>
> Shravya Nethula,
>
> BigData Developer,
>
>
> Hyderabad.
>
> --
> *From:* Ilya Kasnacheev 
> *Sent:* Monday, September 21, 2020 3:04 PM
> *To:* user@ignite.apache.org 
> *Subject:* Re: Issue in Alter Table - Drop Column functionality
>
> Hello!
>
> Sqlline can use both thin JDBC and thick JDBC driver. In this case it uses
> thick JDBC driver.
>
> Do you have a reproducer project to demonstrate the issue? I can't get it
> to show up:
>
> public class ClientNode {
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite

Re: Issue in Alter Table - Drop Column functionality

2020-09-24 Thread Shravya Nethula 
Hi Ilya,

Yes, the code that you sent is working fine.

But can you please try the below code, by executing CREATE TABLE and ALTER 
TABLE in separate programs. In this scenario, its not working.
public class ClientNode {

public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY KEY, 
name VARCHAR, age INT);"));
foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) VALUES 
(1, 'Vassisualy', 45);"));
//foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN 
(age);"));

//System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll());

} }

public class ClientNode {
public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

//foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY 
KEY, name VARCHAR, age INT);"));
//foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) 
VALUES (1, 'Vassisualy', 45);"));
foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN (age);"));

System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll())

} }

This second program is giving the following error:
Exception in thread "main" javax.cache.CacheException
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:750)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:424)
at net.aline.cloudedh.base.database.extra.ClientNode.main(ClientNode.java:28)
Caused by: class 
org.apache.ignite.internal.processors.query.IgniteSQLException: null
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:888)
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommand(CommandProcessor.java:418)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeCommand(IgniteH2Indexing.java:1048)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1130)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2406)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2402)
at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2919)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2422)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2460)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2396)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2323)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:802)
... 3 more
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.query.h2.CommandProcessor.runCommandH2(CommandProcessor.java:834)
... 15 more





Regards,

Shravya Nethula,

BigData Developer,

[cid:377ac046-a8f4-47d0-b6d7-74fce693a1d0]

Hyderabad.


From: Ilya Kasnacheev 
Sent: Monday, September 21, 2020 3:04 PM
To: user@ignite.apache.org 
Subject: Re: Issue in Alter Table - Drop Column functionality

Hello!

Sqlline can use both thin JDBC and thick JDBC driver. In this case it uses 
thick JDBC driver.

Do you have a reproducer project to demonstrate the issue? I can't get it to 
show up:


public class ClientNode {
public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

foo.query(new SqlFieldsQuery("CREATE TABLE person (id INT PRIMARY KEY, 
name VARCHAR, age INT);"));
foo.query(new SqlFieldsQuery("INSERT INTO person (id, name, age) VALUES 
(1, 'Vassisualy', 45);"));
foo.query(new SqlFieldsQuery("ALTER TABLE person DROP COLUMN (age);"));

System.err.println(foo.query(new SqlFieldsQuery("SELECT * FROM 
person;")).getAll());
}
}

gives

[[1, Vassisualy]]

Regards,
--
Ilya Kasnacheev