[GitHub] ignite pull request #2480: IGNITE-5860 make client reconnect on router's sus...

[dmekhanikov]

Github user dmekhanikov closed the pull request at:

https://github.com/apache/ignite/pull/2480


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-6138) JDBC driver metadata queries operate on cache/type instead of schema/table

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-6138:
---

 Summary: JDBC driver metadata queries operate on cache/type 
instead of schema/table
 Key: IGNITE-6138
 URL: https://issues.apache.org/jira/browse/IGNITE-6138
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.1
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


It has to use toUpperCase() to address the mismatch which isn't a correct 
approach

Some additional information should be passed down from MetadataTask in 
GridCacheSqlMetadata.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6139) JDBC driver should return actual values for get*Version()

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-6139:
---

 Summary: JDBC driver should return actual values for get*Version()
 Key: IGNITE-6139
 URL: https://issues.apache.org/jira/browse/IGNITE-6139
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.1
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


Right now it returns:
Database version 1.0 (suggested - actual version from server nodes)
JDBC version 1.0 (suggested - 4.1, that's what's in Java 7)
Driver version 1.0 (suggested - actual version of running Ignite code)

Database product name is "Ignite Cache", probably keep that.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Shell script to stop Ignite node

[Sergey Kozlov]

Hi

Looks like we're trying reinvent a bicycle.

First we have to admit that current installation procedure is very poor
("unzip and run"). As result we need scripts to manage Apache Ignite
process(es)

We should to move to better integration under *nix system (at least),
namely:

1. Provide a script to *nix for start/stop/restart/reload Apache Ignite as
system service (a deamon). At the beginning we can focus on Ubuntu and
RedHat (CentOS)

2. Provide the start confugration for Apache Ignite where provide
non-spring options like number of Apache Ignite instances, JVM options,
work/log directories etc (at might be ini file)

3. Provide platform-adopted packages (*.pkg, *deb) in the own repository
that will install (upgrade) Apache Ignite in proper directories, for
instance: configuration in /etc/ignite/config, visorcmd in /usr/sbin, start
scripts in /etc/init.d, work directory in /var/lib/ignite, logs in
/var/log/ignite/. It will significally reduce the efforts to install Apache
Ignite for newbies ("just add the reporsitory and run package manager")



On Tue, Aug 22, 2017 at 2:16 AM, Dmitriy Setrakyan 
wrote:

> Completely agree. Why do we have Visor CLI join the cluster at all? It
> should be issuing REST requests to the cluster remotely. Essentially, every
> command should be a REST request/response.
>
> Is this possible?
>
> D.
>
> On Mon, Aug 21, 2017 at 12:25 PM, Denis Magda  wrote:
>
> > Igniters,
> >
> > I would propose to make this a feature of Visor CLI.
> >
> > Recently we’ve added the script to control a cluster activation [1]
> > explaining that Visor CLI is not flexible for that. If we keep moving
> this
> > way at some point there will be the whole zoo of scripts.
> >
> > Why don’t we improve Visor CLI letting it connect to the cluster in the
> > simplest way and embed the proposed feature in it?
> >
> > [1] http://apache-ignite-developers.2346864.n4.nabble.
> > com/Control-sh-script-and-cluster-activation-td20821.html
> > That’s the second
> >
> > —
> > Denis
> >
> > > On Aug 21, 2017, at 11:44 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > >
> > > +1
> > >
> > > -Val
> > >
> > > On Mon, Aug 21, 2017 at 9:06 AM, Yakov Zhdanov 
> > wrote:
> > >
> > >> Guys,
> > >>
> > >> Currently to stop node on a particular server (not using visor or web
> > >> console) I need to run "jps" and then "kill" with a pid.
> > >>
> > >> What if we create ignite-stop.sh which will display the list of
> started
> > >> nodes on local host and allow user to choose one to stop. Also it
> should
> > >> support --all param to stop all and probably a signal.
> > >>
> > >> Thoughts?
> > >>
> > >> --Yakov
> > >>
> >
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

[jira] [Created] (IGNITE-6140) JDBC thin: implement DataSource interface

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6140:


 Summary: JDBC thin: implement DataSource interface
 Key: IGNITE-6140
 URL: https://issues.apache.org/jira/browse/IGNITE-6140
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.2


Implemented {{DataSource}} interface is the required by JDBC compliance.
Also it us useful for external connection pool implementation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6141) JDBC thin: support BLOB and CLOB types

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6141:


 Summary: JDBC thin: support BLOB and CLOB types
 Key: IGNITE-6141
 URL: https://issues.apache.org/jira/browse/IGNITE-6141
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.2


Support BLOB and CLOB types by JDBC thin driver.
It is very useful types. jdbc2 driver has supported one already (IGNITE-5203).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6142) JDBC thin: support ARRAY type

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6142:


 Summary: JDBC thin: support ARRAY type
 Key: IGNITE-6142
 URL: https://issues.apache.org/jira/browse/IGNITE-6142
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


Support SQL ARRAY type by JDBC thin driver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6143) JDBC thin: support streams in ResultSet and PreparedStatement

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6143:


 Summary: JDBC thin: support streams in ResultSet and 
PreparedStatement
 Key: IGNITE-6143
 URL: https://issues.apache.org/jira/browse/IGNITE-6143
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


The most of the methods:
- {{PreparedStatement.setXXXStream()}
- {{ResultSet.getXXXStream}}
must be implemented. It is the requirement of the JDBC compliance.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6144) JDBC thin: add tests for escape syntax required for JDBC compliance

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6144:


 Summary: JDBC thin: add tests for escape syntax required for JDBC 
compliance
 Key: IGNITE-6144
 URL: https://issues.apache.org/jira/browse/IGNITE-6144
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


JDBC compliance requires support of escape syntax for types (e.g. date, 
timestamp etc: {{{d '-mm-dd'}}}) and functions (e.g.: {{{fn  
(argument list)}}} ).
We have to add unit test for this feature.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6145) JDBC thin: support connection pooling

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6145:


 Summary: JDBC thin: support connection pooling
 Key: IGNITE-6145
 URL: https://issues.apache.org/jira/browse/IGNITE-6145
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


W have to support connection pool to JDBC compliance.
At the very least we must test ourselves with well-known pooling providers 
(DBCP, c3p0). 
This is blocked by IGNITE-6140





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6146) JDBC thin: improve error handling, supports vendorCode and SQLState

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6146:


 Summary: JDBC thin: improve error handling, supports vendorCode 
and SQLState
 Key: IGNITE-6146
 URL: https://issues.apache.org/jira/browse/IGNITE-6146
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


{{SQLException}} must provide useful information about an error via the 
{{vendorCode}} and {{SQLState}} properties.
See more: [SQLState 
messages|ftp://ftp.software.ibm.com/ps/products/db2/info/vr6/htm/db2m0/db2state.htm#HDRSTTMSG].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6147) JDBC thin: improve error handling, support SQLWarning

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6147:


 Summary: JDBC thin: improve error handling, support SQLWarning
 Key: IGNITE-6147
 URL: https://issues.apache.org/jira/browse/IGNITE-6147
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


SQLWarning must be produced in appropriate cases.
All implementations of the JDBC interfaces must support {{getWarnings}} and 
{{clearWarnings}} methods.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6148) DBC thin: improve error handling, support categorized SQLExceptions

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-6148:


 Summary: DBC thin: improve error handling, support categorized 
SQLExceptions
 Key: IGNITE-6148
 URL: https://issues.apache.org/jira/browse/IGNITE-6148
 Project: Ignite
  Issue Type: Task
  Components: jdbc
Affects Versions: 2.1
Reporter: Taras Ledkov


Current implementation of the JDBC thin driver throws pure SQLException with 
string message only. Appropriate subclasses of SQLException must be thrown in 
special cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6149) Create prototype application

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-6149:


 Summary: Create prototype application
 Key: IGNITE-6149
 URL: https://issues.apache.org/jira/browse/IGNITE-6149
 Project: Ignite
  Issue Type: Sub-task
Reporter: Semen Boikov
Assignee: Semen Boikov


Need create simple prototype application to verify major concepts:
- which data should be stored on coordinator and on data nodes
- filtering algorithm for getAll and sql operations
- clean up of committed versions



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6150) ignite-osgi-karaf-licenses.txt is absent in a build

[Ksenia Rybakova (JIRA)]

Ksenia Rybakova created IGNITE-6150:
---

 Summary: ignite-osgi-karaf-licenses.txt is absent in a build
 Key: IGNITE-6150
 URL: https://issues.apache.org/jira/browse/IGNITE-6150
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.9
Reporter: Ksenia Rybakova


optional/ignite-osgi-karaf/licenses/ignite-osgi-karaf-licenses.txt file is 
absent in a build.
While building there is a warning:
{noformat}
[23:57:23][Step 30/31] [INFO] An Ant BuildException has occured: Warning: Could 
not find file 
/var/lib/teamcity/data/work/374178e7af8a6b6/incubator-ignite/modules/osgi-karaf/target/classes/META-INF/licenses.txt
 to copy.
[23:57:23][Step 30/31] around Ant part ..
 @ 4:270 in 
/var/lib/teamcity/data/work/374178e7af8a6b6/incubator-ignite/modules/osgi-karaf/target/antrun/build-main.xml
[23:57:23][Step 30/31] 
/var/lib/teamcity/data/work/374178e7af8a6b6/incubator-ignite/modules/osgi-karaf/target/antrun/build-main.xml:4:
 Warning: Could not find file 
/var/lib/teamcity/data/work/374178e7af8a6b6/incubator-ignite/modules/osgi-karaf/target/classes/META-INF/licenses.txt
 to copy.
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.taskdefs.Copy.copySingleFile(Copy.java:619)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.taskdefs.Copy.execute(Copy.java:444)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
[23:57:23][Step 30/31]  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
[23:57:23][Step 30/31]  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[23:57:23][Step 30/31]  at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[23:57:23][Step 30/31]  at java.lang.reflect.Method.invoke(Method.java:606)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[23:57:23][Step 30/31]  at org.apache.tools.ant.Task.perform(Task.java:348)
[23:57:23][Step 30/31]  at org.apache.tools.ant.Target.execute(Target.java:390)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.Target.performTasks(Target.java:411)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
[23:57:23][Step 30/31]  at 
org.apache.tools.ant.Project.executeTarget(Project.java:1368)
[23:57:23][Step 30/31]  at 
org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
[23:57:23][Step 30/31]  at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
[23:57:23][Step 30/31]  at 
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
[23:57:23][Step 30/31]  at 
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
[23:57:23][Step 30/31]  at 
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
[23:57:23][Step 30/31]  at 
org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
[23:57:23][Step 30/31]  at 
org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
[23:57:23][Step 30/31]  at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
[23:57:23][Step 30/31]  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
[23:57:23][Step 30/31]  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[23:57:23][Step 30/31]  at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[23:57:23][Step 30/31]  at java.lang.reflect.Method.invoke(Method.java:606)
[23:57:23][Step 30/31]  at 
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
[23:57:23][Step 30/31]  at 
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
[23:57:23][Step 30/31]  at 
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
[23:57:23][Step 30/31]  at 
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite: configuration changes at runtime

[Yakov Zhdanov]

>I think the main question I have is whether this configuration change will
>survive restarts. Part of me says it should, and the other part says it
>shouldn't.

Dmitry,

I think that configuration changes should survive restarts if persistence
is enabled. Moreover, newly joining nodes should get the current settings
of the running nodes.

--Yakov

Re: Ignite: configuration changes at runtime

[Yakov Zhdanov]

>Not really. What if the user configured Ignite from Java API, without any
>XML. What will be saved? To prevent this ambiguity, I think the cleanest
>way is to ask user to save the configuration to a file explicitly.

I think it does not matter too much how the grid was configured. In any
case IgniteConfiguration object is created. And we will be allowing to
change only very few parameters that are mostly of primitive type or enum.

Once changed configuration is saved to metastore. When restarting we should
apply it by default or allow user to override it with provided one.

--Yakov

Re: Cluster auto activation design proposal

[Sergey Chugunov]

Dmitriy,

As I understand you use the term "minimalActivationNodes" as a synonym for
BaselineTopology concept.
In that case I agree with you that we can replace both "establish*" methods
with a simple setter method (see below in summary).

Summing up the whole discussion I see the functionality as following:

New concept BaselineTopology is introduced. The main features it enables
are:

   1. automatic activation of cluster;

   2. easy management of cluster topology changes (planned nodes
   maintenance, adding new nodes etc);

   3. eliminating of rebalancing traffic on short-term node failures.


Use cases to create BLT:

   1. user starts up new cluster of desired number of nodes and activates
   it using existing API. BLT is created with all nodes presented in the
   cluster at the moment of activation, no API is needed;

   2. user prepares BLT using web-console or visor CMD tools and sets it to
   the cluster. New API setter is needed:
   Ignite.activation().setBaselineTopology(BaselineTopology blt);

   3. user provides via static configuration a list of nodes that are
   expected to be in the cluster.
   User starts nodes one by one; when all preconfigured nodes are started
   cluster is activated and BLT is created.
   As list of nodes may be huge it is provided via separate file to avoid
   flooding main configuration.


Igniters, does this description match with your understanding of
functionality? If it does I'll create a set of tickets and start working on
implementation.

Thanks,
Sergey.


On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan 
wrote:

> I still do not see why anyone would explicitly call these 2 methods:
>
> *Ignite::activation::establishBaselineTopology();*
> *Ignite::activation::establishBaselineTopology(BaselineTopology bltTop);*
>
> For example, if a web console, or some other admin process, want to
> automatically set currently started nodes as the baseline topology,
> shouldn't they just call a setter for minimalActivationNodes?
>
> D.
>
> On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev 
> wrote:
>
> > API is proposed in the head of the thread by Sergey, as I understood:
> > __
> >
> > API for BaselineTopology manipulation may look like this:
> >
> > *Ignite::activation::establishBaselineTopology();*
> > *Ignite::activation::establishBaselineTopology(BaselineTopology
> bltTop);*
> >
> > Both methods will establish BT and activate cluster once it is
> established.
> >
> > The first one allows user to establish BT using current topology. If any
> > changes happen to the topology during establishing process, user will be
> > notified and allowed to proceed or abort the procedure.
> >
> > Second method allows to use some monitoring'n'management tools like
> > WebConsole where user can prepare a list of nodes, using them create a BT
> > and send to the cluster a command to finally establish it.
> >
> > From high level BaselineTopology entity contains only collection of
> nodes:
> >
> > *BaselineTopology {*
> > *  Collection nodes;*
> > *}*
> >
> > *TopologyNode* here contains information about node - its consistent id
> and
> > set of user attributes used to calculate affinity function.
> > 
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-
> > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > design-proposal-tp20295p21066.html
> > Sent from the Apache Ignite Developers mailing list archive at
> Nabble.com.
> >
>

[jira] [Created] (IGNITE-6151) ODBC: Implement SQL_ATTR_CONNECTION_DEAD connection attribute

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6151:
---

 Summary: ODBC: Implement SQL_ATTR_CONNECTION_DEAD connection 
attribute
 Key: IGNITE-6151
 URL: https://issues.apache.org/jira/browse/IGNITE-6151
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego
 Fix For: 2.2


Currently, there is no way to check whether the connection to the server node 
is dead. Implementation of {{SQL_ATTR_CONNECTION_DEAD}} would solve this issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6152) Wrong response from grid via jdbc driver

[Ilya Suntsov (JIRA)]

Ilya Suntsov created IGNITE-6152:


 Summary: Wrong response from grid via jdbc driver
 Key: IGNITE-6152
 URL: https://issues.apache.org/jira/browse/IGNITE-6152
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.1
 Environment: OS X 10.10.5
java version "1.8.0_91"
Reporter: Ilya Suntsov
 Fix For: 2.2


Now I'm testing command line tool - *sqlline* ([link to 
github|https://github.com/julianhyde/sqlline]) for execute sql queries to AI 
grid via thin jdbc driver.

When I was trying to get column list from AI grid and H2 database I got 2 
different responses from the same table.
Ignite:
{noformat}
0: jdbc:ignite:thin://127.0.0.1/> !columns person
++++
|   TABLE_CAT|  TABLE_SCHEM   |   
TABLE_NAME   |
++++
|| PUBLIC | PERSON  
   |
|| PUBLIC | PERSON  
   |
|| PUBLIC | PERSON  
   |
++++
{noformat}  

H2:
{noformat}
+---+-++-+-+---+-+---+++-+-++---+--+---+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME |  DATA_TYPE  | TYPE_NAME 
| COLUMN_SIZE | BUFFER_LENGTH | DECIMAL_DIGITS | NUM_PREC_RADIX |  NULLABLE   | 
REMARKS | COLUMN_DEF | SQL_DATA_TYPE | SQL_DATETIME_SUB | CHAR_OCTE |
+---+-++-+-+---+-+---+++-+-++---+--+---+
| TEST" | PUBLIC  | PERSON | ID  | -5  | BIGINT
| 19  | 19| 0  | 10 | 0   | 
|| -5| 0| 19|
| TEST" | PUBLIC  | PERSON | NAME| 12  | VARCHAR   
| 2147483647  | 2147483647| 0  | 10 | 1   | 
|| 12| 0| 214748364 |
| TEST" | PUBLIC  | PERSON | CITY_ID | -5  | BIGINT
| 19  | 19| 0  | 10 | 0   | 
|| -5| 0| 19|
+---+-++-+-+---+-+---+++-+-++---+--+---+
{noformat}






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6153) ODBC: Implement SQLCancel

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6153:
---

 Summary: ODBC: Implement SQLCancel
 Key: IGNITE-6153
 URL: https://issues.apache.org/jira/browse/IGNITE-6153
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


ODBC core level conformance requires implementation of the {{SQLCancel}}. See 
https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlcancel-function 
for details.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6154) Incorrect check checkpoint pages collection

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-6154:
--

 Summary: Incorrect check checkpoint pages collection
 Key: IGNITE-6154
 URL: https://issues.apache.org/jira/browse/IGNITE-6154
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Govorukhin
Assignee: Dmitriy Govorukhin
Priority: Critical
 Fix For: 2.2


There is incorrect check !F.empty(collection) in checkpoint thread.
There should be a full check all elements, because collection is collection of 
GridMultiCollectionWrapper, and we must check all mutlicollections.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6155) Add GC Date Stamps to yardstick gc log

[Oleg Ostanin (JIRA)]

Oleg Ostanin created IGNITE-6155:


 Summary: Add GC Date Stamps to yardstick gc log
 Key: IGNITE-6155
 URL: https://issues.apache.org/jira/browse/IGNITE-6155
 Project: Ignite
  Issue Type: Improvement
Reporter: Oleg Ostanin


We need to add -XX:+PrintGCDateStamps to Jvm options in yardstick config files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6156) ODBC: Implement SQL_ATTR_QUERY_TIMEOUT statement attribute

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6156:
---

 Summary: ODBC: Implement SQL_ATTR_QUERY_TIMEOUT statement attribute
 Key: IGNITE-6156
 URL: https://issues.apache.org/jira/browse/IGNITE-6156
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


Need to implement {{SQL_ATTR_QUERY_TIMEOUT}} statement attribute. See 
https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlsetstmtattr-function
 for details.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2497: IGNITE-6154 Incorrect check checkpoint pages coll...

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/2497

IGNITE-6154 Incorrect check checkpoint pages collection



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-6154

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2497.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2497


commit ec357391b7bba564168802ae390bdf827e0aac20
Author: Dmitriy Govorukhin 
Date:   2017-08-22T13:34:31Z

IGNITE-6154 fix incorrect check checkpoint pages




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2498: IGNITE-6155 added new jvm flag for printing gc da...

[oleg-ostanin]

GitHub user oleg-ostanin opened a pull request:

https://github.com/apache/ignite/pull/2498

IGNITE-6155 added new jvm flag for printing gc date stamps



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6155

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2498.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2498


commit 03211d2f8f2e52886bfba7887beccc3dcc3e5f97
Author: oleg-ostanin 
Date:   2017-08-22T13:39:31Z

IGNITE-6155 added new jvm flag for printing gc date stamps




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-6157) C++: Query example: Incorrect output if run example with standalone node

[Irina Zaporozhtseva (JIRA)]

Irina Zaporozhtseva created IGNITE-6157:
---

 Summary: C++: Query example: Incorrect output if run example with 
standalone node
 Key: IGNITE-6157
 URL: https://issues.apache.org/jira/browse/IGNITE-6157
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 1.9
Reporter: Irina Zaporozhtseva
Priority: Minor


C++: Query example: Incorrect output if run example with standalone node

without standalone node:

{code}
Following people have 'Master' in their resumes: 
1 : Person [orgId=1, lastName=Doe, firstName=John, salary=2000, resume=John Doe 
has Master Degree.]
4 : Person [orgId=2, lastName=Smith, firstName=Jane, salary=2000, resume=Jane 
Smith has Master Degree.]

Following people have 'Bachelor' in their resumes: 
2 : Person [orgId=1, lastName=Doe, firstName=Jane, salary=1000, resume=Jane Doe 
has Bachelor Degree.]
3 : Person [orgId=2, lastName=Smith, firstName=John, salary=1000, resume=John 
Smith has Bachelor Degree.]
{code}

with standalone node (rows are repeated):
{code}
Following people have 'Master' in their resumes: 
1 : Person [orgId=1, lastName=Doe, firstName=John, salary=2000, resume=John Doe 
has Master Degree.]
1 : Person [orgId=1, lastName=Doe, firstName=John, salary=2000, resume=John Doe 
has Master Degree.]
4 : Person [orgId=2, lastName=Smith, firstName=Jane, salary=2000, resume=Jane 
Smith has Master Degree.]

Following people have 'Bachelor' in their resumes: 
2 : Person [orgId=1, lastName=Doe, firstName=Jane, salary=1000, resume=Jane Doe 
has Bachelor Degree.]
3 : Person [orgId=2, lastName=Smith, firstName=John, salary=1000, resume=John 
Smith has Bachelor Degree.]
2 : Person [orgId=1, lastName=Doe, firstName=Jane, salary=1000, resume=Jane Doe 
has Bachelor Degree.]
3 : Person [orgId=2, lastName=Smith, firstName=John, salary=1000, resume=John 
Smith has Bachelor Degree.]
{code}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6158) ODBC: Implement proper support of SQLPrimaryKeys

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6158:
---

 Summary: ODBC: Implement proper support of SQLPrimaryKeys
 Key: IGNITE-6158
 URL: https://issues.apache.org/jira/browse/IGNITE-6158
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


Currently, {{SQLPrimaryKeys}} always returns empty result set. Need to 
implement properly. See details at 
https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlprimarykeys-function.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6159) ODBC: Implement SQLStatistics

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6159:
---

 Summary: ODBC: Implement SQLStatistics
 Key: IGNITE-6159
 URL: https://issues.apache.org/jira/browse/IGNITE-6159
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


Implement {{SQLStatistics}} for ODBC driver. Details: 
https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlstatistics-function



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6160) ODBC: Implement asynchronous mode.

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6160:
---

 Summary: ODBC: Implement asynchronous mode.
 Key: IGNITE-6160
 URL: https://issues.apache.org/jira/browse/IGNITE-6160
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


See for details: 
https://docs.microsoft.com/en-us/sql/odbc/reference/develop-app/asynchronous-execution-notification-method



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6161) ODBC: Make sure that driver supports ODBC API 3.8

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6161:
---

 Summary: ODBC: Make sure that driver supports ODBC API 3.8
 Key: IGNITE-6161
 URL: https://issues.apache.org/jira/browse/IGNITE-6161
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


Currently, we only support ODBC 3.0. Some details on what should be done can be 
found here:
https://docs.microsoft.com/en-us/sql/odbc/reference/develop-driver/upgrading-a-3-5-driver-to-a-3-8-driver
https://docs.microsoft.com/en-us/sql/odbc/reference/what-s-new-in-odbc-3-8



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Yardstick framework for Ignite 2.1

[Aleksei Zaitsev]

If newer versions of Ignite delivers with benchmarks I think it's redundant to 
support one more project.

21.08.2017, 16:17, "Dmitriy Setrakyan" :
> Igniters,
>
> We should either update this repository or delete it. Why have a repository
> with outdated benchmarks.
>
> Thoughts?
>
> D.
>
> On Mon, Aug 21, 2017 at 7:14 AM, Aleksei Zaitsev 
> wrote:
>
>>  Thanks, looks like that's what I need.
>>
>>  21.08.2017, 12:17, "Nikolai Tikhonov" :
>>  > Hello,
>>  >
>>  > Yes, this repository contains benchmarks for old Apache Ignite and
>>  > yardstick version. The last versions Apache Ignite distributed with
>>  > benchmarks. You can download there
>>  > https://ignite.apache.org/download.cgi#binaries and found them and
>>  > instruction in /benchmarks folder.
>>  >
>>  > On Mon, Aug 21, 2017 at 10:54 AM,  wrote:
>>  >
>>  >> I am handling with yardstick benchmark framework for Ignite[1], but the
>>  >> latest version available for 1.9. In versions 2.0-2.1 were made many
>>  back
>>  >> incompatible changes. Is there any newer version that is working with
>>  >> Ignite 2.1 not to do a double job?
>>  >>
>>  >> [1] https://github.com/apacheignite/yardstick-ignite/

[jira] [Created] (IGNITE-6162) ODBC: Implement support for multiple result sets

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-6162:
---

 Summary: ODBC: Implement support for multiple result sets
 Key: IGNITE-6162
 URL: https://issues.apache.org/jira/browse/IGNITE-6162
 Project: Ignite
  Issue Type: Improvement
  Components: odbc
Affects Versions: 2.1
Reporter: Igor Sapego


Implement support of multiple result sets. Some details can be found here:
https://docs.microsoft.com/en-us/sql/odbc/reference/develop-app/multiple-results



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Yardstick framework for Ignite 2.1

[Dmitriy Setrakyan]

On Tue, Aug 22, 2017 at 7:19 AM, Aleksei Zaitsev 
wrote:

> If newer versions of Ignite delivers with benchmarks I think it's
> redundant to support one more project.
>

Well, there are google links. Can we somehow copy the new benchmarks into
the yardstick repo?


>
> 21.08.2017, 16:17, "Dmitriy Setrakyan" :
> > Igniters,
> >
> > We should either update this repository or delete it. Why have a
> repository
> > with outdated benchmarks.
> >
> > Thoughts?
> >
> > D.
> >
> > On Mon, Aug 21, 2017 at 7:14 AM, Aleksei Zaitsev  >
> > wrote:
> >
> >>  Thanks, looks like that's what I need.
> >>
> >>  21.08.2017, 12:17, "Nikolai Tikhonov" :
> >>  > Hello,
> >>  >
> >>  > Yes, this repository contains benchmarks for old Apache Ignite and
> >>  > yardstick version. The last versions Apache Ignite distributed with
> >>  > benchmarks. You can download there
> >>  > https://ignite.apache.org/download.cgi#binaries and found them and
> >>  > instruction in /benchmarks folder.
> >>  >
> >>  > On Mon, Aug 21, 2017 at 10:54 AM,  wrote:
> >>  >
> >>  >> I am handling with yardstick benchmark framework for Ignite[1], but
> the
> >>  >> latest version available for 1.9. In versions 2.0-2.1 were made many
> >>  back
> >>  >> incompatible changes. Is there any newer version that is working
> with
> >>  >> Ignite 2.1 not to do a double job?
> >>  >>
> >>  >> [1] https://github.com/apacheignite/yardstick-ignite/
>

Criteria query to web console

[Alexey Kuznetsov]

Hi, All.

Recently, Yakov opened issue:  Criteria query to web console [1]

We can create criteria based filter that could be passed to ScanQuery over
BinaryObjecs.

I think it make sens to implement this filter as first-class citizen
of org.apache.ignite.cache.query package and also support it from Web
Console. But it could be re-used directly from code also.

I think that we should implement a set of predicates to support AND, OR and
NOT logical operations to group several predicates.

We should support following operations:
Numbers:  ==, <, >, <=, >=, !=
Strings: equals, startsWith, endsWith, contains, matchRegExp (and also with
IgnoreCase mode).
Dates: ==, <, >, <=, >=, !=, between
All: isNull, isDefined*

*isDefined - Here I mean a check that some property is present in
BinaryObject

What do you think?
If I missed some operations, please advice what could be added.

Also, I think that a separate issue should be created for such "rules
engine for BinaryObjects".
Make sense?


[1] https://issues.apache.org/jira/browse/IGNITE-6132


-- 
Alexey Kuznetsov

Re: Scan Queries: rows, updates and binary builders

[Vladimir Ozerov]

Agree in general. All cache access methods should be transactional -
whether this is IgniteCache, SQL or scan. It should be possible to make
in-place update during scanning as well.

On Tue, Aug 22, 2017 at 3:08 AM, Dmitriy Setrakyan 
wrote:

> On Mon, Aug 21, 2017 at 9:01 AM, Yakov Zhdanov 
> wrote:
>
> > Guys,
> >
> > I was asked how effectively change many rows in cache many times.
> > Currently, this can done via ScanQuery - we send affinityCall(), inside
> > callable we iterate over local primary cache partitions, filter entries
> by
> > predicate and then do cache puts for rows that are wanted to change.
> >
> > I want to suggest more convenient API. Please share your thoughts.
> >
> > 1. Key Value pair in scan query is replaced with a single object
> IgniteRow
> > which is basically a set of name-value pairs - the union of ones from key
> > and value. If field names are not unique for a key-value pair then this
> > pair is omitted with warning.
> >
>
> I am not sure what this would achieve. Can you explain why this is better
> than the key-value API? Are you trying to avoid an extra cache lookup for
> puts?
>
>
> >
> > 2. IgniteRow should be mutable. We can allow to change any field in the
> row
> > and store results back to cache. If field belongs to cache key, then new
> > key should be inserted and previous one removed. Optionally we can
> support
> > throwing exception if key belongs to another node/partition after
> mutation.
> >
>
> Is this essentially the same as binary builder interface?
>
>
> >
> > 3. Such updates can be enlisted to ongoing transaction. For simplicity,
> let
> > them be local transactions for the node we are running scan query on.
> > However, I would not bother with this for now.
> >
>
> Makes sense.
>
>
> >
> > 4. I think it is inconvenient to convert binary object to builder, change
> > field and serialize back to binary object. How about having BinaryObject
> > replace(String fldName, Obj newVal)? If it is a simple replace then it
> can
> > be done directly in the array or a copy of the initial array which seems
> to
> > be times more efficient. Imagine we change an int field? Or a string to
> > another string of the same length? This should also be applied to
> > IgniteRow.
> >
>
> Will the replace operation return a builder, or another BinaryObject? Seems
> a bit over-complicated to me. Are we trying to save on a few lines of code?
>
>
> >
> > --Yakov
> >
>

Re: Criteria query to web console

[Vladimir Ozerov]

My strong recommendation here is to think carefully of all potential use
cases first, so that we end up with consistent and extendable API. This
filters could be useful for both ScanQuery, platforms and 3-rd party
clients. Hence, "query" package is definitely not the right place. I would
said that this is our "expression language" or so, and put these filters to
"org.apache.ignite.el" package.

On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov 
wrote:

> Hi, All.
>
> Recently, Yakov opened issue:  Criteria query to web console [1]
>
> We can create criteria based filter that could be passed to ScanQuery over
> BinaryObjecs.
>
> I think it make sens to implement this filter as first-class citizen
> of org.apache.ignite.cache.query package and also support it from Web
> Console. But it could be re-used directly from code also.
>
> I think that we should implement a set of predicates to support AND, OR and
> NOT logical operations to group several predicates.
>
> We should support following operations:
> Numbers:  ==, <, >, <=, >=, !=
> Strings: equals, startsWith, endsWith, contains, matchRegExp (and also with
> IgnoreCase mode).
> Dates: ==, <, >, <=, >=, !=, between
> All: isNull, isDefined*
>
> *isDefined - Here I mean a check that some property is present in
> BinaryObject
>
> What do you think?
> If I missed some operations, please advice what could be added.
>
> Also, I think that a separate issue should be created for such "rules
> engine for BinaryObjects".
> Make sense?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6132
>
>
> --
> Alexey Kuznetsov
>

[jira] [Created] (IGNITE-6163) Upgrade to Spark 2.2.0

[Denis Magda (JIRA)]

Denis Magda created IGNITE-6163:
---

 Summary: Upgrade to Spark 2.2.0
 Key: IGNITE-6163
 URL: https://issues.apache.org/jira/browse/IGNITE-6163
 Project: Ignite
  Issue Type: Improvement
Reporter: Denis Magda
Priority: Blocker
 Fix For: 2.2


Apache Ignite's Spark integration module has to be updated to Spark 2.2.0 to 
avoid issues like this:
https://github.com/dmagda/IgniteSparkIoT/issues/1

In short, Spark 2.1.0 and Spark 2.2.0 modules can't be mixed together, 
otherwise, the exceptions like this will pop up:

{code}
Exception in thread "main" java.lang.NoSuchMethodError: 
org.apache.spark.ui.SparkUI.setStreamingJobProgressListener(Lorg/apache/spark/scheduler/SparkListener;)V
at 
org.apache.spark.streaming.ui.StreamingTab.(StreamingTab.scala:41)
at 
org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:192)
at 
org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:85)
at 
org.apache.spark.streaming.api.java.JavaStreamingContext.(JavaStreamingContext.scala:138)
at 
org.apache.ignite.iot.SparkStreamerStartup.main(SparkStreamerStartup.java:71)
17/08/22 18:27:25 INFO SparkContext: Invoking stop() from shutdown hook
17/08/22 18:27:25 INFO SparkUI: Stopped Spark web UI at http://10.0.1.6:4040
{code} 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Upgrade to Spark 2.2.0

[Denis Magda]

Seems it’s time to upgrade ignite-spark module once again. Soon we should start 
getting more issues related to Spark 2.2.0 incompatibility.

—
Denis

> Begin forwarded message:
> 
> From: "Denis Magda (JIRA)" 
> Subject: [jira] [Created] (IGNITE-6163) Upgrade to Spark 2.2.0
> Date: August 22, 2017 at 5:26:00 PM PDT
> To: dev@ignite.apache.org
> Reply-To: dev@ignite.apache.org
> 
> https://issues.apache.org/jira/browse/IGNITE-6163 
> 

Re: Upgrade to Spark 2.2.0

[Dmitriy Setrakyan]

+1

On Tue, Aug 22, 2017 at 5:30 PM, Denis Magda  wrote:

> Seems it’s time to upgrade ignite-spark module once again. Soon we should
> start getting more issues related to Spark 2.2.0 incompatibility.
>
> —
> Denis
>
> > Begin forwarded message:
> >
> > From: "Denis Magda (JIRA)" 
> > Subject: [jira] [Created] (IGNITE-6163) Upgrade to Spark 2.2.0
> > Date: August 22, 2017 at 5:26:00 PM PDT
> > To: dev@ignite.apache.org
> > Reply-To: dev@ignite.apache.org
> >
> > https://issues.apache.org/jira/browse/IGNITE-6163 <
> https://issues.apache.org/jira/browse/IGNITE-6163>
>

[jira] [Created] (IGNITE-6164) H2 dependency is not resolved from ignite-indexing:2.1.0

[Pranas Baliuka (JIRA)]

Pranas Baliuka created IGNITE-6164:
--

 Summary: H2 dependency is not resolved from ignite-indexing:2.1.0
 Key: IGNITE-6164
 URL: https://issues.apache.org/jira/browse/IGNITE-6164
 Project: Ignite
  Issue Type: Bug
Reporter: Pranas Baliuka
Priority: Minor


I've tested the first code sample from Ignite book code 
https://github.com/srecon/ignite-book-code-samples with dependecies:

{code}

UTF-8
2.1.0



junit
junit
3.8.1
test


org.apache.ignite
ignite-core
${ignite.version}


org.apache.ignite
ignite-spring


org.apache.ignite
ignite-indexing


{code}

Runtime exception indicates what transitive dependency:
{code}

com.h2database
h2
1.4.195
runtime

{code}

is not resolved from the {{ignite-indexing}} . Consider fixing



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Add-on/Extension Request: Please add GA Grid (beta) to Ignite website

[Prachi Garg]

Turik,

GA Grid is added to the "Addons and Related Solutions" section of the
Ignite website - https://ignite.apache.org/addons.html

Thank you for your contribution!

-P

On Thu, Aug 17, 2017 at 1:38 PM, Denis Magda  wrote:

> Hi Turik,
>
> Sure, we will do this with pleasure.
>
> Prachi, could you add the component to the mentioned page?
>
> —
> Denis
>
> > On Aug 16, 2017, at 9:16 PM, techbysample  wrote:
> >
> > Forum Admin,
> >
> > Please add project:
> >
> > GA Grid (beta): Distributive in Memory Genetic Algorithm component for
> > Ignite
> >
> > to "Addons and Extensions" section of Apache Ignite website:
> >
> > https://ignite.apache.org/addons.html
> >
> >  com/file/n20947/GAGrid_Overview.png>
> >
> > (F)itness Calculation, (C)rossover, and (M)utation operations are
> modeled as
> > a ComputeTask for distributive behavior. The ComputeTask is split into
> > multiple ComputeJobs, (ie: Fn,Cn,Mn) assigned to respective nodes, and
> > executed in parallel.
> >
> > All of these ComputeTasks leverage Apache Ignite's Affinity Colocation to
> > route ComputeJobs to respective nodes where Chromosomes are stored in
> cache.
> >
> > Additional information about GA Grid (beta) found here:
> >
> > https://github.com/techbysample/gagrid
> >
> > Please advise.
> >
> > Best,
> > Turik Campbell
> >
> >
> >
> > --
> > View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/Ignite-Add-on-Extension-
> Request-Please-add-GA-Grid-beta-to-Ignite-website-tp20947.html
> > Sent from the Apache Ignite Developers mailing list archive at
> Nabble.com.
>
>

Re: Criteria query to web console

[Dmitriy Setrakyan]

I am a bit confused. How about "select * from SomeTable where myCriteria >
1"?

On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov 
wrote:

> My strong recommendation here is to think carefully of all potential use
> cases first, so that we end up with consistent and extendable API. This
> filters could be useful for both ScanQuery, platforms and 3-rd party
> clients. Hence, "query" package is definitely not the right place. I would
> said that this is our "expression language" or so, and put these filters to
> "org.apache.ignite.el" package.
>
> On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov 
> wrote:
>
> > Hi, All.
> >
> > Recently, Yakov opened issue:  Criteria query to web console [1]
> >
> > We can create criteria based filter that could be passed to ScanQuery
> over
> > BinaryObjecs.
> >
> > I think it make sens to implement this filter as first-class citizen
> > of org.apache.ignite.cache.query package and also support it from Web
> > Console. But it could be re-used directly from code also.
> >
> > I think that we should implement a set of predicates to support AND, OR
> and
> > NOT logical operations to group several predicates.
> >
> > We should support following operations:
> > Numbers:  ==, <, >, <=, >=, !=
> > Strings: equals, startsWith, endsWith, contains, matchRegExp (and also
> with
> > IgnoreCase mode).
> > Dates: ==, <, >, <=, >=, !=, between
> > All: isNull, isDefined*
> >
> > *isDefined - Here I mean a check that some property is present in
> > BinaryObject
> >
> > What do you think?
> > If I missed some operations, please advice what could be added.
> >
> > Also, I think that a separate issue should be created for such "rules
> > engine for BinaryObjects".
> > Make sense?
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> >
> >
> > --
> > Alexey Kuznetsov
> >
>

Re: Ignite: configuration changes at runtime

[Dmitriy Setrakyan]

On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov  wrote:

> >Not really. What if the user configured Ignite from Java API, without any
> >XML. What will be saved? To prevent this ambiguity, I think the cleanest
> >way is to ask user to save the configuration to a file explicitly.
>
> I think it does not matter too much how the grid was configured. In any
> case IgniteConfiguration object is created. And we will be allowing to
> change only very few parameters that are mostly of primitive type or enum.
>
> Once changed configuration is saved to metastore. When restarting we should
> apply it by default or allow user to override it with provided one.
>

I would prefer that users explicitly specify which configuration file to
use on startup. This will prevent any magic. Still think that explicit call
to saveConfigurationFile(...) is the best approach.

Re: Ignite: configuration changes at runtime

[Nick Pordash]

The notion of saving changes to a configuration file does not make sense
for lots of use cases, Kubernetes is one example. Anything in the
container's filesystem is considered transient since it will get destroyed
when a pod is restarted.

I would think that runtime changes should be managed in the cluster
metadata somewhere instead of expecting them to be exported elsewhere like
a file. If new nodes join the cluster then runtime changes received as a
part of a node join operation would get applied prior to node activation
from completing. These would override any node configuration provided on
node startup, regardless of how it was configured.

IMO if someone wants to make runtime changes AND propagate those changes to
wherever they are doing external configuration management then perhaps that
should be left as an exercise for the user. There are so many ways that
this could be done and often what is sitting on the filesystem would easily
get overridden by some other mechanism (f.e ansible, salt, puppet, chef,
container orchestration systems, etc).

-Nick

On Tue, Aug 22, 2017, 6:54 PM Dmitriy Setrakyan 
wrote:

> On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov 
> wrote:
>
> > >Not really. What if the user configured Ignite from Java API, without
> any
> > >XML. What will be saved? To prevent this ambiguity, I think the cleanest
> > >way is to ask user to save the configuration to a file explicitly.
> >
> > I think it does not matter too much how the grid was configured. In any
> > case IgniteConfiguration object is created. And we will be allowing to
> > change only very few parameters that are mostly of primitive type or
> enum.
> >
> > Once changed configuration is saved to metastore. When restarting we
> should
> > apply it by default or allow user to override it with provided one.
> >
>
> I would prefer that users explicitly specify which configuration file to
> use on startup. This will prevent any magic. Still think that explicit call
> to saveConfigurationFile(...) is the best approach.
>

Re: Cluster auto activation design proposal

[Dmitriy Setrakyan]

Sergey, I am still confused. What is the BaselineTopology interface in your
example? I thought that you agreed with me that we simply need a setter for
activation nodes, no?

D.

On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov 
wrote:

> Dmitriy,
>
> As I understand you use the term "minimalActivationNodes" as a synonym for
> BaselineTopology concept.
> In that case I agree with you that we can replace both "establish*" methods
> with a simple setter method (see below in summary).
>
> Summing up the whole discussion I see the functionality as following:
>
> New concept BaselineTopology is introduced. The main features it enables
> are:
>
>1. automatic activation of cluster;
>
>2. easy management of cluster topology changes (planned nodes
>maintenance, adding new nodes etc);
>
>3. eliminating of rebalancing traffic on short-term node failures.
>
>
> Use cases to create BLT:
>
>1. user starts up new cluster of desired number of nodes and activates
>it using existing API. BLT is created with all nodes presented in the
>cluster at the moment of activation, no API is needed;
>
>2. user prepares BLT using web-console or visor CMD tools and sets it to
>the cluster. New API setter is needed:
>Ignite.activation().setBaselineTopology(BaselineTopology blt);
>
>3. user provides via static configuration a list of nodes that are
>expected to be in the cluster.
>User starts nodes one by one; when all preconfigured nodes are started
>cluster is activated and BLT is created.
>As list of nodes may be huge it is provided via separate file to avoid
>flooding main configuration.
>
>
> Igniters, does this description match with your understanding of
> functionality? If it does I'll create a set of tickets and start working on
> implementation.
>
> Thanks,
> Sergey.
>
>
> On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan 
> wrote:
>
> > I still do not see why anyone would explicitly call these 2 methods:
> >
> > *Ignite::activation::establishBaselineTopology();*
> > *Ignite::activation::establishBaselineTopology(BaselineTopology
> bltTop);*
> >
> > For example, if a web console, or some other admin process, want to
> > automatically set currently started nodes as the baseline topology,
> > shouldn't they just call a setter for minimalActivationNodes?
> >
> > D.
> >
> > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> admitr...@gridgain.com>
> > wrote:
> >
> > > API is proposed in the head of the thread by Sergey, as I understood:
> > > __
> > >
> > > API for BaselineTopology manipulation may look like this:
> > >
> > > *Ignite::activation::establishBaselineTopology();*
> > > *Ignite::activation::establishBaselineTopology(BaselineTopology
> > bltTop);*
> > >
> > > Both methods will establish BT and activate cluster once it is
> > established.
> > >
> > > The first one allows user to establish BT using current topology. If
> any
> > > changes happen to the topology during establishing process, user will
> be
> > > notified and allowed to proceed or abort the procedure.
> > >
> > > Second method allows to use some monitoring'n'management tools like
> > > WebConsole where user can prepare a list of nodes, using them create a
> BT
> > > and send to the cluster a command to finally establish it.
> > >
> > > From high level BaselineTopology entity contains only collection of
> > nodes:
> > >
> > > *BaselineTopology {*
> > > *  Collection nodes;*
> > > *}*
> > >
> > > *TopologyNode* here contains information about node - its consistent id
> > and
> > > set of user attributes used to calculate affinity function.
> > > 
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-ignite-
> > > developers.2346864.n4.nabble.com/Cluster-auto-activation-
> > > design-proposal-tp20295p21066.html
> > > Sent from the Apache Ignite Developers mailing list archive at
> > Nabble.com.
> > >
> >
>

Re: Ignite: configuration changes at runtime

[Denis Magda]

I’m fully share Nick’s point of view on this. If you change any configuration 
parameter in runtime then go ahead and update your static configuration or a 
set of DDL statements that are only used during a cluster restart.

—
Denis

> On Aug 22, 2017, at 7:13 PM, Nick Pordash  wrote:
> 
> The notion of saving changes to a configuration file does not make sense
> for lots of use cases, Kubernetes is one example. Anything in the
> container's filesystem is considered transient since it will get destroyed
> when a pod is restarted.
> 
> I would think that runtime changes should be managed in the cluster
> metadata somewhere instead of expecting them to be exported elsewhere like
> a file. If new nodes join the cluster then runtime changes received as a
> part of a node join operation would get applied prior to node activation
> from completing. These would override any node configuration provided on
> node startup, regardless of how it was configured.
> 
> IMO if someone wants to make runtime changes AND propagate those changes to
> wherever they are doing external configuration management then perhaps that
> should be left as an exercise for the user. There are so many ways that
> this could be done and often what is sitting on the filesystem would easily
> get overridden by some other mechanism (f.e ansible, salt, puppet, chef,
> container orchestration systems, etc).
> 
> -Nick
> 
> On Tue, Aug 22, 2017, 6:54 PM Dmitriy Setrakyan 
> wrote:
> 
>> On Tue, Aug 22, 2017 at 4:43 AM, Yakov Zhdanov 
>> wrote:
>> 
 Not really. What if the user configured Ignite from Java API, without
>> any
 XML. What will be saved? To prevent this ambiguity, I think the cleanest
 way is to ask user to save the configuration to a file explicitly.
>>> 
>>> I think it does not matter too much how the grid was configured. In any
>>> case IgniteConfiguration object is created. And we will be allowing to
>>> change only very few parameters that are mostly of primitive type or
>> enum.
>>> 
>>> Once changed configuration is saved to metastore. When restarting we
>> should
>>> apply it by default or allow user to override it with provided one.
>>> 
>> 
>> I would prefer that users explicitly specify which configuration file to
>> use on startup. This will prevent any magic. Still think that explicit call
>> to saveConfigurationFile(...) is the best approach.
>> 

Re: Policy for update third-party dependencies

[Nick Pordash]

Hi Val,

Pretty much, with obvious exceptions being integration modules with other
projects. If the dependency is well isolated, then shading could be
beneficial.

I've also had to do this for client libraries operating inside other
frameworks (I've had to shade netty to avoid conflicting with user code).
It's a good alternative since relying on things like OSGi isn't all that
practical due to lack of widespread adoption.

-Nick

On Mon, Aug 21, 2017, 10:48 AM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Hi Nick,
>
> Do you suggest to build and deploy uber-jars that has no external
> dependencies?
>
> -Val
>
> On Sun, Aug 20, 2017 at 1:02 PM, Nick Pordash 
> wrote:
>
> > If the dependency is not exposed by the public API then another
> alternative
> > is to simply shade the artifact and then this becomes a non-issue for
> > users.
> >
> > Considering Ignite is a platform that executes user code via compute and
> > service grid I personally think it would be good to minimize the number
> of
> > dependencies that can potentially conflict with user code.
> >
> > -Nick
> >
> > On Sun, Aug 20, 2017, 11:51 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > Keep in mind that some projects can use *older* version of third-party
> > > libraries as well, and dependency upgrade can break them. In other
> words,
> > > dependency upgrade is in many cases an incompatible change for us, so
> we
> > > should do this with care.
> > >
> > > Unless there is a specific reason to upgrade a specific dependency, I
> > think
> > > it's better to postpone it until major version.
> > >
> > > -Val
> > >
> > > On Sun, Aug 20, 2017 at 5:04 AM 李玉珏@163 <18624049...@163.com> wrote:
> > >
> > > > If the third party library is incompatible with the new version and
> the
> > > > old version (such as lucene3.5.0-5.5.2), and the dependent version of
> > > > Ignite is older, it may cause conflicts in the user's system.
> > > > For such scenarios, I think that updating third-party dependencies's
> > > > major version is valuable.
> > > >
> > > >
> > > > 在 2017/8/17 上午8:26, Denis Magda 写道:
> > > > > I would respond why do we need to update? Some bug, new
> capabilities,
> > > > security breach? Alexey K., please shed some light on this.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > >> On Aug 16, 2017, at 5:12 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >
> > > > wrote:
> > > > >>
> > > > >> On Wed, Aug 16, 2017 at 5:02 PM, Denis Magda 
> > > wrote:
> > > > >>
> > > > >>> Honestly, I wouldn’t touch a dependency if it works like a charm
> > and
> > > > >>> nobody requested us to migrate to a new version.
> > > > >>>
> > > > >>> Why do you need to update Apache Common coded?
> > > > >>>
> > > > >> Not sure I agree. Why not update it?
> > > > >>
> > > > >>
> > > > >>>
> > > > >>> —
> > > > >>> Denis
> > > > >>>
> > > >  On Aug 16, 2017, at 10:36 AM, Alexey Kuznetsov <
> > > akuznet...@apache.org
> > > > >
> > > > >>> wrote:
> > > >  Done
> > > > 
> > > >  https://issues.apache.org/jira/browse/IGNITE-6090
> > > > 
> > > >  On Wed, Aug 16, 2017 at 8:01 PM, Dmitriy Setrakyan <
> > > > >>> dsetrak...@apache.org>
> > > >  wrote:
> > > > 
> > > > > The answer is Yes, we should update. Jira ticket assigned to
> the
> > > next
> > > > > release should be enough in my view.
> > > > >
> > > > > D.
> > > > >
> > > > > On Wed, Aug 16, 2017 at 2:38 AM, Alexey Kuznetsov <
> > > > >>> akuznet...@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hi, All!
> > > > >>
> > > > >> Do we have any policy for updating third-party dependencies?
> > > > >>
> > > > >> For example, I found that we are using very old  Apache Common
> > > codec
> > > > > v.1.6
> > > > >> (released in 2011)
> > > > >> And latest is Apache Common codec v.1.10
> > > > >>
> > > > >> Do we need to update to new versions from time to time?
> > > > >> And how?
> > > > >>
> > > > >> Just create JIRA issue, update pom.xml and run all tests on
> TC -
> > > > will
> > > > >>> be
> > > > >> enough?
> > > > >>
> > > > >> --
> > > > >> Alexey Kuznetsov
> > > > >>
> > > > 
> > > > 
> > > >  --
> > > >  Alexey Kuznetsov
> > > > >>>
> > > >
> > > >
> > > >
> > >
> >
>

Re: Criteria query to web console

[Alexey Kuznetsov]

Vladimir,

Thanks  for your feedback, I think it make sense.

Dmitriy,

No, critera based filter wull work only for ScanQuery, see ScanQuery
constructor with filter:
ScanQuery(@Nullable IgniteBiPredicate filter)

This should be implemented with buider, smth like this:

CriteriaBuilder builder = new CriteriaBuilder();

IgniteBiPredicate filter =
builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(TCriteria(...)).build();

ScanQuery q = new ScanQuery(filter);

cache.query(q)

And we could have nice Web UI for this: IgniteBiPredicate filter =
builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(TCriteria(...)).build();


On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan 
wrote:

> I am a bit confused. How about "select * from SomeTable where myCriteria >
> 1"?
>
> On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov 
> wrote:
>
> > My strong recommendation here is to think carefully of all potential use
> > cases first, so that we end up with consistent and extendable API. This
> > filters could be useful for both ScanQuery, platforms and 3-rd party
> > clients. Hence, "query" package is definitely not the right place. I
> would
> > said that this is our "expression language" or so, and put these filters
> to
> > "org.apache.ignite.el" package.
> >
> > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov  >
> > wrote:
> >
> > > Hi, All.
> > >
> > > Recently, Yakov opened issue:  Criteria query to web console [1]
> > >
> > > We can create criteria based filter that could be passed to ScanQuery
> > over
> > > BinaryObjecs.
> > >
> > > I think it make sens to implement this filter as first-class citizen
> > > of org.apache.ignite.cache.query package and also support it from Web
> > > Console. But it could be re-used directly from code also.
> > >
> > > I think that we should implement a set of predicates to support AND, OR
> > and
> > > NOT logical operations to group several predicates.
> > >
> > > We should support following operations:
> > > Numbers:  ==, <, >, <=, >=, !=
> > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and also
> > with
> > > IgnoreCase mode).
> > > Dates: ==, <, >, <=, >=, !=, between
> > > All: isNull, isDefined*
> > >
> > > *isDefined - Here I mean a check that some property is present in
> > > BinaryObject
> > >
> > > What do you think?
> > > If I missed some operations, please advice what could be added.
> > >
> > > Also, I think that a separate issue should be created for such "rules
> > > engine for BinaryObjects".
> > > Make sense?
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> >
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Re: Criteria query to web console

[Dmitriy Setrakyan]

I still don't get it. If I need a filter, then I will just use SQL.

On Tue, Aug 22, 2017 at 8:46 PM, Alexey Kuznetsov 
wrote:

> Vladimir,
>
> Thanks  for your feedback, I think it make sense.
>
> Dmitriy,
>
> No, critera based filter wull work only for ScanQuery, see ScanQuery
> constructor with filter:
> ScanQuery(@Nullable IgniteBiPredicate filter)
>
> This should be implemented with buider, smth like this:
>
> CriteriaBuilder builder = new CriteriaBuilder();
>
> IgniteBiPredicate filter =
> builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(
> TCriteria(...)).build();
>
> ScanQuery q = new ScanQuery(filter);
>
> cache.query(q)
>
> And we could have nice Web UI for this: IgniteBiPredicate filter =
> builder.add(TCriteria(...)).and(TCriteria(...)).or(TCriteria(...)).andNot(
> TCriteria(...)).build();
>
>
> On Wed, Aug 23, 2017 at 8:46 AM, Dmitriy Setrakyan 
> wrote:
>
> > I am a bit confused. How about "select * from SomeTable where myCriteria
> >
> > 1"?
> >
> > On Tue, Aug 22, 2017 at 9:16 AM, Vladimir Ozerov 
> > wrote:
> >
> > > My strong recommendation here is to think carefully of all potential
> use
> > > cases first, so that we end up with consistent and extendable API. This
> > > filters could be useful for both ScanQuery, platforms and 3-rd party
> > > clients. Hence, "query" package is definitely not the right place. I
> > would
> > > said that this is our "expression language" or so, and put these
> filters
> > to
> > > "org.apache.ignite.el" package.
> > >
> > > On Tue, Aug 22, 2017 at 6:30 PM, Alexey Kuznetsov <
> akuznet...@apache.org
> > >
> > > wrote:
> > >
> > > > Hi, All.
> > > >
> > > > Recently, Yakov opened issue:  Criteria query to web console [1]
> > > >
> > > > We can create criteria based filter that could be passed to ScanQuery
> > > over
> > > > BinaryObjecs.
> > > >
> > > > I think it make sens to implement this filter as first-class citizen
> > > > of org.apache.ignite.cache.query package and also support it from Web
> > > > Console. But it could be re-used directly from code also.
> > > >
> > > > I think that we should implement a set of predicates to support AND,
> OR
> > > and
> > > > NOT logical operations to group several predicates.
> > > >
> > > > We should support following operations:
> > > > Numbers:  ==, <, >, <=, >=, !=
> > > > Strings: equals, startsWith, endsWith, contains, matchRegExp (and
> also
> > > with
> > > > IgnoreCase mode).
> > > > Dates: ==, <, >, <=, >=, !=, between
> > > > All: isNull, isDefined*
> > > >
> > > > *isDefined - Here I mean a check that some property is present in
> > > > BinaryObject
> > > >
> > > > What do you think?
> > > > If I missed some operations, please advice what could be added.
> > > >
> > > > Also, I think that a separate issue should be created for such "rules
> > > > engine for BinaryObjects".
> > > > Make sense?
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6132
> > > >
> > > >
> > > > --
> > > > Alexey Kuznetsov
> > > >
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>