[jira] [Created] (IGNITE-14457) Update Ignite 3 binary build structure
Valentin Kulichenko created IGNITE-14457:
Summary: Update Ignite 3 binary build structure
Key: IGNITE-14457
URL: https://issues.apache.org/jira/browse/IGNITE-14457
Project: Ignite
Issue Type: Task
Components: build
Affects Versions: 3.0.0-alpha1
Reporter: Valentin Kulichenko
Fix For: 3.0.0-alpha2
In the first alpha release, we provided binaries as separate single-file
downloads for Unix and Windows. This approach doesn't work for two reasons:
# Website download for the Unix binary doesn't work properly, because the file
doesn't have an extension. This doesn't seem to have a reasonable solution.
# Binaries that are downloaded from the website are required to include
{{NOTICE}} and {{LICENSE}} files.
See IGNITE-13978 and INFRA-21332 for more details.
As a solution, let's switch to a ZIP file containing the following:
* {{ignite}} (Unix CLI tool)
* {{ignite.exe}} (Windows CLI tool)
* {{NOTICE}}
* {{LICENSE}}
* {{README}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14151) Set up commit status publisher on TC
Valentin Kulichenko created IGNITE-14151: Summary: Set up commit status publisher on TC Key: IGNITE-14151 URL: https://issues.apache.org/jira/browse/IGNITE-14151 Project: Ignite Issue Type: Task Reporter: Valentin Kulichenko Assignee: Peter Ivanov 'Run All' build should publish the status to the appropriate PR on GitHub. We will then update the process to allow merges only with successful builds. See here: https://www.jetbrains.com/help/teamcity/commit-status-publisher.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14060) UsageTest#multiRootConfigurationTest fails in main
Valentin Kulichenko created IGNITE-14060: Summary: UsageTest#multiRootConfigurationTest fails in main Key: IGNITE-14060 URL: https://issues.apache.org/jira/browse/IGNITE-14060 Project: Ignite Issue Type: Bug Affects Versions: 3.0.0-alpha1 Reporter: Valentin Kulichenko [~sergey-chugunov] Sergey, looks like you created this test. Please take a look. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13978) Ignite 3 alpha Unix binary opens on click instead of being downloaded
Valentin Kulichenko created IGNITE-13978:
Summary: Ignite 3 alpha Unix binary opens on click instead of
being downloaded
Key: IGNITE-13978
URL: https://issues.apache.org/jira/browse/IGNITE-13978
Project: Ignite
Issue Type: Bug
Components: website
Reporter: Valentin Kulichenko
Fix For: 3.0.0-alpha1
1. Go to binary downloads: https://ignite.apache.org/download.cgi#binaries
2. Locate the 3.0.0 alpha and click on the download for Unix.
3. Expected: {{ignite}} file is downloaded. Actual: the file opens up in the
same window.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13950) IgniteCliInterfaceTest can fail on Windows
Valentin Kulichenko created IGNITE-13950:
Summary: IgniteCliInterfaceTest can fail on Windows
Key: IGNITE-13950
URL: https://issues.apache.org/jira/browse/IGNITE-13950
Project: Ignite
Issue Type: Bug
Reporter: Valentin Kulichenko
{{IgniteCliInterfaceTest}} does not take into account that line separators can
differ between different operating systems. Under Windows, it's typically
{{\r\n}} instead of just {{\n}}, which causes multiple test cases to fail.
To make sure these tests pass on every OS, we should compare a multi-line
output line by line (i.e. convert both expected and actual string values to
lists of lines and verify that these lists are the same).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13948) Extend the Ignite 3 Getting Started guide by showing how to start a cluster
Valentin Kulichenko created IGNITE-13948: Summary: Extend the Ignite 3 Getting Started guide by showing how to start a cluster Key: IGNITE-13948 URL: https://issues.apache.org/jira/browse/IGNITE-13948 Project: Ignite Issue Type: Improvement Components: documentation Affects Versions: 3.0.0-alpha1 Reporter: Valentin Kulichenko The current version of the guide shows how to start a single node. In the future versions, it should demonstrate how to create a cluster (at least two nodes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13943) Change the default port for REST
Valentin Kulichenko created IGNITE-13943: Summary: Change the default port for REST Key: IGNITE-13943 URL: https://issues.apache.org/jira/browse/IGNITE-13943 Project: Ignite Issue Type: Improvement Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 3.0.0-alpha1 Port 8080 is often consumed by other apps, so it doesn't seem to be a good choice for the default REST port. Let's choose something more Ignite-specific. Otherwise, there is a big chance a user will not be able to go through the Getting Started guide (all the management commands will require explicit endpoint). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13942) [CLI] Highlighting is inconsistent across different terminals
Valentin Kulichenko created IGNITE-13942: Summary: [CLI] Highlighting is inconsistent across different terminals Key: IGNITE-13942 URL: https://issues.apache.org/jira/browse/IGNITE-13942 Project: Ignite Issue Type: Bug Reporter: Valentin Kulichenko Colors and the highlighting used by the CLI tool differ from terminal to terminal. Need to investigate and see how this can be fixed/improved. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13934) Some of the source files are missing license headers
Valentin Kulichenko created IGNITE-13934: Summary: Some of the source files are missing license headers Key: IGNITE-13934 URL: https://issues.apache.org/jira/browse/IGNITE-13934 Project: Ignite Issue Type: Improvement Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 3.0.0-alpha1 Need to add the license check to the build process, and fix all the missing headers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13933) Add an ability to customize documentation source repo
Valentin Kulichenko created IGNITE-13933: Summary: Add an ability to customize documentation source repo Key: IGNITE-13933 URL: https://issues.apache.org/jira/browse/IGNITE-13933 Project: Ignite Issue Type: Improvement Components: website Reporter: Valentin Kulichenko Fix For: 3.0.0-alpha1 Documentation deployment script current has the original Ignite repo hardcoded: https://github.com/apache/ignite-website/blob/master/_docs/build.sh#L42 Ignite 3.x is located in a different repo: https://github.com/apache/ignite-3 Suggestion: add a parameter that would allow specifying the version (2.x or 3.x). The script should then pick the corresponding repo URL and fetch the sources from there. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13927) Custom Maven repositories do not work
Valentin Kulichenko created IGNITE-13927:
Summary: Custom Maven repositories do not work
Key: IGNITE-13927
URL: https://issues.apache.org/jira/browse/IGNITE-13927
Project: Ignite
Issue Type: Bug
Reporter: Valentin Kulichenko
Fix For: 3.0.0-alpha1
I've created a staging [1] for 3.0.0-alpha1 and tried to install from this
repo, but it failed to work:
{noformat}
Valentin-Kulichenko-MacBook-Pro-1772:apache-ignite-3 vkulichenko$
./modules/cli/target/ignite init
--repo=https://repository.apache.org/content/repositories/orgapacheignite-1494/
Creating directories... Done!
+++
| Binaries Directory |
/Users/vkulichenko/GridGain/apache-ignite-3/modules/cli/target/ignite-bin |
+++
| Work Directory |
/Users/vkulichenko/GridGain/apache-ignite-3/modules/cli/target/ignite-work |
+++
Installing org.apache.ignite:ignite-runner:3.0.0-alpha1...
|>
|0%
[unresolved dependency: org.apache.ignite#ignite-runner;3.0.0-alpha1: not found]
{noformat}
Downloading through {{add module}} doesn't work either:
{noformat}
Valentin-Kulichenko-MacBook-Pro-1772:apache-ignite-3 vkulichenko$
./modules/cli/target/ignite module add
mvn:org.apache.ignite:ignite-runner:3.0.0-alpha1
--repo=https://repository.apache.org/content/repositories/orgapacheignite-1494
Installing org.apache.ignite:ignite-runner:3.0.0-alpha1...
|>
|0%
[unresolved dependency: org.apache.ignite#ignite-runner;3.0.0-alpha1: not found]
{noformat}
[1] https://repository.apache.org/content/repositories/orgapacheignite-1494/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13925) Configure Maven deployment
Valentin Kulichenko created IGNITE-13925: Summary: Configure Maven deployment Key: IGNITE-13925 URL: https://issues.apache.org/jira/browse/IGNITE-13925 Project: Ignite Issue Type: Task Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 3.0.0-alpha1 Need to properly configure GPG and deploy plugins to support Maven deployment for the release. Staging repo URL: https://repository.apache.org/service/local/staging/deploy/maven2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13923) 'init' command doesn't support custom Maven URL
Valentin Kulichenko created IGNITE-13923:
Summary: 'init' command doesn't support custom Maven URL
Key: IGNITE-13923
URL: https://issues.apache.org/jira/browse/IGNITE-13923
Project: Ignite
Issue Type: Bug
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 3.0.0-alpha1
This functionality is available for the {{module add}} command, but not for the
{{init}} command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13916) POM structure is incorrect
Valentin Kulichenko created IGNITE-13916: Summary: POM structure is incorrect Key: IGNITE-13916 URL: https://issues.apache.org/jira/browse/IGNITE-13916 Project: Ignite Issue Type: Bug Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 3.0.0-alpha1 The current POM structure has the following flaws: # Some of the modules are not inherited from the root POM. # Some of the modules still use Java 8, although we've transitioned to Java 11 earlier. # Source JARs are not generated. This needs to be fixed before the release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13899) Remove demo artifacts from the CLI module
Valentin Kulichenko created IGNITE-13899:
Summary: Remove demo artifacts from the CLI module
Key: IGNITE-13899
URL: https://issues.apache.org/jira/browse/IGNITE-13899
Project: Ignite
Issue Type: Sub-task
Reporter: Valentin Kulichenko
The CLI module is currently named {{cli-demo}} and also contains the
{{demo-module-all}} submodule.
This needs to be cleaned up. Let's also check for any other demo artifacts that
might be there.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13898) Change default location for ignite-bin and ignite-work folders
Valentin Kulichenko created IGNITE-13898:
Summary: Change default location for ignite-bin and ignite-work
folders
Key: IGNITE-13898
URL: https://issues.apache.org/jira/browse/IGNITE-13898
Project: Ignite
Issue Type: Sub-task
Reporter: Valentin Kulichenko
Currently, {{ignite-bin}} and {{ignite-work}} folders are created in the
current directory. This might be confusing because the script can be run out of
any folder, which makes the behavior unpredictable.
Let's create these folders somewhere under {{$HOME}} instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13894) Improve look&feel of the CLI tool
Valentin Kulichenko created IGNITE-13894: Summary: Improve look&feel of the CLI tool Key: IGNITE-13894 URL: https://issues.apache.org/jira/browse/IGNITE-13894 Project: Ignite Issue Type: Sub-task Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Need to go through all the commands and fix/improve from usability improvement. This includes the following: * Names of commands, options, and parameters. * Command output. * Text colors and styles. * etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13784) List of modules should indicate which modules are currently installed
Valentin Kulichenko created IGNITE-13784:
Summary: List of modules should indicate which modules are
currently installed
Key: IGNITE-13784
URL: https://issues.apache.org/jira/browse/IGNITE-13784
Project: Ignite
Issue Type: Sub-task
Reporter: Valentin Kulichenko
The {{module list}} command shows which optional Ignite modules are available
and can be used. But there is no way to know which ones are already installed.
It would be useful to have some sort of marker in the table indicating that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13783) Prevent starting nodes with the same consistent ID
Valentin Kulichenko created IGNITE-13783:
Summary: Prevent starting nodes with the same consistent ID
Key: IGNITE-13783
URL: https://issues.apache.org/jira/browse/IGNITE-13783
Project: Ignite
Issue Type: Sub-task
Reporter: Valentin Kulichenko
Currently, it is possible to start two or more nodes with the same consistent
ID. Trying to stop using that ID succeeds for all the nodes, but an error
appears in the output: {{Stop of node n1 was failed}}.
Here is the full output:
{noformat}
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$ ./ignite node start
n1
Started ignite node.
PID: 40644
Consistent Id: n1
Log file: /Users/vkulichenko/GridGain/cli-demo/ignite-work/n1.log
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$ ./ignite node start
n1
Started ignite node.
PID: 40648
Consistent Id: n1
Log file: /Users/vkulichenko/GridGain/cli-demo/ignite-work/n1.log
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$ ./ignite node list
+---+---+-+
| PID | Consistent Id | Log
|
+---+---+-+
| 40644 | n1|
/Users/vkulichenko/GridGain/cli-demo/ignite-work/n1.log |
+---+---+-+
| 40648 | n1|
/Users/vkulichenko/GridGain/cli-demo/ignite-work/n1.log |
+---+---+-+
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$ ./ignite node stop n1
Stop of node n1 was failed
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$ ./ignite node list
No running nodes
Valentin-Kulichenko-MacBook-Pro-1772:cli-demo vkulichenko$
{noformat}
We should have a mechanism that prevents starting multiple nodes with the same
ID.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13782) [CLI Tool] Manual folder removal causes exceptions
Valentin Kulichenko created IGNITE-13782:
Summary: [CLI Tool] Manual folder removal causes exceptions
Key: IGNITE-13782
URL: https://issues.apache.org/jira/browse/IGNITE-13782
Project: Ignite
Issue Type: Bug
Reporter: Valentin Kulichenko
# Run {{ignite init}}
# Delete {{ignite-bin}} and {{ignite-work}} folders
# Run {{ignite node start n1}}
Steps result in the following exception:
{nofomat}
Exception in thread "main" java.lang.RuntimeException:
java.nio.file.NoSuchFileException:
/Users/vkulichenko/GridGain/cli-demo/ignite-bin/2.9.0/cli
at
org.apache.ignite.internal.v2.builtins.SystemPathResolver.list(SystemPathResolver.java:39)
at
org.apache.ignite.internal.v2.spec.IgniteCliSpec.loadSubcommands(IgniteCliSpec.java:81)
at
org.apache.ignite.internal.v2.spec.IgniteCliSpec.lambda$main$0(IgniteCliSpec.java:64)
at java.base/java.util.Optional.ifPresent(Optional.java:176)
at
org.apache.ignite.internal.v2.spec.IgniteCliSpec.main(IgniteCliSpec.java:64)
Caused by: java.nio.file.NoSuchFileException:
/Users/vkulichenko/GridGain/cli-demo/ignite-bin/2.9.0/cli
at
java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at
java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at
java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:412)
at java.base/java.nio.file.Files.newDirectoryStream(Files.java:476)
at java.base/java.nio.file.Files.list(Files.java:3765)
at
org.apache.ignite.internal.v2.builtins.SystemPathResolver.list(SystemPathResolver.java:28)
... 4 more
{nofomat}
{{ignite init}} fails with the same exception as well, so the only way to fix
this is to manually delete the {{.ignitecfg}} file, which the user is not aware
of.
We should have the support for re-initialization, and suggest to do that in
case the described scenario occures.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13740) Create a "Getting Started" documentation page for Ignite 3.0
Valentin Kulichenko created IGNITE-13740: Summary: Create a "Getting Started" documentation page for Ignite 3.0 Key: IGNITE-13740 URL: https://issues.apache.org/jira/browse/IGNITE-13740 Project: Ignite Issue Type: Bug Components: documentation Reporter: Valentin Kulichenko Need to create a starting page that describes: # Installation and upgrade process for manual download and RPM/DEB options. # How to add optional modules (both Ignite and user-provided). # How to start a node. # Any other basic ops appropriate for the "Getting Started" page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13739) Document the new CLI tool
Valentin Kulichenko created IGNITE-13739: Summary: Document the new CLI tool Key: IGNITE-13739 URL: https://issues.apache.org/jira/browse/IGNITE-13739 Project: Ignite Issue Type: Sub-task Components: documentation Reporter: Valentin Kulichenko Need to create a documentation page describing how to use the command-line tool. We should provide detailed explanations on all the supported commands, as well as their parameters. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13738) Document the new documentation format
Valentin Kulichenko created IGNITE-13738: Summary: Document the new documentation format Key: IGNITE-13738 URL: https://issues.apache.org/jira/browse/IGNITE-13738 Project: Ignite Issue Type: Sub-task Components: documentation Reporter: Valentin Kulichenko IEP: https://cwiki.apache.org/confluence/display/IGNITE/IEP-55+Unified+Configuration -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13671) Support collocation based on multiple fields
Valentin Kulichenko created IGNITE-13671:
Summary: Support collocation based on multiple fields
Key: IGNITE-13671
URL: https://issues.apache.org/jira/browse/IGNITE-13671
Project: Ignite
Issue Type: Improvement
Reporter: Valentin Kulichenko
Fix For: 3.0
In Ignite 2.x there are two ways to configure collocation: @AffinityKeyMapped
annotation and CacheKeyConfiguration bean. Both allow specifying *only one
field* to be used for collocation.
Let's say there is a key class that looks like this:
{code:java}
class MyCacheKey {
int a;
int b;
int c;
}{code}
In this case, there is no way to collocate the data based on a pair of fields
(e.g. a and b).
In 3.0, we should have an API that would allow to specify two or more fields to
be used for collocation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13416) Ignite 3 test
Valentin Kulichenko created IGNITE-13416: Summary: Ignite 3 test Key: IGNITE-13416 URL: https://issues.apache.org/jira/browse/IGNITE-13416 Project: Ignite Issue Type: Bug Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13209) JavaIgniteCatalogExample doesn't work with a standalone Spark cluster
Valentin Kulichenko created IGNITE-13209:
Summary: JavaIgniteCatalogExample doesn't work with a standalone
Spark cluster
Key: IGNITE-13209
URL: https://issues.apache.org/jira/browse/IGNITE-13209
Project: Ignite
Issue Type: Bug
Components: spark
Affects Versions: 2.8.1
Reporter: Valentin Kulichenko
To reproduce the issue:
# Start Spark master and slave as described here:
[http://spark.apache.org/docs/latest/spark-standalone.html]
# Change the master URL in the {{JavaIgniteCatalogExample}} from "local" to
the one just started.
# Run the example.
Updated code that creates the {{IgniteSparkSession}}:
{code:java}
String libs =
"/Users/vkulichenko/GridGain/releases/apache-ignite-2.8.1-bin/libs";
IgniteSparkSession igniteSession = IgniteSparkSession.builder()
.appName("Spark Ignite catalog example")
.master("spark://Valentin-Kulichenko-MacBook-Pro-1772.local:7077")
.config("spark.executor.instances", "2")
.config("spark.executor.extraClassPath", libs + "/*" + ":" + libs +
"/ignite-spark/*:" + libs + "/ignite-spring/*")
.igniteConfig(CONFIG)
.getOrCreate();
{code}
Execution fails with this exception:
{noformat}
[2020-07-02 15:50:27,627][ERROR][task-result-getter-3][TaskSetManager] Task 0
in stage 0.0 failed 4 times; aborting job
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to
stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task 0.3 in stage 0.0 (TID 3, 10.0.0.11, executor 0): class
org.apache.ignite.IgniteIllegalStateException: Ignite instance with provided
name doesn't exist. Did you call Ignition.start(..) to start an Ignite
instance? [name=testing]
at org.apache.ignite.internal.IgnitionEx.grid(IgnitionEx.java:1351)
at org.apache.ignite.Ignition.ignite(Ignition.java:528)
at org.apache.ignite.spark.impl.package$.ignite(package.scala:65)
at
org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:238)
at
org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:235)
at org.apache.ignite.spark.Once.apply(IgniteContext.scala:222)
at org.apache.ignite.spark.IgniteContext.ignite(IgniteContext.scala:144)
at
org.apache.ignite.spark.impl.IgniteSQLDataFrameRDD.compute(IgniteSQLDataFrameRDD.scala:65)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-9732) Add joins to Spark Dataframe examples
Valentin Kulichenko created IGNITE-9732:
---
Summary: Add joins to Spark Dataframe examples
Key: IGNITE-9732
URL: https://issues.apache.org/jira/browse/IGNITE-9732
Project: Ignite
Issue Type: Improvement
Components: examples, spark
Affects Versions: 2.6
Reporter: Valentin Kulichenko
Fix For: 2.7
{{IgniteDataFrameExample}} creates two tables - {{city}} and {{person}}, but
only {{person}} is actually used. Need to add join examples.
Would also be great to demonstrate the fact that optimization is working and
joins are executed in Ignite, not Spark (using {{explain()}}, maybe?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9200) Continuous query notifications can be missed if entry processor is retried due to binary type registration
Valentin Kulichenko created IGNITE-9200: --- Summary: Continuous query notifications can be missed if entry processor is retried due to binary type registration Key: IGNITE-9200 URL: https://issues.apache.org/jira/browse/IGNITE-9200 Project: Ignite Issue Type: Improvement Components: cache Reporter: Valentin Kulichenko Assignee: Mikhail Cherkasov Fix For: 2.7 Attachments: InvokeAllContinuousQueryTest.java Due to fix introduced in IGNITE-8926, entry processor will be retried if it triggers binary meta data registration. However, such retries may cause loss of continuous query notifications. Reproducer attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9113) Allocate memory for a data region when first cache assigned to this region is created
Valentin Kulichenko created IGNITE-9113: --- Summary: Allocate memory for a data region when first cache assigned to this region is created Key: IGNITE-9113 URL: https://issues.apache.org/jira/browse/IGNITE-9113 Project: Ignite Issue Type: Improvement Components: cache Affects Versions: 2.6 Reporter: Valentin Kulichenko Fix For: 2.7 Currently we do not create any regions or allocate any offheap memory on client nodes unless it's explicitly configured. This is good behavior, however there is a usability issue caused by the fact that many users have the same config file for both server and clients. This can lead to unexpected excessive memory usage on client side and forces users to maintain two config files in most cases. Same issue is applied to server nodes that do not store any data (e.g. nodes running only services). It's better to allocate memory dynamically, when first cache assigned to a data region is created. More detailed discussion here: http://apache-ignite-developers.2346864.n4.nabble.com/Data-regions-on-client-nodes-td32834.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8154) Add an ability to provide ExceptionListener to JmsStreamer
Valentin Kulichenko created IGNITE-8154:
---
Summary: Add an ability to provide ExceptionListener to JmsStreamer
Key: IGNITE-8154
URL: https://issues.apache.org/jira/browse/IGNITE-8154
Project: Ignite
Issue Type: Improvement
Components: streaming
Affects Versions: 2.4
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.5
JMS API allows to provide custom {{ExceptionListener}} that is invoked when an
exception occurs within JMS queue/topic. Currently, {{JmsStreamer}} doesn't use
this in any way which creates two issues:
* If there is an exception, it's lost and not even logged.
* User can't provide their own listener.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-8139) Need to have better control on the size of SQL on heap cache
Valentin Kulichenko created IGNITE-8139: --- Summary: Need to have better control on the size of SQL on heap cache Key: IGNITE-8139 URL: https://issues.apache.org/jira/browse/IGNITE-8139 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.4 Reporter: Valentin Kulichenko Assignee: Vladimir Ozerov Fix For: 2.5 Currently, if SQL on heap cache is enabled, there is no clear way to control the size of this cache. Looks like its evictions basically depend on evictions in off-heap, so it's really to get OOME. Not very usable. The easiest improvement would be to have a knob (configuration parameter or system property) that would control the size of underlying concurrent map that stores cached rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8027) Expired entry appears back in case of node restart with persistence enabled
Valentin Kulichenko created IGNITE-8027: --- Summary: Expired entry appears back in case of node restart with persistence enabled Key: IGNITE-8027 URL: https://issues.apache.org/jira/browse/IGNITE-8027 Project: Ignite Issue Type: Bug Components: persistence Affects Versions: 2.4 Reporter: Valentin Kulichenko Fix For: 2.5 Detailed description of the issue and a reproducer can be found in this thread: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Expiry-Inconsistency-with-Native-Persistence-td20390.html In short, if entry expires, it can then be read again after node restart. This is reproduced ONLY if node is killed with 'kill -9', in case of graceful stop everything works fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7993) Striped pool can't be disabled
Valentin Kulichenko created IGNITE-7993:
---
Summary: Striped pool can't be disabled
Key: IGNITE-7993
URL: https://issues.apache.org/jira/browse/IGNITE-7993
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.4
Reporter: Valentin Kulichenko
Fix For: 2.5
Javadoc for {{IgniteConfiguration#setStripedPoolSize}} states that striped pool
can be disabled by providing value less or equal than zero:
{noformat}
If set to non-positive value then requests get processed in system pool.
{noformat}
However, doing that prevents node from startup, it fails with the following
exception:
{noformat}
Caused by: class org.apache.ignite.IgniteCheckedException: Invalid stripedPool
thread pool size (must be greater than 0), actual value: 0
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.validateThreadPoolSize(IgnitionEx.java:2061)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1799)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1716)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1144)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:664)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
at org.apache.ignite.Ignition.start(Ignition.java:322)
... 7 more
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7798) Give user an ability to check driver metrics in Cassandra store
Valentin Kulichenko created IGNITE-7798:
---
Summary: Give user an ability to check driver metrics in Cassandra
store
Key: IGNITE-7798
URL: https://issues.apache.org/jira/browse/IGNITE-7798
Project: Ignite
Issue Type: Improvement
Components: cassandra
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.5
Cassandra store uses generic client driver to connect to the cluster. The
driver provides {{Metrics}} object which can be useful for monitoring, however
there is no way for user to acquire it. Need to find a way to expose this
information to public API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7743) JDBC driver allows to connect to non existent schema
Valentin Kulichenko created IGNITE-7743:
---
Summary: JDBC driver allows to connect to non existent schema
Key: IGNITE-7743
URL: https://issues.apache.org/jira/browse/IGNITE-7743
Project: Ignite
Issue Type: Bug
Components: jdbc
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.5
Currently, if one creates a cache without DDL (via {{QueryEntity}} or
{{indexedTypes}}), separate schema for this cache is created. Schema name is
case sensitive, so to connect to it with JDBC driver, it's required to provide
the name in quotes. Here is how it looks like in SqlLine:
{noformat}
./bin/sqlline.sh -u
jdbc:ignite:thin://127.0.0.1/\"CacheQueryExamplePersons\"{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7712) Add an ability to globally enable 'lazy' flag for SQL queries
Valentin Kulichenko created IGNITE-7712:
---
Summary: Add an ability to globally enable 'lazy' flag for SQL
queries
Key: IGNITE-7712
URL: https://issues.apache.org/jira/browse/IGNITE-7712
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.5
We have the {{lazy}} flag that can be set when a particular query is executed,
it's disabled by default.
In some cases it can be useful to enable this flag globally, especially it
makes sense for the tools like Visor and Web Console.
Let's do the following:
* Set it to {{true}} by default in Web Console.
* Add a system property that will allow to do the same on node level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7667) Improve services failover and load balancing
Valentin Kulichenko created IGNITE-7667: --- Summary: Improve services failover and load balancing Key: IGNITE-7667 URL: https://issues.apache.org/jira/browse/IGNITE-7667 Project: Ignite Issue Type: Improvement Components: managed services Affects Versions: 2.3 Reporter: Valentin Kulichenko Currently Ignite services lack proper failover and load balancing capabilities. For example, if there are several node singletons, there is completely no control on which nodes they are deployed or redeployed. Also if all of them are deployed on a single node cluster, adding more nodes does not trigger load balancing, which makes it not scalable. We need to come out with a mechanism to support this so that user can define behavior in different scenarios, probably something similar to what we have in Compute Grid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7641) Add CacheEntry#ttl method
Valentin Kulichenko created IGNITE-7641:
---
Summary: Add CacheEntry#ttl method
Key: IGNITE-7641
URL: https://issues.apache.org/jira/browse/IGNITE-7641
Project: Ignite
Issue Type: Improvement
Components: cache
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Ignite provides a way to specify an expiry policy on per entry level, but there
is no way to know the current TTL for a particular key.
We can add {{CacheEntry#ttl()}} and/or {{IgniteCache#ttl(K key)}} method that
will provide this information. Looks like it's already available via
{{GridCacheMapEntry#ttl()}}, so we just need to properly expose it to public
API.
Here is the user forum discussion about this:
http://apache-ignite-users.70518.x6.nabble.com/Get-TTL-of-the-specific-K-V-entry-td19817.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7337) Spark Data Frames: support saving a data frame in Ignite
Valentin Kulichenko created IGNITE-7337:
---
Summary: Spark Data Frames: support saving a data frame in Ignite
Key: IGNITE-7337
URL: https://issues.apache.org/jira/browse/IGNITE-7337
Project: Ignite
Issue Type: New Feature
Components: spark
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Assignee: Nikolay Izhikov
Priority: Critical
Fix For: 2.4
Once Ignite data source for Spark is implemented, we need to add an ability to
store a data frame in Ignite. Most likely if should be enough to provide
implementation for the following traits:
* {{InsertableRelation}}
* {{CreatableRelationProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7336) Improve peer class loading documentation
Valentin Kulichenko created IGNITE-7336: --- Summary: Improve peer class loading documentation Key: IGNITE-7336 URL: https://issues.apache.org/jira/browse/IGNITE-7336 Project: Ignite Issue Type: Improvement Components: documentation Affects Versions: 2.3 Reporter: Valentin Kulichenko Assignee: Prachi Garg [Zero Deployment|https://apacheignite.readme.io/docs/zero-deployment] page should be updated with the following: * Applicability of the feature (where it can and can't be used). * Warning about anonymous classes and lambdas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7285) Add default query timeout
Valentin Kulichenko created IGNITE-7285:
---
Summary: Add default query timeout
Key: IGNITE-7285
URL: https://issues.apache.org/jira/browse/IGNITE-7285
Project: Ignite
Issue Type: Improvement
Components: cache, sql
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
Currently it's possible to provide timeout only on query level. It would be
very useful to have default timeout value provided on cache startup. Let's add
{{CacheConfiguration#defaultQueryTimeout}} configuration property.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7284) Introduce DEV_ONLY marker to IgniteLogger
Valentin Kulichenko created IGNITE-7284:
---
Summary: Introduce DEV_ONLY marker to IgniteLogger
Key: IGNITE-7284
URL: https://issues.apache.org/jira/browse/IGNITE-7284
Project: Ignite
Issue Type: Improvement
Components: general
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
Need to add markers support to {{IgniteLogger}} and then introduce {{DEV_ONLY}}
marker for warnings that can be suppressed in prod environments.
Problem and solution are discussed in more detail here:
http://apache-ignite-developers.2346864.n4.nabble.com/Suppressing-quot-development-quot-warning-td25195.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7217) Add abilities to monitor custom thread pools
Valentin Kulichenko created IGNITE-7217:
---
Summary: Add abilities to monitor custom thread pools
Key: IGNITE-7217
URL: https://issues.apache.org/jira/browse/IGNITE-7217
Project: Ignite
Issue Type: Improvement
Components: general
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
We have a periodic metrics logger that prints out different stats including the
ones for public and system thread pools:
{noformat}
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=0, idle=8, qSize=0]
{noformat}
However, is user configures a custom thread pools via
{{IgniteConfiguration#setExecutorConfiguration}}, stats for these thread pools
are not added. We also do not register MBeans for these pools.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7207) Exchange worker dies if index contains non existent field
Valentin Kulichenko created IGNITE-7207:
---
Summary: Exchange worker dies if index contains non existent field
Key: IGNITE-7207
URL: https://issues.apache.org/jira/browse/IGNITE-7207
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
If {{QueryEntity}} contains an index with a non existent field, {{createCache}}
first throws this exception which is correct:
{noformat}
[2017-12-14
18:51:23,627][ERROR][exchange-worker-#42%null%][CacheAffinitySharedManager]
Failed to initialize cache. Will try to rollback cache start routine.
[cacheName=test]
class org.apache.ignite.IgniteCheckedException: Field not found: a
at
org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
at
org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:631)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:648)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:584)
at
org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:542)
at
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:438)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:687)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:836)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1816)
at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:751)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:882)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:588)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2278)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "main" javax.cache.CacheException: class
org.apache.ignite.IgniteCheckedException: Field not found: a
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1327)
at
org.apache.ignite.internal.IgniteKernal.createCache(IgniteKernal.java:2817)
at Test.main(Test.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class org.apache.ignite.IgniteCheckedException: Field not found: a
at
org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
at
org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:631)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:648)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:584)
at
org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:542)
at
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:438)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:687)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:836)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1816)
at
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:751)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchange
[jira] [Created] (IGNITE-7179) Create events for cluster activation/deactivation
Valentin Kulichenko created IGNITE-7179: --- Summary: Create events for cluster activation/deactivation Key: IGNITE-7179 URL: https://issues.apache.org/jira/browse/IGNITE-7179 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.3 Reporter: Valentin Kulichenko -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7167) Optimize 'select count(*) from Table'
Valentin Kulichenko created IGNITE-7167:
---
Summary: Optimize 'select count(*) from Table'
Key: IGNITE-7167
URL: https://issues.apache.org/jira/browse/IGNITE-7167
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Currently query like {{select count(*) from Table}} effectively scans the cache
and take a lot of time for large datasets. Probably makes sense to optimize it
to use {{IgniteCache#size}} directly when possible.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7092) Deprecate embedded mode in IgniteRDD
Valentin Kulichenko created IGNITE-7092: --- Summary: Deprecate embedded mode in IgniteRDD Key: IGNITE-7092 URL: https://issues.apache.org/jira/browse/IGNITE-7092 Project: Ignite Issue Type: Improvement Components: spark Affects Versions: 2.3 Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 2.4 Currently we claim to support IgniteRDD in two modes: standalone and embedded. Standalone means there is a separately running Ignite cluster, and Spark start client node(s) to interact with it. In embedded node everything runs within Spark, including Ignite server nodes that are started embedded into Spark executors. The latter case doesn't really work, mainly because the lifecycle of Spark executors is not very predictable - Spark can start and stop them while application is running. In case Ignite cluster is used to store data (which is usually the case), this causes unnecessary rebalancing or even unexpected data loss. I propose to deprecate and eventually discontinue the embedded mode. Luckily, standalone mode is the default one, so we can simply print out a clear warning if one switches to embedded mode, and also mention this in the docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7055) Text query for a particular field not working
Valentin Kulichenko created IGNITE-7055:
---
Summary: Text query for a particular field not working
Key: IGNITE-7055
URL: https://issues.apache.org/jira/browse/IGNITE-7055
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Lucene queries allow to specify a specific field name to search in [1], however
this doesn't seem to work in latest versions of Ignite.
To reproduce, modify {{CacheQueryExample#textQuery}} to use Lucene field
expression:
{code}
QueryCursor> masters =
cache.query(new TextQuery(Person.class, "resume:Master"));
{code}
This query returns empty result.
[1]
http://lucene.apache.org/core/5_5_2/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Fields
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7054) S3 IP finder: support client side encryption
Valentin Kulichenko created IGNITE-7054:
---
Summary: S3 IP finder: support client side encryption
Key: IGNITE-7054
URL: https://issues.apache.org/jira/browse/IGNITE-7054
Project: Ignite
Issue Type: Improvement
Components: s3
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
In case client side encryption [1] is used, it may be required to use
{{AmazonS3EncryptionClient}} instead of regular {{AmazonS3Client}}. We need to
add this option to the S3 IP finder, along with any applicable configuration
parameters.
[1]
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7053) S3 IP finder: support server side encryption
Valentin Kulichenko created IGNITE-7053:
---
Summary: S3 IP finder: support server side encryption
Key: IGNITE-7053
URL: https://issues.apache.org/jira/browse/IGNITE-7053
Project: Ignite
Issue Type: Improvement
Components: s3
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
In case server side encryption [1] is used, it may be required to specify the
algorithm in request headers when saving information in a bucket (can be done
via {{ObjectMetadata#setSSEAlgorithm}} method). To support this we need to add
corresponding configuration property to the S3 IP finder.
[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7052) S3 IP finder: add an ability to provide endpoint address
Valentin Kulichenko created IGNITE-7052:
---
Summary: S3 IP finder: add an ability to provide endpoint address
Key: IGNITE-7052
URL: https://issues.apache.org/jira/browse/IGNITE-7052
Project: Ignite
Issue Type: Improvement
Components: s3
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
By default S3 client detects region automatically by sending special request to
{{us-west-1}}. In case environment is restricted to some other region, this
leads to connection timeout exception.
The issue can be solved by providing a specific region endpoint via
{{AmazonS3Client#setEndpoint}} method. To support this we need to add
{{endpoint}} configuration property to the IP finder.
List of S3 region endpoints:
http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7029) Add an ability to provide multiple connection addresses for thin JDBC driver
Valentin Kulichenko created IGNITE-7029: --- Summary: Add an ability to provide multiple connection addresses for thin JDBC driver Key: IGNITE-7029 URL: https://issues.apache.org/jira/browse/IGNITE-7029 Project: Ignite Issue Type: Improvement Components: jdbc Affects Versions: 2.3 Reporter: Valentin Kulichenko Currently we allow only to provide one address when connecting via thin JDBC driver. This has to issues: * If node driver is connected to goes down, the driver stops working. * Driver has to always go though the same node - this is a bottleneck. As a simple solution we can allow to provide multiple addresses, like MySQL does for example: https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-usagenotes-j2ee-concepts-managing-load-balanced-connections.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6994) Need to document PartitionLossPolicy
Valentin Kulichenko created IGNITE-6994:
---
Summary: Need to document PartitionLossPolicy
Key: IGNITE-6994
URL: https://issues.apache.org/jira/browse/IGNITE-6994
Project: Ignite
Issue Type: Improvement
Components: documentation
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
Since 2.0 we have a feature that makes cache(s) unavailable in case of data
loss; exact behavior is controlled by {{PartitionLossPolicy}}:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/PartitionLossPolicy.html
However, there is no mentioning in documentation about this. Need to provide
explanation of how and when it should be used and provide configuration
examples.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6897) Visor doesn't show the list of caches after persistence enabled cluster restart
Valentin Kulichenko created IGNITE-6897:
---
Summary: Visor doesn't show the list of caches after persistence
enabled cluster restart
Key: IGNITE-6897
URL: https://issues.apache.org/jira/browse/IGNITE-6897
Project: Ignite
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Components: cache, visor
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Fix For: 2.4
# Start a cluster with persistence enabled.
# Create a cache dynamically, put some data.
# Restart the cluster.
# Connect with Visor and execute {{cache}} command to get list of caches.
# No caches are shown.
# Try to access the cache from code (e.g. execute a query).
# Now caches are shown in Visor.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6847) Add an ability to provide custom per operation context
Valentin Kulichenko created IGNITE-6847:
---
Summary: Add an ability to provide custom per operation context
Key: IGNITE-6847
URL: https://issues.apache.org/jira/browse/IGNITE-6847
Project: Ignite
Issue Type: Improvement
Security Level: Public (Viewable by anyone)
Components: general
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Sometimes it's useful to attach custom context information to a distributed
operation. For example, an application that uses Ignite cluster may want to
provide its own ID or name so that it could then be used on other nodes for
auditing purposes.
To support this, we can introduce {{withTag(Serializable tag)}} method on
{{IgniteCache}}, {{IgniteCompute}} and other APIs. If provided, tag is attached
to every remote request and propagated to log messages, events, etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6846) Add metrics for entry processor invocations
Valentin Kulichenko created IGNITE-6846:
---
Summary: Add metrics for entry processor invocations
Key: IGNITE-6846
URL: https://issues.apache.org/jira/browse/IGNITE-6846
Project: Ignite
Issue Type: Improvement
Security Level: Public (Viewable by anyone)
Components: cache
Affects Versions: 2.3
Reporter: Valentin Kulichenko
{{CacheMetrics}} object has multiple metrics for various cache operations like
{{get}}, {{put}} and {{remove}}, but nothing for {{invoke}}/{{EntryProcessor}}.
It makes sense to add such metrics, for example:
* Total number of `invoke` operations executed.
* Number of `invoke` operations that included updates.
* Number of read-only `invoke` operations.
* Min/max/avg execution time.
* ...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6807) NPE is thrown if local query is executed on client
Valentin Kulichenko created IGNITE-6807:
---
Summary: NPE is thrown if local query is executed on client
Key: IGNITE-6807
URL: https://issues.apache.org/jira/browse/IGNITE-6807
Project: Ignite
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Components: sql
Affects Versions: 2.3
Reporter: Valentin Kulichenko
If a local query is executed on client node by mistake, ugly NPE is thrown:
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.segmentsCount(H2TreeIndex.java:162)
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase.threadLocalSegment(GridH2IndexBase.java:172)
at
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find(H2TreeIndex.java:177)
at org.h2.index.BaseIndex.find(BaseIndex.java:128)
at org.h2.index.IndexCursor.find(IndexCursor.java:169)
at org.h2.table.TableFilter.next(TableFilter.java:468)
at
org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1452)
at org.h2.result.LazyResult.hasNext(LazyResult.java:79)
at org.h2.result.LazyResult.next(LazyResult.java:59)
at org.h2.command.dml.Select.queryFlat(Select.java:519)
at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)
at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:114)
at org.h2.command.dml.Query.query(Query.java:352)
at org.h2.command.dml.Query.query(Query.java:333)
at org.h2.command.CommandContainer.query(CommandContainer.java:113)
at org.h2.command.Command.executeQuery(Command.java:201)
... 21 more
{noformat}
We should detect this situation and throw proper exception instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6575) IgniteCache#get can return stale value when executed on backup
Valentin Kulichenko created IGNITE-6575:
---
Summary: IgniteCache#get can return stale value when executed on
backup
Key: IGNITE-6575
URL: https://issues.apache.org/jira/browse/IGNITE-6575
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.2
Reporter: Valentin Kulichenko
If write sync mode is {{PRIMARY_SYNC}} and {{readFromBackup=true}} (both values
are defaults), calling {{get}} right after {{put}} for the same key in the same
thread can return stale value. This happens because even local backup is
updated asynchronously.
This is very confusing behavior for a user, would be good to make sure to
update local backup synchronously even in {{PRIMARY_SYNC}} mode.
More details here:
http://apache-ignite-developers.2346864.n4.nabble.com/PRIMARY-SYNC-readFromBackup-semantics-td22900.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6362) NPE in Log4J2Logger
Valentin Kulichenko created IGNITE-6362:
---
Summary: NPE in Log4J2Logger
Key: IGNITE-6362
URL: https://issues.apache.org/jira/browse/IGNITE-6362
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Fix For: 2.3
When I start a node with {{Log4J2Logger}} configured and verbose mode
({{-DIGNITE_QUIET=false}}, it fails with NPE:
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.logging.log4j.core.config.LoggerConfig.(LoggerConfig.java:145)
at
org.apache.logging.log4j.core.config.LoggerConfig.createLogger(LoggerConfig.java:523)
at
org.apache.ignite.logger.log4j2.Log4J2Logger.createConsoleLogger(Log4J2Logger.java:380)
at
org.apache.ignite.logger.log4j2.Log4J2Logger.addConsoleAppenderIfNeeded(Log4J2Logger.java:338)
at
org.apache.ignite.logger.log4j2.Log4J2Logger.(Log4J2Logger.java:145)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
... 34 more
{noformat}
This is caused by the fact that {{Log4J2Logger#createConsoleLogger}} method
invokes {{LoggerConfig.createLogger}} providing {{null}} as {{Configuration}}
object, which unconditionally causes NPE. Need to provide some default
configuration instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6346) Distributed set does not work in REPLICATED mode
Valentin Kulichenko created IGNITE-6346:
---
Summary: Distributed set does not work in REPLICATED mode
Key: IGNITE-6346
URL: https://issues.apache.org/jira/browse/IGNITE-6346
Project: Ignite
Issue Type: Bug
Components: data structures
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Fix For: 2.2
When {{IgniteSet}} is created with {{REPLICATED}} cache mode, any attempt to
read it fails with exception:
{noformat}
Exception in thread "main" class org.apache.ignite.IgniteException: Cluster
group is empty.
at
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:48)
at ReplicatedSet.main(ReplicatedSet.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: class
org.apache.ignite.internal.cluster.ClusterGroupEmptyCheckedException: Cluster
group is empty.
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.execute0(GridCacheQueryAdapter.java:481)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.execute(GridCacheQueryAdapter.java:442)
at
org.apache.ignite.internal.processors.datastructures.GridCacheSetImpl.iterator0(GridCacheSetImpl.java:420)
at
org.apache.ignite.internal.processors.datastructures.GridCacheSetImpl.iterator(GridCacheSetImpl.java:375)
at
org.apache.ignite.internal.processors.datastructures.GridCacheSetProxy.iterator(GridCacheSetProxy.java:342)
... 6 more
{noformat}
To reproduce run the following code:
{code}
Ignition.start(new IgniteConfiguration().setIgniteInstanceName("server-1"));
Ignition.start(new IgniteConfiguration().setIgniteInstanceName("server-2"));
Ignition.setClientMode(true);
Ignite ignite = Ignition.start();
IgniteSet set = ignite.set("my-set", new
CollectionConfiguration().setCacheMode(CacheMode.REPLICATED));
for (String s : set) {
System.out.println(s);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6219) IgniteCache#loadCache executes local load in caller thread
Valentin Kulichenko created IGNITE-6219:
---
Summary: IgniteCache#loadCache executes local load in caller thread
Key: IGNITE-6219
URL: https://issues.apache.org/jira/browse/IGNITE-6219
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Priority: Critical
Fix For: 2.2
{{IgniteCache#loadCache}} method broadcasts an internal task under the hood. If
one of the jobs are local (i.e. if {{loadCache}} is invoked on server node),
this job is executed in a caller thread, potentially *before all or some remote
requests are sent*. Since data loading is generally long running process, its
duration doubles in this scenario.
Possible solution is to check the list of nodes before task execution, and if
local node is there, execute on remote nodes first, and only then submit to
local node. This way we make sure that remote nodes never wait for the local
node.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6135) java.sql.Date is serialized using OptimizedMarshaller
Valentin Kulichenko created IGNITE-6135:
---
Summary: java.sql.Date is serialized using OptimizedMarshaller
Key: IGNITE-6135
URL: https://issues.apache.org/jira/browse/IGNITE-6135
Project: Ignite
Issue Type: Bug
Components: binary
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Fix For: 2.2
For some reason, if an object has a field of {{java.sql.Date}}, it's serialized
with {{OptimizedMarshaller}}. It should be a first class citizen, similar to
{{java.util.Date}}.
In addition, it's possible to write a field using builder like this:
{code}
builder.setField(name, val, java.util.Date.class)
{code}
where {{val}} is instance of {{java.sql.Date}}. This leads to an exception
during deserialization, because {{java.util.Date}} would be expected.
More context and code reproducing the issue can be found here:
http://apache-ignite-users.70518.x6.nabble.com/JDBC-store-Date-deserialization-problem-td16276.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5981) IgniteContext starts server nodes on executors
Valentin Kulichenko created IGNITE-5981:
---
Summary: IgniteContext starts server nodes on executors
Key: IGNITE-5981
URL: https://issues.apache.org/jira/browse/IGNITE-5981
Project: Ignite
Issue Type: Bug
Components: spark
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Fix For: 2.2
Currently {{IgniteContext#ignite()}} method creates client node only if it is
invoked on driver. However, in case standalone mode is used, this should happen
also executors (or basically anywhere within Spark).
Most likely, it would be enough to replace this line:
{code}
if (sparkContext != null) igniteCfg.setClientMode(true)
{code}
with this:
{code}
if (standalone || sparkContext != null) igniteCfg.setClientMode(true)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5947) ClassCastException when two-dimensional array is fetched from cache
Valentin Kulichenko created IGNITE-5947:
---
Summary: ClassCastException when two-dimensional array is fetched
from cache
Key: IGNITE-5947
URL: https://issues.apache.org/jira/browse/IGNITE-5947
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.1
Reporter: Valentin Kulichenko
Priority: Critical
Fix For: 2.2
When an instance of {{Object[][]}} is put into cache, and then read from there,
the following exception is thrown:
{noformat}
Exception in thread "main" java.lang.ClassCastException: [Ljava.lang.Object;
cannot be cast to [[Ljava.lang.Object;
{noformat}
Reproducer attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5836) AffinityFunctionContext should provide consistent previous assignment
Valentin Kulichenko created IGNITE-5836:
---
Summary: AffinityFunctionContext should provide consistent
previous assignment
Key: IGNITE-5836
URL: https://issues.apache.org/jira/browse/IGNITE-5836
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Priority: Critical
Fix For: 2.2
Currently each cache maintains its own {{AffinityFunctionContext}}. This leads
to the case that {{previousAssignment}} can be different for two caches created
on different topology versions. In particular, this broke
{{FairAffinityFunction}} which was removed because of that.
We should do the following:
* Make sure that {{previousAssignment}} is consistent for all caches with same
affinity function, regardless of topology versions caches were created on.
* Add mechanism to enforce equality check for affinity functions. We probably
will need to force user to implement {{equals}} for cache node filter and
backup filter.
* When above is done, bring back {{FairAffinityFunction}}.
This is also discussed on dev list:
http://apache-ignite-developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction-td19987.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5796) Improve cache memory metrics
Valentin Kulichenko created IGNITE-5796: --- Summary: Improve cache memory metrics Key: IGNITE-5796 URL: https://issues.apache.org/jira/browse/IGNITE-5796 Project: Ignite Issue Type: Improvement Components: cache Affects Versions: 2.0 Reporter: Valentin Kulichenko Fix For: 2.2 It seems that presently all metrics we have for page memory are about number of entries and number of pages. It would be useful to have metrics showing how much memory in bytes is allocated: * Per memory policy. * Per cache. * Separately for data, for indexes and probably other structures that consume memory. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5781) Visor throws ClassCastException if cache store implementation is other than CacheJdbcPojoStore
Valentin Kulichenko created IGNITE-5781:
---
Summary: Visor throws ClassCastException if cache store
implementation is other than CacheJdbcPojoStore
Key: IGNITE-5781
URL: https://issues.apache.org/jira/browse/IGNITE-5781
Project: Ignite
Issue Type: Bug
Components: visor
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Fix For: 2.2
Issue is reported on user list:
http://apache-ignite-users.70518.x6.nabble.com/Problem-with-Visor-and-Cassandra-Cache-Store-td15076.html
There is an obvious bug in the code. {{VisorCacheJdbcType#list}} method checks
the type of store factory like this:
{code}
if (factory != null || factory instanceof CacheJdbcPojoStoreFactory) {
CacheJdbcPojoStoreFactory jdbcFactory = (CacheJdbcPojoStoreFactory) factory;
{code}
It should be {{&&}} instead of {{||}}, because otherwise condition will be
{{true}} for any factory that is not {{null}}. Even better if {{factory !=
null}} is removed completely as {{instanceof}} returns {{false}} for {{null}}
values anyway.
However, it's not clear to me why this scenario is reproduced only in certain
conditions (see mailing list thread for details). It's possible that there is
another hidden bug, this needs to be investigated.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5655) Introduce pluggable string encoder/decoder
Valentin Kulichenko created IGNITE-5655:
---
Summary: Introduce pluggable string encoder/decoder
Key: IGNITE-5655
URL: https://issues.apache.org/jira/browse/IGNITE-5655
Project: Ignite
Issue Type: New Feature
Components: binary
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Currently binary marshaller encodes strings in UTF-8. However, sometimes it
makes sense to customize this.
Need to:
# Create {{BinaryStringEncoder}} interface with {{encode}} and {{decode}}
methods that will convert string to byte array and back.
# Create default implementation based on UTF-8.
# Add {{stringEncoder}} configuration property to {{BinaryConfiguration}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5626) SQL TRIM function is not working properly
Valentin Kulichenko created IGNITE-5626:
---
Summary: SQL TRIM function is not working properly
Key: IGNITE-5626
URL: https://issues.apache.org/jira/browse/IGNITE-5626
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Fix For: 2.1
{{TRIM}} function trims only one char instead of the whole provided string.
Reproducer attached.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5607) HttpSessionBindingListener is not supported for clustered web session
Valentin Kulichenko created IGNITE-5607:
---
Summary: HttpSessionBindingListener is not supported for clustered
web session
Key: IGNITE-5607
URL: https://issues.apache.org/jira/browse/IGNITE-5607
Project: Ignite
Issue Type: Bug
Components: websession
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Fix For: 2.1
Ignite's implementation of {{HttpSession}} ignores values implementing
{{HttpSessionBindingListener}}.
{{WebSession#setAttribute}} and {{WebSession#removeAttribute}} should be
modified accordingly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5592) Remove IgniteCache#localEvict method
Valentin Kulichenko created IGNITE-5592: --- Summary: Remove IgniteCache#localEvict method Key: IGNITE-5592 URL: https://issues.apache.org/jira/browse/IGNITE-5592 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.0 Reporter: Valentin Kulichenko Fix For: 2.1 The method doesn't make much sense in 2.0. Before 2.0 it could be used to evict from on-heap memory to off-heap or swap. Currently off-heap always has the data and there is no swap, so the method should be removed to avoid confusion. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5467) Exception in communication SPI can stall the cluster
Valentin Kulichenko created IGNITE-5467:
---
Summary: Exception in communication SPI can stall the cluster
Key: IGNITE-5467
URL: https://issues.apache.org/jira/browse/IGNITE-5467
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Fix For: 2.1
Test attached.
If there is an exception in {{CommunicationSpi#sendMessage}} while sending
response for a cache operation, it can stall whole cluster forever (operation
doesn't complete, new nodes can't join, etc.).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-5444) Collections.singletonList is not properly serialized by binary marshaller
Valentin Kulichenko created IGNITE-5444:
---
Summary: Collections.singletonList is not properly serialized by
binary marshaller
Key: IGNITE-5444
URL: https://issues.apache.org/jira/browse/IGNITE-5444
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Fix For: 2.1
Attachments: BinaryObjectSingletonList.java
Test attached. {{Collections.singletonList}} is apparently serialized by
optimized marshaller, because when deserialized, it doesn't return collection
of binary objects, but rather collection of deserialized objects.
Most likely the reason for this is that {{Collections.singletonList}} is not
recognized by {{BinaryUtils.knownCollection}} method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-5270) Cassandra store should support binary objects with POJO strategy
Valentin Kulichenko created IGNITE-5270: --- Summary: Cassandra store should support binary objects with POJO strategy Key: IGNITE-5270 URL: https://issues.apache.org/jira/browse/IGNITE-5270 Project: Ignite Issue Type: Improvement Components: cassandra Affects Versions: 2.0 Reporter: Valentin Kulichenko Fix For: 2.2 Currently Cassandra store requires to have POJO classes on server nodes when POJO strategy is used. We should improve it and allow to support binary objects directly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-5103) TcpDiscoverySpi ignores maxMissedClientHeartbeats property
Valentin Kulichenko created IGNITE-5103:
---
Summary: TcpDiscoverySpi ignores maxMissedClientHeartbeats property
Key: IGNITE-5103
URL: https://issues.apache.org/jira/browse/IGNITE-5103
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Fix For: 2.1
Test scenario is the following:
* Start one or more servers.
* Start a client node.
* Suspend client process using {{-SIGSTOP}} signal.
* Wait for {{maxMissedClientHeartbeats*heartbeatFrequency}}.
* Client node is expected to be removed from topology, but server nodes don't
do that.
Attached is the unit test reproducing the same by stopping the heartbeat sender
thread on the client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-5099) Injection doesn't work for StreamReceiver in case it's local
Valentin Kulichenko created IGNITE-5099:
---
Summary: Injection doesn't work for StreamReceiver in case it's
local
Key: IGNITE-5099
URL: https://issues.apache.org/jira/browse/IGNITE-5099
Project: Ignite
Issue Type: Bug
Components: streaming
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Fix For: 2.1
Attachments: VisitorTest.java
If {{StreamReceiver}} is invoked on the same node where
{{IgniteDataStreamer.addData}} is invoked, resource injection doesn't happen.
Reproducer attached.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-5081) SecurityPermissionSetBuilder duplicates permission if it's appended more than once
Valentin Kulichenko created IGNITE-5081:
---
Summary: SecurityPermissionSetBuilder duplicates permission if
it's appended more than once
Key: IGNITE-5081
URL: https://issues.apache.org/jira/browse/IGNITE-5081
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.1
If the same permission is appended to builder more than once, builder
duplicates it in the resulting collection. This doesn't actually break
anything, but seems to be redundant.
Example:
{code}
new SecurityPermissionSetBuilder()
.appendSystemPermissions(ADMIN_VIEW)
.appendSystemPermissions(ADMIN_VIEW, ADMIN_QUERY)
.build();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-5080) SecurityBasicPermissionSet implementation is incomplete
Valentin Kulichenko created IGNITE-5080:
---
Summary: SecurityBasicPermissionSet implementation is incomplete
Key: IGNITE-5080
URL: https://issues.apache.org/jira/browse/IGNITE-5080
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.1
There are several issues with {{SecurityBasicPermissionSet}}:
* It doesn't implement {{hashCode}} and {{equals}} methods. This makes it
impossible to use it as part of validation token.
* {{Collection}} fields are not marked with {{@GridToStringInclude}}
annotation, so {{toString}} method doesn't actually print out all the
information.
* {{systemPermissions}} method returns empty collection instead of {{null}} by
default. This actually means 'no system permissions' even if
{{defaultAllowAll}} is true.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4991) Do not print out system properties when IGNITE_TO_STRING_INCLUDE_SENSITIVE is set
Valentin Kulichenko created IGNITE-4991:
---
Summary: Do not print out system properties when
IGNITE_TO_STRING_INCLUDE_SENSITIVE is set
Key: IGNITE-4991
URL: https://issues.apache.org/jira/browse/IGNITE-4991
Project: Ignite
Issue Type: Improvement
Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.1
{{IgniteKernal#ackSystemProperties}} and {{IgniteKernal#ackVmArguments}} print
out system properties that can contain sensitive data. This print outs should
be disabled when {{IGNITE_TO_STRING_INCLUDE_SENSITIVE}} system property is set
to {{true}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4954) Timeout in ignite-cassandra SessionPool is not configurable
Valentin Kulichenko created IGNITE-4954:
---
Summary: Timeout in ignite-cassandra SessionPool is not
configurable
Key: IGNITE-4954
URL: https://issues.apache.org/jira/browse/IGNITE-4954
Project: Ignite
Issue Type: Bug
Components: ignite-cassandra
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Priority: Critical
Fix For: 2.0
Cassandra cache store implementation uses {{SessionPool}} class which kills
idle sessions after 5 minutes. Need to make this timeout configurable.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4948) Web session clustering fails to work with Spring Security
Valentin Kulichenko created IGNITE-4948:
---
Summary: Web session clustering fails to work with Spring Security
Key: IGNITE-4948
URL: https://issues.apache.org/jira/browse/IGNITE-4948
Project: Ignite
Issue Type: Bug
Components: websession
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.0
{{WebSessionFilter.doFilterV2}} method do not refresh the local instance of
session after {{chain.doFilter}}. But it's possible that session is replaced by
some other filter (e.g. due to login). And if after this replacement some
attributes are added, they do not end up in cache.
dev@ list discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-2741-spring-session-design-td14560.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4933) Need to add an option to write all updates in write-behind store
Valentin Kulichenko created IGNITE-4933:
---
Summary: Need to add an option to write all updates in
write-behind store
Key: IGNITE-4933
URL: https://issues.apache.org/jira/browse/IGNITE-4933
Project: Ignite
Issue Type: Improvement
Components: cache
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Fix For: 2.0
Currently if there are several updates for the same key, write behind store
will write only one of them within a single batch. I.e. second update will
override the first one if the latter is not written to DB yet.
This is good from performance and memory consumption standpoint, however
sometimes it's required to write all updates that happen in cache.
We can add parameter to {{CacheConfiguration}} that will switch this behavior
and implement it in {{GridCacheWriteBehindStore}}. Default behavior should
remain the same.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4891) Key is deserialized during transactional get() even if withKeepBinary is set
Valentin Kulichenko created IGNITE-4891: --- Summary: Key is deserialized during transactional get() even if withKeepBinary is set Key: IGNITE-4891 URL: https://issues.apache.org/jira/browse/IGNITE-4891 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 1.9 Reporter: Valentin Kulichenko Priority: Critical Fix For: 2.0 This test reproduces the issue: https://github.com/olegskoblya/tx-binary-bug -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-4832) Service is deployed on client when service configuration is provided on startup
Valentin Kulichenko created IGNITE-4832:
---
Summary: Service is deployed on client when service configuration
is provided on startup
Key: IGNITE-4832
URL: https://issues.apache.org/jira/browse/IGNITE-4832
Project: Ignite
Issue Type: Bug
Components: managed services
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Fix For: 2.0
In case service configuration is provided on startup (i.e. as a part of
{{IgniteConfiguration}}), the service can be deployed on the client node which
is incorrect. Client nodes should be filtered out by default, like it's done in
{{IgniteServices.deployXXX(..)}} methods.
Test reproducing the behavior is attached.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4831) Add an option to disable MBeans
Valentin Kulichenko created IGNITE-4831:
---
Summary: Add an option to disable MBeans
Key: IGNITE-4831
URL: https://issues.apache.org/jira/browse/IGNITE-4831
Project: Ignite
Issue Type: Improvement
Components: general
Affects Versions: 1.8
Reporter: Valentin Kulichenko
There are multiple MBeans registered by Ignite and there is no way to avoid
their registration (in case they're not allowed for security reasons for
example). We should have a system property that will disable MBeans.
In addition, if MBean registration throws a {{RuntimeException}}, this
exception is not properly handled. For example, if it happens during cache
creation, {{createCache}} method hangs forever.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4812) Disable EventStorageSpi by default
Valentin Kulichenko created IGNITE-4812:
---
Summary: Disable EventStorageSpi by default
Key: IGNITE-4812
URL: https://issues.apache.org/jira/browse/IGNITE-4812
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.9
Reporter: Valentin Kulichenko
Fix For: 2.0
Current default implementation of {{EventStorageSpi}} is
{{MemoryEventStorageSpi}}. It consumes a lot of memory, while is used only for
events querying (i.e. not listening), which is a fairly rare use case.
Need to:
* Introduce {{NoOpEventStorageSpi}}.
* Make {{NoOpEventStorageSpi}} the default one.
* Throw an exception when {{IgniteEvents#localQuery}} or {{#remoteQuery}}
method is called with default setting. Exception message should explain how to
fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4802) Create separate thread pool for services
Valentin Kulichenko created IGNITE-4802: --- Summary: Create separate thread pool for services Key: IGNITE-4802 URL: https://issues.apache.org/jira/browse/IGNITE-4802 Project: Ignite Issue Type: Improvement Components: managed services Affects Versions: 1.9 Reporter: Valentin Kulichenko Fix For: 2.0 Currently if a service is invoked via service proxy, it will be executed on a remote node in its public thread pool. This means that it's unsafe to execute a compute job or closure from a service as it can cause starvation. Need to create a separate pool for services to overcome this limitation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-4762) Support transactional updates in JDBC driver
Valentin Kulichenko created IGNITE-4762: --- Summary: Support transactional updates in JDBC driver Key: IGNITE-4762 URL: https://issues.apache.org/jira/browse/IGNITE-4762 Project: Ignite Issue Type: Improvement Components: jdbc-driver, SQL Affects Versions: 1.8 Reporter: Valentin Kulichenko Currently transactions are not supported by SQL and therefore by JDBC driver. However, it seems to be possible to provide support for multiple updates in a single transaction. We can add a special flag to JDBC driver to allow this. Any SELECT queries will still be executed out of transactional scope. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-4748) jdbc2.JdbcStatement does not implement setQueryTimeout method
Valentin Kulichenko created IGNITE-4748: --- Summary: jdbc2.JdbcStatement does not implement setQueryTimeout method Key: IGNITE-4748 URL: https://issues.apache.org/jira/browse/IGNITE-4748 Project: Ignite Issue Type: Bug Components: jdbc-driver Affects Versions: 1.8 Reporter: Valentin Kulichenko Priority: Critical Fix For: 1.9 The method was implemented in the older version of JDBC driver, but was missed in the newer version (still throws exception). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-4738) Support REGEXP_LIKE and REGEXP_REPLACE SQL functions
Valentin Kulichenko created IGNITE-4738: --- Summary: Support REGEXP_LIKE and REGEXP_REPLACE SQL functions Key: IGNITE-4738 URL: https://issues.apache.org/jira/browse/IGNITE-4738 Project: Ignite Issue Type: Improvement Components: SQL Affects Versions: 1.8 Reporter: Valentin Kulichenko Fix For: 2.0 These functions are supported by H2 in their latest version, but not in the version Ignite depends on. Most likely we just need to upgrade H2. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (IGNITE-4658) GridCacheProcessor.createMissingCaches() should be called in a separate thread
Valentin Kulichenko created IGNITE-4658:
---
Summary: GridCacheProcessor.createMissingCaches() should be called
in a separate thread
Key: IGNITE-4658
URL: https://issues.apache.org/jira/browse/IGNITE-4658
Project: Ignite
Issue Type: Improvement
Components: cache
Affects Versions: 1.8
Reporter: Valentin Kulichenko
{{GridCacheProcessor.createMissingCaches()}} method is called during query
execution if not all caches are created on the client. If this happen inside a
transaction, query fails due to fact that cache can't be created within a
transaction.
To work around this and improve usability, this method should start a separate
thread where caches will be created, and synchronously join on this thread.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4643) JdbcDatabaseMetadata.getIndexInfo() method not working
Valentin Kulichenko created IGNITE-4643:
---
Summary: JdbcDatabaseMetadata.getIndexInfo() method not working
Key: IGNITE-4643
URL: https://issues.apache.org/jira/browse/IGNITE-4643
Project: Ignite
Issue Type: Bug
Components: jdbc-driver
Affects Versions: 1.8
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 1.9
{{JdbcDatabaseMetadata.getIndexInfo()}} method does not update metadata before
creating result set. So if it's a first method invocation on this instance of
{{JdbcDatabaseMetadata}}, it fails with NPE. Need to create tests and add
{{updateMetaData()}} call in the beginning of the method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4642) Support enforceJoinOrder flag for JDBC driver
Valentin Kulichenko created IGNITE-4642:
---
Summary: Support enforceJoinOrder flag for JDBC driver
Key: IGNITE-4642
URL: https://issues.apache.org/jira/browse/IGNITE-4642
Project: Ignite
Issue Type: Improvement
Components: jdbc-driver
Reporter: Valentin Kulichenko
There is a useful {{enforceJoinOrder}} flag on {{SqlFieldsQuery}}, but it's not
exposed on JDBC driver.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4641) Refresh client attributes during reconnect
Valentin Kulichenko created IGNITE-4641:
---
Summary: Refresh client attributes during reconnect
Key: IGNITE-4641
URL: https://issues.apache.org/jira/browse/IGNITE-4641
Project: Ignite
Issue Type: Improvement
Components: general
Affects Versions: 1.8
Reporter: Valentin Kulichenko
Fix For: 1.9
Currently, when client disconnects and reconnects, it is forced to do that with
the exact same set of attributes. However, there can be a component that wants
to update attribute(s) in {{onDisconnected}} method, so that node reconnects
with new attributes. To make sure this happens we need to refresh attributes
before reconnecting. Most likely this should happen in
{{TcpDiscoveryNode.clientReconnectNode()}} method which should acquire
attributes via {{GridKernalContext.nodeAttributes()}}.
Typical use case is a security token that can expire. Security processor
implementation may want to refresh the token in case of disconnect, but this is
not possible now.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (IGNITE-4555) Make KeyValuePersistenceSettings configuration class Spring compatible
Valentin Kulichenko created IGNITE-4555:
---
Summary: Make KeyValuePersistenceSettings configuration class
Spring compatible
Key: IGNITE-4555
URL: https://issues.apache.org/jira/browse/IGNITE-4555
Project: Ignite
Issue Type: Improvement
Components: ignite-cassandra
Affects Versions: 1.8
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 1.9
Currently {{KeyValuePersistenceSettings}} uses a special XML format which is
OK, but limits usability in following cases:
* Spring is used for all other components of Ignite.
* User wants to start dynamic cache and does not know persistence configuration
in advance.
We should add getter and setter methods to {{KeyValuePersistenceSettings}} and
make it possible to create via Spring or directly in code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4525) Near reader is not created when value is loaded from store
Valentin Kulichenko created IGNITE-4525: --- Summary: Near reader is not created when value is loaded from store Key: IGNITE-4525 URL: https://issues.apache.org/jira/browse/IGNITE-4525 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 1.8 Reporter: Valentin Kulichenko Fix For: 2.0 When client with a near cache get value that is not in cache yet, it's loaded from store and the value is saved in near cache. However reader on primary node is not created, and when it's updated on server side, near cache is not synchronized. Test attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4520) Web console: allow agent to connect through proxy
Valentin Kulichenko created IGNITE-4520:
---
Summary: Web console: allow agent to connect through proxy
Key: IGNITE-4520
URL: https://issues.apache.org/jira/browse/IGNITE-4520
Project: Ignite
Issue Type: Improvement
Components: wizards
Affects Versions: 1.8
Reporter: Valentin Kulichenko
Priority: Critical
Fix For: 2.0
We currently assume that web agent will have direct access to console server.
Sometimes this is not true and any external Internet access is done through
SOCKS or HTTP proxy, potentially with authentication.
We should add settings required for proxy (type, address, login, password) and
implement the support. {{java.net.Proxy}} class can be used for this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4513) Improve DEBUG logging for continuous queries
Valentin Kulichenko created IGNITE-4513: --- Summary: Improve DEBUG logging for continuous queries Key: IGNITE-4513 URL: https://issues.apache.org/jira/browse/IGNITE-4513 Project: Ignite Issue Type: Improvement Components: cache Affects Versions: 1.8 Reporter: Valentin Kulichenko Assignee: Valentin Kulichenko Fix For: 2.0 There are several reports from users claiming that sometimes there is a huge delay between remote filter and local listener notifications, sometimes local listener notifications are even seem to be lost. Configuration and code looks correct, so it's hard to tell if it's a bug or just a misuse. We need to improve DEBUG level logging for continuous queries to allow better analysis of such cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4465) Read-through is not properly working with multiple gets executed in parallel
Valentin Kulichenko created IGNITE-4465:
---
Summary: Read-through is not properly working with multiple gets
executed in parallel
Key: IGNITE-4465
URL: https://issues.apache.org/jira/browse/IGNITE-4465
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 1.8
Reporter: Valentin Kulichenko
Priority: Critical
Fix For: 1.9
The issue is sporadic and very hard to isolate, however I managed to create a
test that reproduces it. Basically, the scenario is the following:
* We have one server and one client.
* The client creates a cache with read through enabled.
* The client then executes bunch of jobs (number of jobs is bigger than number
of threads in server's public pool) asynchronously in parallel.
* {{CacheStore.load()}} method is called more than once and invocations go one
after another (sometimes even in the same thread!) with an interval of a bit
more than one second, which is the duration of load (there is a
{{Thread.sleep(1000)}} in the implementation.
* In addition, statistics show that number of misses go up to 50 which is
number of jobs. I would not expect more than 16 there. First 16 jobs executed
in parallel can all register miss at the same time before load happens. But all
consecutive execution must read the value from cache.
Test is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4450) Explicit lock is not released when node that acquired it is killed
Valentin Kulichenko created IGNITE-4450: --- Summary: Explicit lock is not released when node that acquired it is killed Key: IGNITE-4450 URL: https://issues.apache.org/jira/browse/IGNITE-4450 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 1.8 Reporter: Valentin Kulichenko Priority: Critical Fix For: 1.9 Test is attached. The scenario is the following: # Start first node and create a transactional cache. # Start second node and acquire a lock. # Kill second node after several seconds without doing unlock. # Try to start third node. It can't join because the lock is still held for some reason. The result if hanged cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
