[jira] [Created] (IGNITE-3745) ODBC: Escape sequences: Support TIME escape sequence.
Vladimir Ozerov created IGNITE-3745: --- Summary: ODBC: Escape sequences: Support TIME escape sequence. Key: IGNITE-3745 URL: https://issues.apache.org/jira/browse/IGNITE-3745 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov See https://msdn.microsoft.com/en-us/library/ms712360(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3747) ODBC: Escape sequences: Support TIMESTAMP escape sequence.
Vladimir Ozerov created IGNITE-3747: --- Summary: ODBC: Escape sequences: Support TIMESTAMP escape sequence. Key: IGNITE-3747 URL: https://issues.apache.org/jira/browse/IGNITE-3747 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov See https://msdn.microsoft.com/en-us/library/ms712360(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3752) ODBC: Scalar functions: Support CAST function.
Vladimir Ozerov created IGNITE-3752: --- Summary: ODBC: Scalar functions: Support CAST function. Key: IGNITE-3752 URL: https://issues.apache.org/jira/browse/IGNITE-3752 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov See https://msdn.microsoft.com/en-us/library/ms713608(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3737) ODBC: Support scalar functions,
Vladimir Ozerov created IGNITE-3737: --- Summary: ODBC: Support scalar functions, Key: IGNITE-3737 URL: https://issues.apache.org/jira/browse/IGNITE-3737 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 1.8 The following functions must be supported by our driver: https://msdn.microsoft.com/en-us/library/ms711813(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3743) ODBC: Escape sequences: Support procedure call escape sequence.
Vladimir Ozerov created IGNITE-3743: --- Summary: ODBC: Escape sequences: Support procedure call escape sequence. Key: IGNITE-3743 URL: https://issues.apache.org/jira/browse/IGNITE-3743 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3740) ODBC: Escape sequences: Support interval escapse sequence.
Vladimir Ozerov created IGNITE-3740: --- Summary: ODBC: Escape sequences: Support interval escapse sequence. Key: IGNITE-3740 URL: https://issues.apache.org/jira/browse/IGNITE-3740 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 See https://msdn.microsoft.com/en-us/library/ms710906(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3742) ODBC: Escape sequences: Support OUTER JOIN escapse sequence.
Vladimir Ozerov created IGNITE-3742: --- Summary: ODBC: Escape sequences: Support OUTER JOIN escapse sequence. Key: IGNITE-3742 URL: https://issues.apache.org/jira/browse/IGNITE-3742 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov See https://msdn.microsoft.com/en-us/library/ms710299(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3749) ODBC: Scalar functions: Support numeric functions.
Vladimir Ozerov created IGNITE-3749: --- Summary: ODBC: Scalar functions: Support numeric functions. Key: IGNITE-3749 URL: https://issues.apache.org/jira/browse/IGNITE-3749 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 1) All the functions from the list must be supported: https://msdn.microsoft.com/en-us/library/ms711743(v=vs.85).aspx 2) Separate test for every function must be added. 3) If some of them cannot be supported immediately, then JIRA ticket must be created and linked to this ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3738) ODBC: Escape functions: Ensure that escape sequence token is parsed correctly irrespective of whitespaces.
Vladimir Ozerov created IGNITE-3738:
---
Summary: ODBC: Escape functions: Ensure that escape sequence token
is parsed correctly irrespective of whitespaces.
Key: IGNITE-3738
URL: https://issues.apache.org/jira/browse/IGNITE-3738
Project: Ignite
Issue Type: Task
Components: odbc
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Andrew Mashenkov
Fix For: 1.8
The following escape sequences must be supported and parsed correctly:
1) {fn X}
2) { fn X}
3) {fn X}
4) { \n X}
Etc. That is, irrespective of whitespaces, everything must be parsed correctly.
I propose to rely on {{Character.isWhitespace}}, not on regexp. That is, once {
is found, we should simply skip all following whitespaces, then get the token,
then possibly skip more whitespaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3750) ODBC: Scalar functions: Support time/date/interval functions
Vladimir Ozerov created IGNITE-3750: --- Summary: ODBC: Scalar functions: Support time/date/interval functions Key: IGNITE-3750 URL: https://issues.apache.org/jira/browse/IGNITE-3750 Project: Ignite Issue Type: Sub-task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 1) All the functions from the list must be supported: https://msdn.microsoft.com/en-us/library/ms714639(v=vs.85).aspx 2) Separate test for every function must be added. 3) If some of them cannot be supported immediately, then JIRA ticket must be created and linked to this ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3801) ODBC: Add test for OUTER JOIN escape sequence and ensure it is marked as supported.
Vladimir Ozerov created IGNITE-3801: --- Summary: ODBC: Add test for OUTER JOIN escape sequence and ensure it is marked as supported. Key: IGNITE-3801 URL: https://issues.apache.org/jira/browse/IGNITE-3801 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Priority: Minor Fix For: 1.8 IGNITE-3742 is ready, so that we are able to parse OJ escape sequences in Java. We need to add tests for it in ODBC layer and ensure this feature is marked as supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3890) IGFS: Flatten IgfsInputStream inheritance hierarchy.
Vladimir Ozerov created IGNITE-3890: --- Summary: IGFS: Flatten IgfsInputStream inheritance hierarchy. Key: IGNITE-3890 URL: https://issues.apache.org/jira/browse/IGNITE-3890 Project: Ignite Issue Type: Task Components: IGFS Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 It is too complex for now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3902) Hadoop: unreliable context classloader manipulations.
Vladimir Ozerov created IGNITE-3902: --- Summary: Hadoop: unreliable context classloader manipulations. Key: IGNITE-3902 URL: https://issues.apache.org/jira/browse/IGNITE-3902 Project: Ignite Issue Type: Bug Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3887) IGFS: Encapsulate IgfsPath root .
Vladimir Ozerov created IGNITE-3887:
---
Summary: IGFS: Encapsulate IgfsPath root .
Key: IGNITE-3887
URL: https://issues.apache.org/jira/browse/IGNITE-3887
Project: Ignite
Issue Type: Task
Components: IGFS
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Priority: Minor
Fix For: 1.8
Currently we have some trash in {{IgfsPath}} class:
1) {{root()}} method;
2) {{new IgfsPath()}} which also creates a root;
3) {{isSame()}} method which simply delegates to {{equals()}}.
Let's deprecate all these things and create consistent public static final ROOT
singleton.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3909) Hadoop: invalid classloader definition delegation logic.
Vladimir Ozerov created IGNITE-3909:
---
Summary: Hadoop: invalid classloader definition delegation logic.
Key: IGNITE-3909
URL: https://issues.apache.org/jira/browse/IGNITE-3909
Project: Ignite
Issue Type: Bug
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Priority: Critical
Fix For: 1.8
Currently we load explicitly only some {{org.apache.ignite}} and
{{org.apache.hadoop}} classes.
For this reason 3rd-party classes are always loaded by parent classloader, what
leads to various weird exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3916) Hadoop: Create additional module.
Vladimir Ozerov created IGNITE-3916:
---
Summary: Hadoop: Create additional module.
Key: IGNITE-3916
URL: https://issues.apache.org/jira/browse/IGNITE-3916
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 1.8
The plan is as follows:
1) {{ignite-hadoop}} will have helpers and interfaces, but *will not* have
direct Hadoop dependency. This module will be loaded in runtime by the core.
2) {{ignite-hadoop-impl}} will have Hadoop-dependent implementations and have
direct dependency on Hadoop libraries. It will be accessed only through
{{HadoopClassLoader}} interface and should not be located in normal classpath
of the {{core}} module.
In this ticket we should only create additional module, move all current Hadoop
classes there, and ensure correct dependencies
{{core <- hadoop <- hadoop-impl}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3912) Hadoop: rework HadoopClassLoader infrastructure
Vladimir Ozerov created IGNITE-3912:
---
Summary: Hadoop: rework HadoopClassLoader infrastructure
Key: IGNITE-3912
URL: https://issues.apache.org/jira/browse/IGNITE-3912
Project: Ignite
Issue Type: Task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Priority: Critical
Fix For: 1.8
This is an umbrella ticket to host tickets related to {{HadoopClassLoader}}
rework logic.
In general, we should achieve clean and consistent separation of ignite-core on
the one hand, and (ignite-hadoop + Hadoop dependencies) on the other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3914) Hadoop: Move HadoopClassLoader to core module.
Vladimir Ozerov created IGNITE-3914:
---
Summary: Hadoop: Move HadoopClassLoader to core module.
Key: IGNITE-3914
URL: https://issues.apache.org/jira/browse/IGNITE-3914
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Ivan Veselovsky
Fix For: 1.8
Implementation plan:
1) We will use notion of "helper". See {{IgfsHelper}} interface as example to
get used to existing code base.
2) {{HadoopClassLoaderUtils}} must be converted to an interface
{{HadoopHelper}}. It will have 4 methods - current {{public static}} methods.
3) There will be two implementations - {{HadoopNoopHelper}} and
{{HadoopHelperImpl}}. The first will be located in {{core}} module, the second
one in {{hadoop}} module.
4) {{HadoopHelper}} is wired up with {{GridKernalContextImpl}} in the same way
as it is done for {{IgfsHelper}}. The only difference is that we will
instantiate {{HadoopHelperImpl}} if Hadoop module is in the classpath. See
other components initialization logic for reference.
5) {{HadoopClassLoader}} will accept helper in constructor. This will require
some refactoring to places where it is used (4 currently). The only problematic
place is {{HadoopV2Job}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3875) Create separate thread pool for data streamer.
Vladimir Ozerov created IGNITE-3875: --- Summary: Create separate thread pool for data streamer. Key: IGNITE-3875 URL: https://issues.apache.org/jira/browse/IGNITE-3875 Project: Ignite Issue Type: Task Components: streaming Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Andrew Mashenkov Fix For: 1.8 Currently data streamer requests are submitted to PUBLIC pool. Because of this it is not safe to run streamer from compute tasks which is very inconvenient. We should create separate thread pool for streamer and route streamer jobs to it. Compatibility must be preserved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4006) Hadoop: integrate with ResourceManager.
Vladimir Ozerov created IGNITE-4006: --- Summary: Hadoop: integrate with ResourceManager. Key: IGNITE-4006 URL: https://issues.apache.org/jira/browse/IGNITE-4006 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 2.0 When jobs are executed through Hadoop, users may find useful historical info (such as logs) in Resource Manager. This doesn't work for Ignite. Looks like we need to investigate how to integrate w/ native Resource Manager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4005) Hadoop: introduce default job counters.
Vladimir Ozerov created IGNITE-4005: --- Summary: Hadoop: introduce default job counters. Key: IGNITE-4005 URL: https://issues.apache.org/jira/browse/IGNITE-4005 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 1.8 When job is executed through Accelerator, little to no counters are printed at the end of the job. We should ensure that more or less the same counters are printed out as in case of standard Apache Hadoop engine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4041) Consistent thread pool resolution for IO and closures.
Vladimir Ozerov created IGNITE-4041: --- Summary: Consistent thread pool resolution for IO and closures. Key: IGNITE-4041 URL: https://issues.apache.org/jira/browse/IGNITE-4041 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3949) Hadoop: change class loading strategy: exclude only classes with direct Hadoop dependencies.
Vladimir Ozerov created IGNITE-3949: --- Summary: Hadoop: change class loading strategy: exclude only classes with direct Hadoop dependencies. Key: IGNITE-3949 URL: https://issues.apache.org/jira/browse/IGNITE-3949 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3906) Hadoop: add optional envvar to set additional libraries to classpath.
Vladimir Ozerov created IGNITE-3906: --- Summary: Hadoop: add optional envvar to set additional libraries to classpath. Key: IGNITE-3906 URL: https://issues.apache.org/jira/browse/IGNITE-3906 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3928) Hadoop: print warning if Hadoop is found in classpath of parent classloader.
Vladimir Ozerov created IGNITE-3928:
---
Summary: Hadoop: print warning if Hadoop is found in classpath of
parent classloader.
Key: IGNITE-3928
URL: https://issues.apache.org/jira/browse/IGNITE-3928
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Ivan Veselovsky
Fix For: 1.8
Normally parent classloader should not have Hadoop in classpath. Otherwise we
may end up in JAR hell situation.
We need to probe parent classloader for Hadoop and print a warning if Hadoop is
likely to be in the classpath. E.g. we can probe for {{FileSystem}} or
{{Configuration}} class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3929) Hadoop: Fix native libraries loading mechanism.
Vladimir Ozerov created IGNITE-3929:
---
Summary: Hadoop: Fix native libraries loading mechanism.
Key: IGNITE-3929
URL: https://issues.apache.org/jira/browse/IGNITE-3929
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Ivan Veselovsky
Fix For: 1.8
Currently we do the following to load native libraries:
1) Ask parent class loader (normally it is app class loader) to load
{{org.apache.hadoop.util.NativeCodeLoader}} class. It will force loading of
relevant Hadoop native libraries.
2) Then hack into parent class loader internals, find loaded library, and wire
it up to current classloader.
This approach will not work any more because parent classloader normally will
not have Hadoop in classpath.
We should provide alternative solution. One idea is to hack into
{{findLibrary}} method: when it is called we can create alternative library
name and point it, e.g., to symlink of a target library.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3960) Hadoop: add libraries necessary for mapr to the list of predefined libs.
Vladimir Ozerov created IGNITE-3960: --- Summary: Hadoop: add libraries necessary for mapr to the list of predefined libs. Key: IGNITE-3960 URL: https://issues.apache.org/jira/browse/IGNITE-3960 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3961) IGFS: Support direct PROXY mode invocation in method: affinity
Vladimir Ozerov created IGNITE-3961:
---
Summary: IGFS: Support direct PROXY mode invocation in method:
affinity
Key: IGNITE-3961
URL: https://issues.apache.org/jira/browse/IGNITE-3961
Project: Ignite
Issue Type: Sub-task
Components: IGFS
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
Fix For: 1.8
See
{{org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem.getFileBlockLocations}}
for reference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3954) Hadoop: Create separate class loader for test suites.
Vladimir Ozerov created IGNITE-3954: --- Summary: Hadoop: Create separate class loader for test suites. Key: IGNITE-3954 URL: https://issues.apache.org/jira/browse/IGNITE-3954 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 This class loader should force loading of Hadoop classes and hadoop-impl module. This way tests will be more isolated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3943) Hadoop: change scripts.
Vladimir Ozerov created IGNITE-3943:
---
Summary: Hadoop: change scripts.
Key: IGNITE-3943
URL: https://issues.apache.org/jira/browse/IGNITE-3943
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Ivan Veselovsky
Fix For: 1.8
1) Remove and do not use {{HadoopClasspathMain}}.
2) Do not include {{hadoop-impl}} module into node class path.
3) {{HadoopClasspathUtils.classpathForClassLoader}} must include
{{hadoop-impl}} in class path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3942) Hadoop: HadoopProcesor must be created using HadoopClassLoader.
Vladimir Ozerov created IGNITE-3942: --- Summary: Hadoop: HadoopProcesor must be created using HadoopClassLoader. Key: IGNITE-3942 URL: https://issues.apache.org/jira/browse/IGNITE-3942 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3941) Hadoop: Introduce common HadoopClassLoader.
Vladimir Ozerov created IGNITE-3941: --- Summary: Hadoop: Introduce common HadoopClassLoader. Key: IGNITE-3941 URL: https://issues.apache.org/jira/browse/IGNITE-3941 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3920) Hadoop: remove PayloadAware interface.
Vladimir Ozerov created IGNITE-3920:
---
Summary: Hadoop: remove PayloadAware interface.
Key: IGNITE-3920
URL: https://issues.apache.org/jira/browse/IGNITE-3920
Project: Ignite
Issue Type: Task
Components: hadoop, IGFS
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Ivan Veselovsky
Fix For: 1.8
When IGNITE-3376 is ready, we will be able to execute {{PROXY}} operations
directly from {{IgfsImpl}}. It means that we no longer need
{{HadoopPayloadAware}} interface, and we no longer need to pass {{IgfsPaths}}
to the client.
Let's remove them all together.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3918) Hadoop: move interfaces from "hadoop-impl" to "hadoop" module.
Vladimir Ozerov created IGNITE-3918: --- Summary: Hadoop: move interfaces from "hadoop-impl" to "hadoop" module. Key: IGNITE-3918 URL: https://issues.apache.org/jira/browse/IGNITE-3918 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3926) Exception thrown from static job initializer hangs remote future.
Vladimir Ozerov created IGNITE-3926:
---
Summary: Exception thrown from static job initializer hangs remote
future.
Key: IGNITE-3926
URL: https://issues.apache.org/jira/browse/IGNITE-3926
Project: Ignite
Issue Type: Bug
Components: compute
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Andrew Mashenkov
Fix For: 1.8
Test case:
1) Create a job/closure class with static initializer. Throw some exception
from it (good candidate is {{NoClassDefFoundError}} which simulates a
dependency which is missing at runtime).
2) Start standalone Ignite node.
3) Try to running a job on this node from remote process.
4) Observe that a future in remote process hangs because the first node fails
to process error correctly.
Proposed fix:
We need to improve our error handling for our {{GridJobExecuteRequest}}.
Possibly adding additional try-catch ensuring that no exceptions/errors are
ignored.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3925) Output process ID to the log during node start.
Vladimir Ozerov created IGNITE-3925:
---
Summary: Output process ID to the log during node start.
Key: IGNITE-3925
URL: https://issues.apache.org/jira/browse/IGNITE-3925
Project: Ignite
Issue Type: Task
Components: general
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Andrew Mashenkov
Priority: Minor
Fix For: 1.8
It might be very useful to know process ID of Ignite node. However, we do not
output this info to the log. Let's add it.
I would output it after {{VM total memory}} message.
Since there is no reliable way to get PID in Java, we do our best to get it,
but never throw and error if it is impossible for some reason.
Start point: {{IgniteKernal.start}} and various {{IgniteKernal.ack*}} methods.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3757) ODBC: Add tests for SQL_AGGREGATE_FUNCTIONS.
Vladimir Ozerov created IGNITE-3757: --- Summary: ODBC: Add tests for SQL_AGGREGATE_FUNCTIONS. Key: IGNITE-3757 URL: https://issues.apache.org/jira/browse/IGNITE-3757 Project: Ignite Issue Type: Task Components: c, odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 We need to ensure that all SQL_AGGREGATE_FUNCTIONS [1] are really supported. Let's add tests for all mentioned functions. [1] https://msdn.microsoft.com/en-us/library/ms711681(v=vs.85).aspx#Anchor_16 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3758) ODBC: Add tests for SQL_SQL92_DATETIME_FUNCTIONS.
Vladimir Ozerov created IGNITE-3758: --- Summary: ODBC: Add tests for SQL_SQL92_DATETIME_FUNCTIONS. Key: IGNITE-3758 URL: https://issues.apache.org/jira/browse/IGNITE-3758 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 Let's add tests for SQL_SQL92_DATETIME_FUNCTIONS [1] Affected functions: SQL_SDF_CURRENT_DATE SQL_SDF_CURRENT_TIME SQL_SDF_CURRENT_TIMESTAMP [1] https://msdn.microsoft.com/en-us/library/ms711681(v=vs.85).aspx#Anchor_16 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3759) ODBC: Add tests for SQL_DATETIME_LITERALS.
Vladimir Ozerov created IGNITE-3759: --- Summary: ODBC: Add tests for SQL_DATETIME_LITERALS. Key: IGNITE-3759 URL: https://issues.apache.org/jira/browse/IGNITE-3759 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 Let's add tests for SQL_DATETIME_LITERALS [1] Affected functions: SQL_DL_SQL92_DATE SQL_DL_SQL92_TIME SQL_DL_SQL92_TIMESTAMP SQL_DL_SQL92_INTERVAL_YEAR SQL_DL_SQL92_INTERVAL_MONTH SQL_DL_SQL92_INTERVAL_DAY SQL_DL_SQL92_INTERVAL_HOUR SQL_DL_SQL92_INTERVAL_MINUTE SQL_DL_SQL92_INTERVAL_SECOND SQL_DL_SQL92_INTERVAL_YEAR_TO_MONTH SQL_DL_SQL92_INTERVAL_DAY_TO_HOUR SQL_DL_SQL92_INTERVAL_DAY_TO_MINUTE SQL_DL_SQL92_INTERVAL_DAY_TO_SECOND SQL_DL_SQL92_INTERVAL_HOUR_TO_MINUTE SQL_DL_SQL92_INTERVAL_HOUR_TO_SECOND SQL_DL_SQL92_INTERVAL_MINUTE_TO_SECOND [1] https://msdn.microsoft.com/en-us/library/ms711681(v=vs.85).aspx#Anchor_16 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3760) ODBC: Add tests for SQL_SQL92_STRING_FUNCTIONS.
Vladimir Ozerov created IGNITE-3760: --- Summary: ODBC: Add tests for SQL_SQL92_STRING_FUNCTIONS. Key: IGNITE-3760 URL: https://issues.apache.org/jira/browse/IGNITE-3760 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 We need to ensure that all SQL_AGGREGATE_FUNCTIONS [1] are really supported. Let's add tests for all mentioned functions. SQL_SSF_CONVERT SQL_SSF_LOWER SQL_SSF_UPPER SQL_SSF_SUBSTRING SQL_SSF_TRANSLATE SQL_SSF_TRIM_BOTH SQL_SSF_TRIM_LEADING SQL_SSF_TRIM_TRAILING [1] https://msdn.microsoft.com/en-us/library/ms711681(v=vs.85).aspx#Anchor_16 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3764) ODBC: Support and test implicit functions (>, <, _, +, -, etc.).
Vladimir Ozerov created IGNITE-3764: --- Summary: ODBC: Support and test implicit functions (>, <, _, +, -, etc.). Key: IGNITE-3764 URL: https://issues.apache.org/jira/browse/IGNITE-3764 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Fix For: 1.8 The following functions must be supported and tested with all supported data types: != % + \- / < < <= == > >= -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3980) Process failing Binary Object Queries tests.
Vladimir Ozerov created IGNITE-3980: --- Summary: Process failing Binary Object Queries tests. Key: IGNITE-3980 URL: https://issues.apache.org/jira/browse/IGNITE-3980 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3981) GridTestProperties should be test instance member.
Vladimir Ozerov created IGNITE-3981: --- Summary: GridTestProperties should be test instance member. Key: IGNITE-3981 URL: https://issues.apache.org/jira/browse/IGNITE-3981 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3977) Process failing OSGI tests.
Vladimir Ozerov created IGNITE-3977: --- Summary: Process failing OSGI tests. Key: IGNITE-3977 URL: https://issues.apache.org/jira/browse/IGNITE-3977 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3976) Process failing SPI tests.
Vladimir Ozerov created IGNITE-3976: --- Summary: Process failing SPI tests. Key: IGNITE-3976 URL: https://issues.apache.org/jira/browse/IGNITE-3976 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3978) Process failing AWS tests.
Vladimir Ozerov created IGNITE-3978: --- Summary: Process failing AWS tests. Key: IGNITE-3978 URL: https://issues.apache.org/jira/browse/IGNITE-3978 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3972) Continuous query events could be lost on backup node when primary leaves.
Vladimir Ozerov created IGNITE-3972:
---
Summary: Continuous query events could be lost on backup node when
primary leaves.
Key: IGNITE-3972
URL: https://issues.apache.org/jira/browse/IGNITE-3972
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Andrew Mashenkov
Fix For: 1.8
Consider the following scenario:
1) One node in topology;
2) PARTITIONED cache with 1 backup;
3) Continuous query is set on the cache;
If another node joins the cluster, it will handle some updates. If it leaves
the topology, the first node must flush it's own events from backup queue thus
ensuring that no events are lost.
But this doesn't happen because
{{GridContinuousProcessor.addBackupNotification}} method perform lookup only on
remote infos map, while handler is local and hence located in local infos map.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3974) Process failing ZooKeeper tests.
Vladimir Ozerov created IGNITE-3974: --- Summary: Process failing ZooKeeper tests. Key: IGNITE-3974 URL: https://issues.apache.org/jira/browse/IGNITE-3974 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3975) Process failing streamer tests.
Vladimir Ozerov created IGNITE-3975: --- Summary: Process failing streamer tests. Key: IGNITE-3975 URL: https://issues.apache.org/jira/browse/IGNITE-3975 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4001) Ignite thread pools must have timeouts for idle threads.
Vladimir Ozerov created IGNITE-4001:
---
Summary: Ignite thread pools must have timeouts for idle threads.
Key: IGNITE-4001
URL: https://issues.apache.org/jira/browse/IGNITE-4001
Project: Ignite
Issue Type: Task
Components: general
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 1.8
Classes to look at:
{{IgnitionEx}} - 7 pools;
{{GridIoManager}} - affinity pool;
{{IgfsImpl}} - dual pool;
{{IgfsIpcHandler}} - pool to process incoming requests;
{{OdbcProcessor}} - pool to process incoming requests;
{{ServerImpl}} - utility pool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-3988) Process failing cloud tests.
Vladimir Ozerov created IGNITE-3988: --- Summary: Process failing cloud tests. Key: IGNITE-3988 URL: https://issues.apache.org/jira/browse/IGNITE-3988 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3989) Process failing JTA tests.
Vladimir Ozerov created IGNITE-3989: --- Summary: Process failing JTA tests. Key: IGNITE-3989 URL: https://issues.apache.org/jira/browse/IGNITE-3989 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3993) Process failing client nodes tests.
Vladimir Ozerov created IGNITE-3993: --- Summary: Process failing client nodes tests. Key: IGNITE-3993 URL: https://issues.apache.org/jira/browse/IGNITE-3993 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3992) Process failing Hibernate tests.
Vladimir Ozerov created IGNITE-3992: --- Summary: Process failing Hibernate tests. Key: IGNITE-3992 URL: https://issues.apache.org/jira/browse/IGNITE-3992 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3990) Process failing Spring tests.
Vladimir Ozerov created IGNITE-3990: --- Summary: Process failing Spring tests. Key: IGNITE-3990 URL: https://issues.apache.org/jira/browse/IGNITE-3990 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3991) Process failing IGFS tests.
Vladimir Ozerov created IGNITE-3991: --- Summary: Process failing IGFS tests. Key: IGNITE-3991 URL: https://issues.apache.org/jira/browse/IGNITE-3991 Project: Ignite Issue Type: Sub-task Components: general Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3987) ODBC: Improve error output when query parsing failed.
Vladimir Ozerov created IGNITE-3987: --- Summary: ODBC: Improve error output when query parsing failed. Key: IGNITE-3987 URL: https://issues.apache.org/jira/browse/IGNITE-3987 Project: Ignite Issue Type: Task Components: odbc Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Igor Sapego Priority: Minor Fix For: 1.8 Currently if an error occurred we only prints the top-level message, like "Failed to parse query ...". The problem is that we do not explain users why exactly it failed. Looks like we need to add more info on Java side when sending error response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-3969) Hadoop: Add installation instructions for MapR to readme.io.
Vladimir Ozerov created IGNITE-3969: --- Summary: Hadoop: Add installation instructions for MapR to readme.io. Key: IGNITE-3969 URL: https://issues.apache.org/jira/browse/IGNITE-3969 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4003) Slow or faulty client can stall the whole cluster.
Vladimir Ozerov created IGNITE-4003:
---
Summary: Slow or faulty client can stall the whole cluster.
Key: IGNITE-4003
URL: https://issues.apache.org/jira/browse/IGNITE-4003
Project: Ignite
Issue Type: Bug
Components: cache, general
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Priority: Critical
Fix For: 1.8
Steps to reproduce:
1) Start two server nodes and some data to cache.
2) Start a client from Docker subnet, which is not visible from the outside.
Client will join the cluster.
3) Try to put something to cache or start another node to force rabalance.
Cluster is stuck at this moment. Root cause - servers are constantly trying to
establish outgoing connection to the client, but fail as Docker subnet is not
visible from the outside. It may stop virtually all cluster operations.
Typical thread dump:
{code}
org.apache.ignite.IgniteCheckedException: Failed to send message (node may have
left the grid or TCP connection cannot be established due to firewall issues)
[node=TcpDiscoveryNode [id=a15d74c2-1ec2-4349-9640-aeacd70d8714,
addrs=[127.0.0.1, 172.17.0.6], sockAddrs=[/127.0.0.1:0, /127.0.0.1:0,
/172.17.0.6:0], discPort=0, order=7241, intOrder=3707,
lastExchangeTime=1474096941045, loc=false, ver=1.5.23#20160526-sha1:259146da,
isClient=true], topic=T4 [topic=TOPIC_CACHE,
id1=949732fd-1360-3a58-8d9e-0ff6ea6182cc,
id2=a15d74c2-1ec2-4349-9640-aeacd70d8714, id3=2], msg=GridContinuousMessage
[type=MSG_EVT_NOTIFICATION, routineId=7e13c48e-6933-48b2-9f15-8d92007930db,
data=null, futId=null], policy=2]
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1129)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1347)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1227)
~[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1198)
~[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1180)
~[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendNotification(GridContinuousProcessor.java:841)
~[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.addNotification(GridContinuousProcessor.java:800)
~[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.onEntryUpdate(CacheContinuousQueryHandler.java:787)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.access$700(CacheContinuousQueryHandler.java:91)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler$1.onEntryUpdated(CacheContinuousQueryHandler.java:412)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:343)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.onEntryUpdated(CacheContinuousQueryManager.java:250)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:3476)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture$MiniFuture.onResult(GridDhtForceKeysFuture.java:548)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onResult(GridDhtForceKeysFuture.java:207)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.processForceKeyResponse(GridDhtPreloader.java:636)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.access$1000(GridDhtPreloader.java:81)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$3.onMessage(GridDhtPreloader.java:202)
[ignite-core-1.5.23.jar:1.5.23]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader$3.onMessage(GridDhtPreloader.java:200)
[ignite-core-1.5.23.jar:1.5.23]
at
[jira] [Created] (IGNITE-4112) Client is stopped if communication to servers is lost during (re)connect routine.
Vladimir Ozerov created IGNITE-4112:
---
Summary: Client is stopped if communication to servers is lost
during (re)connect routine.
Key: IGNITE-4112
URL: https://issues.apache.org/jira/browse/IGNITE-4112
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Fix For: 1.8
Consider the following situation for the client which is currently disconnected:
1) Link to servers is restored;
2) Clients connects successfully;
3) Link is broken again after some time.
Result: client is disconnected.
But!
1) Link to servers is restored;
2) Clients is re-connecting;
3) Link is broken again before reconnect is finished.
Result: client might be stopped (for new client it will exit from
{{Ignition.start}} routine with exception).
Apparently, we need to make (re)connect routine "atomic" with respect to
concurrent network events. That is, we either reconnect all components
successfully, or client is forcefully moved to disconnected state again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4215) Introduce global identity resolver for binary objects.
Vladimir Ozerov created IGNITE-4215: --- Summary: Introduce global identity resolver for binary objects. Key: IGNITE-4215 URL: https://issues.apache.org/jira/browse/IGNITE-4215 Project: Ignite Issue Type: Task Components: binary Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Alexander Paschenko Fix For: 2.0 Currently identity resolver can only be set on per-type level. We need to introduce it on global level as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4086) Binary objets: Do not use BinaryField when comparing objects with the same schema.
Vladimir Ozerov created IGNITE-4086: --- Summary: Binary objets: Do not use BinaryField when comparing objects with the same schema. Key: IGNITE-4086 URL: https://issues.apache.org/jira/browse/IGNITE-4086 Project: Ignite Issue Type: Task Components: binary Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 1.8 Most often we perform comparisons of objects with the same schema. When it is so, there is no need to compare them field-by-field. Instead, we should do the following: 1) Skip header 2) Skip footer 3) Compare byte-by-byte everything in between. Please ensure that byte-by-byte comparison is implemented efficiently. E.g. that we actually compare bigger chunks (e.g. 8 bytes at once). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4085) Binary objets: Compare fields of well-known types without deserialization.
Vladimir Ozerov created IGNITE-4085:
---
Summary: Binary objets: Compare fields of well-known types without
deserialization.
Key: IGNITE-4085
URL: https://issues.apache.org/jira/browse/IGNITE-4085
Project: Ignite
Issue Type: Task
Components: binary
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Fix For: 1.8
It is not efficient to use {{BinaryField.value}} directly when comparing binary
objects because it leads to "materialization" of the field.
Instead, when well-known type is met, we should simply find field start in both
objects and perform usual byte-by-byte comparison.
E.g., for string it will be:
1) Find position
2) Compare lengths (readInt)
3) Compare contents byte-by-byte.
The same goes for other types:
1) Primitive arrays
2) UUID
3) Data/TimeStamp.
Please ensure that byte-by-byte comparison is performed efficiently, e.g. that
we compare bigger chunks (e.g. 8 bytes at once).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4105) Create separate thread pool for SQL queries.
Vladimir Ozerov created IGNITE-4105: --- Summary: Create separate thread pool for SQL queries. Key: IGNITE-4105 URL: https://issues.apache.org/jira/browse/IGNITE-4105 Project: Ignite Issue Type: Task Components: SQL Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Andrew Mashenkov Fix For: 1.8 If several long-running queries are executed, all concurrent cache operations will be blocked because there will be no threads in system pool to server cache requests/responses. Let's introduce separate thread pool for SQL queries execution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4089) Hadoop: Add MapR installation instructions to readme.io.
Vladimir Ozerov created IGNITE-4089: --- Summary: Hadoop: Add MapR installation instructions to readme.io. Key: IGNITE-4089 URL: https://issues.apache.org/jira/browse/IGNITE-4089 Project: Ignite Issue Type: Task Components: documentation, hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4092) Implement prototype for thread-per-partition approach.
Vladimir Ozerov created IGNITE-4092: --- Summary: Implement prototype for thread-per-partition approach. Key: IGNITE-4092 URL: https://issues.apache.org/jira/browse/IGNITE-4092 Project: Ignite Issue Type: Task Components: cache Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 Rationale: - Decrease contention on thread-pool blocking queue head/tail; - Minimize amount of thread parks/unparks; - Decrease overhead on partition RW locking. Implementation: - Implemented striped thread pool; - Every thread has separate non-blocking queue; - Blocking back-off strategy if queue is empty. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4270) Hadoop: optionally stripe mapper output for every partition.
Vladimir Ozerov created IGNITE-4270: --- Summary: Hadoop: optionally stripe mapper output for every partition. Key: IGNITE-4270 URL: https://issues.apache.org/jira/browse/IGNITE-4270 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 Currently we have R maps for M mappers, where R is number of reducers. For this reason many mappers writes to concurrent offheap data structure, loosing time on concurrency burden. Let's add an option to create R * M maps, so that every mapper has dedicated map for every reducer. This will eliminate almost all concurrency overhead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4271) Hadoop messages must use "direct marshallable" infrastructure.
Vladimir Ozerov created IGNITE-4271: --- Summary: Hadoop messages must use "direct marshallable" infrastructure. Key: IGNITE-4271 URL: https://issues.apache.org/jira/browse/IGNITE-4271 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7, 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4273) Hadoop: implement head-based data structures.
Vladimir Ozerov created IGNITE-4273: --- Summary: Hadoop: implement head-based data structures. Key: IGNITE-4273 URL: https://issues.apache.org/jira/browse/IGNITE-4273 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 We store output offheap what causes a lot of offheap <-> heap transitions. Also it doesn't allow is to use raw comparator. Let's add new data structures which utilize heap arrays. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4276) Hadoop: control shuffle jobs "sleep" backpressure with property.
Vladimir Ozerov created IGNITE-4276: --- Summary: Hadoop: control shuffle jobs "sleep" backpressure with property. Key: IGNITE-4276 URL: https://issues.apache.org/jira/browse/IGNITE-4276 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 Currently we simply sleep between shuffle job iterations for 5 milliseconds (hard-coded). Let's get more control around it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4277) Hadoop: support raw comparator.
Vladimir Ozerov created IGNITE-4277: --- Summary: Hadoop: support raw comparator. Key: IGNITE-4277 URL: https://issues.apache.org/jira/browse/IGNITE-4277 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4278) Hadoop: investigate why small array copying is slow on PPC.
Vladimir Ozerov created IGNITE-4278: --- Summary: Hadoop: investigate why small array copying is slow on PPC. Key: IGNITE-4278 URL: https://issues.apache.org/jira/browse/IGNITE-4278 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4272) Hadoop: shuffle messages must always be processed asynchronously wtr to shuffle job or mapper.
Vladimir Ozerov created IGNITE-4272: --- Summary: Hadoop: shuffle messages must always be processed asynchronously wtr to shuffle job or mapper. Key: IGNITE-4272 URL: https://issues.apache.org/jira/browse/IGNITE-4272 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 Currently we run "GridIoManager.send()" from shuffle job. If it hit local node, then message will be processed synchronously, stalling shuffling process for other mappers. Need to ensure that it is always routed to internal pool instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4281) Hadoop: decouple mapper and reducer maps.
Vladimir Ozerov created IGNITE-4281: --- Summary: Hadoop: decouple mapper and reducer maps. Key: IGNITE-4281 URL: https://issues.apache.org/jira/browse/IGNITE-4281 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.7, 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4274) Hadoop: control shuffle message buffer size through property.
Vladimir Ozerov created IGNITE-4274: --- Summary: Hadoop: control shuffle message buffer size through property. Key: IGNITE-4274 URL: https://issues.apache.org/jira/browse/IGNITE-4274 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 Currently it is hard-coded to 128Kb. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4275) Hadoop: create separate property to avoid sorting of mapper output.
Vladimir Ozerov created IGNITE-4275: --- Summary: Hadoop: create separate property to avoid sorting of mapper output. Key: IGNITE-4275 URL: https://issues.apache.org/jira/browse/IGNITE-4275 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 Currently it is "merged" with reducer sorting flag. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4286) Hadoop: use plain property names.
Vladimir Ozerov created IGNITE-4286: --- Summary: Hadoop: use plain property names. Key: IGNITE-4286 URL: https://issues.apache.org/jira/browse/IGNITE-4286 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4262) Hadoop Accelerator performance improvements.
Vladimir Ozerov created IGNITE-4262: --- Summary: Hadoop Accelerator performance improvements. Key: IGNITE-4262 URL: https://issues.apache.org/jira/browse/IGNITE-4262 Project: Ignite Issue Type: Task Components: hadoop Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4263) Hadoop: abstract out offheap/heap memory management.
Vladimir Ozerov created IGNITE-4263: --- Summary: Hadoop: abstract out offheap/heap memory management. Key: IGNITE-4263 URL: https://issues.apache.org/jira/browse/IGNITE-4263 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 2.0 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4295) GridUnsafe: implement specialized methods for every kind of copy operation.
Vladimir Ozerov created IGNITE-4295: --- Summary: GridUnsafe: implement specialized methods for every kind of copy operation. Key: IGNITE-4295 URL: https://issues.apache.org/jira/browse/IGNITE-4295 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 2.0 1) copy(OFFHEAP, HEAP) 2) copy(HEAP, OFFHEAP) 3) copy(OFFHEAP, OFFHEAP) 4) copy(HEAP, HEAP) 5) copy(T[], T[]) must be avoided and replaced with System.arrayCopy(). 6) Add optional threshold. If we copy too small memory chunk which size is below the threshold, then resort to byte-by-byte copying, as it will be faster. E.g. on PowerPC with OpenJDK 8, copying of <100 bytes of data is faster on byte-by-byte basis. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4294) Allow unaligned memory access flag override in GridUnsafe.
Vladimir Ozerov created IGNITE-4294: --- Summary: Allow unaligned memory access flag override in GridUnsafe. Key: IGNITE-4294 URL: https://issues.apache.org/jira/browse/IGNITE-4294 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 This way we can avoid byte-by-byte read/write if arch is not x86, but it permits unaligned access still. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4305) GridNearAtomicSingleUpdateInvokeRequest cannot unmarshal invoke arguments properly.
Vladimir Ozerov created IGNITE-4305:
---
Summary: GridNearAtomicSingleUpdateInvokeRequest cannot unmarshal
invoke arguments properly.
Key: IGNITE-4305
URL: https://issues.apache.org/jira/browse/IGNITE-4305
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Semen Boikov
Fix For: 1.8
Steps to reproduce:
1) Set binary marshaller as deafult for
{{GridCacheBinaryAtomicEntryProcessorDeploymentSelfTest}}:
{code}
static {
GridTestProperties.setProperty(GridTestProperties.MARSH_CLASS_NAME,
BinaryMarshaller.class.getName());
GridTestProperties.setProperty(GridTestProperties.ENTRY_PROCESSOR_CLASS_NAME,
"org.apache.ignite.tests.p2p.CacheDeploymentBinaryEntryProcessor");
}
{code}
2) Run the test, observe failures in {{testInvokeDeployment*}} tests.
3) Go to {{GridNearAtomicSingleUpdateFuture.mapSingleUpdate}} and force single
transform to use old {{GridNearAtomicFullUpdateRequest}}
4) Re-run test. No all pass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4300) Hadoop: get rid of shuffle job.
Vladimir Ozerov created IGNITE-4300: --- Summary: Hadoop: get rid of shuffle job. Key: IGNITE-4300 URL: https://issues.apache.org/jira/browse/IGNITE-4300 Project: Ignite Issue Type: Sub-task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 1) When mapper detects that -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4309) IgniteLockExample hangs in multi-node mode.
Vladimir Ozerov created IGNITE-4309:
---
Summary: IgniteLockExample hangs in multi-node mode.
Key: IGNITE-4309
URL: https://issues.apache.org/jira/browse/IGNITE-4309
Project: Ignite
Issue Type: Bug
Components: data structures
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Semen Boikov
Priority: Critical
Fix For: 1.8
Steps to reproduce:
1) Start CacheExamplesMultiNodeSelfTest.testCacheLockExample.
2) Observe that example hangs and the following metrics printout appears form
time to time:
{code}
^-- Public thread pool [active=2, idle=0, qSize=0]
{code}
or
{code}
^-- Public thread pool [active=3, idle=0, qSize=0]
{code}
Logs with thread dumps on TC:
http://195.239.208.174/viewLog.html?buildId=368311=IgniteTests_IgniteExamples=buildLog#_focus=24510
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4310) Add release notes for Ignite 1.8.
Vladimir Ozerov created IGNITE-4310: --- Summary: Add release notes for Ignite 1.8. Key: IGNITE-4310 URL: https://issues.apache.org/jira/browse/IGNITE-4310 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Blocker Fix For: 1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4301) Hadoop: put shuffle ack to the top of send queue.
Vladimir Ozerov created IGNITE-4301: --- Summary: Hadoop: put shuffle ack to the top of send queue. Key: IGNITE-4301 URL: https://issues.apache.org/jira/browse/IGNITE-4301 Project: Ignite Issue Type: Sub-task Components: general, hadoop Affects Versions: 1.7, 1.8 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Fix For: 2.0 This shuffle acks will be processed ASAP, hence minimizing wait during completion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4240) Review GridInternal usages.
Vladimir Ozerov created IGNITE-4240:
---
Summary: Review GridInternal usages.
Key: IGNITE-4240
URL: https://issues.apache.org/jira/browse/IGNITE-4240
Project: Ignite
Issue Type: Task
Components: compute
Affects Versions: 1.7, 1.8
Reporter: Vladimir Ozerov
Fix For: 2.0
We have {{@GridInternal}} annotation which manages which thread will execute
particular job - public pool or management pool. This annotation is applied to
some internal Ignite jobs.
There is strong evidence that it doesn't work in all places where we set it:
1) Some annotation is set on tasks and this is where everything appears to be
ok.
2) Some annotation is set on jobs. E.g {{GridCacheAdapter.SizeJob}}. In this
case annotation was ignored by parent {{GridCacheAdapter.SizeTask}} what caused
starvation described in IGNITE-4239. There are more places where we do that.
3) Sometimes it is set on closures or jobs which are passed to
{{GridClosureProcessor}}. E.g. looks at {{GridAffinityUtils.AffinityJob}}. It
is passed to {{GridClosureProcessor.callAsyncNoFailover}} where execution is
forced to happen in system pool (sick!).
To summarize: {{@GridInternal}} jobs and closures are expected to be executed
in management pool only, but in reality they can be executed in system, public
or management pool.
We need unified solution for this problem, so that any task, job, or closure
with this annotation is processed properly. Suggested solution:
1) If task is marked internal, all it's jobs must be executed in management
pool.
2) If certain job is marked internal, it must be executed in management pool.
3) If closure is marked internal, it must be executed in management pool.
4) All methods (~10) having {{boolean sys}} argument in
{{GridClosureProcessor}} must be reviewed as follows:
- If there are not public usages outside that class or all such usages have
{{system = false}}, then remove methods with flags and delegate decision to
task processor;
- If there are public usages ({{runLocalSafe}}, {{callLocalSafe}}), split every
such methods in two. First will always execute in system pool, second will make
decision based on annotation. This way we will not make mistakes in future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4237) Implement interface to local Ignite instance over IPC
Vladimir Ozerov created IGNITE-4237: --- Summary: Implement interface to local Ignite instance over IPC Key: IGNITE-4237 URL: https://issues.apache.org/jira/browse/IGNITE-4237 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.8 Reporter: Vladimir Ozerov Fix For: 2.0 This way we can cover some important cases: 1) User has short-lived processes which require access to Ignite. 2) User do not want or cannot have JVM in the same process. No failover is needed here at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4171) DML: Print warning when no identity is set for type.
Vladimir Ozerov created IGNITE-4171: --- Summary: DML: Print warning when no identity is set for type. Key: IGNITE-4171 URL: https://issues.apache.org/jira/browse/IGNITE-4171 Project: Ignite Issue Type: Sub-task Components: SQL Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Alexander Paschenko Priority: Minor Fix For: 1.8 When no identity for type is set, we should do the following: 1) Pick array-based identity instance and use it for hash code generation 2) Print a warning, once per type, that array-based identity was used automatically, and resulting hash code unlikely to match hash function of a real type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4149) Consider adding bitmap index to SQL engine.
Vladimir Ozerov created IGNITE-4149: --- Summary: Consider adding bitmap index to SQL engine. Key: IGNITE-4149 URL: https://issues.apache.org/jira/browse/IGNITE-4149 Project: Ignite Issue Type: Task Components: SQL Affects Versions: 1.7 Reporter: Vladimir Ozerov Fix For: 2.0 Bitmap index can help us a lot when there are only several distinct values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4145) "No query result found for request" exception when running multiple queries concurrently.
Vladimir Ozerov created IGNITE-4145:
---
Summary: "No query result found for request" exception when
running multiple queries concurrently.
Key: IGNITE-4145
URL: https://issues.apache.org/jira/browse/IGNITE-4145
Project: Ignite
Issue Type: Bug
Components: SQL
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Assignee: Andrew Mashenkov
Fix For: 1.8
When many queries are executed simultaneously in multi-node cluster, the
following exception might appear from time to time:
{code}
Exception in thread "qry-exec-20" javax.cache.CacheException: Failed to fetch
data from node: 3432842f-2fa2-44b9-b69b-cadba286d317
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor$3.fetchNextPage(GridReduceQueryExecutor.java:298)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridMergeIndex.fetchNextPage(GridMergeIndex.java:229)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridMergeIndexUnsorted$1.hasNext(GridMergeIndexUnsorted.java:106)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridMergeIndex$IteratorCursor.next(GridMergeIndex.java:351)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridMergeIndex$FetchingCursor.next(GridMergeIndex.java:382)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:640)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$4.iterator(IgniteH2Indexing.java:1057)
at
org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:81)
at
org.apache.ignite.yardstick.ringcentral.AdgRunner$QueryExecutor.run(AdgRunner.java:394)
at java.lang.Thread.run(Thread.java:745)
Suppressed: javax.cache.CacheException: Failed to execute map query on
the node: c51bc4f2-7038-4df6-88a9-eb87506c2f86, class
javax.cache.CacheException:No query result found for request:
GridQueryNextPageRequest [qryReqId=93, qry=0, pageSize=1024]
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.fail(GridReduceQueryExecutor.java:259)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.onFail(GridReduceQueryExecutor.java:249)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.onMessage(GridReduceQueryExecutor.java:230)
at
org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor$1.onMessage(GridReduceQueryExecutor.java:178)
at
org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage(GridIoManager.java:1900)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1080)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:708)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:101)
at
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:671)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
{code}
Quick analysis shows that next-page request could possibly arrive before
original query message is processed. Need to confirm it with deeper debugging
first.
The best way to reproduce it:
1) Generate some data
2) Start several data nodes (e.g. 4)
3) Start client node
4) Concurrently execute multiple SQL queries from the client in different
threads.
P.S.: May be cache start message gets delayed in IO manager during start?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4150) B-Tree index cannot be used efficiently with IN clause.
Vladimir Ozerov created IGNITE-4150:
---
Summary: B-Tree index cannot be used efficiently with IN clause.
Key: IGNITE-4150
URL: https://issues.apache.org/jira/browse/IGNITE-4150
Project: Ignite
Issue Type: Task
Components: SQL
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Fix For: 2.0
Consider the following query:
{code}
SELECT * FROM table
WHERE a = ? AND b IN (?, ?)
{code}
If there is an index {{(a, b)}}, it will not be used properly: only column
{{a}} will be used. This will leads to multiple unnecessary comparisons.
Most obvious way to fix that - use temporary table and {{JOIN}}. However, this
approach doesn't work well when there are multiple {{IN}}s.
Proper solution would be to hack deeper into H2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4054) Add debug to trace HadoopMapReducePlanner results.
Vladimir Ozerov created IGNITE-4054: --- Summary: Add debug to trace HadoopMapReducePlanner results. Key: IGNITE-4054 URL: https://issues.apache.org/jira/browse/IGNITE-4054 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.7 Reporter: Vladimir Ozerov Assignee: Vladimir Ozerov Priority: Minor Fix For: 1.8 This is necessary to track planning process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4053) Task error is always printed to the console.
Vladimir Ozerov created IGNITE-4053:
---
Summary: Task error is always printed to the console.
Key: IGNITE-4053
URL: https://issues.apache.org/jira/browse/IGNITE-4053
Project: Ignite
Issue Type: Bug
Components: compute
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Priority: Minor
Fix For: 1.8
Currently if something wrong happens to the task, it prints error to the
console:
{code}
java.lang.Throwable.printStackTrace(java.lang.Throwable$PrintStreamOrWriter)
(line: 653)
java.lang.Throwable.printStackTrace(java.io.PrintStream) (line: 643)
org.apache.ignite.internal.util.IgniteUtils.error(org.apache.ignite.IgniteLogger,
java.lang.Object, java.lang.Object, java.lang.Throwable) (line: 4171)
org.apache.ignite.internal.util.IgniteUtils.error(org.apache.ignite.IgniteLogger,
java.lang.Object, java.lang.Throwable) (line: 4189)
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(org.apache.ignite.lang.IgniteInClosure)
(line: 269)
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners()
(line: 250)
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(java.lang.Object,
java.lang.Throwable, boolean) (line: 380)
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(java.lang.Object,
java.lang.Throwable)
org.apache.ignite.internal.processors.task.GridTaskWorker.finishTask(java.lang.Object,
java.lang.Throwable, boolean) (line: 1475)
org.apache.ignite.internal.processors.task.GridTaskWorker.finishTask(java.lang.Object,
java.lang.Throwable) (line: 1443)
org.apache.ignite.internal.processors.task.GridTaskWorker$4.apply() (line: 988)
org.apache.ignite.internal.processors.task.GridTaskWorker$4.apply()
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(java.lang.ClassLoader,
org.apache.ignite.lang.IgniteOutClosure) (line: 6516)
org.apache.ignite.internal.processors.task.GridTaskWorker.result(org.apache.ignite.compute.ComputeJobResult,
java.util.List)
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(org.apache.ignite.internal.GridJobExecuteResponse)
(line: 810)
{code}
This happens because there is no logger in the future, hence error is
unconditionally printed to the console.
We need to fix that. Probably we can inject logger into the task future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4060) RoundRobinLoadBalancingSpi doesn't work after client reconnect.
Vladimir Ozerov created IGNITE-4060:
---
Summary: RoundRobinLoadBalancingSpi doesn't work after client
reconnect.
Key: IGNITE-4060
URL: https://issues.apache.org/jira/browse/IGNITE-4060
Project: Ignite
Issue Type: Bug
Components: compute
Affects Versions: 1.7
Reporter: Vladimir Ozerov
Fix For: 1.8
Attachments: RoundRodbinErrorReproducer.java
If {{IgntieCompute.apply()}} is called on a client after reconnect, it fails.
Reproducer code is attached. The problem is that {{RoundRobinLoadBalancingSpi}}
(which is default one) is not notified about topology reset.
Sample stack trace:
{code}
class org.apache.ignite.IgniteException: Task topology does not have alive
nodes: [TcpDiscoveryNode [id=50f56f53-bbf6-4d66-bfc5-02d801241e14,
addrs=[127.0.0.1], sockAddrs=[/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1476269284697, loc=false,
ver=1.7.0#19700101-sha1:, isClient=false]]
at
org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinGlobalLoadBalancer.checkBalancerNodes(RoundRobinGlobalLoadBalancer.java:256)
at
org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinGlobalLoadBalancer.getBalancedNode(RoundRobinGlobalLoadBalancer.java:201)
at
org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinLoadBalancingSpi.getBalancedNode(RoundRobinLoadBalancingSpi.java:307)
at
org.apache.ignite.internal.managers.loadbalancer.GridLoadBalancerManager.getBalancedNode(GridLoadBalancerManager.java:81)
at
org.apache.ignite.internal.managers.loadbalancer.GridLoadBalancerManager$1.getBalancedNode(GridLoadBalancerManager.java:99)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$T8.map(GridClosureProcessor.java:1616)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$T8.map(GridClosureProcessor.java:1587)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$2.call(GridTaskWorker.java:519)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$2.call(GridTaskWorker.java:517)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6509)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:516)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:679)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:403)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor.callAsync(GridClosureProcessor.java:630)
at
org.apache.ignite.internal.IgniteComputeImpl.apply(IgniteComputeImpl.java:451)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4399) Merge IgfsSecondaryFileSystemV2 and IgfsSecondaryFileSystem
Vladimir Ozerov created IGNITE-4399:
---
Summary: Merge IgfsSecondaryFileSystemV2 and
IgfsSecondaryFileSystem
Key: IGNITE-4399
URL: https://issues.apache.org/jira/browse/IGNITE-4399
Project: Ignite
Issue Type: Task
Components: IGFS
Affects Versions: 2.0
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
Fix For: 2.0
Let's move all methods from {{IgfsSecondaryFileSystemV2}} to
{{IgfsSecondaryFileSystemV2}} and remove V2 altogether.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4405) Hadoop: support "readLine" method in HadoopDataInStream and HadoopDirectDataInput classes.
Vladimir Ozerov created IGNITE-4405: --- Summary: Hadoop: support "readLine" method in HadoopDataInStream and HadoopDirectDataInput classes. Key: IGNITE-4405 URL: https://issues.apache.org/jira/browse/IGNITE-4405 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Taras Ledkov Fix For: 2.0 They are simply not implemented now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-4407) IgniteUuid is a single point of contention for the whole framework.
Vladimir Ozerov created IGNITE-4407:
---
Summary: IgniteUuid is a single point of contention for the whole
framework.
Key: IGNITE-4407
URL: https://issues.apache.org/jira/browse/IGNITE-4407
Project: Ignite
Issue Type: Task
Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Priority: Minor
Fix For: 2.0
It uses static {{AtomicLong.incrementAndGet}}. We can use thread-locals to
minimize contention.
Similar problem is in {{GridCacheIoManager}} and {{GridCacheMvccCandidate}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4426) Hadoop: allow optional classloader reuse between tasks.
Vladimir Ozerov created IGNITE-4426:
---
Summary: Hadoop: allow optional classloader reuse between tasks.
Key: IGNITE-4426
URL: https://issues.apache.org/jira/browse/IGNITE-4426
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 2.0
*Problem*
Currently we create separate {{HadoopClassLoader}} for each and every MR task.
It has serious performance implications:
1) Every task have to reload classes form disk;
2) Every task starts from "cold" state with not-compiled classes;
3) Very high permgen/metaspace and code cache regions usage.
We can observe, that every task classloader works with the same JARs. It means
that the only case when we really need to isolate tasks through classloaders is
when they depend on some static data, which should not be shared.
*Proposal*
- Share {{HadoopClassLoader}} between tasks by default;
- Allow to optionally disable this opto.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4428) Hadoop: move HadoopMapReducePlanner and dependent classes to public space.
Vladimir Ozerov created IGNITE-4428: --- Summary: Hadoop: move HadoopMapReducePlanner and dependent classes to public space. Key: IGNITE-4428 URL: https://issues.apache.org/jira/browse/IGNITE-4428 Project: Ignite Issue Type: Task Components: hadoop Affects Versions: 1.8 Reporter: Vladimir Ozerov Assignee: Taras Ledkov Priority: Critical Fix For: 2.0 Currently these classes are located in private package. Need to move the to public. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
