[jira] [Assigned] (IGNITE-4007) SQL: QueryMetrics.minimumTime() is always zero.
[
https://issues.apache.org/jira/browse/IGNITE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Semen Boikov reassigned IGNITE-4007:
Assignee: Alexey Kuznetsov (was: Semen Boikov)
Reviewed, looks good.
> SQL: QueryMetrics.minimumTime() is always zero.
> ---
>
> Key: IGNITE-4007
> URL: https://issues.apache.org/jira/browse/IGNITE-4007
> Project: Ignite
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.6
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
> Fix For: 1.8
>
>
> QueryMetrics.minimumTime() is always zero, because internal AtomicInteger
> initialized with zero and code that change minTime looks like
> {code}minTime.setIfLess(duration);{code}
> Obviously that execution time is always >=0 and any number is always is
> greate than zero and metrics never updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (IGNITE-3998) IGFS: uncomment testCreateConsistencyMultithreaded
[ https://issues.apache.org/jira/browse/IGNITE-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-3998: Priority: Trivial (was: Major) > IGFS: uncomment testCreateConsistencyMultithreaded > --- > > Key: IGNITE-3998 > URL: https://issues.apache.org/jira/browse/IGNITE-3998 > Project: Ignite > Issue Type: Bug > Components: IGFS >Affects Versions: 1.6 >Reporter: Ivan Veselovsky >Assignee: Vladimir Ozerov >Priority: Trivial > Fix For: 1.8 > > > Test > org.apache.ignite.internal.processors.igfs.IgfsAbstractSelfTest#testCreateConsistencyMultithreaded > was commented out for unknown reason. > Uncomment it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (IGNITE-3998) IGFS: uncomment testCreateConsistencyMultithreaded
[ https://issues.apache.org/jira/browse/IGNITE-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov closed IGNITE-3998. --- > IGFS: uncomment testCreateConsistencyMultithreaded > --- > > Key: IGNITE-3998 > URL: https://issues.apache.org/jira/browse/IGNITE-3998 > Project: Ignite > Issue Type: Bug > Components: IGFS >Affects Versions: 1.6 >Reporter: Ivan Veselovsky >Assignee: Vladimir Ozerov >Priority: Trivial > Fix For: 1.8 > > > Test > org.apache.ignite.internal.processors.igfs.IgfsAbstractSelfTest#testCreateConsistencyMultithreaded > was commented out for unknown reason. > Uncomment it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535348#comment-15535348 ] Alexey Kuznetsov commented on IGNITE-3609: -- Looks good. Merged to master. One last question - it seems that mutations should be documented on readme.io ? > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535348#comment-15535348 ] Alexey Kuznetsov edited comment on IGNITE-3609 at 9/30/16 7:55 AM: --- Looks good. Merged to master. One last question - it seems that mutations should be documented on readme.io ? Please close this issue if issue is done. was (Author: kuaw26): Looks good. Merged to master. One last question - it seems that mutations should be documented on readme.io ? > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-2881) SPI queries not working
[
https://issues.apache.org/jira/browse/IGNITE-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535376#comment-15535376
]
Vladimir Ozerov commented on IGNITE-2881:
-
Re-running test on TC right now. Will merge if it is fine.
> SPI queries not working
> ---
>
> Key: IGNITE-2881
> URL: https://issues.apache.org/jira/browse/IGNITE-2881
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Assignee: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.8
>
> Attachments: SpiQueryTest.java
>
>
> {{SpiQuery}} functionality looks completely broken right now, any query
> execution fails with the exception shown below. Also I didn't find a single
> test for it, they should be added.
> I'm attaching the simple example that reproduce the issue.
> {noformat}
> Caused by: class org.apache.ignite.IgniteCheckedException: Received next page
> request after iterator was removed. Consider increasing maximum number of
> stored iterators (see GridCacheConfiguration.getMaximumQueryIteratorCount()
> configuration property).
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.executeFieldsQuery(GridCacheQueryManager.java:666)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runFieldsQuery(GridCacheQueryManager.java:1168)
> ... 7 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (IGNITE-4007) SQL: QueryMetrics.minimumTime() is always zero.
[
https://issues.apache.org/jira/browse/IGNITE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kuznetsov closed IGNITE-4007.
> SQL: QueryMetrics.minimumTime() is always zero.
> ---
>
> Key: IGNITE-4007
> URL: https://issues.apache.org/jira/browse/IGNITE-4007
> Project: Ignite
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.6
>Reporter: Alexey Kuznetsov
>Assignee: Alexey Kuznetsov
> Fix For: 1.8
>
>
> QueryMetrics.minimumTime() is always zero, because internal AtomicInteger
> initialized with zero and code that change minTime looks like
> {code}minTime.setIfLess(duration);{code}
> Obviously that execution time is always >=0 and any number is always is
> greate than zero and metrics never updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (IGNITE-3133) Unable to get a reference to a cache inside TX on a node that is filtered out by a node filter
[
https://issues.apache.org/jira/browse/IGNITE-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Govorukhin reassigned IGNITE-3133:
--
Assignee: Dmitriy Govorukhin
> Unable to get a reference to a cache inside TX on a node that is filtered out
> by a node filter
> --
>
> Key: IGNITE-3133
> URL: https://issues.apache.org/jira/browse/IGNITE-3133
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Denis Magda
>Assignee: Dmitriy Govorukhin
> Fix For: 1.8
>
> Attachments: CacheNodeFilter.java, CacheNodeFilterExample.java
>
>
> It's impossible to get a reference to a cache (Ignite.cache("name")) inside
> of a transaction on a node that is filtered out with
> CacheConfiguration.nodeFilter (doesn't hold cache data). The following
> exception happens.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:527)
> ... 32 more
> Caused by: java.io.InvalidObjectException: Failed to find cache for name:
> red_cache
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.readResolve(GridCacheContext.java:2051)
> ... 37 more
> Caused by: java.lang.IllegalStateException: Failed to find cache for name:
> red_cache
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.readResolve(GridCacheContext.java:2046)
> ... 37 more
> {noformat}
> To reproduce:
> - start ExampleNodeStartup
> - start attached CacheNodeFilterExample
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4008) Nodes logs can mixed if started on same jvm
Andrew Mashenkov created IGNITE-4008:
Summary: Nodes logs can mixed if started on same jvm
Key: IGNITE-4008
URL: https://issues.apache.org/jira/browse/IGNITE-4008
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 1.7, 1.6
Reporter: Andrew Mashenkov
i've started two nodes in one JVM with DEBUG log level and then shutdown second
node. Information from second node appears in first node log file.
Some messages can be identified by prefixes
{noformat}
sys-#82%grid-0% - ...
sys-#82%grid-1% -
{noformat}
but who knows which node generated rest messages
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3667) IGFS: Local secondary: Ensure that user context is propagated properly to underlying FS operation.
[
https://issues.apache.org/jira/browse/IGNITE-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535557#comment-15535557
]
Taras Ledkov commented on IGNITE-3667:
--
HDFS uses the user parameter only for java security with JAAS. The operation
with the native file systems are performed with process owner's permissions. So
hdfs doesn't set the default user for file operation.
> IGFS: Local secondary: Ensure that user context is propagated properly to
> underlying FS operation.
> --
>
> Key: IGNITE-3667
> URL: https://issues.apache.org/jira/browse/IGNITE-3667
> Project: Ignite
> Issue Type: Task
> Components: IGFS
>Affects Versions: 1.6
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 1.8
>
>
> If operation was initiated by user X, then it should be processed on behalf
> of this user, not on behalf of Ignite process owner.
> See {{IgniteHadoopIgfsSecondaryFileSystem.fileSystemForUser()}} for reference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4009) OptimizedObjectOutputStream checks for error message of removed IgniteEmailProcessor
Roman Shtykh created IGNITE-4009:
Summary: OptimizedObjectOutputStream checks for error message of
removed IgniteEmailProcessor
Key: IGNITE-4009
URL: https://issues.apache.org/jira/browse/IGNITE-4009
Project: Ignite
Issue Type: Bug
Reporter: Roman Shtykh
Assignee: Roman Shtykh
Priority: Minor
I wonder if we need a catch part with {{if
(CONVERTED_ERR.contains(t.getMessage()))}} of {{writeObjectOverride(...)}} at
all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (IGNITE-3972) Continuous query events could be lost on backup node when primary leaves.
[
https://issues.apache.org/jira/browse/IGNITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Mashenkov updated IGNITE-3972:
-
Attachment: stable-node.log
flapping-node.log
Logs for transactional partitioned full-sync cache with 1 backup attached.
- Stable node has data putting process run and continuous query for validate
putted data run.
- Flapping node just restarts.
Event with key=27 is lost after flapping node leaved topology.
> Continuous query events could be lost on backup node when primary leaves.
> -
>
> Key: IGNITE-3972
> URL: https://issues.apache.org/jira/browse/IGNITE-3972
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 1.7
>Reporter: Vladimir Ozerov
>Assignee: Andrew Mashenkov
> Fix For: 1.8
>
> Attachments: flapping-node.log, stable-node.log
>
>
> Consider the following scenario:
> 1) One node in topology;
> 2) PARTITIONED cache with 1 backup;
> 3) Continuous query is set on the cache;
> If another node joins the cluster, it will handle some updates. If it leaves
> the topology, the first node must flush it's own events from backup queue
> thus ensuring that no events are lost.
> But this doesn't happen because
> {{GridContinuousProcessor.addBackupNotification}} method perform lookup only
> on remote infos map, while handler is local and hence located in local infos
> map.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4010) Web Console: Implement responsive layout
Andrey Novikov created IGNITE-4010: -- Summary: Web Console: Implement responsive layout Key: IGNITE-4010 URL: https://issues.apache.org/jira/browse/IGNITE-4010 Project: Ignite Issue Type: Task Components: wizards Affects Versions: 1.7 Reporter: Andrey Novikov Priority: Minor Fix For: 1.8 Add responsive width for body. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (IGNITE-4010) Web Console: Implement responsive layout
[ https://issues.apache.org/jira/browse/IGNITE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Afanasiev reassigned IGNITE-4010: --- Assignee: Maxim Afanasiev > Web Console: Implement responsive layout > > > Key: IGNITE-4010 > URL: https://issues.apache.org/jira/browse/IGNITE-4010 > Project: Ignite > Issue Type: Task > Components: wizards >Affects Versions: 1.7 >Reporter: Andrey Novikov >Assignee: Maxim Afanasiev >Priority: Minor > Fix For: 1.8 > > > Add responsive width for body. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3767) Web Console: create reusable dialog for node selection
[ https://issues.apache.org/jira/browse/IGNITE-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535706#comment-15535706 ] Maxim Afanasiev commented on IGNITE-3767: - Fixed UI. Please review. > Web Console: create reusable dialog for node selection > -- > > Key: IGNITE-3767 > URL: https://issues.apache.org/jira/browse/IGNITE-3767 > Project: Ignite > Issue Type: Task > Components: wizards >Affects Versions: 1.7 >Reporter: Alexey Kuznetsov >Assignee: Maxim Afanasiev > Fix For: 1.8 > > Attachments: mockup.png > > > In several places (for example on SQL screen) we need to select some node > where some task will be executed. > We need a reusable dialog that will display all nodes and will allow to > select only one and return that node to calling code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (IGNITE-3767) Web Console: create reusable dialog for node selection
[ https://issues.apache.org/jira/browse/IGNITE-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Afanasiev updated IGNITE-3767: Assignee: Andrey Novikov (was: Maxim Afanasiev) > Web Console: create reusable dialog for node selection > -- > > Key: IGNITE-3767 > URL: https://issues.apache.org/jira/browse/IGNITE-3767 > Project: Ignite > Issue Type: Task > Components: wizards >Affects Versions: 1.7 >Reporter: Alexey Kuznetsov >Assignee: Andrey Novikov > Fix For: 1.8 > > Attachments: mockup.png > > > In several places (for example on SQL screen) we need to select some node > where some task will be executed. > We need a reusable dialog that will display all nodes and will allow to > select only one and return that node to calling code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-481) Add tests for Metrics to the file system tests infrastructure
[ https://issues.apache.org/jira/browse/IGNITE-481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533904#comment-15533904 ] Ivan Veselovsky edited comment on IGNITE-481 at 9/30/16 12:24 PM: -- 1) merged 1.6.9 into this branch ; 2) adapted and refactored block tests to be more generic to be able to handle PROXY mode also. 3) problem: in proxy mode 1 was added upon multiple blocks write instead of real number of blocks -- fixed. 4) problem: due to prefetch feature number of read blocks was wrong in some cases. Currently fixed by setting prefetch = 0 in the test. Better should be fixed expectation accordingly. 5) TODO: In org.apache.ignite.internal.processors.hadoop.impl.igfs.Hadoop1DualAbstractTest test block writes to primary and secondary fs are counted both in total counter, while in other IGFS-IGFS tests are not . this currently causes the test to fail. 6) org.apache.ignite.internal.processors.igfs.IgfsOutputStreamImpl#close org.apache.ignite.internal.processors.igfs.IgfsOutputStreamProxyImpl#close in error execution path were throwing exception *before* updateMetricsOnClose() invocation, so the number of locks get incorrect in concurrent tests -- fixed. was (Author: iveselovskiy): 1) merged 1.6.9 into this branch ; 2) adapted and refactored block tests to be more generic to be able to handle PROXY mode also. 3) problem: in proxy mode 1 was added upon multiple blocks write instead of real number of blocks -- fixed. 4) problem: due to prefetch feature number of read blocks was wrong in some cases. Currently fixed by setting prefetch = 0 in the test. Better should be fixed expectation accordingly. 5) TODO: In org.apache.ignite.internal.processors.hadoop.impl.igfs.Hadoop1DualAbstractTest test block writes to primary and secondary fs are counted both in total counter, while in other IGFS-IGFS tests are not . this currently causes the test to fail. > Add tests for Metrics to the file system tests infrastructure > - > > Key: IGNITE-481 > URL: https://issues.apache.org/jira/browse/IGNITE-481 > Project: Ignite > Issue Type: Task > Components: hadoop >Affects Versions: sprint-2 >Reporter: Ivan Veselovsky >Assignee: Ivan Veselovsky >Priority: Minor > Fix For: 1.8 > > > Need to add tests for org.apache.ignite.igfs.IgfsMetrics to the filesystem > tests. > See org.apache.ignite.IgniteFileSystem#metrics , > org.apache.ignite.IgniteFileSystem#resetMetrics . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-3877) Clarify if IgfsFile -> FileStatus conversion should treat groupBlockSize as blockSize
[
https://issues.apache.org/jira/browse/IGNITE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15485008#comment-15485008
]
Ivan Veselovsky edited comment on IGNITE-3877 at 9/30/16 12:27 PM:
---
It appears that block size in IgfsFile and in IgfsFileInfo are different for
the same file (code sample below).
This happens because of
org.apache.ignite.internal.processors.igfs.IgfsImpl#create0 , where block size
is simply taken from cfg.getBlockSize(), but all the other considerations are
ignored. It looks like this is okay to have block size different for primary
and underlying file systems (due to the group size feature), but from primary
Fs viewpoint the block size should be consistent. So, I guess,
org.apache.ignite.internal.processors.igfs.IgfsImpl#create0() should be fixed:
block size there should be obtained in the same way as in
org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile)
, that is , via groupBlockSize().
{code}
igfs.create(file1, 256, true, null, 1, 256, null).close();
IgfsFile f1 = igfs.info(file1);
int blockSize = f1.blockSize();
IgfsEntryInfo info = ((IgfsImpl)igfs).meta.infoForPath(file1);
final int blockSize2 = info.blockSize();
assert blockSize == blockSize2; // this fails
{code}
was (Author: iveselovskiy):
It appears that block size in IgfsFile and in IgfsFileInfo are different for
the same file (code sample below).
This happens because of
org.apache.ignite.internal.processors.igfs.IgfsImpl#create0 , where block siae
is simply taken from cfg.getBlockSize() , but all the other considerations are
ignored. It looks like this is okay to have block size different for primary
and underlying file systems (due to the group size feature), but from promary
Fs viewpoint the block size should be consistent. So, I guess,
org.apache.ignite.internal.processors.igfs.IgfsImpl#create0() should be fixed:
block size there should be taken from in the same way as in
org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile)
, that is , via groupBlockSize().
{code}
igfs.create(file1, 256, true, null, 1, 256, null).close();
IgfsFile f1 = igfs.info(file1);
int blockSize = f1.blockSize();
IgfsEntryInfo info = ((IgfsImpl)igfs).meta.infoForPath(file1);
final int blockSize2 = info.blockSize();
assert blockSize == blockSize2; // this fails
{code}
> Clarify if IgfsFile -> FileStatus conversion should treat groupBlockSize as
> blockSize
> -
>
> Key: IGNITE-3877
> URL: https://issues.apache.org/jira/browse/IGNITE-3877
> Project: Ignite
> Issue Type: Bug
> Components: IGFS
>Affects Versions: 1.6
>Reporter: Ivan Veselovsky
>Assignee: Ivan Veselovsky
> Fix For: 1.8
>
>
> During Metrics tests repairing test
> org.apache.ignite.igfs.Hadoop1DualAbstractTest#testMetricsBlock revealed the
> following problem:
> org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile)
> method treats groupBlockSize as blockSize for Hadoop FileStatus.
> groupBlockSize can be several times larger than blockSize, so blockSize in
> status gets different to that in original IgfsFile .
> changing file.groupBlockSize() to file.blockSize() fixes problem in metrics
> tests, but creates problems in Hadoop tests that are bound to splits
> calculation, since split calculation related to blockSizes.
> Ned to
> 1) clarify if the treatment of groupBlcokSize was intentional.
> 2) fix either metrics tests or Hadoop tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (IGNITE-3877) Clarify if IgfsFile -> FileStatus conversion should treat groupBlockSize as blockSize
[ https://issues.apache.org/jira/browse/IGNITE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Veselovsky updated IGNITE-3877: Description: During Metrics tests repairing test org.apache.ignite.igfs.Hadoop1DualAbstractTest#testMetricsBlock revealed the following problem: org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile) method treats groupBlockSize as blockSize for Hadoop FileStatus. groupBlockSize can be several times larger than blockSize, so blockSize in status gets different to that in original IgfsFile . changing file.groupBlockSize() to file.blockSize() fixes problem in metrics tests, but creates problems in Hadoop tests that are bound to splits calculation, since split calculation related to blockSizes. Need to 1) clarify if the treatment of groupBlcokSize was intentional. 2) fix either metrics tests or Hadoop tests. was: During Metrics tests repairing test org.apache.ignite.igfs.Hadoop1DualAbstractTest#testMetricsBlock revealed the following problem: org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile) method treats groupBlockSize as blockSize for Hadoop FileStatus. groupBlockSize can be several times larger than blockSize, so blockSize in status gets different to that in original IgfsFile . changing file.groupBlockSize() to file.blockSize() fixes problem in metrics tests, but creates problems in Hadoop tests that are bound to splits calculation, since split calculation related to blockSizes. Ned to 1) clarify if the treatment of groupBlcokSize was intentional. 2) fix either metrics tests or Hadoop tests. > Clarify if IgfsFile -> FileStatus conversion should treat groupBlockSize as > blockSize > - > > Key: IGNITE-3877 > URL: https://issues.apache.org/jira/browse/IGNITE-3877 > Project: Ignite > Issue Type: Bug > Components: IGFS >Affects Versions: 1.6 >Reporter: Ivan Veselovsky >Assignee: Ivan Veselovsky > Fix For: 1.8 > > > During Metrics tests repairing test > org.apache.ignite.igfs.Hadoop1DualAbstractTest#testMetricsBlock revealed the > following problem: > org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem#convert(org.apache.ignite.igfs.IgfsFile) > method treats groupBlockSize as blockSize for Hadoop FileStatus. > groupBlockSize can be several times larger than blockSize, so blockSize in > status gets different to that in original IgfsFile . > changing file.groupBlockSize() to file.blockSize() fixes problem in metrics > tests, but creates problems in Hadoop tests that are bound to splits > calculation, since split calculation related to blockSizes. > Need to > 1) clarify if the treatment of groupBlcokSize was intentional. > 2) fix either metrics tests or Hadoop tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (IGNITE-3667) IGFS: Local secondary: Ensure that user context is propagated properly to underlying FS operation.
[
https://issues.apache.org/jira/browse/IGNITE-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taras Ledkov updated IGNITE-3667:
-
Comment: was deleted
(was: HDFS uses the user parameter only for java security with JAAS. The
operation with the native file systems are performed with process owner's
permissions. So hdfs doesn't set the default user for file operation.)
> IGFS: Local secondary: Ensure that user context is propagated properly to
> underlying FS operation.
> --
>
> Key: IGNITE-3667
> URL: https://issues.apache.org/jira/browse/IGNITE-3667
> Project: Ignite
> Issue Type: Task
> Components: IGFS
>Affects Versions: 1.6
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Fix For: 1.8
>
>
> If operation was initiated by user X, then it should be processed on behalf
> of this user, not on behalf of Ignite process owner.
> See {{IgniteHadoopIgfsSecondaryFileSystem.fileSystemForUser()}} for reference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4011) Automatically compute hash codes for newly built binary objects
Alexander Paschenko created IGNITE-4011: --- Summary: Automatically compute hash codes for newly built binary objects Key: IGNITE-4011 URL: https://issues.apache.org/jira/browse/IGNITE-4011 Project: Ignite Issue Type: Task Components: binary, cache Reporter: Alexander Paschenko Assignee: Alexander Paschenko Fix For: 1.8 For binary keys built automatically inside SQL engine during INSERT or MERGE, we need to compute hash codes automatically because in this case the user does not interact with any builders and can't set hash code explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (IGNITE-3840) Continue investigation: High memory utilization when OFFHEAP_TIERED mode and expiry policy are enabled.
[ https://issues.apache.org/jira/browse/IGNITE-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Mashenkov reassigned IGNITE-3840: Assignee: Andrew Mashenkov (was: Semen Boikov) > Continue investigation: High memory utilization when OFFHEAP_TIERED mode and > expiry policy are enabled. > --- > > Key: IGNITE-3840 > URL: https://issues.apache.org/jira/browse/IGNITE-3840 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 1.5.0.final >Reporter: Andrew Mashenkov >Assignee: Andrew Mashenkov >Priority: Critical > Labels: community, important > Fix For: 1.8 > > Attachments: IgniteExpiryIssue.java, config.xml > > > Continue investigation of high memory utilization. > The problem is originally reported by Neil Wightman: > http://apache-ignite-users.70518.x6.nabble.com/Off-Heap-cache-using-lots-of-heap-memory-td3414.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (IGNITE-3875) Create separate thread pool for data streamer.
[ https://issues.apache.org/jira/browse/IGNITE-3875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Mashenkov updated IGNITE-3875: - Assignee: Vladimir Ozerov (was: Andrew Mashenkov) > Create separate thread pool for data streamer. > -- > > Key: IGNITE-3875 > URL: https://issues.apache.org/jira/browse/IGNITE-3875 > Project: Ignite > Issue Type: Task > Components: streaming >Affects Versions: 1.7 >Reporter: Vladimir Ozerov >Assignee: Vladimir Ozerov > Fix For: 1.8 > > > Currently data streamer requests are submitted to PUBLIC pool. Because of > this it is not safe to run streamer from compute tasks which is very > inconvenient. > We should create separate thread pool for streamer and route streamer jobs to > it. Compatibility must be preserved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536403#comment-15536403 ] Igor Rudyak edited comment on IGNITE-3609 at 9/30/16 4:29 PM: -- Alexey, Mutations is rather internal feature - to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation. Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. was (Author: irudyak): Alexey, Mutations is rather internal feature to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation. Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536403#comment-15536403 ] Igor Rudyak commented on IGNITE-3609: - Alexey, Mutations is rather internal feature to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation. Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536403#comment-15536403 ] Igor Rudyak edited comment on IGNITE-3609 at 9/30/16 4:31 PM: -- Alexey, Mutations is rather internal feature - to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation (if all these changes were made inside Ignite transaction). Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. was (Author: irudyak): Alexey, Mutations is rather internal feature - to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation. Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536403#comment-15536403 ] Igor Rudyak edited comment on IGNITE-3609 at 9/30/16 4:32 PM: -- Alexey, Mutations is rather internal feature - to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation (if you are using *TRANSACTIONAL* caches). Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. was (Author: irudyak): Alexey, Mutations is rather internal feature - to accumulate all the changes made inside transaction. Users never interact with mutations directly - they just call *put/putAll/remove/removeAll* methods of Ignite cache and all required mutation objects will be created inside CassandraCacheStore implementation (if all these changes were made inside Ignite transaction). Thus, I don't think that it makes sense to document mutations on readme.io cause it's rather internal feature. However, it makes sense to add documentation providing more details about how changes made inside Ignite transactions will be persisted into Cassandra. I am going to add this documentation a bit later. > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (IGNITE-3609) Utilize Cassandra logged batches for transactions
[ https://issues.apache.org/jira/browse/IGNITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Rudyak closed IGNITE-3609. --- > Utilize Cassandra logged batches for transactions > - > > Key: IGNITE-3609 > URL: https://issues.apache.org/jira/browse/IGNITE-3609 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Igor Rudyak >Assignee: Igor Rudyak > Fix For: 1.8 > > > Utilize Cassandra logged batches to atomically persist transaction changes in > persistent store -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (IGNITE-4012) Compile issues with v1.7.0/1.8.0 - missing dependency & H2 issue
[
https://issues.apache.org/jira/browse/IGNITE-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve White updated IGNITE-4012:
Summary: Compile issues with v1.7.0/1.8.0 - missing dependency & H2 issue
(was: Compile issues with v1.7.0/1.8.0 - missing dependency & H2)
> Compile issues with v1.7.0/1.8.0 - missing dependency & H2 issue
>
>
> Key: IGNITE-4012
> URL: https://issues.apache.org/jira/browse/IGNITE-4012
> Project: Ignite
> Issue Type: Bug
> Components: build
>Affects Versions: 1.7, 1.8
>Reporter: Steve White
>
> Hi, I'd just like to mention a couple of build issues - not sure if you're
> aware.
> If I build Apache Ignite tags/1.7.0 (1.7.0-SNAPSHOT) & master
> (1.8.0-SNAPSHOT) I get:
> {code}
> Missing dependencies:
> [ERROR] Failed to execute goal on project ignite-core: Could not resolve
> dependencies for project org.apache.ignite:ignite-core:jar:1.7.0-SNAPSHOT:
> The following artifacts could not be resolved:
> org.apache.ignite.binary:test1:jar:1.1,
> org.apache.ignite.binary:test2:jar:1.1:
> {code}
> If I comment the ignite-core:test1,test2 dependencies out I get a bunch of
> compile time errors in the indexing module referencing h2, just to mention a
> couple:
> {code}
> GridMergeIndex.java:[298,16] method getCostRangeIndex in class
> org.h2.index.BaseIndex cannot be applied to given types;
> [ERROR] required:
> int[],long,org.h2.table.TableFilter[],int,org.h2.result.SortOrder,boolean,java.util.HashSet
> [ERROR] found:
> int[],long,org.h2.table.TableFilter[],int,org.h2.result.SortOrder,boolean
> [ERROR] reason: actual and formal argument lists differ in length
> GridH2TreeIndex.java:[49,8]
> org.apache.ignite.internal.processors.query.h2.opt.GridH2TreeIndex is not
> abstract and does
> not override abstract method
> getCost(org.h2.engine.Session,int[],org.h2.table.TableFilter[],int,org.h2.result.SortOrder,java.util.HashSet)
> in org.h2.index.Index
> {code}
> Restricting building a subset of modules works, e.g:
> {code}
> mvn clean install -DskipTests -Dmaven.javadoc.skip=true -pl
> modules/apache-license-gen,modules/tools,modules/core,modules/osgi,modules/spring,modules/slf4j
> {code}
> But I really need indexing. My next step is to be drop the pom to an earlier
> version of H2 to try get it built. My reasons for building from source are to
> include a number of additional OSGi package imports (TBC).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-4012) Compile issues with v1.7.0/1.8.0 - missing dependency & H2
Steve White created IGNITE-4012:
---
Summary: Compile issues with v1.7.0/1.8.0 - missing dependency & H2
Key: IGNITE-4012
URL: https://issues.apache.org/jira/browse/IGNITE-4012
Project: Ignite
Issue Type: Bug
Components: build
Affects Versions: 1.7, 1.8
Reporter: Steve White
Hi, I'd just like to mention a couple of build issues - not sure if you're
aware.
If I build Apache Ignite tags/1.7.0 (1.7.0-SNAPSHOT) & master (1.8.0-SNAPSHOT)
I get:
{code}
Missing dependencies:
[ERROR] Failed to execute goal on project ignite-core: Could not resolve
dependencies for project org.apache.ignite:ignite-core:jar:1.7.0-SNAPSHOT: The
following artifacts could not be resolved:
org.apache.ignite.binary:test1:jar:1.1, org.apache.ignite.binary:test2:jar:1.1:
{code}
If I comment the ignite-core:test1,test2 dependencies out I get a bunch of
compile time errors in the indexing module referencing h2, just to mention a
couple:
{code}
GridMergeIndex.java:[298,16] method getCostRangeIndex in class
org.h2.index.BaseIndex cannot be applied to given types;
[ERROR] required:
int[],long,org.h2.table.TableFilter[],int,org.h2.result.SortOrder,boolean,java.util.HashSet
[ERROR] found:
int[],long,org.h2.table.TableFilter[],int,org.h2.result.SortOrder,boolean
[ERROR] reason: actual and formal argument lists differ in length
GridH2TreeIndex.java:[49,8]
org.apache.ignite.internal.processors.query.h2.opt.GridH2TreeIndex is not
abstract and does
not override abstract method
getCost(org.h2.engine.Session,int[],org.h2.table.TableFilter[],int,org.h2.result.SortOrder,java.util.HashSet)
in org.h2.index.Index
{code}
Restricting building a subset of modules works, e.g:
{code}
mvn clean install -DskipTests -Dmaven.javadoc.skip=true -pl
modules/apache-license-gen,modules/tools,modules/core,modules/osgi,modules/spring,modules/slf4j
{code}
But I really need indexing. My next step is to be drop the pom to an earlier
version of H2 to try get it built. My reasons for building from source are to
include a number of additional OSGi package imports (TBC).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3897) Test transactional behaviour of caches during rolling restart
[ https://issues.apache.org/jira/browse/IGNITE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536566#comment-15536566 ] Yakov Zhdanov commented on IGNITE-3897: --- Hi John! Thanks for contribution. I have just commented in the PR at github. > Test transactional behaviour of caches during rolling restart > - > > Key: IGNITE-3897 > URL: https://issues.apache.org/jira/browse/IGNITE-3897 > Project: Ignite > Issue Type: Test > Components: cache >Affects Versions: 1.7 >Reporter: John Levey >Assignee: John Levey > > Create a test that validates the following conditions against a running > cluster that is undergoing a rolling restart: > * Validate a continuous aggregate sum against the cluster during restarts > * Validate transaction handling against the cluster during restarts > See also https://issues.apache.org/jira/browse/IGNITE-3456 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (IGNITE-481) Add tests for Metrics to the file system tests infrastructure
[ https://issues.apache.org/jira/browse/IGNITE-481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533904#comment-15533904 ] Ivan Veselovsky edited comment on IGNITE-481 at 9/30/16 7:23 PM: - 1) merged 1.6.9 into this branch ; 2) adapted and refactored block tests to be more generic to be able to handle PROXY mode also. 3) problem: in proxy mode 1 was added upon multiple blocks write instead of real number of blocks -- fixed. 4) problem: due to prefetch feature number of read blocks was wrong in some cases. Currently fixed by setting prefetch = 0 in the test. Better should be fixed expectation accordingly. 5) org.apache.ignite.internal.processors.igfs.IgfsOutputStreamImpl#close org.apache.ignite.internal.processors.igfs.IgfsOutputStreamProxyImpl#close in error execution path were throwing exception *before* updateMetricsOnClose() invocation, so the number of locks get incorrect in concurrent tests -- fixed. was (Author: iveselovskiy): 1) merged 1.6.9 into this branch ; 2) adapted and refactored block tests to be more generic to be able to handle PROXY mode also. 3) problem: in proxy mode 1 was added upon multiple blocks write instead of real number of blocks -- fixed. 4) problem: due to prefetch feature number of read blocks was wrong in some cases. Currently fixed by setting prefetch = 0 in the test. Better should be fixed expectation accordingly. 5) TODO: In org.apache.ignite.internal.processors.hadoop.impl.igfs.Hadoop1DualAbstractTest test block writes to primary and secondary fs are counted both in total counter, while in other IGFS-IGFS tests are not . this currently causes the test to fail. 6) org.apache.ignite.internal.processors.igfs.IgfsOutputStreamImpl#close org.apache.ignite.internal.processors.igfs.IgfsOutputStreamProxyImpl#close in error execution path were throwing exception *before* updateMetricsOnClose() invocation, so the number of locks get incorrect in concurrent tests -- fixed. > Add tests for Metrics to the file system tests infrastructure > - > > Key: IGNITE-481 > URL: https://issues.apache.org/jira/browse/IGNITE-481 > Project: Ignite > Issue Type: Task > Components: hadoop >Affects Versions: sprint-2, 1.6 >Reporter: Ivan Veselovsky >Assignee: Vladimir Ozerov >Priority: Minor > Fix For: 1.8 > > > Need to add tests for org.apache.ignite.igfs.IgfsMetrics to the filesystem > tests. > See org.apache.ignite.IgniteFileSystem#metrics , > org.apache.ignite.IgniteFileSystem#resetMetrics . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
