[jira] [Created] (IGNITE-9819) SQL: document recomendation to use TIMESTAMP instead of DATE
Vladimir Ozerov created IGNITE-9819: --- Summary: SQL: document recomendation to use TIMESTAMP instead of DATE Key: IGNITE-9819 URL: https://issues.apache.org/jira/browse/IGNITE-9819 Project: Ignite Issue Type: Task Components: documentation Reporter: Vladimir Ozerov Assignee: Artem Budnikov Fix For: 2.7 We support both DATE (date only) and TIMESTAMP (date + time) data types. Unfortunately, DATE data type is serialized/deserialized very inefficiently, what leads to performance degradation. We need to warn users in documentation, that it is better to use TIMESTAMP than DATE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4901: IGNITE-9728: fix setup IGNITE_HOME for tests
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4901 ---
[GitHub] ignite pull request #4925: IGNITE-9814: JDK11: HadoopClassLoader#nativeLibra...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4925 ---
[MTCGA]: new failures in builds [2027969] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master CachePartitionedTest.TestReplaceBinary https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=4633785065299579521&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master CachePartitionedTest.TestWithExpiryPolicyZeroNegative https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-8600205843164804928&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master CachePartitionedTest.TestLock https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8335296346751082042&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master CachePartitionedTest.TestEvict https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-6867183422951066051&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master CachePartitionedTest.TestEvictAllKeys https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-6749476979732882312&branch=%3Cdefault%3E&tab=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 06:18:44 09-10-2018
Absence of unsigned integer read/write support in IBinaryRawReader/Writer
I’m using Ignite IBinarizable raw serialization in Ignite v2.6 with C# client. I notice there is no support for unsigned short, int and long integer types (both single values and arrays). I also noticed that Decimal, DateTime and Guid read/write methods only support nullable values. Currently I have code that writes a ushort value into an into to preserve its value range. Similarly I have non-nullable Guid values and need to do the nullable dance to on the read side to transform them back to non-nullable. Is there a particular reason these are not supported? Thanks, Raymond.
[GitHub] ignite pull request #4928: IGNITE-8870: Example of usage of encrypted cache.
GitHub user nizhikov opened a pull request: https://github.com/apache/ignite/pull/4928 IGNITE-8870: Example of usage of encrypted cache. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nizhikov/ignite IGNITE-8870 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4928.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4928 commit 11e6259232803e2c21aeb2947ce3bb76d29659e8 Author: Nikolay Izhikov Date: 2018-10-08T18:13:26Z IGNITE-8870: Example of usage of encrypted cache. ---
[GitHub] ignite pull request #4927: IGNITE-9818 Fix javadoc for annotation AffinityKe...
GitHub user Max-Pudov opened a pull request: https://github.com/apache/ignite/pull/4927 IGNITE-9818 Fix javadoc for annotation AffinityKeyMapped You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9818 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4927.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4927 commit ca94001b851afc7656b9f623c470f6cc2902715a Author: Max-Pudov Date: 2018-10-08T17:22:57Z IGNITE-9818 Fix javadoc for annotation AffinityKeyMapped ---
[jira] [Created] (IGNITE-9818) Fix javadoc for annotation AffinityKeyMapped which cannot be applied to a method anymore
Maxim Pudov created IGNITE-9818: --- Summary: Fix javadoc for annotation AffinityKeyMapped which cannot be applied to a method anymore Key: IGNITE-9818 URL: https://issues.apache.org/jira/browse/IGNITE-9818 Project: Ignite Issue Type: Bug Components: documentation Affects Versions: 2.0 Reporter: Maxim Pudov Assignee: Maxim Pudov Fix For: 2.8 In the past @AffinityKeyMapped could have been used on methods, but since ignite-4950 it isn't. Javadoc need to be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9817) Update documentation and examples for Spark SQL Table Specification
Nikolay Izhikov created IGNITE-9817: --- Summary: Update documentation and examples for Spark SQL Table Specification Key: IGNITE-9817 URL: https://issues.apache.org/jira/browse/IGNITE-9817 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov We should update documentation and examples according to the results of IGNITE-9228. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4551: IGNITE-9228 Spark SQL Table Schema Specification
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4551 ---
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223397571 ## File path: ignite-tc-helper-web/src/main/webapp/js/testfails-2.1.js ## @@ -328,7 +357,7 @@ function triggerBuild(serverId, suiteId, branchName, top, observe, ticketId) { url: 'rest/build/trigger', data: { "serverId": serverId, -"suiteId": suiteId, +"suiteIdList": suiteId, Review comment: suiteId -> suiteIds This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: Mark dirty for DataPage: small changes in huge objects.
I agree that such option is hard to explain and will complicate data storage tuning (which is already not simple). The problem is that we don't divide pages to overflow/non-overflow so far. We need to see benchmark results first - there's a chance that negative effects will be insignificant and option won't be needed at all. Otherwise, we may come up with a heuristic that will minimize negative effect, e.g. apply bytewise comparison only for data pages with only one payload item. Best Regards, Ivan Rakov On 08.10.2018 18:25, Vladimir Ozerov wrote: Can we use this mode for overflow pages and do not use for normal entries which fir a single page? In general users try to avoid fine-grained tuning options, as they are very complex to understand. We should try to avoid any new configuration options. On Mon, Oct 8, 2018 at 5:51 PM Ivan Rakov wrote: Huge +1. Page dirty flag is set in PageMemoryImpl#writeUnlockPage body. Caller passes "markDirty=true" boolean flag if he assumes that page content may have changed (dirty flag will be set even if page content remained intact). Instead of this, we can dump page content to thread-local buffer after successful write lock and compare it bytewise with new content on write unlock. I believe, this logic should be introduced as a separate data storage mode as it have both positive and negative effects. Positive: Small updates of large entries will produce much less dirty pages. It can dramatically boost performance of updates - especially when SQL update of single field is performed over large objects. Negative: CPU consumption and latency will be increased. We'll need some time to copy and compare page content. Anyway, lack of disk IOPS hits us much more often than lack of CPU - benchmarks will show whether such impact will be perceptible. Let's file a ticket for this task unless there are any objections. Best Regards, Ivan Rakov On 08.10.2018 16:18, Dmitriy Pavlov wrote: Hi Igniters, I'd like to share a case which was implemented in the previous version of TC Bot. It is a kind of REST responses cache : Response { Long tsRefreshed; // timestamp of the last call to real service List builds; // a huge list of builds, most times it is not changed. } And it seems timestamp (ts) offset in all entries pages is constant and it requires 8 bytes. Data in builds storage will require a number of pages in the durable memory, probably >10-20 pages. So if REST (real service) responds with the same builds content only TS is updated. After that, I did cache.put(restParms, reponse). So my question is, will such update, which affects only 1 field causes mark dirty for 1 page or for 20? I feel according to checkpoints amount that we mark all pages as dirty even if the content is not modified. If so, I would like to suggest a slight change to Ignite: for data pages mark as only that pages, which has a modification in its content. I understand that previous implementation in the Bot was quite naive (now it is changed), but still, what if we will check for modifications by mem-compare before marking? Mark dirty now seems to cause additional data to be flushed to disk on next checkpoint. I would appreciate if Native Persistence Experts can help me to find a place in the code, where such updates are performed? (Maybe I miss something). Sincerely, Dmitriy Pavlov
Blog post "Introduction to the Apache(R) Ignite™ community structure"
Hi Apache Ignite Enthusiasts, I'm happy to announce a new post about Apache Ignite https://www.gridgain.com/resources/blog/introduction-apacher-ignitetm-community-structure - "Introduction to the Apache® Ignite™ community structure." This post is a translated version of the Habr post. Thanks to Tom D. for translation of original post. There I explain the roles, rules, and benefits of participating in the Apache Ignite community. As always, comments and feedback are strongly appreciated. Sincerely, Dmitriy Pavlov
[jira] [Created] (IGNITE-9816) Add FailureHandler call on binary_meta error
Dmitriy Gladkikh created IGNITE-9816: Summary: Add FailureHandler call on binary_meta error Key: IGNITE-9816 URL: https://issues.apache.org/jira/browse/IGNITE-9816 Project: Ignite Issue Type: Improvement Reporter: Dmitriy Gladkikh This is a critical situation, corresponding exception should be propagated to handler to make necessary actions. Need to add a call to the FailureHandler if the specified error occurs. {code:java} 2018-09-07 14:40:53.912[ERROR][sys-stripe-14-#15%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.t.IgniteTxHandler] Failed completing transaction [commit=true, tx=GridDhtTxLocal[xid=31fa1856661--08ce-7bae--008f, xidVersion=GridCacheVersion [topVer=147749806, order=1539301289747, nodeOrder=143], concurrency=PESSIMISTIC, isolation=READ_COMMITTED, state=COMMITTING, invalidate=false, rollbackOnly=false, nodeId=0262783b-1e7d-42bc-a839-04a562f10e44, timeout=20, duration=171]] org.apache.ignite.IgniteException: Failed to create string representation of binary object. at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1018) at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:758) at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:706) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1505) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:940) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:649) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:851) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:795) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.localFinish(GridDhtTxLocal.java:583) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:464) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitDhtLocalAsync(GridDhtTxLocal.java:505) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:942) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:821) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:777) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:99) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:191) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:189) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:496) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.ignite.IgniteException: Failed to create string representation of binary object. at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1018)
Re: Mark dirty for DataPage: small changes in huge objects.
Can we use this mode for overflow pages and do not use for normal entries which fir a single page? In general users try to avoid fine-grained tuning options, as they are very complex to understand. We should try to avoid any new configuration options. On Mon, Oct 8, 2018 at 5:51 PM Ivan Rakov wrote: > Huge +1. > > Page dirty flag is set in PageMemoryImpl#writeUnlockPage body. Caller > passes "markDirty=true" boolean flag if he assumes that page content may > have changed (dirty flag will be set even if page content remained > intact). Instead of this, we can dump page content to thread-local > buffer after successful write lock and compare it bytewise with new > content on write unlock. > > I believe, this logic should be introduced as a separate data storage > mode as it have both positive and negative effects. > > Positive: > Small updates of large entries will produce much less dirty pages. It > can dramatically boost performance of updates - especially when SQL > update of single field is performed over large objects. > > Negative: > CPU consumption and latency will be increased. We'll need some time to > copy and compare page content. Anyway, lack of disk IOPS hits us much > more often than lack of CPU - benchmarks will show whether such impact > will be perceptible. > > Let's file a ticket for this task unless there are any objections. > > Best Regards, > Ivan Rakov > > On 08.10.2018 16:18, Dmitriy Pavlov wrote: > > Hi Igniters, > > > > I'd like to share a case which was implemented in the previous version of > > TC Bot. It is a kind of REST responses cache : > > Response { > >Long tsRefreshed; // timestamp of the last call to real service > >List builds; // a huge list of builds, most times it is not > > changed. > > } > > > > And it seems timestamp (ts) offset in all entries pages is constant and > it > > requires 8 bytes. Data in builds storage will require a number of pages > in > > the durable memory, probably >10-20 pages. > > > > So if REST (real service) responds with the same builds content only TS > is > > updated. After that, I did cache.put(restParms, reponse). > > > > So my question is, will such update, which affects only 1 field causes > mark > > dirty for 1 page or for 20? I feel according to checkpoints amount that > we > > mark all pages as dirty even if the content is not modified. If so, I > would > > like to suggest a slight change to Ignite: for data pages mark as only > that > > pages, which has a modification in its content. > > > > I understand that previous implementation in the Bot was quite naive (now > > it is changed), but still, what if we will check for modifications by > > mem-compare before marking? Mark dirty now seems to cause additional data > > to be flushed to disk on next checkpoint. > > > > I would appreciate if Native Persistence Experts can help me to find a > > place in the code, where such updates are performed? (Maybe I miss > > something). > > > > Sincerely, > > Dmitriy Pavlov > > > >
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223392275 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/tcbot/visa/TcBotTriggerAndSignOffService.java ## @@ -77,7 +78,7 @@ @NotNull public String triggerBuildAndObserve( @Nullable String srvId, @Nullable String branchForTc, -@Nullable String suiteId, +@Nullable String suiteIdList, Review comment: Rename this method to `triggerBuildsAndObserve`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223387780 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java ## @@ -0,0 +1,100 @@ +package org.apache.ignite.ci.observer; Review comment: Missed license. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223329723 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java ## @@ -0,0 +1,100 @@ +package org.apache.ignite.ci.observer; + +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.user.ICredentialsProv; + +public class BuildsInfo{ Review comment: Need space before opening brace. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223397571 ## File path: ignite-tc-helper-web/src/main/webapp/js/testfails-2.1.js ## @@ -328,7 +357,7 @@ function triggerBuild(serverId, suiteId, branchName, top, observe, ticketId) { url: 'rest/build/trigger', data: { "serverId": serverId, -"suiteId": suiteId, +"suiteIdList": suiteId, Review comment: suiteId -> suiteIds This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223332273 ## File path: ignite-tc-helper-web/src/main/webapp/js/testfails-2.1.js ## @@ -355,59 +384,82 @@ function triggerBuild(serverId, suiteId, branchName, top, observe, ticketId) { }); } -function triggerBuilds(serverId, suiteIdList, branchName, top) { -var res = "Trigger builds at server: " + serverId + "" + -"Branch:" + branchName + "Top: " + top + ""; +function triggerBuilds(serverId, suiteIdList, branchName, top, observe, ticketId) { +var possibleBlockersNotFound = suiteIdList.length === 0; +var res = ""; -var partsOfStr = suiteIdList.split(','); +if (possibleBlockersNotFound) { Review comment: Remove braces for one-line `if` statement. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223332768 ## File path: ignite-tc-helper-web/src/main/webapp/js/testfails-2.1.js ## @@ -355,59 +384,82 @@ function triggerBuild(serverId, suiteId, branchName, top, observe, ticketId) { }); } -function triggerBuilds(serverId, suiteIdList, branchName, top) { -var res = "Trigger builds at server: " + serverId + "" + -"Branch:" + branchName + "Top: " + top + ""; +function triggerBuilds(serverId, suiteIdList, branchName, top, observe, ticketId) { +var possibleBlockersNotFound = suiteIdList.length === 0; +var res = ""; -var partsOfStr = suiteIdList.split(','); +if (possibleBlockersNotFound) { +res = "Possible Blockers not found!"; +} +else { +res = "Trigger builds at server: " + serverId + "" + +"Branch:" + branchName + "Top: " + top + ""; + +var partsOfStr = suiteIdList.split(','); -for (var i = 0; i < partsOfStr.length; i++) { -var suite = partsOfStr[i]; -res += "Suite ID: " + suite + ""; +for (var i = 0; i < partsOfStr.length; i++) { +var suite = partsOfStr[i]; +res += "Suite ID: " + suite + ""; +} } + var triggerConfirm = $("#triggerConfirm"); triggerConfirm.html(res); -triggerConfirm.dialog({ -modal: true, -buttons: { -"Run": function() { -$(this).dialog("close"); - -var queueAtTop = isDefinedAndFilled(top) && top -$.ajax({ -url: 'rest/build/triggerBuilds', -data: { -"serverId": serverId, -"suiteIdList": suiteIdList, -"branchName": branchName, -"top": queueAtTop -}, -success: function(result) { -var dialog = $("#triggerDialog"); - -dialog.html("Trigger builds at server: " + serverId + "" + -" Suites " + suiteIdList + "Branch:" + branchName + "Top: " + top + -" Result: " + result.result); -dialog.dialog({ -modal: true, -buttons: { -"Ok": function() { -$(this).dialog("close"); +if (possibleBlockersNotFound){ Review comment: Add space before opening brace. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223387956 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java ## @@ -0,0 +1,100 @@ +package org.apache.ignite.ci.observer; + +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.user.ICredentialsProv; + +public class BuildsInfo{ +/** Finished. */ +public static final String FINISHED = "finished"; + +/** Server id. */ +public final String srvId; + +/** Build type id. */ +public final String buildTypeId; + +/** Branch name. */ +public final String branchName; + +/** Prov. */ Review comment: Credentials. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223329487 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java ## @@ -0,0 +1,100 @@ +package org.apache.ignite.ci.observer; + +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.user.ICredentialsProv; + +public class BuildsInfo{ +/** Finished. */ +public static final String FINISHED = "finished"; + +/** Server id. */ +public final String srvId; + +/** Build type id. */ +public final String buildTypeId; + +/** Branch name. */ +public final String branchName; + +/** Prov. */ +public final ICredentialsProv prov; + +/** JIRA ticket full name. */ +public final String ticket; + +/** Finished builds. */ +private final Map finishedBuilds = new HashMap<>(); + +/** + * @param srvId Server id. + * @param prov Prov. + * @param ticket Ticket. + * @param builds Builds. + */ +public BuildsInfo(String srvId, ICredentialsProv prov, String ticket, Build[] builds) { +this.srvId = srvId; +this.prov = prov; +this.ticket = ticket; +this.buildTypeId = builds.length > 1 ? "IgniteTests24Java8_RunAll" : builds[0].buildTypeId; +this.branchName = builds[0].branchName; + +for (Build build : builds) +finishedBuilds.put(build, false); +} + +/** + * @param teamcity Teamcity. + */ +public boolean isFinished(IAnalyticsEnabledTeamcity teamcity) { +for (Map.Entry entry : finishedBuilds.entrySet()){ +if (!entry.getValue()){ +Build build = teamcity.getBuild(entry.getKey().getId()); +entry.setValue(build.state.equals(FINISHED)); +} +} + +return !finishedBuilds.containsValue(false); +} + +/** + * Return builds count. + */ +public int buildsCount(){ +return finishedBuilds.size(); +} + +/** + * Return finished builds count. + */ +public int finishedBuildsCount(){ +return (int)finishedBuilds.values().stream().filter(v -> v).count(); +} + +/** {@inheritDoc} */ +@Override public boolean equals(Object o) { +if (this == o) +return true; + +if (!(o instanceof BuildsInfo)) +return false; + +BuildsInfo info = (BuildsInfo)o; + +return Objects.equals(srvId, info.srvId) && +Objects.equals(buildTypeId, info.buildTypeId) && +Objects.equals(branchName, info.branchName) && +Objects.equals(prov, info.prov) && +Objects.equals(ticket, info.ticket) && +Objects.equals(finishedBuilds, info.finishedBuilds); +} + +/** {@inheritDoc} */ +@Override public int hashCode() { + Review comment: Remove empty line. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223395846 ## File path: ignite-tc-helper-web/src/main/webapp/js/testfails-2.1.js ## @@ -214,14 +236,21 @@ function addBlockersData(server, settings) { return ""; var blockers = ""; +var blockersList = ""; Review comment: Variable is never used. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223388354 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/web/rest/TriggerBuild.java ## @@ -71,14 +72,16 @@ public SimpleResult triggerBuild( if (!prov.hasAccess(srvId)) throw ServiceUnauthorizedException.noCreds(srvId); +if (isNullOrEmpty(suiteIdList)) +return new SimpleResult("Error: nothing to run"); Review comment: Missed dot. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button
SomeFire commented on a change in pull request #27: IGNITE-9770 Add 'Re-run possible blockers' button URL: https://github.com/apache/ignite-teamcity-bot/pull/27#discussion_r223329630 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/observer/BuildsInfo.java ## @@ -0,0 +1,100 @@ +package org.apache.ignite.ci.observer; + +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.user.ICredentialsProv; + +public class BuildsInfo{ +/** Finished. */ +public static final String FINISHED = "finished"; + +/** Server id. */ +public final String srvId; + +/** Build type id. */ +public final String buildTypeId; + +/** Branch name. */ +public final String branchName; + +/** Prov. */ +public final ICredentialsProv prov; + +/** JIRA ticket full name. */ +public final String ticket; + +/** Finished builds. */ +private final Map finishedBuilds = new HashMap<>(); + +/** + * @param srvId Server id. + * @param prov Prov. + * @param ticket Ticket. + * @param builds Builds. + */ +public BuildsInfo(String srvId, ICredentialsProv prov, String ticket, Build[] builds) { +this.srvId = srvId; +this.prov = prov; +this.ticket = ticket; +this.buildTypeId = builds.length > 1 ? "IgniteTests24Java8_RunAll" : builds[0].buildTypeId; +this.branchName = builds[0].branchName; + +for (Build build : builds) +finishedBuilds.put(build, false); +} + +/** + * @param teamcity Teamcity. + */ +public boolean isFinished(IAnalyticsEnabledTeamcity teamcity) { +for (Map.Entry entry : finishedBuilds.entrySet()){ +if (!entry.getValue()){ Review comment: Need space before opening brace. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: Mark dirty for DataPage: small changes in huge objects.
Huge +1. Page dirty flag is set in PageMemoryImpl#writeUnlockPage body. Caller passes "markDirty=true" boolean flag if he assumes that page content may have changed (dirty flag will be set even if page content remained intact). Instead of this, we can dump page content to thread-local buffer after successful write lock and compare it bytewise with new content on write unlock. I believe, this logic should be introduced as a separate data storage mode as it have both positive and negative effects. Positive: Small updates of large entries will produce much less dirty pages. It can dramatically boost performance of updates - especially when SQL update of single field is performed over large objects. Negative: CPU consumption and latency will be increased. We'll need some time to copy and compare page content. Anyway, lack of disk IOPS hits us much more often than lack of CPU - benchmarks will show whether such impact will be perceptible. Let's file a ticket for this task unless there are any objections. Best Regards, Ivan Rakov On 08.10.2018 16:18, Dmitriy Pavlov wrote: Hi Igniters, I'd like to share a case which was implemented in the previous version of TC Bot. It is a kind of REST responses cache : Response { Long tsRefreshed; // timestamp of the last call to real service List builds; // a huge list of builds, most times it is not changed. } And it seems timestamp (ts) offset in all entries pages is constant and it requires 8 bytes. Data in builds storage will require a number of pages in the durable memory, probably >10-20 pages. So if REST (real service) responds with the same builds content only TS is updated. After that, I did cache.put(restParms, reponse). So my question is, will such update, which affects only 1 field causes mark dirty for 1 page or for 20? I feel according to checkpoints amount that we mark all pages as dirty even if the content is not modified. If so, I would like to suggest a slight change to Ignite: for data pages mark as only that pages, which has a modification in its content. I understand that previous implementation in the Bot was quite naive (now it is changed), but still, what if we will check for modifications by mem-compare before marking? Mark dirty now seems to cause additional data to be flushed to disk on next checkpoint. I would appreciate if Native Persistence Experts can help me to find a place in the code, where such updates are performed? (Maybe I miss something). Sincerely, Dmitriy Pavlov
Re: Apache Ignite Teamcity Bot - Master Trends
Hi, Dmitrii! Thanks for the feedback, I will definitely add these parameters to the table. By the way, it would be interesting to know how many people on average use the TC Bot every day. Does this figure increase? With the updated UI TC Bot has become more user-friendly. And I’m sure it will help it become more popular with the contributors. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [MTCGA]: new failures in builds [1999976] needs to be handled
Hi Igniters, I see this test is still failing. Who can address this? Vladimir, could you please step in? Sincerely, Dmitriy Pavlov чт, 4 окт. 2018 г. в 2:01, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're grateful that you were > a volunteer to make the contribution to this project, but things change and > you may no longer be able to finalize your contribution. > Could you respond to this email and indicate if you wish to continue and > fix test failures or step down and some committer may revert you commit. > > *New test failure in master > CacheMvccReplicatedCoordinatorFailoverTest.testAccountsTxScan_Server_Backups0_CoordinatorFails > > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8681833606515480088&branch=%3Cdefault%3E&tab=testDetails > Changes may lead to failure were done by > - ppozerov > http://ci.ignite.apache.org/viewModification.html?modId=833688&personal=false > - vozerov > http://ci.ignite.apache.org/viewModification.html?modId=833660&personal=false > - andrey.mashenkov > http://ci.ignite.apache.org/viewModification.html?modId=833652&personal=false > - oignatenko > http://ci.ignite.apache.org/viewModification.html?modId=833632&personal=false > - tledkov > http://ci.ignite.apache.org/viewModification.html?modId=833628&personal=false > - mr.weider > http://ci.ignite.apache.org/viewModification.html?modId=833601&personal=false > > - Here's a reminder of what contributors were agreed to do > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > - Should you have any questions please contact > dev@ignite.apache.org > > Best Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 02:01:34 04-10-2018 >
[GitHub] ignite pull request #4886: IGNITE-8803 Corrected output for quiet mode
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4886 ---
[GitHub] ignite pull request #4926: IGNITE-9694 Tests that read operations are not bl...
GitHub user ibessonov opened a pull request: https://github.com/apache/ignite/pull/4926 IGNITE-9694 Tests that read operations are not blocked by exchange You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9694 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4926.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4926 commit 5708547a5a1803e34d60312aaf09e53b9e2bc872 Author: ibessonov Date: 2018-10-02T11:50:27Z IGNITE-9694 Test prototype commit ac1a299eea5c0dac45f00dd4914874b8eedde74e Author: ibessonov Date: 2018-10-04T16:08:58Z Merge remote-tracking branch 'professional/ignite-9694' into ignite-9694 commit 9497d524a1292f89b12ac0ec2c1dfdc48abc9869 Author: ibessonov Date: 2018-10-04T16:05:07Z IGNITE-9694 Further test improvements, not finished yet. commit bf40518499e7b1b9721f3706108eb2b344873336 Author: ibessonov Date: 2018-10-05T09:57:58Z IGNITE-9694 Snapshots test fixed. Other improvements, overall it's more stable now. commit 0aacf55466bc8fa15925950a91c2c016b74541d4 Author: ibessonov Date: 2018-10-05T16:14:28Z IGNITE-9694 Previously commented tests are stable now. commit 97c21b425d704893b162e29df31040fc39e6dc06 Author: ibessonov Date: 2018-10-08T10:27:48Z IGNITE-9694 "stop client" tests commented, everything else seems fine now. commit 4037d4f092ba5f8e5096c25908292333929681bc Author: ibessonov Date: 2018-10-08T10:35:56Z IGNITE-9694 "stop client" test commented in the other way. commit 97a371663477d2e19d4c042f353c50b2a440171a Author: ibessonov Date: 2018-10-08T11:49:27Z IGNITE-9694 Table name generator replaced with the better one. commit ed68ba1fd2e279a55402a155a7422d7a628332c1 Author: ibessonov Date: 2018-10-08T11:53:54Z IGNITE-9694 Minor error in javadoc. commit dc422b09585956baeabb8b8dc7fdb93322551f14 Author: ibessonov Date: 2018-10-08T12:59:56Z IGNITE-9694 Default warmup timeout increased. ---
[GitHub] ignite pull request #4925: IGNITE-9814: JDK11: HadoopClassLoader#nativeLibra...
GitHub user tledkov-gridgain opened a pull request: https://github.com/apache/ignite/pull/4925 IGNITE-9814: JDK11: HadoopClassLoader#nativeLibraries fails because has invalid assumption for ClassLoader type You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9814 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4925.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4925 commit d1ae537821df250905da63e2195c18d730059fc7 Author: tledkov-gridgain Date: 2018-10-08T13:43:28Z IGNITE-9814: : HadoopClassLoader#nativeLibraries fails because has invalid assumption for ClassLoader type ---
Mark dirty for DataPage: small changes in huge objects.
Hi Igniters, I'd like to share a case which was implemented in the previous version of TC Bot. It is a kind of REST responses cache : Response { Long tsRefreshed; // timestamp of the last call to real service List builds; // a huge list of builds, most times it is not changed. } And it seems timestamp (ts) offset in all entries pages is constant and it requires 8 bytes. Data in builds storage will require a number of pages in the durable memory, probably >10-20 pages. So if REST (real service) responds with the same builds content only TS is updated. After that, I did cache.put(restParms, reponse). So my question is, will such update, which affects only 1 field causes mark dirty for 1 page or for 20? I feel according to checkpoints amount that we mark all pages as dirty even if the content is not modified. If so, I would like to suggest a slight change to Ignite: for data pages mark as only that pages, which has a modification in its content. I understand that previous implementation in the Bot was quite naive (now it is changed), but still, what if we will check for modifications by mem-compare before marking? Mark dirty now seems to cause additional data to be flushed to disk on next checkpoint. I would appreciate if Native Persistence Experts can help me to find a place in the code, where such updates are performed? (Maybe I miss something). Sincerely, Dmitriy Pavlov
Re: [TC Bot] Proposal of improvement
Hi Nikolay, Igniters, Please check the newest version of the Bot. It contains huge UI refactoring and simplified navigation to open PR and its test results. Just couple seconds is needed to find a PR now. Thanks to Dmitrii Ryabov, who developed initial GitHub integration. It is more or less reused, and PRs are cached in the Ignite instance. As always, I appreciate the feedback. Sincerely, Dmitriy Pavlov вт, 25 сент. 2018 г. в 13:00, Dmitriy Pavlov : > JIRA ticket is some elementary change that needs to be reviewed. We don't > review the patch, we review ticket (with motivation, implementation > details, history of discussion), so reviewer will look at a ticket first. > > PR does not have a mark, that it is ready to be merged. Some PRs are > created just for research, but Patch Available ticket is something that is > ready to be in the product. > > So if we concentrate on a ticket, from the point of view of a new > contributor, > - he or she creates a branch, PR and sets ticket to PA, > - and the bot will do all necessary mechanic work. > > No issues with asking newcomers to run (proper) tests. A newbie needs only > to follow the first steps in How To Contribute. A reviewer will see a > ticket with the bot Visa after 2-3 hours after setting of PA state. > > But only one concern I have here, I'm not sure I have spare cycles to do > this project. I'd rather move towards it in step-by-step mode, doing small > changes in each version. Any assistance is appreciated here. > > вт, 25 сент. 2018 г. в 11:25, Nikolay Izhikov : > >> Hello, Dmitriy >> >> > What about the case when committer creates ignite-9679 branch and tests >> it> without PR? >> >> It means, committer is experienced enough to run tests via Team City >> interface :). >> >> > So scanning seems to be possible only in JIRA >> >> I don't understand you here. >> You can retrieve comments filtered by *date*. >> You don't have to scan all 1000 PR's one by one. >> Anyway 1000 PR doesn't sound like big issue for me. >> >> My vote goes strong to GiHub user interface. >> I think we should have closer integration with GitHub, not Jira. >> >> Jira is about tickets and project management. >> GitHub is about code, commits and patches. >> We test patch, not ticket. >> >> >> В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет: >> > Hi Nikolay, >> > >> > What about the case when committer creates ignite-9679 branch and tests >> it >> > without PR? >> > >> > We have 1100+ open PRs and less than 100 open tickets. So scanning >> seems to >> > be possible only in JIRA. Mention probably will work for GitHub, but it >> > needs to be researched. >> > >> > Two open PRs is not a valid situation in the majority of cases and How >> To >> > Contribute asks to avoid it. The bot can ignore closed PRs and the bot >> can >> > expect there is only one open PR per ticket. >> > >> > Sincerely, >> > Dmitriy Pavlov >> > >> > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov : >> > >> > > Hello, Dmitriy. >> > > >> > > > But it could be a lot of work to handle mentions (probably from the >> > > >> > > email> account and inbox). >> > > >> > > Actually, it can be done via GitHub REST API [1]. >> > > It has 'since' param, so getting new GitHub comments is a very basic >> task. >> > > >> > > > Patch available ticket >> > > >> > > I think we shouldn't take a ticket as an entity that should be tested. >> > > For me, it's a PR. >> > > >> > > Moreover, it's a common case when we have several PR in a ticket. >> > > And it's a common case when both of them has to be tested. >> > > >> > > My vote goes to the closer integration with GitHub. >> > > >> > > [1] >> > > >> https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository >> > > >> > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: >> > > > Hi Nikolay, >> > > > >> > > > The idea makes perfect sense for me, and we should definitely take >> the >> > > >> > > best >> > > > practices from other big Apache projects. >> > > > >> > > > But it could be a lot of work to handle mentions (probably from the >> email >> > > > account and inbox). >> > > > >> > > > I would like to suggest the following algorithm: >> > > > >> > > > Patch available ticket, which was never checked by the bot will be >> > > > processed in the following steps: >> > > > 1. check existing run all (by PR or by branch name), if found go to >> the >> > > > step 3 >> > > > 2. run-all to be triggered by PR >> > > > 3. results should be analyzed for the presence of possible >> blockers. If >> > > > there is no blockers go to step 5. >> > > > 4. re-run of particular suites containing possible blockers should >> be >> > > > applied to try to get success for very rare flaky failures (<1%). >> Go to 3 >> > > > (this go to should be done only once). >> > > > 5. comment should be added to JIRA ticket containing information >> about >> > > > results. >> > > > >> > > > If a ticket was processed by bot early (probably author added some >> fixes) >> > > > but still in PA s
Re: Apache Ignite Teamcity Bot - Master Trends
Hi Nikolai, Cool! It seems we're making progress. Failed tests failures median goes down 54 -> 29 during the current week. I very liked how you did charts, so it is quite visual what test failures were at the timeline. Can we add total run time (duration) as a metric? And how much runs we've managed to do. What do you think? Sincerely, Dmitriy Pavlov пт, 5 окт. 2018 г. в 12:13, Nikolai Kulagin : > Hello, Igniters! > > On the Apache Ignite Teamcity Bot, there is a new page - Master Trends > [1]. On the page, you can view and compare the statistics of "Run All" for > the master branch for a period of time. Master Trends page contains > information about the main parameters of the builds, such as the number of > failed tests, the number of timeouts, JVM crashes, and so on. For a time > period, the minimum quantity, the median, the maximum quantity, and the > line chart for the entire selected interval are considered. > > You can read more about Master Trends and about Apache Ignite Teamcity > Bot on the wiki page [2]. > > [1] https://goo.gl/4Tj1mo [https://goo.gl/4Tj1mo] > [2] > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot > [ > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot > ]
[jira] [Created] (IGNITE-9815) [TC Bot] Muted tests shouldn't considered as blocker
Nikolay Izhikov created IGNITE-9815: --- Summary: [TC Bot] Muted tests shouldn't considered as blocker Key: IGNITE-9815 URL: https://issues.apache.org/jira/browse/IGNITE-9815 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Currently, TC bot doesn't analyze stack trace of test fail [1]. If some test failed with the link to Ignite ticket it shouldn't be considered as a blocker. Link with examples of such behaviour. https://issues.apache.org/jira/browse/IGNITE-9272?focusedCommentId=16641617&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16641617 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4924: IGNITE-9784 Automatically adjust max WAL history ...
GitHub user agoncharuk opened a pull request: https://github.com/apache/ignite/pull/4924 IGNITE-9784 Automatically adjust max WAL history size â¦mance drop You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9784 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4924.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4924 commit 1eff3881e3459819d755ac5b591d73f18ba4253d Author: Alexey Goncharuk Date: 2018-10-08T12:08:26Z IGNITE-9784 Automatically adjust max WAL history size to avoid performance drop ---
[jira] [Created] (IGNITE-9814) JDK11: HadoopClassLoader#nativeLibraries fails because has invalid assumption for ClassLoader type
Taras Ledkov created IGNITE-9814: Summary: JDK11: HadoopClassLoader#nativeLibraries fails because has invalid assumption for ClassLoader type Key: IGNITE-9814 URL: https://issues.apache.org/jira/browse/IGNITE-9814 Project: Ignite Issue Type: Bug Components: hadoop Affects Versions: 2.6 Reporter: Taras Ledkov Assignee: Taras Ledkov Error: {code} [2018-10-08 08:43:40,204][ERROR][main][IgniteKernal%grid_secondary] Exception during start processors, node will be stopped and close connections java.lang.ClassCastException: class java.util.concurrent.ConcurrentHashMap cannot be cast to class java.util.Vector (java.util.concurrent.ConcurrentHashMap and java.util.Vector are in module java.base of loader 'bootstrap') at org.apache.ignite.internal.processors.hadoop.HadoopClassLoader.nativeLibraries(HadoopClassLoader.java:270) at org.apache.ignite.internal.processors.hadoop.HadoopClassLoader.initializeNativeLibraries0(HadoopClassLoader.java:216) at org.apache.ignite.internal.processors.hadoop.HadoopClassLoader.initializeNativeLibraries(HadoopClassLoader.java:172) at org.apache.ignite.internal.processors.hadoop.HadoopClassLoader.(HadoopClassLoader.java:141) at org.apache.ignite.internal.processors.hadoop.HadoopHelperImpl.commonClassLoader(HadoopHelperImpl.java:78) at org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.start(HadoopJobTracker.java:163) at org.apache.ignite.internal.processors.hadoop.HadoopProcessor.start(HadoopProcessor.java:90) at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1692) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1022) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2033) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1728) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1156) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:676) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:601) at org.apache.ignite.Ignition.start(Ignition.java:323) at org.apache.ignite.internal.processors.igfs.IgfsAbstractBaseSelfTest.startGridWithIgfs(IgfsAbstractBaseSelfTest.java:394) at org.apache.ignite.internal.processors.hadoop.impl.igfs.Hadoop1DualAbstractTest.startUnderlying(Hadoop1DualAbstractTest.java:118) at org.apache.ignite.internal.processors.hadoop.impl.igfs.Hadoop1DualAbstractTest.createSecondaryFileSystemStack(Hadoop1DualAbstractTest.java:84) at org.apache.ignite.internal.processors.igfs.IgfsAbstractBaseSelfTest.beforeTestsStarted(IgfsAbstractBaseSelfTest.java:272) at org.apache.ignite.testframework.junits.GridAbstractTest.setUp(GridAbstractTest.java:621) at org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.setUp(GridCommonAbstractTest.java:494) at junit.framework.TestCase.runBare(TestCase.java:139) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at junit.framework.TestSuite.runTest(TestSuite.java:255) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4918: IGNITE-9684: JDK9: pass JVM options to created pr...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4918 ---
[GitHub] ignite pull request #4834: ignite-8507: Print a warning if SQL index inline ...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4834 ---
[MTCGA]: new failures in builds [1922022, 1922021] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master org.apache.ignite.visor.testsuites.VisorConsoleSelfTestSuite.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8731124613132278951&branch=%3Cdefault%3E&tab=testDetails No changes in the build *Recently contributed test failed in master org.apache.ignite.scalar.testsuites.ScalarSelfTestSuite.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-490302067567913317&branch=%3Cdefault%3E&tab=testDetails No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 13:51:07 08-10-2018
[jira] [Created] (IGNITE-9813) Unstablle behaviour in org.apache.ignite.internal.processors.query.GridQueryProcessor#processSchemaOperationLocal
Taras Ledkov created IGNITE-9813: Summary: Unstablle behaviour in org.apache.ignite.internal.processors.query.GridQueryProcessor#processSchemaOperationLocal Key: IGNITE-9813 URL: https://issues.apache.org/jira/browse/IGNITE-9813 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.6 Reporter: Taras Ledkov The test {{DynamicColumnsAbstractConcurrentSelfTest#testDropColumnCoordinatorChange}} periodically fails with exception {{SchemaOperationException: cache not found}} because we don't check the node state at the {{GridQueryProcessor#processSchemaOperationLocal}}. The node's stop must be checked here (e.g. by {{GridQueryProcessor#busyLock}}). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4923: IGNITE-9726 (for mass run)
GitHub user avplatonov opened a pull request: https://github.com/apache/ignite/pull/4923 IGNITE-9726 (for mass run) Just for test running on TC You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9726-mass-run Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4923.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4923 commit d02093c29d106b4ff4900ed7013d819880992e92 Author: Alexey Platonov Date: 2018-09-27T14:46:22Z fix suite lock due to missing of interruption in child threads commit 40c3206752e2615603a85e0dbe56c2c0bc3266be Author: Alexey Platonov Date: 2018-09-27T14:46:34Z Merge branch 'master' of https://github.com/apache/ignite into ignite-9726 commit e8e69e2c298ec44c390f62f13027de4eb45bbdb0 Author: Alexey Platonov Date: 2018-10-08T09:57:01Z skip tests for checking fix ---
Re: jira issue filling rules
Vyacheslav, Thanks for links, I just wanted to remind these rules for all. Not all Igniters use these rules in work. On Mon, Oct 8, 2018 at 12:17 PM Vyacheslav Daradur wrote: > Hi Igniters, > > Dmitriy Govorukhin, the naming of GitHub PR [1] and Upsource review > [2] already described in the wiki. > > About the naming of Jira issue, I'd suggest adding rules in related > part [3] in the wiki. > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-Creation > [2] > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewWithUpsource > [3] > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-TicketCreation > On Mon, Oct 8, 2018 at 12:16 PM Maxim Muzafarov > wrote: > > > > Dmitry, > > > > I fully support you! > > > > It's never too much remind these simple rules. It's not obvious at first > > glance but placing comment > > 'why' and 'how' particular JIRA issue was done will help much for the > > further code investigations. > > > > I'm not sure that pinning `link` to the TC result is a good practice. It > > becomes obsolete too quickly. > > Having the TC.Bot visa with ~no blockers found~ and the brief description > > of implementation details should be enough. > > > > Thoughts? > > > > On Sun, 7 Oct 2018 at 18:53 Dmitriy Pavlov > wrote: > > > > > ++1 from my side, as always. > > > > > > I want to add GitHub as a possible place of discussion, but any place > we've > > > selected to run review should be mentioned in JIRA ticket. E.g., I > left a > > > couple of comments in PR; please take a look. > > > > > > вс, 7 окт. 2018 г. в 18:01, Dmitriy Govorukhin < > > > dmitriy.govoruk...@gmail.com > > > >: > > > > > > > Hi folks, > > > > > > > > Recently time ago, I noticed that many Jira issues created and > filled in > > > > different ways, > > > > so someone does not fill issue description, someone does not attach > links > > > > in the links section instead, add links to comments and etc. > > > > I want to star discussion regards to Jira issue filling rules. > > > > > > > > I suggest, > > > > > > > > The name should include a short description problem > > > > The description should contain: > > > > (if bug) *Steps to reproduce* (java reproducer) or TC link to failed > > > tests. > > > > (if feature) *Idea and solution* > > > > PR should start with IGNITE- (JIRA bot automatically link your PR > > > with > > > > JIRA Issue) > > > > TC link should be in the* links section, *link name example: TC Run > all | > > > > TC Run pds | TC {suit name} > > > > Upsource link should be in the* links section* (optional, if some > > > reviewer > > > > needs), > > > > link name example: IGNT-CR-*** or CR-*** > > > > All discussions related code should be in JIRA issue comments or > Upsource > > > > review > > > > When you answer someone use @username for Upsource or ~username for > JIRA > > > > > > > > Comments are welcome! > > > > > > > > > -- > > -- > > Maxim Muzafarov > > > > -- > Best Regards, Vyacheslav D. >
Re: jira issue filling rules
Hi Igniters, Dmitriy Govorukhin, the naming of GitHub PR [1] and Upsource review [2] already described in the wiki. About the naming of Jira issue, I'd suggest adding rules in related part [3] in the wiki. [1] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-Creation [2] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewWithUpsource [3] https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-TicketCreation On Mon, Oct 8, 2018 at 12:16 PM Maxim Muzafarov wrote: > > Dmitry, > > I fully support you! > > It's never too much remind these simple rules. It's not obvious at first > glance but placing comment > 'why' and 'how' particular JIRA issue was done will help much for the > further code investigations. > > I'm not sure that pinning `link` to the TC result is a good practice. It > becomes obsolete too quickly. > Having the TC.Bot visa with ~no blockers found~ and the brief description > of implementation details should be enough. > > Thoughts? > > On Sun, 7 Oct 2018 at 18:53 Dmitriy Pavlov wrote: > > > ++1 from my side, as always. > > > > I want to add GitHub as a possible place of discussion, but any place we've > > selected to run review should be mentioned in JIRA ticket. E.g., I left a > > couple of comments in PR; please take a look. > > > > вс, 7 окт. 2018 г. в 18:01, Dmitriy Govorukhin < > > dmitriy.govoruk...@gmail.com > > >: > > > > > Hi folks, > > > > > > Recently time ago, I noticed that many Jira issues created and filled in > > > different ways, > > > so someone does not fill issue description, someone does not attach links > > > in the links section instead, add links to comments and etc. > > > I want to star discussion regards to Jira issue filling rules. > > > > > > I suggest, > > > > > > The name should include a short description problem > > > The description should contain: > > > (if bug) *Steps to reproduce* (java reproducer) or TC link to failed > > tests. > > > (if feature) *Idea and solution* > > > PR should start with IGNITE- (JIRA bot automatically link your PR > > with > > > JIRA Issue) > > > TC link should be in the* links section, *link name example: TC Run all | > > > TC Run pds | TC {suit name} > > > Upsource link should be in the* links section* (optional, if some > > reviewer > > > needs), > > > link name example: IGNT-CR-*** or CR-*** > > > All discussions related code should be in JIRA issue comments or Upsource > > > review > > > When you answer someone use @username for Upsource or ~username for JIRA > > > > > > Comments are welcome! > > > > > > -- > -- > Maxim Muzafarov -- Best Regards, Vyacheslav D.
Re: jira issue filling rules
Dmitry, I fully support you! It's never too much remind these simple rules. It's not obvious at first glance but placing comment 'why' and 'how' particular JIRA issue was done will help much for the further code investigations. I'm not sure that pinning `link` to the TC result is a good practice. It becomes obsolete too quickly. Having the TC.Bot visa with ~no blockers found~ and the brief description of implementation details should be enough. Thoughts? On Sun, 7 Oct 2018 at 18:53 Dmitriy Pavlov wrote: > ++1 from my side, as always. > > I want to add GitHub as a possible place of discussion, but any place we've > selected to run review should be mentioned in JIRA ticket. E.g., I left a > couple of comments in PR; please take a look. > > вс, 7 окт. 2018 г. в 18:01, Dmitriy Govorukhin < > dmitriy.govoruk...@gmail.com > >: > > > Hi folks, > > > > Recently time ago, I noticed that many Jira issues created and filled in > > different ways, > > so someone does not fill issue description, someone does not attach links > > in the links section instead, add links to comments and etc. > > I want to star discussion regards to Jira issue filling rules. > > > > I suggest, > > > > The name should include a short description problem > > The description should contain: > > (if bug) *Steps to reproduce* (java reproducer) or TC link to failed > tests. > > (if feature) *Idea and solution* > > PR should start with IGNITE- (JIRA bot automatically link your PR > with > > JIRA Issue) > > TC link should be in the* links section, *link name example: TC Run all | > > TC Run pds | TC {suit name} > > Upsource link should be in the* links section* (optional, if some > reviewer > > needs), > > link name example: IGNT-CR-*** or CR-*** > > All discussions related code should be in JIRA issue comments or Upsource > > review > > When you answer someone use @username for Upsource or ~username for JIRA > > > > Comments are welcome! > > > -- -- Maxim Muzafarov
[GitHub] ignite pull request #4845: IGNITE-9710 Ignite watchdog service handles longr...
Github user daradurvs closed the pull request at: https://github.com/apache/ignite/pull/4845 ---
[GitHub] ignite pull request #4875: IGNITE-9742 Illegal change of DiscoveryCustomEven...
Github user daradurvs closed the pull request at: https://github.com/apache/ignite/pull/4875 ---
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223283252 ## File path: ignite-tc-helper-web/src/main/webapp/comparison.html ## @@ -173,6 +185,139 @@ return parseFloat(string.substring(string.indexOf("-") + 2, string.lastIndexOf("-") - 1)); } +var changeDisplay = function(id) { +$("#" + id).toggle(); +} + +function mergeSuits(results) { +let mergedSuites = new Set(); + +for (let key of Object.keys(results)) { +for (let suite of Object.keys(results[key])) +mergedSuites.add(suite); +} + +return Array.from(mergedSuites); +} + +function printTests(results) { +$("#testsTable tr:not(:first-child)").remove(); + +let markedRow = true; + +for (let s of mergeSuits(results).sort()) { +let suiteName = s.split('_').filter((value, index) => index != 0).join('_'); + +let backgroundColor = markedRow ? "background-color:#fafaff" : ''; + +let testsCntCells = ''; + +let testsCells = ''; + +for (let key of Object.keys(results)) { +let obj = results[key]; + +let testLength = !obj.hasOwnProperty(s) || obj[s].length == 0 ? +'' : obj[s].length; + +testsCntCells = testsCntCells + '' + +testLength + ''; + +testsCells = testsCells + '' ++ getSuiteTestsHtml(results, s, key) + '' +} + +$("#testsTable > tbody:last-child").append('' + +'' + suiteName + +'' + testsCntCells + ''); + +$("#testsTable tbody").append('' + +'' + testsCells + ''); + +markedRow = !markedRow; +} +} + +function generateCompareTestsResults(results) { +let compareTestsResults = {}; + +for (let key of Object.keys(results)) { +let uniqueObj = {}; + +for (let suite of Object.keys(results[key])) { +let allTests = []; + +for (let key2 of Object.keys(results)) { +if (key == key2) +continue; + +allTests = allTests.concat(results[key2][suite]); +} + +let uniqTests = results[key][suite].filter(function(test) { +return allTests.indexOf(test) == -1; +}); + +if (uniqTests.length != 0) +uniqueObj[suite] = uniqTests; +} + +compareTestsResults[key] = uniqueObj; +} + +return compareTestsResults; +} + +function getSuiteTestsHtml(results, suite, key) { +if (!results[key].hasOwnProperty(suite) || results[key][suite].length == 0) +return ''; + +let res = ''; + +for (let v of results[key][suite].sort()) { +let list = v.toString().split("."); + +if (list.length < 2) +list = v.toString().split(":"); + +let testName = list.pop(); + Review comment: Remove empty line between `let` statements. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223284955 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/web/model/hist/BuildsHistory.java ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.ci.web.model.hist; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.text.DateFormat; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import javax.servlet.ServletContext; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.ITcHelper; +import org.apache.ignite.ci.tcbot.chain.BuildChainProcessor; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences; +import org.apache.ignite.ci.user.ICredentialsProv; +import org.apache.ignite.ci.web.CtxListener; +import org.apache.ignite.ci.web.model.current.BuildStatisticsSummary; +import org.apache.ignite.ci.web.rest.exception.ServiceUnauthorizedException; +import org.apache.ignite.ci.web.rest.parms.FullQueryParams; + +import static com.google.common.base.Strings.isNullOrEmpty; +import static org.apache.ignite.ci.web.rest.build.GetBuildTestFailures.BUILDS_STATISTICS_SUMMARY_CACHE_NAME; + +/** + * Builds History: includes statistic for every build and merged failed unmuted tests in specified time interval. + */ + +public class BuildsHistory { +/** */ +private String srvId; + +/** */ +private String projectId; + +/** */ +private String buildTypeId; + +/** */ +private String branchName; + +/** */ +private Date sinceDateFilter; + +/** */ +private Date untilDateFilter; + +/** */ +private Map> mergedTestsBySuites = new HashMap<>(); + +/** */ +public List buildsStatistics = new ArrayList<>(); + +/** */ +public String mergedTestsJson; + +/** */ +public void initialize(ICredentialsProv prov, ServletContext context) { +if (!prov.hasAccess(srvId)) +throw ServiceUnauthorizedException.noCreds(srvId); + +ITcHelper tcHelper = CtxListener.getTcHelper(context); + +IAnalyticsEnabledTeamcity teamcity = tcHelper.server(srvId, prov); + +int[] finishedBuildsIds = teamcity.getBuildNumbersFromHistory(buildTypeId, branchName, +sinceDateFilter, untilDateFilter); + +initBuildsStatistics(teamcity, prov, context, finishedBuildsIds); + +initBuildsMergedFailedTests(teamcity, finishedBuildsIds); +} + +/** */ +private void initBuildsStatistics(IAnalyticsEnabledTeamcity teamcity, ICredentialsProv prov, +ServletContext context, int[] buildIds) { +for (int buildId : buildIds) { +FullQueryParams buildParams = new FullQueryParams(); + +buildParams.setBuildId(buildId); + +buildParams.setBranch(branchName); + +buildParams.setServerId(srvId); + +BuildStatisticsSummary buildsStatistic = CtxListener.getBackgroundUpdater(context).get( +BUILDS_STATISTICS_SUMMARY_CACHE_NAME, prov, buildParams, +(k) -> { +BuildStatisticsSummary stat = new BuildStatisticsSummary(buildId); + +stat.initialize(teamcity); + +return stat; +}, false); + +if (buildsStatistic != null && !buildsStatistic.isFakeStub) +buildsStatistics.add(buildsStatistic); +} +} + +/** */ +private void initBuildsMergedFailedTests(IAnalyticsEnabledTeamcity teamcity, int[] buildIds) { +for (int buildId : buildIds) { +Build build = teamcity.getBuild(teamcity.getBuildHrefById(buildId)); + +TestOccurrences testOccurren
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223286978 ## File path: ignite-tc-helper-web/src/main/webapp/comparison.html ## @@ -173,6 +185,139 @@ return parseFloat(string.substring(string.indexOf("-") + 2, string.lastIndexOf("-") - 1)); } +var changeDisplay = function(id) { +$("#" + id).toggle(); +} + +function mergeSuits(results) { Review comment: How looks `results` in this and other methods? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223271317 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/web/model/hist/BuildsHistory.java ## @@ -0,0 +1,256 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.ci.web.model.hist; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.text.DateFormat; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import javax.servlet.ServletContext; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.ITcHelper; +import org.apache.ignite.ci.tcbot.chain.BuildChainProcessor; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences; +import org.apache.ignite.ci.user.ICredentialsProv; +import org.apache.ignite.ci.web.CtxListener; +import org.apache.ignite.ci.web.model.current.BuildStatisticsSummary; +import org.apache.ignite.ci.web.rest.exception.ServiceUnauthorizedException; +import org.apache.ignite.ci.web.rest.parms.FullQueryParams; + +import static com.google.common.base.Strings.isNullOrEmpty; +import static org.apache.ignite.ci.web.rest.build.GetBuildTestFailures.BUILDS_STATISTICS_SUMMARY_CACHE_NAME; + +/** + * Builds History: includes statistic for every build and merged failed unmuted tests in specified time interval. + */ + Review comment: Redundant empty line. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223279959 ## File path: ignite-tc-helper-web/src/main/webapp/comparison.html ## @@ -173,6 +185,139 @@ return parseFloat(string.substring(string.indexOf("-") + 2, string.lastIndexOf("-") - 1)); } +var changeDisplay = function(id) { +$("#" + id).toggle(); +} + +function mergeSuits(results) { +let mergedSuites = new Set(); + +for (let key of Object.keys(results)) { +for (let suite of Object.keys(results[key])) +mergedSuites.add(suite); +} + +return Array.from(mergedSuites); +} + +function printTests(results) { +$("#testsTable tr:not(:first-child)").remove(); + +let markedRow = true; + +for (let s of mergeSuits(results).sort()) { +let suiteName = s.split('_').filter((value, index) => index != 0).join('_'); + Review comment: Redundant empty lines between `let` statements. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223286626 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/web/rest/build/GetBuildTestFailures.java ## @@ -159,75 +155,63 @@ public TestFailuresSummary getBuildTestFails( } @GET -@Path("history") -public List getBuildsHistory( +@Produces(MediaType.TEXT_PLAIN) +@Path("testRef") +public String getTestRef( +@NotNull @QueryParam("testName") String name, +@NotNull @QueryParam("suiteName") String suiteName, @Nullable @QueryParam("server") String srv, -@Nullable @QueryParam("buildType") String buildType, -@Nullable @QueryParam("branch") String branch, -@Nullable @QueryParam("sinceDate") String sinceDate, -@Nullable @QueryParam("untilDate") String untilDate) -throws ServiceUnauthorizedException { -String srvId = isNullOrEmpty(srv) ? "apache" : srv; -String buildTypeId = isNullOrEmpty(buildType) ? "IgniteTests24Java8_RunAll" : buildType; -String branchName = isNullOrEmpty(branch) ? "refs/heads/master" : branch; -Date sinceDateFilter = isNullOrEmpty(sinceDate) ? null : dateParse(sinceDate); -Date untilDateFilter = isNullOrEmpty(untilDate) ? null : dateParse(untilDate); - -final BackgroundUpdater updater = CtxListener.getBackgroundUpdater(ctx); - -final ITcHelper tcHelper = CtxListener.getTcHelper(ctx); +@Nullable @QueryParam("projectId") String projectId) +throws InterruptedException, ExecutionException, ServiceUnauthorizedException { +final ITcHelper helper = CtxListener.getTcHelper(ctx); final ICredentialsProv prov = ICredentialsProv.get(req); -IAnalyticsEnabledTeamcity teamcity = tcHelper.server(srvId, prov); +String project = projectId == null ? "IgniteTests24Java8" : projectId; -int[] finishedBuilds = teamcity.getBuildNumbersFromHistory(buildTypeId, branchName, sinceDateFilter, untilDateFilter); +String srvId = srv == null ? "apache" : srv; -List buildsStatistics = new ArrayList<>(); +if (!prov.hasAccess(srvId)) +throw ServiceUnauthorizedException.noCreds(srvId); -for (int i = 0; i < finishedBuilds.length; i++) { -int buildId = finishedBuilds[i]; +IAnalyticsEnabledTeamcity teamcity = helper.server(srvId, prov); -FullQueryParams param = new FullQueryParams(); -param.setBuildId(buildId); -param.setBranch(branchName); -param.setServerId(srvId); +FullQueryParams key = new FullQueryParams(); -BuildStatisticsSummary buildsStatistic = updater.get( -BUILDS_STATISTICS_SUMMARY_CACHE_NAME, prov, param, -(k) -> getBuildStatisticsSummaryNoCache(srvId, buildId), false); +key.setTestName(name); Review comment: Redundant empty lines between `set` calls. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223279740 ## File path: ignite-tc-helper-web/src/main/webapp/comparison.html ## @@ -173,6 +185,139 @@ return parseFloat(string.substring(string.indexOf("-") + 2, string.lastIndexOf("-") - 1)); } +var changeDisplay = function(id) { +$("#" + id).toggle(); +} + +function mergeSuits(results) { +let mergedSuites = new Set(); + +for (let key of Object.keys(results)) { +for (let suite of Object.keys(results[key])) +mergedSuites.add(suite); +} + +return Array.from(mergedSuites); +} + +function printTests(results) { +$("#testsTable tr:not(:first-child)").remove(); Review comment: Create local variable for `$("#testsTable")`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals
SomeFire commented on a change in pull request #25: IGNITE-9645 [TC Bot] Add comparison of failed tests lists in two date intervals URL: https://github.com/apache/ignite-teamcity-bot/pull/25#discussion_r223274311 ## File path: ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/web/model/hist/BuildsHistory.java ## @@ -0,0 +1,256 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.ci.web.model.hist; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.text.DateFormat; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Date; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import javax.servlet.ServletContext; +import org.apache.ignite.ci.IAnalyticsEnabledTeamcity; +import org.apache.ignite.ci.ITcHelper; +import org.apache.ignite.ci.tcbot.chain.BuildChainProcessor; +import org.apache.ignite.ci.tcmodel.result.Build; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence; +import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences; +import org.apache.ignite.ci.user.ICredentialsProv; +import org.apache.ignite.ci.web.CtxListener; +import org.apache.ignite.ci.web.model.current.BuildStatisticsSummary; +import org.apache.ignite.ci.web.rest.exception.ServiceUnauthorizedException; +import org.apache.ignite.ci.web.rest.parms.FullQueryParams; + +import static com.google.common.base.Strings.isNullOrEmpty; +import static org.apache.ignite.ci.web.rest.build.GetBuildTestFailures.BUILDS_STATISTICS_SUMMARY_CACHE_NAME; + +/** + * Builds History: includes statistic for every build and merged failed unmuted tests in specified time interval. + */ + +public class BuildsHistory { +/** */ +private String srvId; + +/** */ +private String projectId; + +/** */ +private String buildTypeId; + +/** */ +private String branchName; + +/** */ +private Date sinceDateFilter; + +/** */ +private Date untilDateFilter; + +/** */ +private ObjectMapper objectMapper = new ObjectMapper(); + +/** */ +private Map> mergedTestsBySuites = new HashMap<>(); + +/** */ +public List buildsStatistics = new ArrayList<>(); + +/** */ +public String mergedTestsJson; + +/** */ +public void initialize(ICredentialsProv prov, ServletContext context) { +if (!prov.hasAccess(srvId)) +throw ServiceUnauthorizedException.noCreds(srvId); + +ITcHelper tcHelper = CtxListener.getTcHelper(context); + +IAnalyticsEnabledTeamcity teamcity = tcHelper.server(srvId, prov); + +int[] finishedBuildsIds = teamcity.getBuildNumbersFromHistory(buildTypeId, branchName, +sinceDateFilter, untilDateFilter); + +initBuildsStatistics(teamcity, prov, context, finishedBuildsIds); + +initBuildsMergedFailedTests(teamcity, finishedBuildsIds); +} + +/** */ +private void initBuildsStatistics(IAnalyticsEnabledTeamcity teamcity, ICredentialsProv prov, +ServletContext context, int[] buildIds) { +for (int buildId : buildIds) { +FullQueryParams buildParams = new FullQueryParams(); + +buildParams.setBuildId(buildId); +buildParams.setBranch(branchName); +buildParams.setServerId(srvId); + +BuildStatisticsSummary buildsStatistic = CtxListener.getBackgroundUpdater(context).get( +BUILDS_STATISTICS_SUMMARY_CACHE_NAME, prov, buildParams, +(k) -> { +BuildStatisticsSummary stat = new BuildStatisticsSummary(buildId); + +stat.initialize(teamcity); + +return stat; +}, false); + +if (buildsStatistic != null && !buildsStatistic.isFakeStub) +buildsStatistics.add(buildsStatistic); +} +} + +/** */ +private void initBuildsMergedFailedTests(IAnalyticsEnabledTeamcity teamcity, int[] buildIds) { +for (int buildId : buildIds) { +Build build = teamcity.getBuild(teamcity.
[jira] [Created] (IGNITE-9812) Explicit tests with expired SSL certificates
Ilya Kasnacheev created IGNITE-9812: --- Summary: Explicit tests with expired SSL certificates Key: IGNITE-9812 URL: https://issues.apache.org/jira/browse/IGNITE-9812 Project: Ignite Issue Type: Test Components: clients, security Reporter: Ilya Kasnacheev It came to our attention that we have no test which checks that expired SSL certificates do not work indeed. Should get such old certificate from git, check if it will fail to join cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4837: IGNITE-8717
GitHub user antonovsergey93 reopened a pull request: https://github.com/apache/ignite/pull/4837 IGNITE-8717 Cache configuration was moved to meta store. Verification of cache configuration on node joining was added. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8717 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4837.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4837 commit 8aeefa775cac27e34fbebb3799a056d2f1ba282b Author: Ivan Daschinskiy Date: 2018-06-20T09:01:00Z IGNITE-8717: First steps in implementing storing cache configuration in metastore. commit 2e28aba5686f4f694c590487db1da9572f1cc78a Author: Ivan Daschinskiy Date: 2018-06-21T08:30:26Z IGNITE-8717: Second step. Tests passed. commit 9c75281671e1a25afbbf9bc07ec623d0a6031f80 Author: Sergey Antonov Date: 2018-09-19T09:18:47Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite into ignite-8717 commit 4e30a5707b39b650727a4512e0730f30155372f1 Author: Sergey Antonov Date: 2018-09-21T10:22:44Z IGNITE-8717 WIP commit 549073b14a43151ca5715e4d583ea31d07a7bf50 Author: Sergey Antonov Date: 2018-09-21T12:05:20Z IGNITE-8717 WIP commit 19be946aaec837b553f89b6d46528769f29df730 Author: Sergey Antonov Date: 2018-09-24T08:51:46Z IGNITE-8717 Reproducer was added commit 338a3fe1d56f3719f8b258c38d50f0fb8b069861 Author: Sergey Antonov Date: 2018-09-26T11:55:52Z IGNITE-8717 first version of fix commit 26b9e3e0bc39bc35f080e4c7e95ef8b2697d7cfc Author: Sergey Antonov Date: 2018-09-26T12:00:24Z IGNITE-8717 cody style changes in test commit 03fd032f7784c486bfed87f32d5d72cb811bd3ae Author: Sergey Antonov Date: 2018-09-26T12:06:38Z IGNITE-8717 code formatting changes were canceled. commit 1b82cbe75922d738dadb667af661f60400455597 Author: Sergey Antonov Date: 2018-09-26T12:15:24Z IGNITE-8717 code formatting changes were canceled. commit 45007c2fcfcc8087b383365802520600114d0451 Author: Sergey Antonov Date: 2018-09-26T12:18:50Z Revert "IGNITE-8717 code formatting changes were canceled." This reverts commit 1b82cbe75922d738dadb667af661f60400455597. commit 1e505fe43a7e4bfa4128931286e710f7a529d474 Author: Sergey Antonov Date: 2018-09-26T12:28:18Z IGNITE-8717 Unused method were removed after interface API change. commit a1866a67d145c88dfc2cafb9ecfa533204cfe007 Author: Sergey Antonov Date: 2018-09-26T12:33:06Z IGNITE-8717 code formatting changes were canceled. commit b021a9701b54a84b285a881d3a18aa8f23a01c7f Author: Sergey Antonov Date: 2018-09-26T12:42:30Z IGNITE-8717 code formatting changes were canceled. commit 66dd620e648573e79e05c47f97d81142a51d947d Author: Sergey Antonov Date: 2018-09-26T12:50:28Z IGNITE-8717 missed javadoc was added. commit 9d3bb85aff0899637e72b142215b2cc5d13d10ca Author: Sergey Antonov Date: 2018-09-26T12:52:44Z IGNITE-8717 javadoc was changed. commit e33644c7702a6f83cd97de9bc16e72688fdf857e Author: Sergey Antonov Date: 2018-09-26T13:26:33Z IGNITE-8717 Missed cache configurations were added on joining node. commit 2833e05cffbc6e5f40f1fe731aee8c2f2e6be9ad Author: Sergey Antonov Date: 2018-09-26T13:32:41Z IGNITE-8717 WIP commit e5069dd74be55085518d092ee32627edb073f89e Author: Sergey Antonov Date: 2018-09-26T13:33:45Z IGNITE-8717 WIP commit fd05f9c0ddff18e310def78484d78df3c5e50273 Author: Sergey Antonov Date: 2018-09-26T14:53:43Z IGNITE-8717 Tests were added commit 108cc565bea1f06f5886e93e0774a1f7e1a866f2 Author: Sergey Antonov Date: 2018-09-26T15:21:48Z IGNITE-8717 Tests were added commit 60c15dc83ad1400d26b5f6dabba12c937e57bc8f Author: Sergey Antonov Date: 2018-09-26T16:15:50Z Merge branch 'master-mirror' into ignite-8717 commit 43e6cdf6a554f1d3fd4ba8b9c03339c70ba90176 Author: Sergey Antonov Date: 2018-09-26T16:16:54Z IGNITE-8717 change caches registry storage to metastore commit 348ef130193aba37ea0b9428b5ef2f8269dba930 Author: Sergey Antonov Date: 2018-09-27T10:44:29Z IGNITE-8717 WIP tests commit 7adabd8483341848908dff659d7d520763c6f1a9 Author: Sergey Antonov Date: 2018-09-27T12:25:29Z IGNITE-8717 Fixed master merge problems commit 6919d950b3d18024dcd36ce73282165d937abe1f Author: Sergey Antonov Date: 2018-09-27T13:57:58Z IGNITE-8717 Update tests commit 92b7dc2c42e7591dcac4951f62d65522d98eda88 Author: Sergey Antonov Date: 2018-09-27T14:18:34Z IGNITE-8717 update tests commit 7f2a53aa703e3e6eec5bffdd3d281bece27c793f Author: Sergey Antonov Date: 2018-09-27T16:35:21Z IGNITE-8717 first version of fix commit b230ee336fcd180c2f45ff25e9f1c7aa2b4e59fa Author: Sergey Antonov Date: 2018-09-28T12:52:50Z IGNITE-871
[GitHub] ignite pull request #4837: IGNITE-8717
Github user antonovsergey93 closed the pull request at: https://github.com/apache/ignite/pull/4837 ---
[jira] [Created] (IGNITE-9811) Web Console: Do not block "Save" button in case some inputs are invalid
Alexey Kuznetsov created IGNITE-9811: Summary: Web Console: Do not block "Save" button in case some inputs are invalid Key: IGNITE-9811 URL: https://issues.apache.org/jira/browse/IGNITE-9811 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexander Kalinin Let's not block "Save" button in case some inputs invalid. In modern UI trends it is better to set focus to invalid filed and show tooltip with error when user clicked on "Save" button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9810) Potential SQL performance drop between AI 2.6 and AI 2.7
Vladimir Ozerov created IGNITE-9810: --- Summary: Potential SQL performance drop between AI 2.6 and AI 2.7 Key: IGNITE-9810 URL: https://issues.apache.org/jira/browse/IGNITE-9810 Project: Ignite Issue Type: Task Components: sql, yardstick Reporter: Vladimir Ozerov Fix For: 2.7 At the moment Yardstick benchmarks show 5-7% performance drop in several SQL benchmarks. Need to find out whether these numbers are stable, and if yes - isolate problematic commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #4922: Ignite 9560 1
GitHub user dgarus opened a pull request: https://github.com/apache/ignite/pull/4922 Ignite 9560 1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/dgarus/ignite IGNITE-9560_1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4922.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4922 commit c5da193d222f13b33dadbde8dba88b41d057e7a6 Author: d.garus Date: 2018-10-04T14:41:47Z IGNITE-9560 commit 3759e1918cee538710c7de3b5cabaf5f7265eed6 Author: d.garus Date: 2018-10-05T10:58:25Z IGNITE-9560 Added tests for compute case ---