[MTCGA]: new failures in builds [3062722] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New stable failure of a flaky test in master IgniteProjectionStartStopRestartSelfTest.testRestartNodeById https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7244515415928378863=%3Cdefault%3E=testDetails *New stable failure of a flaky test in master IgniteProjectionStartStopRestartSelfTest.testRestartNodesByIds https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-1736033944635181204=%3Cdefault%3E=testDetails *New stable failure of a flaky test in master IgniteProjectionStartStopRestartSelfTest.testRestartNodes https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-9168655272383159616=%3Cdefault%3E=testDetails Changes may lead to failure were done by - mr.weider https://ci.ignite.apache.org/viewModification.html?modId=875407 - slava.koptilin https://ci.ignite.apache.org/viewModification.html?modId=875414 - dpavlov https://ci.ignite.apache.org/viewModification.html?modId=875409 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 06:41:48 12-02-2019
[jira] [Created] (IGNITE-11293) NPE in TcpCommunicationSpi
Ivan Bessonov created IGNITE-11293:
--
Summary: NPE in TcpCommunicationSpi
Key: IGNITE-11293
URL: https://issues.apache.org/jira/browse/IGNITE-11293
Project: Ignite
Issue Type: Bug
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov
Sometimes this exception is happening in "stopAllGrids" method in tests:
{code:java}
java.lang.NullPointerException
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2787)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2717)
at
org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1648)
at
org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1757)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.sendOrderedMessage(GridCacheIoManager.java:1303)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.lambda$null$1(GridDhtPartitionDemander.java:505)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6892)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
Reason is interruption thrown from TcpCommunicationSpi#reserveClient, SPI is
already in invalid state at this point so "log" and some other fields are nulls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11292) There is no way to disable WAL for cache in group
Dmitry Sherstobitov created IGNITE-11292:
Summary: There is no way to disable WAL for cache in group
Key: IGNITE-11292
URL: https://issues.apache.org/jira/browse/IGNITE-11292
Project: Ignite
Issue Type: Bug
Reporter: Dmitry Sherstobitov
Following code doesn't work if cache is in cacheGroup:
{code}ignite.cluster().disableWal(cacheName){code}
cacheName == cacheName:
{code}
Caused by: class org.apache.ignite.IgniteCheckedException: Cannot change WAL
mode because not all cache names belonging to the group are provided
[group=cache_group_1, missingCaches=[cache_group_1_005, cache_group_3_063,
cache_group_1_003, cache_group_3_064, cache_group_1_004, cache_group_3_061,
cache_group_3_062, cache_group_1_001, cache_group_1_002]]
{code}
cacheName == groupName:
{code}
Caused by: class org.apache.ignite.IgniteCheckedException: Cache doesn't exist:
cache_group_1
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11291) Assertion error in time of rebalance completion lead to to critical failure node
Vladislav Pyatkov created IGNITE-11291:
--
Summary: Assertion error in time of rebalance completion lead to
to critical failure node
Key: IGNITE-11291
URL: https://issues.apache.org/jira/browse/IGNITE-11291
Project: Ignite
Issue Type: Improvement
Reporter: Vladislav Pyatkov
{noformat}
java.lang.AssertionError: Got removed exception on entry with dht local
candidate: [IgniteTxEntry [key=KeyCacheObjectImpl [part=11859,
val=3338011748811769508, hasValBytes=true], cacheId=-313938805,
txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=11859, val=3338011748811769508,
hasValBytes=true], cacheId=-313938805], val=[op=UPDATE,
val=com.sbt.bm.ucp.common.dpl.model.party.hashcodes.DPartyHashCode_DPL_PROXY
[idHash=1985166276, hash=1530579783, colocationKey=11859, sync Flag=null,
hashUpdateTime=Sat Feb 09 20:16:55 MSK 2019, lastChangeDate=1549732615042,
partition_DPL_id=4, ownerId=ucp, externalSystem=21,
serializedHashCodesMap={"Address":["581a1367d4f50172c41168747c99105df1d3a4c6","24d3413bc5d3151dd784e088eaf4603e8e86c40b","fd7ce1449cd539a5976d358fb15060820c630f36"],"BirthDate":["a5cc7c1ac9821a6b19a455ae7eac55f4a4475bd7","415fe3810f8c4555a7188fe62275b34cdd1384cd","1311875998ef2aaccc5860ac6f991107ad4fa558"],"BirthPlace":["2fb5ea4cf7e9cfbdf77baae18f26804a10f51d57","76ddc5f2d959d27044c180957d9d0aed7369c0a2"],"Gender":["ad99678bb0e6de91d6a2ce620ba4fa98206aa9b9"],"IndividualIdentification":["c5b73f0087461fa4b980362344d38afddd1677b6","157952428fbe8c5b80da4db12c945cb4ad1f33f8","4d2257f62b1aa9cf290728d147776dd569b226a7"],"IndividualName":["256dba12a3b9d95dc1ada524d1a67cb590eb3ec2","2302eda39fb8f3751f3646542ad54ae717f5bc2c","9b8084de661530d5dc6ea140df793f4aa417a114"],"PartyToPartyGroup":["8d41984edf2e9ff916da0a74b884554cc2fceca9"],"PhoneNumber":["c89cbfb741ba5ea0fa13091a1cc7591c69374c0c","149081529b36fe29bc72bf88c66e51bdab3ae2d7","3c6bc9cc9e47ecd2f79b1506a22799ba69b95227","389830cd794fd6748f46ab7d8d878a2fec75cf63","9d5c3ecd5c951a16745cefd771b79c17bbc8c665","dfd8959391fd18e89505bfde31e0aa9a80513fec"],"Individual":["82dc325513d53990709bafbf1a334b602025ba17","544aab947d3a33787c7feffc93c4a4572e957dca","2728a40ce9759fa7110bc5d0d3c8b5c193210a2d"]},
uid=null, isDeleted=false, isImmutable=false, checksum=null,
id=3338011748811769508, externalClientId=75662144,
colocationId=1216693183554769678]], prevVal=[op=NOOP, val=null],
oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
filters=CacheEntryPredicate[] [], filtersPassed=false, filtersSet=false,
entry=GridCacheMapEntry [key=KeyCacheObjectImpl [part=11859,
val=3338011748811769508, hasValBytes=true], val=null, startVer=1551196730698,
ver=GridCacheVersion [topVer=156972865, order=1546089814280, nodeOrder=65],
hash=777160356, extras=GridCacheObsoleteEntryExtras
[obsoleteVer=GridCacheVersion [topVer=2147483647, order=0, nodeOrder=0]],
flags=2]GridDistributedCacheEntry [super=]GridDhtCacheEntry [rdrs=ReaderId[]
[], part=11859, super=], prepared=1, locked=false, nodeId=null,
locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2,
partUpdateCntr=0, serReadVer=GridCacheVersion[topVer=156972865,
order=1546089814280, nodeOrder=65], xidVer=null]]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.checkReadConflict(GridDhtTxPrepareFuture.java:1164)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1223)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.access$000(GridDhtTxPrepareFuture.java:109)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture$2.apply(GridDhtTxPrepareFuture.java:701)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture$2.apply(GridDhtTxPrepareFuture.java:696)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onDone(GridDhtForceKeysFuture.java:153)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtForceKeysFuture.onDone(GridDhtForceKeysFuture.java:69)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at
Re: Make the TeamCity console quiet.
Maksim, sounds good. 1) Can we just turn off log rollover? 2) I believe, that we need the ability to override this by setting parameter for Run All. On Mon, Feb 11, 2019 at 4:12 PM Maksim Stepachev wrote: > Ivan, > > Yes. It happens because we use the RollingFileAppender for a file logging. > This appender has the next properties: > > > > > Can I increase MaxFileSize by 200MB or more? I suppose these limits were > added for cases when the test writes a log. It makes the guarantee that > logs less than 10*10 = 100MBs. But it's wrong if it writes into the console > too. > > пн, 11 февр. 2019 г. в 13:14, Павлухин Иван : > > > Maksim, > > > > Generally I like the idea. But there is one thing which bothers me a > > little bit. Usually I use "Download full build log" link to download > > log and then examine it as a single file. AFAIK artifact with logs > > contains several files. Could you suggest a way how can I conveniently > > explore files archive as a single file? > > > > пн, 11 февр. 2019 г. в 13:04, Ilya Kasnacheev >: > > > > > > Hello! > > > > > > Can we do IGNITE_QUIET=true for runAlls triggered by bot (along with > > > SCALE_FACTOR) and false for manually triggered builds (or retriggered > > > failures)? > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > пн, 11 февр. 2019 г. в 13:03, Dmitriy Pavlov : > > > > > > > ++1 from my side. It is very interesting how overall RunAll run time > > will > > > > be decreased. > > > > > > > > My previous experience with TeamCity shows that removing pressure > from > > the > > > > console and using adequate logging instead may bring x1.5 - x2 > > performance > > > > boost for tests. > > > > > > > > One more benefit of moving from synced streams to async logging will > > show > > > > us that bugs, which we can't see right now. > > > > > > > > So I absolutely agree to move logging data to a logger. > > > > > > > > пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev < > > maksim.stepac...@gmail.com > > > > >: > > > > > > > > > Igniters, > > > > > > > > > > > > > > > When I was working with flaky tests, I was surprised that one of > the > > > > > reasons for failure was a log appender blocking the console. > > > > > > > > > > > > > > > I suppose it happened because of a TeamCity agent communicating > with > > java > > > > > out through Linux pipe. This is a problem for tests with 1GB log > > history. > > > > > > > > > > > > > > > Our test by default writes logs into 2 sources, such as the console > > and > > > > the > > > > > file. I'm going to change mode for the console at IGNITE_QUIET=true > > and > > > > > write only warn and error logs in it. Also, I’ll include a > > diagnostic log > > > > > into it. > > > > > > > > > > > > > > > You will be able to read the previous log from the Artifacts tab in > > your > > > > > build. > > > > > > > > > > This is a solution which has advantages: > > > > > > > > > > 1. First of all, we’ll cut down the space usage pre-test run by > ~6-7 > > > > times. > > > > > For example - it's 1GB of saved space for 1 suit like "Cache > > (Restarts) > > > > 1". > > > > > > > > > > 2. We’ll be able to use a new space for long history storage for > > builds. > > > > > > > > > > 3. The TeamCity bot won't be lagging when it parses a large file. > > > > > > > > > > 4. Possibly some of tests won't be flaky. > > > > > > > > > > 5. And finally, the time of running my suite will be cut down by > 5-7 > > > > > minutes. > > > > > > > > > > > > > > > Also, I'm going to make a workaround for previous behavior. For > > example, > > > > > this flag will be added to build params. But I should investigate > it. > > > > > > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > >
[jira] [Created] (IGNITE-11290) History of server node IDs should be passed to new nodes with NodeAddedMessage
Sergey Chugunov created IGNITE-11290: Summary: History of server node IDs should be passed to new nodes with NodeAddedMessage Key: IGNITE-11290 URL: https://issues.apache.org/jira/browse/IGNITE-11290 Project: Ignite Issue Type: Bug Reporter: Sergey Chugunov Assignee: Sergey Chugunov Fix For: 2.8 As part of IGNITE-5569 (bounded) history of IDs of all server nodes existed in the cluster is introduced to prevent join requests with duplicate IDs if network glitch happens during node's join process. Initial implementation maintains the history locally on each node and isn't transferred to successfully joined nodes. It is needed to pass it (in NodeAdded messages) to new nodes to cover edge-case scenarios of coordinator failover. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Make the TeamCity console quiet.
Ivan, Yes. It happens because we use the RollingFileAppender for a file logging. This appender has the next properties: Can I increase MaxFileSize by 200MB or more? I suppose these limits were added for cases when the test writes a log. It makes the guarantee that logs less than 10*10 = 100MBs. But it's wrong if it writes into the console too. пн, 11 февр. 2019 г. в 13:14, Павлухин Иван : > Maksim, > > Generally I like the idea. But there is one thing which bothers me a > little bit. Usually I use "Download full build log" link to download > log and then examine it as a single file. AFAIK artifact with logs > contains several files. Could you suggest a way how can I conveniently > explore files archive as a single file? > > пн, 11 февр. 2019 г. в 13:04, Ilya Kasnacheev : > > > > Hello! > > > > Can we do IGNITE_QUIET=true for runAlls triggered by bot (along with > > SCALE_FACTOR) and false for manually triggered builds (or retriggered > > failures)? > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пн, 11 февр. 2019 г. в 13:03, Dmitriy Pavlov : > > > > > ++1 from my side. It is very interesting how overall RunAll run time > will > > > be decreased. > > > > > > My previous experience with TeamCity shows that removing pressure from > the > > > console and using adequate logging instead may bring x1.5 - x2 > performance > > > boost for tests. > > > > > > One more benefit of moving from synced streams to async logging will > show > > > us that bugs, which we can't see right now. > > > > > > So I absolutely agree to move logging data to a logger. > > > > > > пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev < > maksim.stepac...@gmail.com > > > >: > > > > > > > Igniters, > > > > > > > > > > > > When I was working with flaky tests, I was surprised that one of the > > > > reasons for failure was a log appender blocking the console. > > > > > > > > > > > > I suppose it happened because of a TeamCity agent communicating with > java > > > > out through Linux pipe. This is a problem for tests with 1GB log > history. > > > > > > > > > > > > Our test by default writes logs into 2 sources, such as the console > and > > > the > > > > file. I'm going to change mode for the console at IGNITE_QUIET=true > and > > > > write only warn and error logs in it. Also, I’ll include a > diagnostic log > > > > into it. > > > > > > > > > > > > You will be able to read the previous log from the Artifacts tab in > your > > > > build. > > > > > > > > This is a solution which has advantages: > > > > > > > > 1. First of all, we’ll cut down the space usage pre-test run by ~6-7 > > > times. > > > > For example - it's 1GB of saved space for 1 suit like "Cache > (Restarts) > > > 1". > > > > > > > > 2. We’ll be able to use a new space for long history storage for > builds. > > > > > > > > 3. The TeamCity bot won't be lagging when it parses a large file. > > > > > > > > 4. Possibly some of tests won't be flaky. > > > > > > > > 5. And finally, the time of running my suite will be cut down by 5-7 > > > > minutes. > > > > > > > > > > > > Also, I'm going to make a workaround for previous behavior. For > example, > > > > this flag will be added to build params. But I should investigate it. > > > > > > > > > > > -- > Best regards, > Ivan Pavlukhin >
[MTCGA]: new failures in builds [3058159] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New Critical Failure in master Platform .NET (NuGet)* https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNetNuGet=%3Cdefault%3E=buildTypeStatusDiv No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 15:26:49 11-02-2019
[jira] [Created] (IGNITE-11289) BPlusTree.AbstractForwardCursor can use obsolete page
Ivan Pavlukhin created IGNITE-11289:
---
Summary: BPlusTree.AbstractForwardCursor can use obsolete page
Key: IGNITE-11289
URL: https://issues.apache.org/jira/browse/IGNITE-11289
Project: Ignite
Issue Type: Bug
Components: cache
Reporter: Ivan Pavlukhin
There is a possibility that {{AbstractForwardCursor.nextPageId}} refers to a
page which was already excluded from the tree. The problem reproduces with a
scan query execution with put/remove load in background.
Linked PR contains reproducer and some tricks allowing to reproduce the problem
more often. Problem becomes evident when problematic page is taken from
REUSE_BUCKET. But there could be other hidden problems which do no cause any
runtime errors but lead to data inconsistency.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11288) Missing SO_LINGER in TcpDiscovery and TcpCommunicationSpi causing SSLSocket.close() deadlock.
Pavel Voronkin created IGNITE-11288: --- Summary: Missing SO_LINGER in TcpDiscovery and TcpCommunicationSpi causing SSLSocket.close() deadlock. Key: IGNITE-11288 URL: https://issues.apache.org/jira/browse/IGNITE-11288 Project: Ignite Issue Type: Bug Reporter: Pavel Voronkin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Make the TeamCity console quiet.
Maksim, Generally I like the idea. But there is one thing which bothers me a little bit. Usually I use "Download full build log" link to download log and then examine it as a single file. AFAIK artifact with logs contains several files. Could you suggest a way how can I conveniently explore files archive as a single file? пн, 11 февр. 2019 г. в 13:04, Ilya Kasnacheev : > > Hello! > > Can we do IGNITE_QUIET=true for runAlls triggered by bot (along with > SCALE_FACTOR) and false for manually triggered builds (or retriggered > failures)? > > Regards, > -- > Ilya Kasnacheev > > > пн, 11 февр. 2019 г. в 13:03, Dmitriy Pavlov : > > > ++1 from my side. It is very interesting how overall RunAll run time will > > be decreased. > > > > My previous experience with TeamCity shows that removing pressure from the > > console and using adequate logging instead may bring x1.5 - x2 performance > > boost for tests. > > > > One more benefit of moving from synced streams to async logging will show > > us that bugs, which we can't see right now. > > > > So I absolutely agree to move logging data to a logger. > > > > пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev > >: > > > > > Igniters, > > > > > > > > > When I was working with flaky tests, I was surprised that one of the > > > reasons for failure was a log appender blocking the console. > > > > > > > > > I suppose it happened because of a TeamCity agent communicating with java > > > out through Linux pipe. This is a problem for tests with 1GB log history. > > > > > > > > > Our test by default writes logs into 2 sources, such as the console and > > the > > > file. I'm going to change mode for the console at IGNITE_QUIET=true and > > > write only warn and error logs in it. Also, I’ll include a diagnostic log > > > into it. > > > > > > > > > You will be able to read the previous log from the Artifacts tab in your > > > build. > > > > > > This is a solution which has advantages: > > > > > > 1. First of all, we’ll cut down the space usage pre-test run by ~6-7 > > times. > > > For example - it's 1GB of saved space for 1 suit like "Cache (Restarts) > > 1". > > > > > > 2. We’ll be able to use a new space for long history storage for builds. > > > > > > 3. The TeamCity bot won't be lagging when it parses a large file. > > > > > > 4. Possibly some of tests won't be flaky. > > > > > > 5. And finally, the time of running my suite will be cut down by 5-7 > > > minutes. > > > > > > > > > Also, I'm going to make a workaround for previous behavior. For example, > > > this flag will be added to build params. But I should investigate it. > > > > > -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-11287) JDBC Thin: best effort affinity
Alexander Lapin created IGNITE-11287: Summary: JDBC Thin: best effort affinity Key: IGNITE-11287 URL: https://issues.apache.org/jira/browse/IGNITE-11287 Project: Ignite Issue Type: Task Reporter: Alexander Lapin It's an umbrella ticket for implementing [IEP-23|https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients] within the scope of JDBC Thin driver. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Make the TeamCity console quiet.
Hello! Can we do IGNITE_QUIET=true for runAlls triggered by bot (along with SCALE_FACTOR) and false for manually triggered builds (or retriggered failures)? Regards, -- Ilya Kasnacheev пн, 11 февр. 2019 г. в 13:03, Dmitriy Pavlov : > ++1 from my side. It is very interesting how overall RunAll run time will > be decreased. > > My previous experience with TeamCity shows that removing pressure from the > console and using adequate logging instead may bring x1.5 - x2 performance > boost for tests. > > One more benefit of moving from synced streams to async logging will show > us that bugs, which we can't see right now. > > So I absolutely agree to move logging data to a logger. > > пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev >: > > > Igniters, > > > > > > When I was working with flaky tests, I was surprised that one of the > > reasons for failure was a log appender blocking the console. > > > > > > I suppose it happened because of a TeamCity agent communicating with java > > out through Linux pipe. This is a problem for tests with 1GB log history. > > > > > > Our test by default writes logs into 2 sources, such as the console and > the > > file. I'm going to change mode for the console at IGNITE_QUIET=true and > > write only warn and error logs in it. Also, I’ll include a diagnostic log > > into it. > > > > > > You will be able to read the previous log from the Artifacts tab in your > > build. > > > > This is a solution which has advantages: > > > > 1. First of all, we’ll cut down the space usage pre-test run by ~6-7 > times. > > For example - it's 1GB of saved space for 1 suit like "Cache (Restarts) > 1". > > > > 2. We’ll be able to use a new space for long history storage for builds. > > > > 3. The TeamCity bot won't be lagging when it parses a large file. > > > > 4. Possibly some of tests won't be flaky. > > > > 5. And finally, the time of running my suite will be cut down by 5-7 > > minutes. > > > > > > Also, I'm going to make a workaround for previous behavior. For example, > > this flag will be added to build params. But I should investigate it. > > >
[jira] [Created] (IGNITE-11286) Add console prompt for password in control.(sh|bat) script
Andrey Kuznetsov created IGNITE-11286:
-
Summary: Add console prompt for password in control.(sh|bat) script
Key: IGNITE-11286
URL: https://issues.apache.org/jira/browse/IGNITE-11286
Project: Ignite
Issue Type: Task
Affects Versions: 2.7
Reporter: Andrey Kuznetsov
Assignee: Andrey Kuznetsov
For security reasons we are to add interactive alternative to {{--password}}.
Other password-related options already have this alternative, see [1].
[1] https://jira.apache.org/jira/browse/IGNITE-10257
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Make the TeamCity console quiet.
++1 from my side. It is very interesting how overall RunAll run time will be decreased. My previous experience with TeamCity shows that removing pressure from the console and using adequate logging instead may bring x1.5 - x2 performance boost for tests. One more benefit of moving from synced streams to async logging will show us that bugs, which we can't see right now. So I absolutely agree to move logging data to a logger. пн, 11 февр. 2019 г. в 11:14, Maksim Stepachev : > Igniters, > > > When I was working with flaky tests, I was surprised that one of the > reasons for failure was a log appender blocking the console. > > > I suppose it happened because of a TeamCity agent communicating with java > out through Linux pipe. This is a problem for tests with 1GB log history. > > > Our test by default writes logs into 2 sources, such as the console and the > file. I'm going to change mode for the console at IGNITE_QUIET=true and > write only warn and error logs in it. Also, I’ll include a diagnostic log > into it. > > > You will be able to read the previous log from the Artifacts tab in your > build. > > This is a solution which has advantages: > > 1. First of all, we’ll cut down the space usage pre-test run by ~6-7 times. > For example - it's 1GB of saved space for 1 suit like "Cache (Restarts) 1". > > 2. We’ll be able to use a new space for long history storage for builds. > > 3. The TeamCity bot won't be lagging when it parses a large file. > > 4. Possibly some of tests won't be flaky. > > 5. And finally, the time of running my suite will be cut down by 5-7 > minutes. > > > Also, I'm going to make a workaround for previous behavior. For example, > this flag will be added to build params. But I should investigate it. >
[jira] [Created] (IGNITE-11285) Conflicting javadocs for PagesFillFactor
Alexey Kuznetsov created IGNITE-11285:
-
Summary: Conflicting javadocs for PagesFillFactor
Key: IGNITE-11285
URL: https://issues.apache.org/jira/browse/IGNITE-11285
Project: Ignite
Issue Type: Improvement
Reporter: Alexey Kuznetsov
Assignee: Dmitriy Govorukhin
org.apache.ignite.MemoryMetrics#getPagesFillFactor
{code}
/**
* Gets the percentage of space that is still free and can be filled in.
*
* @return The percentage of space that is still free and can be filled in.
*/
public float getPagesFillFactor();
{code}
org.apache.ignite.DataRegionMetrics#getPagesFillFactor
{code}
/**
* Gets the percentage of the used space.
*
* @return The percentage of the used space.
*/
public float getPagesFillFactor();
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Code inspection
Petr, we should have 1 configuration for project, may be 1 configuration
per programming language.
пн, 11 февр. 2019 г., 11:33 Petr Ivanov mr.wei...@gmail.com:
> I was asking about how many build configuration is intended? One for all
> and multiple per module?
>
> With IDEA inspections it was going to be build configuration per module.
>
>
>
>
> > On 11 Feb 2019, at 11:24, Nikolay Izhikov wrote:
> >
> > Hello, Petr.
> >
> > Are you saying that we have not single build task? And each module builds
> > when it required? If yes, then I propose to create a task like "Licence
> > check" which will be run for every patch.
> >
> > My point is that violation of codestyle should be treated as hard as
> > compile error.
> >
> > пн, 11 февр. 2019 г., 11:16 Petr Ivanov mr.wei...@gmail.com:
> >
> >> Is build configuration Inspections [Core] meant to transform into single
> >> all-modules check build configuration (without module subdivision)?
> >>
> >>
> >>> On 11 Feb 2019, at 11:02, Nikolay Izhikov wrote:
> >>>
> >>> Hello, Maxim.
> >>>
> >>> +1 from me for migrating to checkstyle.
> >>>
> >>> Oleg, there is plugin for IDEA with 2mln downloads -
> >>> https://plugins.jetbrains.com/plugin/1065-checkstyle-idea
> >>>
> >>> I propose do the following:
> >>>
> >>> 1. Migrate current checks to checkstyle.
> >>> 2. Apply checks to all Ignite modules. Currently, only core module are
> >>> checked.
> >>> I will review and commit this patch, or do it by my own.
> >>>
> >>> 3. Include code style checks to "Build Apache Ignite" suite. Ignite has
> >> to
> >>> fail to build if patch violates codestyle.
> >>>
> >>> вс, 10 февр. 2019 г. в 07:54, Павлухин Иван :
> >>>
> Hi,
>
> I also think that some warning from IDEA that some code style rule is
> violated is a must-have.
>
> вс, 10 февр. 2019 г. в 01:58, oignatenko :
> >
> > Hi Maxim,
> >
> > I believe that whatever style checks we establish at Teamcity, we
> >> better
> > take care of making it easy for developers to find and fix violations
> >> in
> > their typical dev environment (for Ignite this means, in IDEA). I
> think
> it
> > is important that developers can maintain required style with minimal
> effort
> > on their side.
> >
> > If above is doable then I am 200% for migrating our Teamcity
> >> inspections
> to
> > checkstyle / maven.
> >
> > This is because I am very disappointed observing how it stays broken
> >> for
> so
> > long. And worst of all, even when (if) it is fixed, I feel we will
> always be
> > at risk that it breaks again and that we will have to again wait for
> months
> > for it to be fixed.
> >
> > This is such a stark contrast with my experience regarding checkstyle
> based
> > inspections. These just work and you just never fear that it is going
> >> to
> > break for some obscure reason, this is so much better than what I
> >> observe
> > now.
> >
> > One suggestion in case if we pick checkstyle - I recommend keeping
> its
> > config file somewhere in the project under version control. I used to
> > maintain such a shared style config at one of past jobs and after
> some
> > experimenting it turned out most convenient to have it this way - so
> >> that
> > developers could easily assess and discuss style settings and keep
> >> track
> of
> > changes in these. (note how Kafka folks from your link [5] appear to
> be
> > doing it this way)
> >
> > regards, Oleg
> >
> >
> > Mmuzaf wrote
> >> Igniters,
> >>
> >> I've found that some of the community members have faced with
> >> `[Inspections] Core suite [1]` is not working well enough on TC. The
> >> suite has a `FAILED` status for more than 2 months due to some
> issues
> >> in TeamCity application [2]. Current suite behaviour confuses not
> only
> >> new contributors but also other community members. Moreover, this
> >> suite is no longer checks rules we previously configured. For
> >> instance, in the master branch, I've found 11 `Unused imports` which
> >> should have been caught earlier (e.g. for
> >> {{IgniteCachePutAllRestartTest} [3]).
> >>
> >> I think we should make the next step to enable an automatic code
> style
> >> checks. As an example, we can consider the Apache Kafka code style
> [5]
> >> way and configure for the Ignite project a maven-checkstyle-plugin
> >> with its own maven profile and run it simultaneously with other TC.
> We
> >> can also enable the previously configured inspection rules, so no
> >> coding style violations will be missed.
> >>
> >> I see some advantages of using a maven plugin:
> >> - an IDE agnostic way for code checks
> >> - can be used with different CI and build tools (Jenkins, TC)
> >> - executable from the command line
> >> - the entry single point to
Re: Code inspection
I was asking about how many build configuration is intended? One for all and
multiple per module?
With IDEA inspections it was going to be build configuration per module.
> On 11 Feb 2019, at 11:24, Nikolay Izhikov wrote:
>
> Hello, Petr.
>
> Are you saying that we have not single build task? And each module builds
> when it required? If yes, then I propose to create a task like "Licence
> check" which will be run for every patch.
>
> My point is that violation of codestyle should be treated as hard as
> compile error.
>
> пн, 11 февр. 2019 г., 11:16 Petr Ivanov mr.wei...@gmail.com:
>
>> Is build configuration Inspections [Core] meant to transform into single
>> all-modules check build configuration (without module subdivision)?
>>
>>
>>> On 11 Feb 2019, at 11:02, Nikolay Izhikov wrote:
>>>
>>> Hello, Maxim.
>>>
>>> +1 from me for migrating to checkstyle.
>>>
>>> Oleg, there is plugin for IDEA with 2mln downloads -
>>> https://plugins.jetbrains.com/plugin/1065-checkstyle-idea
>>>
>>> I propose do the following:
>>>
>>> 1. Migrate current checks to checkstyle.
>>> 2. Apply checks to all Ignite modules. Currently, only core module are
>>> checked.
>>> I will review and commit this patch, or do it by my own.
>>>
>>> 3. Include code style checks to "Build Apache Ignite" suite. Ignite has
>> to
>>> fail to build if patch violates codestyle.
>>>
>>> вс, 10 февр. 2019 г. в 07:54, Павлухин Иван :
>>>
Hi,
I also think that some warning from IDEA that some code style rule is
violated is a must-have.
вс, 10 февр. 2019 г. в 01:58, oignatenko :
>
> Hi Maxim,
>
> I believe that whatever style checks we establish at Teamcity, we
>> better
> take care of making it easy for developers to find and fix violations
>> in
> their typical dev environment (for Ignite this means, in IDEA). I think
it
> is important that developers can maintain required style with minimal
effort
> on their side.
>
> If above is doable then I am 200% for migrating our Teamcity
>> inspections
to
> checkstyle / maven.
>
> This is because I am very disappointed observing how it stays broken
>> for
so
> long. And worst of all, even when (if) it is fixed, I feel we will
always be
> at risk that it breaks again and that we will have to again wait for
months
> for it to be fixed.
>
> This is such a stark contrast with my experience regarding checkstyle
based
> inspections. These just work and you just never fear that it is going
>> to
> break for some obscure reason, this is so much better than what I
>> observe
> now.
>
> One suggestion in case if we pick checkstyle - I recommend keeping its
> config file somewhere in the project under version control. I used to
> maintain such a shared style config at one of past jobs and after some
> experimenting it turned out most convenient to have it this way - so
>> that
> developers could easily assess and discuss style settings and keep
>> track
of
> changes in these. (note how Kafka folks from your link [5] appear to be
> doing it this way)
>
> regards, Oleg
>
>
> Mmuzaf wrote
>> Igniters,
>>
>> I've found that some of the community members have faced with
>> `[Inspections] Core suite [1]` is not working well enough on TC. The
>> suite has a `FAILED` status for more than 2 months due to some issues
>> in TeamCity application [2]. Current suite behaviour confuses not only
>> new contributors but also other community members. Moreover, this
>> suite is no longer checks rules we previously configured. For
>> instance, in the master branch, I've found 11 `Unused imports` which
>> should have been caught earlier (e.g. for
>> {{IgniteCachePutAllRestartTest} [3]).
>>
>> I think we should make the next step to enable an automatic code style
>> checks. As an example, we can consider the Apache Kafka code style [5]
>> way and configure for the Ignite project a maven-checkstyle-plugin
>> with its own maven profile and run it simultaneously with other TC. We
>> can also enable the previously configured inspection rules, so no
>> coding style violations will be missed.
>>
>> I see some advantages of using a maven plugin:
>> - an IDE agnostic way for code checks
>> - can be used with different CI and build tools (Jenkins, TC)
>> - executable from the command line
>> - the entry single point to configure new rules
>>
>> I've created the ticket [4] and will prepare PR for it.
>>
>> WDYT?
>>
>> [1]
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
>> [2] https://youtrack.jetbrains.com/issue/TW-58504
>> [3]
>>
Re: Code inspection
Hello, Petr.
Are you saying that we have not single build task? And each module builds
when it required? If yes, then I propose to create a task like "Licence
check" which will be run for every patch.
My point is that violation of codestyle should be treated as hard as
compile error.
пн, 11 февр. 2019 г., 11:16 Petr Ivanov mr.wei...@gmail.com:
> Is build configuration Inspections [Core] meant to transform into single
> all-modules check build configuration (without module subdivision)?
>
>
> > On 11 Feb 2019, at 11:02, Nikolay Izhikov wrote:
> >
> > Hello, Maxim.
> >
> > +1 from me for migrating to checkstyle.
> >
> > Oleg, there is plugin for IDEA with 2mln downloads -
> > https://plugins.jetbrains.com/plugin/1065-checkstyle-idea
> >
> > I propose do the following:
> >
> > 1. Migrate current checks to checkstyle.
> > 2. Apply checks to all Ignite modules. Currently, only core module are
> > checked.
> > I will review and commit this patch, or do it by my own.
> >
> > 3. Include code style checks to "Build Apache Ignite" suite. Ignite has
> to
> > fail to build if patch violates codestyle.
> >
> > вс, 10 февр. 2019 г. в 07:54, Павлухин Иван :
> >
> >> Hi,
> >>
> >> I also think that some warning from IDEA that some code style rule is
> >> violated is a must-have.
> >>
> >> вс, 10 февр. 2019 г. в 01:58, oignatenko :
> >>>
> >>> Hi Maxim,
> >>>
> >>> I believe that whatever style checks we establish at Teamcity, we
> better
> >>> take care of making it easy for developers to find and fix violations
> in
> >>> their typical dev environment (for Ignite this means, in IDEA). I think
> >> it
> >>> is important that developers can maintain required style with minimal
> >> effort
> >>> on their side.
> >>>
> >>> If above is doable then I am 200% for migrating our Teamcity
> inspections
> >> to
> >>> checkstyle / maven.
> >>>
> >>> This is because I am very disappointed observing how it stays broken
> for
> >> so
> >>> long. And worst of all, even when (if) it is fixed, I feel we will
> >> always be
> >>> at risk that it breaks again and that we will have to again wait for
> >> months
> >>> for it to be fixed.
> >>>
> >>> This is such a stark contrast with my experience regarding checkstyle
> >> based
> >>> inspections. These just work and you just never fear that it is going
> to
> >>> break for some obscure reason, this is so much better than what I
> observe
> >>> now.
> >>>
> >>> One suggestion in case if we pick checkstyle - I recommend keeping its
> >>> config file somewhere in the project under version control. I used to
> >>> maintain such a shared style config at one of past jobs and after some
> >>> experimenting it turned out most convenient to have it this way - so
> that
> >>> developers could easily assess and discuss style settings and keep
> track
> >> of
> >>> changes in these. (note how Kafka folks from your link [5] appear to be
> >>> doing it this way)
> >>>
> >>> regards, Oleg
> >>>
> >>>
> >>> Mmuzaf wrote
> Igniters,
>
> I've found that some of the community members have faced with
> `[Inspections] Core suite [1]` is not working well enough on TC. The
> suite has a `FAILED` status for more than 2 months due to some issues
> in TeamCity application [2]. Current suite behaviour confuses not only
> new contributors but also other community members. Moreover, this
> suite is no longer checks rules we previously configured. For
> instance, in the master branch, I've found 11 `Unused imports` which
> should have been caught earlier (e.g. for
> {{IgniteCachePutAllRestartTest} [3]).
>
> I think we should make the next step to enable an automatic code style
> checks. As an example, we can consider the Apache Kafka code style [5]
> way and configure for the Ignite project a maven-checkstyle-plugin
> with its own maven profile and run it simultaneously with other TC. We
> can also enable the previously configured inspection rules, so no
> coding style violations will be missed.
>
> I see some advantages of using a maven plugin:
> - an IDE agnostic way for code checks
> - can be used with different CI and build tools (Jenkins, TC)
> - executable from the command line
> - the entry single point to configure new rules
>
> I've created the ticket [4] and will prepare PR for it.
>
> WDYT?
>
> [1]
>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> [2] https://youtrack.jetbrains.com/issue/TW-58504
> [3]
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCachePutAllRestartTest.java#L29
> [4] https://issues.apache.org/jira/browse/IGNITE-11277
> [5] https://github.com/apache/kafka/tree/trunk/checkstyle
>
> On Fri, 21 Dec 2018 at 16:03, Petr Ivanov
Re: Running single test multiple times on TeamCity
Vlad, Your case is valid as well. And DebugSuite will not help there. But I talk about another case. And I have already reproduced several tests after repetitive execution of a single test on TC. So, my case also seems valid. пн, 11 февр. 2019 г. в 10:43, Vladislav Pyatkov : > > Hi, > > I think more test falling on TC due to context in which running. > If you added it in debug suite, it can stop failing anymore like local run. > > On Mon, Feb 11, 2019 at 9:37 AM Павлухин Иван wrote: > > > Hi, > > > > During a couple of last weeks I was fixing several flaky tests. > > Sometimes it was quite hard to reproduce a test locally. So, one > > option was running a particular test on TC several times in a row. To > > setup such run I did code modifications in several places. > > > > I thought about how to simplify the thing. And I came up with some > > sort of solution which I would like to share. Basically it is custom > > junit runner DebugSuite and a configuration annotation > > DebugSuite.Config which allows to choose a method to run and number of > > executions. You can see a draft in PR [1]. > > > > As always there are several options to solve a problem. One > > alternative way is creating something similar to parameterized build > > job Jenkins employs [2] (I have not checked for TC analog yet) and > > using maven features to run single test repeatedly (have not checked > > as well). But all in all we need to answer following questions: > > 1. Do we need such tool? (Or perhaps we already have something and > > there is no need to reinvent the wheel.) > > 2. What is the best way for us to implement the tool? > > > > [1] https://github.com/apache/ignite/pull/6076 > > [2] https://wiki.jenkins.io/display/JENKINS/Parameterized+Build > > > > -- > > Best regards, > > Ivan Pavlukhin > > > > > -- > Vladislav Pyatkov -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-11284) Web console: Actualize grid configuration: query entity and Pojo store
Vasiliy Sisko created IGNITE-11284: -- Summary: Web console: Actualize grid configuration: query entity and Pojo store Key: IGNITE-11284 URL: https://issues.apache.org/jira/browse/IGNITE-11284 Project: Ignite Issue Type: Improvement Components: wizards Environment: Result for class: org.apache.ignite.cache.QueryEntity Missed fieldsPrecision notNullFields fieldsScale defaultFieldValues Result for class: org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory Missed types Result for class: org.apache.ignite.cache.QueryIndex Missed inlineSize Reporter: Vasiliy Sisko Assignee: Vasiliy Sisko -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Code inspection
Is build configuration Inspections [Core] meant to transform into single
all-modules check build configuration (without module subdivision)?
> On 11 Feb 2019, at 11:02, Nikolay Izhikov wrote:
>
> Hello, Maxim.
>
> +1 from me for migrating to checkstyle.
>
> Oleg, there is plugin for IDEA with 2mln downloads -
> https://plugins.jetbrains.com/plugin/1065-checkstyle-idea
>
> I propose do the following:
>
> 1. Migrate current checks to checkstyle.
> 2. Apply checks to all Ignite modules. Currently, only core module are
> checked.
> I will review and commit this patch, or do it by my own.
>
> 3. Include code style checks to "Build Apache Ignite" suite. Ignite has to
> fail to build if patch violates codestyle.
>
> вс, 10 февр. 2019 г. в 07:54, Павлухин Иван :
>
>> Hi,
>>
>> I also think that some warning from IDEA that some code style rule is
>> violated is a must-have.
>>
>> вс, 10 февр. 2019 г. в 01:58, oignatenko :
>>>
>>> Hi Maxim,
>>>
>>> I believe that whatever style checks we establish at Teamcity, we better
>>> take care of making it easy for developers to find and fix violations in
>>> their typical dev environment (for Ignite this means, in IDEA). I think
>> it
>>> is important that developers can maintain required style with minimal
>> effort
>>> on their side.
>>>
>>> If above is doable then I am 200% for migrating our Teamcity inspections
>> to
>>> checkstyle / maven.
>>>
>>> This is because I am very disappointed observing how it stays broken for
>> so
>>> long. And worst of all, even when (if) it is fixed, I feel we will
>> always be
>>> at risk that it breaks again and that we will have to again wait for
>> months
>>> for it to be fixed.
>>>
>>> This is such a stark contrast with my experience regarding checkstyle
>> based
>>> inspections. These just work and you just never fear that it is going to
>>> break for some obscure reason, this is so much better than what I observe
>>> now.
>>>
>>> One suggestion in case if we pick checkstyle - I recommend keeping its
>>> config file somewhere in the project under version control. I used to
>>> maintain such a shared style config at one of past jobs and after some
>>> experimenting it turned out most convenient to have it this way - so that
>>> developers could easily assess and discuss style settings and keep track
>> of
>>> changes in these. (note how Kafka folks from your link [5] appear to be
>>> doing it this way)
>>>
>>> regards, Oleg
>>>
>>>
>>> Mmuzaf wrote
Igniters,
I've found that some of the community members have faced with
`[Inspections] Core suite [1]` is not working well enough on TC. The
suite has a `FAILED` status for more than 2 months due to some issues
in TeamCity application [2]. Current suite behaviour confuses not only
new contributors but also other community members. Moreover, this
suite is no longer checks rules we previously configured. For
instance, in the master branch, I've found 11 `Unused imports` which
should have been caught earlier (e.g. for
{{IgniteCachePutAllRestartTest} [3]).
I think we should make the next step to enable an automatic code style
checks. As an example, we can consider the Apache Kafka code style [5]
way and configure for the Ignite project a maven-checkstyle-plugin
with its own maven profile and run it simultaneously with other TC. We
can also enable the previously configured inspection rules, so no
coding style violations will be missed.
I see some advantages of using a maven plugin:
- an IDE agnostic way for code checks
- can be used with different CI and build tools (Jenkins, TC)
- executable from the command line
- the entry single point to configure new rules
I've created the ticket [4] and will prepare PR for it.
WDYT?
[1]
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
[2] https://youtrack.jetbrains.com/issue/TW-58504
[3]
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCachePutAllRestartTest.java#L29
[4] https://issues.apache.org/jira/browse/IGNITE-11277
[5] https://github.com/apache/kafka/tree/trunk/checkstyle
On Fri, 21 Dec 2018 at 16:03, Petr Ivanov
>>>
mr.weider@
>>>
wrote:
>
> It seems there is bug in latest 2018.2 TeamCity
> Bug is filed [1]
>
>
> [1] https://youtrack.jetbrains.com/issue/TW-58504
>
>> On 19 Dec 2018, at 11:31, Petr Ivanov
>>>
mr.weider@
>>>
wrote:
>>
>> Investigating problem, stand by.
>>
>>
>>> On 18 Dec 2018, at 19:41, Dmitriy Pavlov
>>>
dpavlov@
>>>
wrote:
>>>
>>> Both patches were applied. Maxim, thank you!
>>>
>>> What about 1. An `Unexpected error during build
Make the TeamCity console quiet.
Igniters, When I was working with flaky tests, I was surprised that one of the reasons for failure was a log appender blocking the console. I suppose it happened because of a TeamCity agent communicating with java out through Linux pipe. This is a problem for tests with 1GB log history. Our test by default writes logs into 2 sources, such as the console and the file. I'm going to change mode for the console at IGNITE_QUIET=true and write only warn and error logs in it. Also, I’ll include a diagnostic log into it. You will be able to read the previous log from the Artifacts tab in your build. This is a solution which has advantages: 1. First of all, we’ll cut down the space usage pre-test run by ~6-7 times. For example - it's 1GB of saved space for 1 suit like "Cache (Restarts) 1". 2. We’ll be able to use a new space for long history storage for builds. 3. The TeamCity bot won't be lagging when it parses a large file. 4. Possibly some of tests won't be flaky. 5. And finally, the time of running my suite will be cut down by 5-7 minutes. Also, I'm going to make a workaround for previous behavior. For example, this flag will be added to build params. But I should investigate it.
Re: Code inspection
Hello, Maxim.
+1 from me for migrating to checkstyle.
Oleg, there is plugin for IDEA with 2mln downloads -
https://plugins.jetbrains.com/plugin/1065-checkstyle-idea
I propose do the following:
1. Migrate current checks to checkstyle.
2. Apply checks to all Ignite modules. Currently, only core module are
checked.
I will review and commit this patch, or do it by my own.
3. Include code style checks to "Build Apache Ignite" suite. Ignite has to
fail to build if patch violates codestyle.
вс, 10 февр. 2019 г. в 07:54, Павлухин Иван :
> Hi,
>
> I also think that some warning from IDEA that some code style rule is
> violated is a must-have.
>
> вс, 10 февр. 2019 г. в 01:58, oignatenko :
> >
> > Hi Maxim,
> >
> > I believe that whatever style checks we establish at Teamcity, we better
> > take care of making it easy for developers to find and fix violations in
> > their typical dev environment (for Ignite this means, in IDEA). I think
> it
> > is important that developers can maintain required style with minimal
> effort
> > on their side.
> >
> > If above is doable then I am 200% for migrating our Teamcity inspections
> to
> > checkstyle / maven.
> >
> > This is because I am very disappointed observing how it stays broken for
> so
> > long. And worst of all, even when (if) it is fixed, I feel we will
> always be
> > at risk that it breaks again and that we will have to again wait for
> months
> > for it to be fixed.
> >
> > This is such a stark contrast with my experience regarding checkstyle
> based
> > inspections. These just work and you just never fear that it is going to
> > break for some obscure reason, this is so much better than what I observe
> > now.
> >
> > One suggestion in case if we pick checkstyle - I recommend keeping its
> > config file somewhere in the project under version control. I used to
> > maintain such a shared style config at one of past jobs and after some
> > experimenting it turned out most convenient to have it this way - so that
> > developers could easily assess and discuss style settings and keep track
> of
> > changes in these. (note how Kafka folks from your link [5] appear to be
> > doing it this way)
> >
> > regards, Oleg
> >
> >
> > Mmuzaf wrote
> > > Igniters,
> > >
> > > I've found that some of the community members have faced with
> > > `[Inspections] Core suite [1]` is not working well enough on TC. The
> > > suite has a `FAILED` status for more than 2 months due to some issues
> > > in TeamCity application [2]. Current suite behaviour confuses not only
> > > new contributors but also other community members. Moreover, this
> > > suite is no longer checks rules we previously configured. For
> > > instance, in the master branch, I've found 11 `Unused imports` which
> > > should have been caught earlier (e.g. for
> > > {{IgniteCachePutAllRestartTest} [3]).
> > >
> > > I think we should make the next step to enable an automatic code style
> > > checks. As an example, we can consider the Apache Kafka code style [5]
> > > way and configure for the Ignite project a maven-checkstyle-plugin
> > > with its own maven profile and run it simultaneously with other TC. We
> > > can also enable the previously configured inspection rules, so no
> > > coding style violations will be missed.
> > >
> > > I see some advantages of using a maven plugin:
> > > - an IDE agnostic way for code checks
> > > - can be used with different CI and build tools (Jenkins, TC)
> > > - executable from the command line
> > > - the entry single point to configure new rules
> > >
> > > I've created the ticket [4] and will prepare PR for it.
> > >
> > > WDYT?
> > >
> > > [1]
> > >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_InspectionsCore_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv
> > > [2] https://youtrack.jetbrains.com/issue/TW-58504
> > > [3]
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCachePutAllRestartTest.java#L29
> > > [4] https://issues.apache.org/jira/browse/IGNITE-11277
> > > [5] https://github.com/apache/kafka/tree/trunk/checkstyle
> > >
> > > On Fri, 21 Dec 2018 at 16:03, Petr Ivanov
> >
> > > mr.weider@
> >
> > > wrote:
> > >>
> > >> It seems there is bug in latest 2018.2 TeamCity
> > >> Bug is filed [1]
> > >>
> > >>
> > >> [1] https://youtrack.jetbrains.com/issue/TW-58504
> > >>
> > >> > On 19 Dec 2018, at 11:31, Petr Ivanov
> >
> > > mr.weider@
> >
> > > wrote:
> > >> >
> > >> > Investigating problem, stand by.
> > >> >
> > >> >
> > >> >> On 18 Dec 2018, at 19:41, Dmitriy Pavlov
> >
> > > dpavlov@
> >
> > > wrote:
> > >> >>
> > >> >> Both patches were applied. Maxim, thank you!
> > >> >>
> > >> >> What about 1. An `Unexpected error during build messages
> processing in
> > >> >> TeamCity`, what can we do as the next step to fix it?
> > >> >>
> > >> >> Sincerely,
> > >> >> Dmitriy Pavlov
> > >> >>[cut]
> > >> >
> > >>
> >
> >
> >
> >
> >
> > --
> > Sent from:
