[
https://issues.apache.org/jira/browse/BEAM-4041?focusedWorklogId=103393&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-103393
]
ASF GitHub Bot logged work on BEAM-4041:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/May/18 14:40
Start Date: 18/May/18 14:40
Worklog Time Spent: 10m
Work Description: DariuszAniszewski opened a new pull request #5425:
[BEAM-4041] Increase timeout for getting K8s LoadBalancer external IP
URL: https://github.com/apache/beam/pull/5425
From some time we're observing that obtaining K8s LoadBalancer IP fails,
even with 3 minutes timeout. This came less often but still in this week (at
least) two jobs suffered this problem:
- `beam_PerformanceTests_MongoDBIO_IT` [build
168](https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_MongoDBIO_IT/168/console)
- `beam_PerformanceTests_HadoopInputFormat` [build
256](https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_HadoopInputFormat/256/console)
and [build
257](https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_HadoopInputFormat/257/console)
Since [PR 1641 to
PerfkiBenchmarker](https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1641)
it's now possible to configure retry-mechanism policy.
This PR increases overall timeout from 3 minutes to 6 minutes.
----------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 103393)
Time Spent: 10m
Remaining Estimate: 0h
> Performance tests fail due to kubernetes load balancer problems
> ---------------------------------------------------------------
>
> Key: BEAM-4041
> URL: https://issues.apache.org/jira/browse/BEAM-4041
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Łukasz Gajowy
> Assignee: Jason Kuster
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, as we added more IOITs to be run on jenkins using kubernetes, some
> of them started to fail randomly, because they couldn't retrieve LoadBalancer
> address. Normally obtaining the address took about one minute. Perfkit waits
> for the address (actively checking for it) for 3 minutes. This should be
> enough for getting the address, yet it recently started to exceed the 3
> minutes limit. I also noticed that this error didn't happen when there were
> fewer tests.
> Example logs:
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/31/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
