[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041172#comment-16041172
]
Jean-Daniel Cryans commented on KUDU-2034:
--
bq. The kudu-jepsen setup has been running with no errors for a long time
already.
This isn't something that's running on ASF infra and the results aren't
published either, right? It'd be nice if we could make the above claim and also
back it.
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> The kudu-jepsen setup has been running with no errors for a long time
> already. Currently, the set of parameters for the back-end components is
> standard -- everything is set to defaults, but the experimental flags are
> enabled. Yes, it made sense to run with the default parameters: originally
> we wanted to make sure the test did not fail with production-alike settings.
> Now we can start exercising 'artificially congested' scenarios.
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043206#comment-16043206
]
Alexey Serbin commented on KUDU-2034:
-
Yup, I was referring to the kudu-jepsen running at the private internal infra,
not on ASF infra. The results of those runs are not published elsewhere.
Thank you for pointing at that -- I'll remove the first statement from the
description.
However, since the kudu-jepsen's run-time flags for the master and tablet
servers is a part of the upstream project, I think it's worth keeping this item
in this JIRA system.
If there is a need to start running the kudu-jepsen on ASF infra, I think it's
worth opening a separate task for that.
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> The kudu-jepsen setup has been running with no errors for a long time
> already. Currently, the set of parameters for the back-end components is
> standard -- everything is set to defaults, but the experimental flags are
> enabled. Yes, it made sense to run with the default parameters: originally
> we wanted to make sure the test did not fail with production-alike settings.
> Now we can start exercising 'artificially congested' scenarios.
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043212#comment-16043212
]
Jean-Daniel Cryans commented on KUDU-2034:
--
bq. The results of those runs are not published elsewhere. Thank you for
pointing at that – I'll remove the first statement from the description.
No need to edit, I was merely expressing a desire to see those results
somewhere public
bq. If there is a need to start running the kudu-jepsen on ASF infra, I think
it's worth opening a separate task for that.
I think we should have a jira about setting this up, yeah. Having this in the
open would inspire more confidence in Kudu's testing.
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> The kudu-jepsen setup has been running with no errors for a long time
> already. Currently, the set of parameters for the back-end components is
> standard -- everything is set to defaults, but the experimental flags are
> enabled. Yes, it made sense to run with the default parameters: originally
> we wanted to make sure the test did not fail with production-alike settings.
> Now we can start exercising 'artificially congested' scenarios.
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139423#comment-16139423
]
Hrishikesh Gadre commented on KUDU-2034:
[~aserbin] Are you working on this task? If not I would love to take a stab at
it.
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> Currently, the set of parameters for the back-end components is almost
> standard -- everything is set to defaults, but the experimental flags are
> enabled and only the following tserver flags are set to something 'special'
> to speed up the test:
> {noformat}
> --heartbeat_interval_ms
> --raft_heartbeat_interval_ms
> --heartbeat_max_failures_before_backoff
> {noformat}
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139435#comment-16139435
]
Alexey Serbin commented on KUDU-2034:
-
[~hgadre] nope, I'm not currently working on this. Please feel free to start
working on this. Do you want me to assign this JIRA item to you?
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> Currently, the set of parameters for the back-end components is almost
> standard -- everything is set to defaults, but the experimental flags are
> enabled and only the following tserver flags are set to something 'special'
> to speed up the test:
> {noformat}
> --heartbeat_interval_ms
> --raft_heartbeat_interval_ms
> --heartbeat_max_failures_before_backoff
> {noformat}
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KUDU-2034) For kudu-jepsen scenario, add parameters to induce more frequent fail-over events on the server side
[
https://issues.apache.org/jira/browse/KUDU-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139525#comment-16139525
]
Hrishikesh Gadre commented on KUDU-2034:
[~aserbin] Yes that would be great.
> For kudu-jepsen scenario, add parameters to induce more frequent fail-over
> events on the server side
>
>
> Key: KUDU-2034
> URL: https://issues.apache.org/jira/browse/KUDU-2034
> Project: Kudu
> Issue Type: Task
>Affects Versions: 1.4.0
>Reporter: Alexey Serbin
> Labels: consistency, kudu-jepsen, newbie
>
> Currently, the set of parameters for the back-end components is almost
> standard -- everything is set to defaults, but the experimental flags are
> enabled and only the following tserver flags are set to something 'special'
> to speed up the test:
> {noformat}
> --heartbeat_interval_ms
> --raft_heartbeat_interval_ms
> --heartbeat_max_failures_before_backoff
> {noformat}
> Let's set parameters to induce frequent re-election/fail-over events at the
> server side. The idea is to bring in set of parameters used by certain tests
> in {{src/kudu/integration-tests}}. E.g., {{catalog_manager_tsk-itest.cc}}
> contains an example of parameters to enable frequent re-elections among
> masters; the raft-related part can be used as a starting point to update
> tserver's parameters for kudu-jepsen.
> An additional step might be injecting latency into tservers' operations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
