[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147884#comment-17147884
]
Berenguer Blasi commented on CASSANDRA-15901:
-
Ok the ones failing on the hostname resolution have gone down from 20-ish to 3
and they look like snowflakes: unsing reflection and other non-std stuff. I'll
fix those tomorrow but looks like we're on track.
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147825#comment-17147825
]
Berenguer Blasi commented on CASSANDRA-15901:
-
Mmmm they seem to failing already but let it run to get an updated list of
failing tests. I'll start digging...
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147803#comment-17147803
]
Michael Semb Wever commented on CASSANDRA-15901:
bq. Do you think you could run the PR against cassandra13 and see if the issue
has been sorted?
Running at https://ci-cassandra.apache.org/job/Cassandra-devbranch-test/153/
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147668#comment-17147668
]
Berenguer Blasi commented on CASSANDRA-15901:
-
[~mck] CI
[runs|https://app.circleci.com/pipelines/github/bereng/cassandra/51/workflows/e5aeb0d3-9bc6-43d1-a45c-8c6f6c3412b1]
seem ok & tests pass locally as well, besides the OOM but circle was on a
partial outage today... Do you think you could run the PR against cassandra13
and see if the issue has been sorted?
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17146160#comment-17146160
]
Berenguer Blasi commented on CASSANDRA-15901:
-
After discussion with [~mck] it seems the underlying root cause is that some
tests don't init the daemon. Hence the config/listen_address are never loaded
and effective. We came up with the following course of action:
* Make all ci-cass agents fail consistently, so any new tests will get
reported.
* Fix all current tests to init the daemon correctly
* Once ci-cass is in good shape we can link the report for the commit into the
Jira ticket
* Explore ways to make that fail locally so you get the failure pre-push
rather than post-commit/ci-cass run
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17146133#comment-17146133
]
Michael Semb Wever commented on CASSANDRA-15901:
Back references:
- example failure stacktrace on
[ci-cassandra.a.o|https://ci-cassandra.apache.org/job/Cassandra-trunk/150/testReport/(root)/_init_/org_apache_cassandra_locator_ReplicaCollectionTest/]
- dev ML
[post|https://lists.apache.org/thread.html/r1a7bc49b0648ec3b4ab9245dc101dc7dfbec51048f83c7128e3989eb%40%3Cdev.cassandra.apache.org%3E]
asking for help
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17146130#comment-17146130
]
Michael Semb Wever commented on CASSANDRA-15901:
Quick interjection while i keep looking…
bq. If it does we'd need to run it against eth0 on all agents and see we indeed
get what we expect and tests run ok
Not all the agents are AWS. Some are virtuals on-premise (e.g. iland). In fact
I only know for certain that the Amazon agents are AWS :shrug:
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15901) Force jenkins tests to run on the private VPC IP
[
https://issues.apache.org/jira/browse/CASSANDRA-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17144997#comment-17144997
]
Berenguer Blasi commented on CASSANDRA-15901:
-
[~mck] This is the POC I came up with. I tested it on a local jenkins and I can
see it being effective:
{noformat}
16:56:29 forceDeviceListenAddress:
16:56:29 [exec] docker0: flags=4099 mtu 1500
16:56:29 [exec] inet 172.17.0.1 netmask 255.255.0.0 broadcast
172.17.255.255
16:56:29 [exec] inet6 fe80::42:26ff:fe25:c1e2 prefixlen 64
scopeid 0x20
16:56:29 [exec] ether 02:42:26:25:c1:e2 txqueuelen 0 (Ethernet)
16:56:29 [exec] RX packets 78645 bytes 5621831 (5.6 MB)
16:56:29 [exec] RX errors 0 dropped 0 overruns 0 frame 0
16:56:29 [exec] TX packets 85630 bytes 1329098560 (1.3 GB)
16:56:29 [exec] TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0
16:56:29 [exec]
16:56:29 [exec] lo: flags=73 mtu 65536
16:56:29 [exec] inet 127.0.0.1 netmask 255.0.0.0
16:56:29 [exec] inet6 ::1 prefixlen 128 scopeid 0x10
16:56:29 [exec] loop txqueuelen 1000 (Local Loopback)
16:56:29 [exec] RX packets 66296 bytes 21093416 (21.0 MB)
16:56:29 [exec] RX errors 0 dropped 0 overruns 0 frame 0
16:56:29 [exec] TX packets 66296 bytes 21093416 (21.0 MB)
16:56:29 [exec] TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0
16:56:29 [exec]
16:56:29 [exec] wlp59s0: flags=4163 mtu
1500
16:56:29 [exec] inet 192.168.1.131 netmask 255.255.255.0
broadcast 192.168.1.255
16:56:29 [exec] inet6 fe80::9c8e:fcad:d881:ffda prefixlen 64
scopeid 0x20
16:56:29 [exec] ether 18:1d:ea:b1:51:48 txqueuelen 1000
(Ethernet)
16:56:29 [exec] RX packets 6031123 bytes 7766582727 (7.7 GB)
16:56:29 [exec] RX errors 0 dropped 0 overruns 0 frame 0
16:56:29 [exec] TX packets 2398356 bytes 1840272042 (1.8 GB)
16:56:29 [exec] TX errors 0 dropped 0 overruns 0 carrier 0
collisions 0
16:56:29 [exec]
16:56:29 [echo] *** 192.168.1.131
{noformat}
You'll notice that a device is being force upon calling {{ifconfig}}. This is
bc many ips can match the regexp. On AWS the private ip is on eth0. So I'd
suggest this course of action:
- Take a look at what I did and see if it makes sense at all
- If it does we'd need to run it against {{eth0}} on all agents and see we
indeed get what we expect and tests run ok
wdyt?
> Force jenkins tests to run on the private VPC IP
>
>
> Key: CASSANDRA-15901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15901
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Many of the ci-cassandra jenkins runs fail on {{ip-10-0-5-5: Name or service
> not known}}. CASSANDRA-15622 addressed some of these but many still remain.
> Currently test C* nodes are either failing or listening on a public ip
> depending on which agent they end up.
> The idea behind this ticket is to make ant force the private VPC ip in the
> cassandra yaml when building, this will force the nodes to listen on the
> correct ip.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
