Re: Cassandra build failing after Central Repository HTTPS
I think I've found the source of the issue. We are using assert-core on our fork of the repository, which is attempting to pull in junit-5.4.0. junit-5.4.0 doesn't seem to be published on either of the https repositories, so ant tries to hit the http as a backup and then fails with the error below. It's curious that Junit was published to the http but not https repository. Either way, thanks for the assistance in debugging! On Wed, Jan 15, 2020 at 8:19 PM Leon Zaruvinsky wrote: > I agree that something feels wonky on the Circle container. I was able to > successfully build locally. > > I SSHed into the container, cleared out .m2/repository and still can't get > it to build. Using ant 1.10.7 with environment: > > openjdk version "1.8.0_222" > OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1~deb9u1-b10) > OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode) > > On 1/15/20, 7:14 PM, "Michael Shuler" wrote: > > I just did a quick `rm -r ~/.m2/repository/` so the build would > download > everything, and with 2.2.14 tag + 63ff65a8dd, the build succeeded for > me > fine, pulling everything from https. Not sure where to go, unless the > circleci container is somehow contaminated with http cached data. > > Michael > > On 1/15/20 6:05 PM, Michael Shuler wrote: > > Bleh, sorry, you updated those, right. > > > > I don't see any other related commits to build.xml nor > > build.properties.default.. Some elderly sub-dependent that has an > http > > URL in it, perhaps? > > > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > > >
Re: Cassandra build failing after Central Repository HTTPS
I agree that something feels wonky on the Circle container. I was able to successfully build locally. I SSHed into the container, cleared out .m2/repository and still can't get it to build. Using ant 1.10.7 with environment: openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1~deb9u1-b10) OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode) On 1/15/20, 7:14 PM, "Michael Shuler" wrote: I just did a quick `rm -r ~/.m2/repository/` so the build would download everything, and with 2.2.14 tag + 63ff65a8dd, the build succeeded for me fine, pulling everything from https. Not sure where to go, unless the circleci container is somehow contaminated with http cached data. Michael On 1/15/20 6:05 PM, Michael Shuler wrote: > Bleh, sorry, you updated those, right. > > I don't see any other related commits to build.xml nor > build.properties.default.. Some elderly sub-dependent that has an http > URL in it, perhaps? > - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra build failing after Central Repository HTTPS
Hey Michael, That’s basically what I did, but still getting the error unfortunately. The first artifact does seem to recognize to use https, but the transitive dependency does not. Thanks > On Jan 15, 2020, at 7:00 PM, Michael Shuler wrote: > > You could cherry-pick sha 63ff65a8dd3a31e500ae5ec6232f1f9eade6fa3d which was > committed after the 2.2.14 release tag. > > https://github.com/apache/cassandra/commit/63ff65a8dd3a31e500ae5ec6232f1f9eade6fa3d > > -- > Kind regards, > Michael > >> On 1/15/20 5:44 PM, Leon Zaruvinsky wrote: >> Hey all, >> I'm having trouble with the building Cassandra 2.2.14 on CircleCI since >> Central Repo has stopped supporting http.* >> * https://central.sonatype.org/articles/2020/Jan/15/501-https-required-error/ >> I've updated the build.properties and build.xml files to use https. >> However, it seems that the ant build starts to use https and then switches >> to http after a few artifacts: >> maven-ant-tasks-retrieve-build: [artifact:dependencies] Downloading: >> junit/junit/4.6/junit-4.6.pom from repository central at >> https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 1K from >> central [artifact:dependencies] Downloading: >> org/assertj/assertj-core/3.12.0/assertj-core-3.12.0.pom from repository >> central at https://repo1.maven.org/maven2 [artifact:dependencies] >> Transferring 13K from central [artifact:dependencies] Downloading: >> org/assertj/assertj-parent-pom/2.2.2/assertj-parent-pom-2.2.2.pom from >> repository central at https://repo1.maven.org/maven2 [artifact:dependencies] >> Transferring 16K from central [artifact:dependencies] Downloading: >> org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom from repository central at >> http://repo1.maven.org/maven2 [artifact:dependencies] Error transferring >> file: Server returned HTTP response code: 501 for URL: >> http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom >> [artifact:dependencies] [WARNING] Unable to get resource >> 'org.junit:junit-bom:pom:5.4.0' from repository central >> (http://repo1.maven.org/maven2): Error transferring file: Server returned >> HTTP response code: 501 for URL: >> http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom >> [artifact:dependencies] An error has occurred while processing the Maven >> artifact tasks. [artifact:dependencies] Diagnosis: [artifact:dependencies] >> [artifact:dependencies] Unable to resolve artifact: Unable to get dependency >> information: Unable to read the metadata file for artifact >> 'org.assertj:assertj-core:jar': POM 'org.junit:junit-bom' not found in >> repository: Unable to download the artifact from any repository >> [artifact:dependencies] [artifact:dependencies] >> org.junit:junit-bom:pom:5.4.0 [artifact:dependencies] >> [artifact:dependencies] from the specified remote repositories: >> [artifact:dependencies] central (http://repo1.maven.org/maven2) >> [artifact:dependencies] [artifact:dependencies] for project >> org.junit:junit-bom [artifact:dependencies] >> org.assertj:assertj-core:jar:3.12.0 [artifact:dependencies] >> [artifact:dependencies] from the specified remote repositories: >> [artifact:dependencies] apache (https://repo.maven.apache.org/maven2), >> [artifact:dependencies] central (https://repo1.maven.org/maven2) >> [artifact:dependencies] [artifact:dependencies] Path to dependency: >> [artifact:dependencies] 1) >> org.apache.cassandra:cassandra-build-deps:jar:2.2.14-SNAPSHOT >> [artifact:dependencies] [artifact:dependencies] >> There are no references to http://repo1.maven.org/maven2 anywhere in my >> repo. One theory is that this reference is being automatically injected >> somewhere, but I'm not sure where or how to stop it. >> Has anyone else encountered this or has suggestions for how to fix? >> Thanks, >> Leon > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra build failing after Central Repository HTTPS
I just did a quick `rm -r ~/.m2/repository/` so the build would download everything, and with 2.2.14 tag + 63ff65a8dd, the build succeeded for me fine, pulling everything from https. Not sure where to go, unless the circleci container is somehow contaminated with http cached data. Michael On 1/15/20 6:05 PM, Michael Shuler wrote: Bleh, sorry, you updated those, right. I don't see any other related commits to build.xml nor build.properties.default.. Some elderly sub-dependent that has an http URL in it, perhaps? - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra build failing after Central Repository HTTPS
Bleh, sorry, you updated those, right. I don't see any other related commits to build.xml nor build.properties.default.. Some elderly sub-dependent that has an http URL in it, perhaps? -- Michael On 1/15/20 5:44 PM, Leon Zaruvinsky wrote: Hey all, I'm having trouble with the building Cassandra 2.2.14 on CircleCI since Central Repo has stopped supporting http.* * https://central.sonatype.org/articles/2020/Jan/15/501-https-required-error/ I've updated the build.properties and build.xml files to use https. However, it seems that the ant build starts to use https and then switches to http after a few artifacts: maven-ant-tasks-retrieve-build: [artifact:dependencies] Downloading: junit/junit/4.6/junit-4.6.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 1K from central [artifact:dependencies] Downloading: org/assertj/assertj-core/3.12.0/assertj-core-3.12.0.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 13K from central [artifact:dependencies] Downloading: org/assertj/assertj-parent-pom/2.2.2/assertj-parent-pom-2.2.2.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 16K from central [artifact:dependencies] Downloading: org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom from repository central at http://repo1.maven.org/maven2 [artifact:dependencies] Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] [WARNING] Unable to get resource 'org.junit:junit-bom:pom:5.4.0' from repository central (http://repo1.maven.org/maven2): Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] An error has occurred while processing the Maven artifact tasks. [artifact:dependencies] Diagnosis: [artifact:dependencies] [artifact:dependencies] Unable to resolve artifact: Unable to get dependency information: Unable to read the metadata file for artifact 'org.assertj:assertj-core:jar': POM 'org.junit:junit-bom' not found in repository: Unable to download the artifact from any repository [artifact:dependencies] [artifact:dependencies] org.junit:junit-bom:pom:5.4.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] central (http://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] for project org.junit:junit-bom [artifact:dependencies] org.assertj:assertj-core:jar:3.12.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] apache (https://repo.maven.apache.org/maven2), [artifact:dependencies] central (https://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] Path to dependency: [artifact:dependencies] 1) org.apache.cassandra:cassandra-build-deps:jar:2.2.14-SNAPSHOT [artifact:dependencies] [artifact:dependencies] There are no references to http://repo1.maven.org/maven2 anywhere in my repo. One theory is that this reference is being automatically injected somewhere, but I'm not sure where or how to stop it. Has anyone else encountered this or has suggestions for how to fix? Thanks, Leon - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: Cassandra build failing after Central Repository HTTPS
You could cherry-pick sha 63ff65a8dd3a31e500ae5ec6232f1f9eade6fa3d which was committed after the 2.2.14 release tag. https://github.com/apache/cassandra/commit/63ff65a8dd3a31e500ae5ec6232f1f9eade6fa3d -- Kind regards, Michael On 1/15/20 5:44 PM, Leon Zaruvinsky wrote: Hey all, I'm having trouble with the building Cassandra 2.2.14 on CircleCI since Central Repo has stopped supporting http.* * https://central.sonatype.org/articles/2020/Jan/15/501-https-required-error/ I've updated the build.properties and build.xml files to use https. However, it seems that the ant build starts to use https and then switches to http after a few artifacts: maven-ant-tasks-retrieve-build: [artifact:dependencies] Downloading: junit/junit/4.6/junit-4.6.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 1K from central [artifact:dependencies] Downloading: org/assertj/assertj-core/3.12.0/assertj-core-3.12.0.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 13K from central [artifact:dependencies] Downloading: org/assertj/assertj-parent-pom/2.2.2/assertj-parent-pom-2.2.2.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 16K from central [artifact:dependencies] Downloading: org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom from repository central at http://repo1.maven.org/maven2 [artifact:dependencies] Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] [WARNING] Unable to get resource 'org.junit:junit-bom:pom:5.4.0' from repository central (http://repo1.maven.org/maven2): Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] An error has occurred while processing the Maven artifact tasks. [artifact:dependencies] Diagnosis: [artifact:dependencies] [artifact:dependencies] Unable to resolve artifact: Unable to get dependency information: Unable to read the metadata file for artifact 'org.assertj:assertj-core:jar': POM 'org.junit:junit-bom' not found in repository: Unable to download the artifact from any repository [artifact:dependencies] [artifact:dependencies] org.junit:junit-bom:pom:5.4.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] central (http://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] for project org.junit:junit-bom [artifact:dependencies] org.assertj:assertj-core:jar:3.12.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] apache (https://repo.maven.apache.org/maven2), [artifact:dependencies] central (https://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] Path to dependency: [artifact:dependencies] 1) org.apache.cassandra:cassandra-build-deps:jar:2.2.14-SNAPSHOT [artifact:dependencies] [artifact:dependencies] There are no references to http://repo1.maven.org/maven2 anywhere in my repo. One theory is that this reference is being automatically injected somewhere, but I'm not sure where or how to stop it. Has anyone else encountered this or has suggestions for how to fix? Thanks, Leon - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Cassandra build failing after Central Repository HTTPS
Hey all, I'm having trouble with the building Cassandra 2.2.14 on CircleCI since Central Repo has stopped supporting http.* * https://central.sonatype.org/articles/2020/Jan/15/501-https-required-error/ I've updated the build.properties and build.xml files to use https. However, it seems that the ant build starts to use https and then switches to http after a few artifacts: maven-ant-tasks-retrieve-build: [artifact:dependencies] Downloading: junit/junit/4.6/junit-4.6.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 1K from central [artifact:dependencies] Downloading: org/assertj/assertj-core/3.12.0/assertj-core-3.12.0.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 13K from central [artifact:dependencies] Downloading: org/assertj/assertj-parent-pom/2.2.2/assertj-parent-pom-2.2.2.pom from repository central at https://repo1.maven.org/maven2 [artifact:dependencies] Transferring 16K from central [artifact:dependencies] Downloading: org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom from repository central at http://repo1.maven.org/maven2 [artifact:dependencies] Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] [WARNING] Unable to get resource 'org.junit:junit-bom:pom:5.4.0' from repository central (http://repo1.maven.org/maven2): Error transferring file: Server returned HTTP response code: 501 for URL: http://repo1.maven.org/maven2/org/junit/junit-bom/5.4.0/junit-bom-5.4.0.pom [artifact:dependencies] An error has occurred while processing the Maven artifact tasks. [artifact:dependencies] Diagnosis: [artifact:dependencies] [artifact:dependencies] Unable to resolve artifact: Unable to get dependency information: Unable to read the metadata file for artifact 'org.assertj:assertj-core:jar': POM 'org.junit:junit-bom' not found in repository: Unable to download the artifact from any repository [artifact:dependencies] [artifact:dependencies] org.junit:junit-bom:pom:5.4.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] central (http://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] for project org.junit:junit-bom [artifact:dependencies] org.assertj:assertj-core:jar:3.12.0 [artifact:dependencies] [artifact:dependencies] from the specified remote repositories: [artifact:dependencies] apache (https://repo.maven.apache.org/maven2), [artifact:dependencies] central (https://repo1.maven.org/maven2) [artifact:dependencies] [artifact:dependencies] Path to dependency: [artifact:dependencies] 1) org.apache.cassandra:cassandra-build-deps:jar:2.2.14-SNAPSHOT [artifact:dependencies] [artifact:dependencies] There are no references to http://repo1.maven.org/maven2 anywhere in my repo. One theory is that this reference is being automatically injected somewhere, but I'm not sure where or how to stop it. Has anyone else encountered this or has suggestions for how to fix? Thanks, Leon
Re: inter dc bandwidth calculation
Thank you. I have an insight now. Regards, Osman On Wed, 15 Jan 2020 at 19:18, Reid Pinchback wrote: > > Oh, duh. Revise that. I was forgetting that multi-dc writes are sent to a > single node in the other dc and tagged to be forwarded to other nodes within > the dc. > > So your quick-and-dirty estimate would be more like (write volume) x 2 to > leave headroom for random other mechanics. > > R > > > On 1/15/20, 11:07 AM, "Reid Pinchback" wrote: > > Message from External Sender > > I would think that it would be largely driven by the replication factor. > It isn't that the sstables are forklifted from one dc to another, it's just > that the writes being made to the memtables are also shipped around by the > coordinator nodes as the writes happen. Operations at the sstable level, > like compactions, are local to the node. > > One potential wrinkle that I'm unclear on, is related to repairs. I > don't know if merkle trees are biased to mostly bounce around only intra-dc, > versus how often they are communicated inter-dc. Note that even queries can > trigger some degree of repair traffic if you have a usage pattern of trying > to read data recently written, because at the bleeding edge of the recent > changes you'll have more cases of rows not having had time to settle to a > consistent state. > > If you want a quick-and-dirty heuristic, I'd probably take (write volume) > x (replication factor) x 2 as a guestimate so you have some headroom for C* > and TCP mechanics, but then monitor to see what your real use is. > > R > > > On 1/15/20, 4:14 AM, "Osman Yozgatlıoğlu" > wrote: > > Message from External Sender > > Hello, > > Is there any way to calculate inter dc bandwidth requirements for > proper operation? > I can't find any info about this subject. > Can we say, how much sstable collected at one dc has to be > transferred to other? > I can calculate bandwidth with generated sstable then. > I have twcs with one hour window. > > Regards, > Osman > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > > > > - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: inter dc bandwidth calculation
Oh, duh. Revise that. I was forgetting that multi-dc writes are sent to a single node in the other dc and tagged to be forwarded to other nodes within the dc. So your quick-and-dirty estimate would be more like (write volume) x 2 to leave headroom for random other mechanics. R On 1/15/20, 11:07 AM, "Reid Pinchback" wrote: Message from External Sender I would think that it would be largely driven by the replication factor. It isn't that the sstables are forklifted from one dc to another, it's just that the writes being made to the memtables are also shipped around by the coordinator nodes as the writes happen. Operations at the sstable level, like compactions, are local to the node. One potential wrinkle that I'm unclear on, is related to repairs. I don't know if merkle trees are biased to mostly bounce around only intra-dc, versus how often they are communicated inter-dc. Note that even queries can trigger some degree of repair traffic if you have a usage pattern of trying to read data recently written, because at the bleeding edge of the recent changes you'll have more cases of rows not having had time to settle to a consistent state. If you want a quick-and-dirty heuristic, I'd probably take (write volume) x (replication factor) x 2 as a guestimate so you have some headroom for C* and TCP mechanics, but then monitor to see what your real use is. R On 1/15/20, 4:14 AM, "Osman Yozgatlıoğlu" wrote: Message from External Sender Hello, Is there any way to calculate inter dc bandwidth requirements for proper operation? I can't find any info about this subject. Can we say, how much sstable collected at one dc has to be transferred to other? I can calculate bandwidth with generated sstable then. I have twcs with one hour window. Regards, Osman - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
Re: inter dc bandwidth calculation
I would think that it would be largely driven by the replication factor. It isn't that the sstables are forklifted from one dc to another, it's just that the writes being made to the memtables are also shipped around by the coordinator nodes as the writes happen. Operations at the sstable level, like compactions, are local to the node. One potential wrinkle that I'm unclear on, is related to repairs. I don't know if merkle trees are biased to mostly bounce around only intra-dc, versus how often they are communicated inter-dc. Note that even queries can trigger some degree of repair traffic if you have a usage pattern of trying to read data recently written, because at the bleeding edge of the recent changes you'll have more cases of rows not having had time to settle to a consistent state. If you want a quick-and-dirty heuristic, I'd probably take (write volume) x (replication factor) x 2 as a guestimate so you have some headroom for C* and TCP mechanics, but then monitor to see what your real use is. R On 1/15/20, 4:14 AM, "Osman Yozgatlıoğlu" wrote: Message from External Sender Hello, Is there any way to calculate inter dc bandwidth requirements for proper operation? I can't find any info about this subject. Can we say, how much sstable collected at one dc has to be transferred to other? I can calculate bandwidth with generated sstable then. I have twcs with one hour window. Regards, Osman - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
inter dc bandwidth calculation
Hello, Is there any way to calculate inter dc bandwidth requirements for proper operation? I can't find any info about this subject. Can we say, how much sstable collected at one dc has to be transferred to other? I can calculate bandwidth with generated sstable then. I have twcs with one hour window. Regards, Osman - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
