Re: [jclouds/jclouds-labs] JCLOUDS-1386 1&1 Baremetal servers (#431)
@andreaturli I have addressed the issues you mentioned, could you now please take a look? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-labs/pull/431#issuecomment-414971479
[jira] [Created] (JCLOUDS-1444) Outage on any endpoint in catalog blocks usage of other endpoints
Gilles Coremans created JCLOUDS-1444:
Summary: Outage on any endpoint in catalog blocks usage of other
endpoints
Key: JCLOUDS-1444
URL: https://issues.apache.org/jira/browse/JCLOUDS-1444
Project: jclouds
Issue Type: Bug
Components: jclouds-compute
Affects Versions: 2.1.1
Environment: OVH OpenStack
Reporter: Gilles Coremans
I was trying to create a template using the ComputeService backed by the nova
API (I was connecting to OVH, which does not have a jClouds provider). However,
one of their regions was down for maintenance. This caused the following error:
{code:java}
SEVERE: Cannot retry after server error, command has exceeded retry limit 5:
[method=org.jclouds.openstack.nova.v2_0.features.ImageApi.public abstract
org.jclouds.collect.PagedIterable
org.jclouds.openstack.nova.v2_0.features.ImageApi.listInDetail()[], request=GET
https://compute.bhs3.cloud.ovh.net/v2/[tenant]/images/detail HTTP/1.1{code}
I feel that jClouds shold handle this case gracefully and simply skip the
offending API, rather than failing with NullPointerExceptions and making it
impossible to create templates as long as any endpoint is down for whatever
reason.
This is the reply one gets when querying the given URL directly:
{code:java}
$ curl https://compute.bhs3.cloud.ovh.net/v2/[tenant]/images/detail
curl: (52) Empty reply from server
{code}
I verified on OVH's site that this region was indeed down for maintenance:
[http://travaux.ovh.net/?do=details&id=32795&PHPSESSID=b0747f889c42d0b97fedeb393cc16b0e|http://travaux.ovh.net/?do=details&id=32795&PHPSESSID=b0747f889c42d0b97fedeb393cc16b0e)]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request.
> +
+ final Image image =
imageInRegionToImage.apply(ImageInRegion.create(Regions.EU_CENTRAL_1.getName(),
ecsImage));
+ assertEquals(ecsImage.id(), image.getProviderId());
+ assertEquals(ecsImage.name(), image.getName());
+ assertEquals(Image.Status.AVAILABLE, image.getStatus());
+ final org.jclouds.compute.domain.OperatingSystem operatingSystem =
image.getOperatingSystem();
+
+ assertEquals(ecsImage.osName(), operatingSystem.getName());
+ assertEquals(ecsImage.description(), operatingSystem.getDescription());
+ assertTrue(operatingSystem.is64Bit());
+ assertEquals(region, image.getLocation());
+ }
+
+ Date parseDate(final String dateString) {
+ return DatatypeConverter.parseDateTime(dateString).getTime();
+ }
done
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#discussion_r211980875
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request. > + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.jclouds.aliyun.ecs.domain; + +import com.google.common.base.Enums; +import com.google.common.base.Joiner; +import com.google.common.base.Optional; + +import static com.google.common.base.Preconditions.checkArgument; + +/** + * The type of the ECS resource. All values must be lowercase. done -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-labs/pull/443#discussion_r211981107
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request.
> +
+ public void testList() {
+ final AtomicInteger found = new AtomicInteger(0);
+
assertTrue(Iterables.all(api().list(Regions.EU_CENTRAL_1.getName()).concat(),
new Predicate() {
+ @Override
+ public boolean apply(VPC input) {
+found.incrementAndGet();
+return !isNullOrEmpty(input.id());
+ }
+ }), "All vpcs must have at least the 'id' field populated");
+ assertTrue(found.get() > 0, "Expected some vpc to be returned");
+ }
+
+ private VPCApi api() {
+ return api.vpcApi();
+ }
added missing
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#discussion_r211981335
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request.
> +
+ public void testList() {
+ final AtomicInteger found = new AtomicInteger(0);
+
assertTrue(Iterables.all(api().list(Regions.EU_CENTRAL_1.getName()).concat(),
new Predicate() {
+ @Override
+ public boolean apply(VSwitch input) {
+found.incrementAndGet();
+return !isNullOrEmpty(input.id());
+ }
+ }), "All vSwitches must have at least the 'id' field populated");
+ assertTrue(found.get() > 0, "Expected some vSwitch to be returned");
+ }
+
+ private VSwitchApi api() {
+ return api.vSwitchApi();
+ }
added missing
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#discussion_r211981309
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request.
> + this.instanceSuspendedPredicate = instanceSuspendedPredicate;
+ this.regionIds = regionIds;
+ this.cleanupResources = cleanupResources;
+ }
+
+ @Override
+ public NodeAndInitialCredentials
createNodeWithGroupEncodedIntoName(String group, String name, Template
template) {
+ String instanceType = template.getHardware().getId();
+ String regionId = template.getLocation().getId();
+ String imageId = template.getImage().getId();
+
+ ECSServiceTemplateOptions templateOptions =
template.getOptions().as(ECSServiceTemplateOptions.class);
+
+ String keyPairName = templateOptions.getKeyPairName();
+ String securityGroupId =
Iterables.getOnlyElement(templateOptions.getGroups());
+ String vSwitchId = templateOptions.getVSwitchId();
I think I've added the necessary code to support this design: vpcApi vSwitchApi
and I've modified the `CreateResourcesThenCreateNodes` accordingly
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#discussion_r211981725
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
andreaturli commented on this pull request.
> +import org.jclouds.http.HttpRetryHandler;
+import org.jclouds.http.HttpUtils;
+import org.jclouds.http.annotation.ClientError;
+import org.jclouds.http.handlers.BackoffLimitedRetryHandler;
+import org.jclouds.json.Json;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.util.Set;
+
+import static org.jclouds.http.HttpUtils.closeClientButKeepContentStream;
+
+/**
+ * Handles Retryable responses with error codes in the 4xx range
+ */
+public class ECSErrorRetryHandler implements HttpRetryHandler {
@nacx I think this implementation reflects better your suggestion, but please
advice if it is not what you meant! thanks
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#pullrequestreview-148517914
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
sorry again as this PR is becoming huge. @nacx I think I've addressed all of your comments now and I've also simplified the RetryHandler for 4xx errors I'm running the live tests to see the result, some of them are expected to fail though - particularly from `ECSComputeServiceLiveTest`. I'll keep you posted Thanks again for your help -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-labs/pull/443#issuecomment-415059572
Re: [jclouds/jclouds-labs] [JCLOUDS-1430] Aliyun ECS (#443)
here's the live tests result
```
Results :
Failed tests:
SshKeyPairApiLiveTest.testImport:58 » IllegalArgument
{"RequestId":"4084B85E-D...
ECSComputeServiceLiveTest.testCorrectAuthException » Test
Method BaseComputeS...
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testCreateAndRunAService:747
» NoSuchElement
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testGet:553 »
NoSuchElement
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testImageById:211
expected [{id=eu-central-1/centos_7_04_64_20G_alibase_20180419.vhd,
providerId=centos_7_04_64_20G_alibase_20180419.vhd,
name=centos_7_04_64_20G_alibase_20180419.vhd, location={scope=REGION,
id=eu-central-1, description=欧洲中部 1 (法兰克福), parent=alibaba-ecs},
os={family=centos, name=CentOS 7.4 64位, version=7.4, description=,
is64Bit=true}, description=, status=AVAILABLE, loginUser=root}] but found [null]
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testListSizes:888->BaseComputeServiceLiveTest.checkVolumes:893
{id=ecs.sn1.medium, providerId=ecs.sn1.medium, name=ecs.sn1.medium,
processors=[{cores=2.0, speed=2.0}], ram=4096, hypervisor=none,
supportsImage=Predicates.alwaysTrue()}
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testOptionToNotBlock:866
» RunNodes
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testWeCanCancelTasks:282
» RunNodes
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testAScriptExecutionAfterBootWithBasicTemplate:227
» RunNodes
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testConcurrentUseOfComputeServiceToCreateNodes:498
» Execution
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testCreateTwoNodesWithOneSpecifiedName:383
» RunNodes
ECSComputeServiceLiveTest>BaseComputeServiceLiveTest.testCreateAnotherNodeWithANewContextToEnsureSharedMemIsntRequired:435
» RunNodes
Tests run: 55, Failures: 12, Errors: 0, Skipped: 7
```
`SshKeyPairApiLiveTest` is a quite weird as by the doc it should work
I'll concentrate on `ECSComputeServiceLiveTest` failures of `{testGet,
testImageById, testListSizes}` as they should be fine.
`testCreateAndRunAService` fails because of jetty on centos7 issue
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/443#issuecomment-415086300
Re: [jclouds/jclouds] JCLOUDS-1339: Support launching an x1 EC2 instance (#1235)
andreaturli commented on this pull request.
> @@ -290,6 +290,19 @@ private EC2HardwareBuilder hs1() {
virtualizationTypes(VirtualizationType.HVM,
VirtualizationType.PARAVIRTUAL);
return this;
}
+
+ private EC2HardwareBuilder x1() {
+ virtualizationTypes(VirtualizationType.HVM);
+
+ // TODO X1 is not deprecated, but it requires that you are using
a VPC
+ // until we have a way for hardware instances to be filtered
based on network
+ // we do NOT want X1 selected automatically.
+ // You get: org.jclouds.aws.AWSResponseException: request POST
https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='2300b99e-ddc0-42ab-b1ed-9d628a161be4', requestToken='null',
code='VPCResourceNotSpecified', message='The specified instance type can only
be used in a VPC. A subnet ID or network interface ID is required to carry out
the request.', context='{Response=, Errors=}'}
+ // A user can explicitly request a x1 if they are also setting up
a VPC.
+ deprecated();
[minor] do you think it is still a valid comment? Look quite unusual to not
deploy on a VPC these days, is it still important to filter those instance types
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1235#pullrequestreview-148559738
Re: [jclouds/jclouds] JCLOUDS-1339: Support launching an x1 EC2 instance (#1235)
thanks @lemmy for your contribution! only a minor comment from my side otherwise looks sensible. This manual process is really tedious, thanks for helping us out! I wish AWS will offer a simple instanceTypes API at some point! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1235#issuecomment-415090459
Re: [jclouds/jclouds] [JCLOUDS-1443] - Fix resolution uri (#1234)
@gaul do you think it is good to be merged now? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1234#issuecomment-415090751
Re: [jclouds/jclouds] Improve Jetty installation for live tests (#1230)
rebuild please -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1230#issuecomment-415091039
Re: [jclouds/jclouds] JCLOUDS-1339: Support launching an x1 EC2 instance (#1235)
lemmy commented on this pull request.
> @@ -290,6 +290,19 @@ private EC2HardwareBuilder hs1() {
virtualizationTypes(VirtualizationType.HVM,
VirtualizationType.PARAVIRTUAL);
return this;
}
+
+ private EC2HardwareBuilder x1() {
+ virtualizationTypes(VirtualizationType.HVM);
+
+ // TODO X1 is not deprecated, but it requires that you are using
a VPC
+ // until we have a way for hardware instances to be filtered
based on network
+ // we do NOT want X1 selected automatically.
+ // You get: org.jclouds.aws.AWSResponseException: request POST
https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='2300b99e-ddc0-42ab-b1ed-9d628a161be4', requestToken='null',
code='VPCResourceNotSpecified', message='The specified instance type can only
be used in a VPC. A subnet ID or network interface ID is required to carry out
the request.', context='{Response=, Errors=}'}
+ // A user can explicitly request a x1 if they are also setting up
a VPC.
+ deprecated();
@andreaturli I just copied this from
https://github.com/jclouds/jclouds/blob/cf67233765855a850e40f4b3d3b3ba320678dbbd/apis/ec2/src/main/java/org/jclouds/ec2/compute/domain/EC2HardwareBuilder.java#L218-L230
assuming the same reasoning applies to X1 instances.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1235#discussion_r212018738
Re: [jclouds/jclouds] [JCLOUDS-1374]This is the first commit of Softlayer LoadBalancer API (#1176)
hi @swaqos any update on that? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1176#issuecomment-415091546
Re: [jclouds/jclouds-labs] JCLOUDS-1386 1&1 Baremetal servers (#431)
andreaturli commented on this pull request.
>
@Inject
OneandoneComputeServiceAdapter(OneAndOneApi api, CleanupResources
cleanupResources,
+ GenerateHardwareRequest generateHardwareRequest,
@Named(POLL_PREDICATE_SERVER) Predicate
waitServerUntilAvailable,
PasswordGenerator.Config passwordGenerator) {
`PasswordGenerator.Config passwordGenerator` is not used, remove it
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs/pull/431#discussion_r212021489
Re: [jclouds/jclouds-labs] JCLOUDS-1386 1&1 Baremetal servers (#431)
andreaturli requested changes on this pull request.
@alibazlamit I've added some more comments but I could not finish up the review
as the code is becoming too complicated to be reviewed. As I told you other
times it would be much easier to add small chunks of code in small PRs.
For example you could focus on a PR that adds `RecoveryImageApi` with Mock and
Live Tests only (+ domain objects needed)
Then we can focus on the baremetal feature itself as it requires a huge number
of changes! Does it sound like a plan?
Thanks again
> @@ -253,7 +253,7 @@ public static ApplianceType fromValue(String v) {
}
public enum OSImageType {
- Standard, Minimal, Personal, ISO_OS, ISO_TOOL, NULL, UNRECOGNIZED;
+ STANDARD, MINIMAL, Personal, ISO_OS, ISO_TOOL, NULL, UNRECOGNIZED;
please make it all capital
> @@ -283,4 +283,22 @@ public String toString() {
return value;
}
}
+
+ public enum ServerType {
+ cloud, baremetal, UNRECOGNIZED;
+
+ public static ServerType fromValue(String v) {
+ return Enums.getIfPresent(ServerType.class, v).or(UNRECOGNIZED);
+ }
+
+ }
+
+ public enum ServerTypeCompatibility {
+ vps, cloud, baremetal, UNRECOGNIZED;
upper case?
> @@ -283,4 +283,22 @@ public String toString() {
return value;
}
}
+
+ public enum ServerType {
+ cloud, baremetal, UNRECOGNIZED;
why not upper case?
> @@ -179,9 +179,46 @@
return new NodeAndInitialCredentials(updateServer,
updateServer.id(), serverCredentials);
}
+ private ServerAppliance findWorkingImage(String imageId,
Types.ServerTypeCompatibility serverType) {
why this name? it seems that it tries to find a `ServerAppliance`, doesn't it?
> -//check if the bootable device has enough size to run the
> appliance(image).
-float minHddSize = volume.getSize();
-if (volume.isBootDevice()) {
- SingleServerAppliance appliance =
api.serverApplianceApi().get(image.getId());
- if (appliance.minHddSize() > volume.getSize()) {
- minHddSize = appliance.minHddSize();
- }
-}
-Hdd.CreateHdd hdd = Hdd.CreateHdd.create(minHddSize,
volume.isBootDevice());
-hdds.add(hdd);
- } catch (Exception ex) {
-throw Throwables.propagate(ex);
-
+ String imageId = image.getId();
+ Hardware hardwareModel =
generateHardwareRequest.isFlavor(hardware.getId());
+ boolean isBaremetal = hardware.getName() == null ? false :
hardware.getName().contains(baremetalModelsKey);
wouldn't be better to introduce a custom `TemplateOptions` where you can pass
in `bareMetal` boolean parameter? see
https://github.com/jclouds/jclouds/blob/master/providers/digitalocean2/src/main/java/org/jclouds/digitalocean2/compute/options/DigitalOcean2TemplateOptions.java#L38
for an example
> - SingleServerAppliance appliance =
> api.serverApplianceApi().get(image.getId());
- if (appliance.minHddSize() > volume.getSize()) {
- minHddSize = appliance.minHddSize();
- }
-}
-Hdd.CreateHdd hdd = Hdd.CreateHdd.create(minHddSize,
volume.isBootDevice());
-hdds.add(hdd);
- } catch (Exception ex) {
-throw Throwables.propagate(ex);
-
+ String imageId = image.getId();
+ Hardware hardwareModel =
generateHardwareRequest.isFlavor(hardware.getId());
+ boolean isBaremetal = hardware.getName() == null ? false :
hardware.getName().contains(baremetalModelsKey);
+ ServerAppliance workingImage;
+
+ //choose the correct image based on the server type baremetal or cloud
move this business logic into its own method
> }
}
- // provision server
- Server server = null;
- Double cores = ComputeServiceUtils.getCores(hardware);
- Double ram = (double) hardware.getRam();
- if (ram < 1024) {
- ram = 0.5;
- } else {
- ram = ram / 1024;
+ //configuring Firewall rules
+ Map portsRange = getPortRangesFromList(inboundPorts);
all the reasonings for the firewalls / inbound ports shouldn't probably happen
in the adapter: firewalls are created once and attached to all the nodes of a
group. See `CreateResourcesThenCreateNodes` as a reference at
https://github.com/jclouds/jclouds/blob/master/providers/azurecompute-arm/src/main/java/org/jclouds/azurecompute/arm/compute/strategy/CreateResourcesThenCreateNodes.java#L138
> - ram = ram / 1024;
+ //configuring Firewall rules
+ Map portsRange = getPortRangesFromList(inboundPorts);
+ List rules = new
ArrayList();
+
+ for (Map.Entry range : portsRange.entrySet()) {
+ FirewallPolicy.Rule.CreatePayload rule =
FirewallPolicy.Rule.CreatePayload.builder()
+ .portFrom(range.getKey())
+ .portTo(range.getValue())
+ .protocol(Types.RuleProtocol
Re: [jclouds/jclouds] [JCLOUDS-1443] - Fix resolution uri (#1234)
I think so, let's get this merged and backported -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1234#issuecomment-415130329
Re: [jclouds/jclouds] JCLOUDS-1339: Support launching an x1 EC2 instance (#1235)
andreaturli commented on this pull request.
> @@ -290,6 +290,19 @@ private EC2HardwareBuilder hs1() {
virtualizationTypes(VirtualizationType.HVM,
VirtualizationType.PARAVIRTUAL);
return this;
}
+
+ private EC2HardwareBuilder x1() {
+ virtualizationTypes(VirtualizationType.HVM);
+
+ // TODO X1 is not deprecated, but it requires that you are using
a VPC
+ // until we have a way for hardware instances to be filtered
based on network
+ // we do NOT want X1 selected automatically.
+ // You get: org.jclouds.aws.AWSResponseException: request POST
https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 400, error:
AWSError{requestId='2300b99e-ddc0-42ab-b1ed-9d628a161be4', requestToken='null',
code='VPCResourceNotSpecified', message='The specified instance type can only
be used in a VPC. A subnet ID or network interface ID is required to carry out
the request.', context='{Response=, Errors=}'}
+ // A user can explicitly request a x1 if they are also setting up
a VPC.
+ deprecated();
Thanks @lemmy
@nacx any thoughts on this?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1235#discussion_r212064403
Re: [jclouds/jclouds] [JCLOUDS-1443] - Fix resolution uri (#1234)
gaul approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/1234#pullrequestreview-148643508
