I would say it's best practice to have the ES data on a ephemeral volume if you need extreme performance. But you loose the ability to snapshot the volume for backup and autoscaling. Nevertheless you can (and I would prefer) to still use a instance with EBS boot volume but add an additional block volume mapping with an ephemeral NVMe device to keep the ES data on.
On Sun, Jul 28, 2019, 09:28 Roni Dromi <ronid1...@gmail.com> wrote: > I've been told that this is the best practice for elasticsearch servers, > it's not tested yet but we want to give it a try. > > On Saturday, July 27, 2019 at 10:58:49 PM UTC+3, Rickard von Essen wrote: >> >> By the way, is there any particular reason you choose instance store EC2 >> over EBS EC2's? >> >> On Sat, Jul 27, 2019, 21:51 Rickard von Essen <rickard...@gmail.com> >> wrote: >> >>> I think that is just because you use the latest generation of instance >>> which has NVMe attached disks. Try to use a previous generation without >>> this and it should work. >>> >>> It is probably not hard to get this working with NVMe instance either >>> but it probably needs some adjustments of some arguments. >>> >>> On Sat, Jul 27, 2019, 18:13 Roni Dromi <roni...@gmail.com> wrote: >>> >>>> Goof catch! >>>> After changing instance type, I'm getting this error - Non-standard >>>> volume device "/dev/nvme0n1p1" (FatalError), >>>> Do you have any idea what may cause this? It's not mentioned before but >>>> I'm using this AMI - >>>> https://aws.amazon.com/marketplace/pp/Centosorg-CentOS-7-x8664-with-Updates-HVM/B00O7WM7QW >>>> Thanks again!! >>>> >>>> On Thursday, July 25, 2019 at 9:59:58 PM UTC+3, Rickard von Essen wrote: >>>>> >>>>> Hi, >>>>> >>>>> It was a couple of years since I last built an instance store AMI, but >>>>> I would first change to be building it from a instance store instance >>>>> (which you aren't, since you are using T2 instances which are EBS only). >>>>> >>>>> On Thu, Jul 25, 2019, 16:04 Roni Dromi <roni...@gmail.com> wrote: >>>>> >>>>>> Hey! >>>>>> Got this error today when I tried ruuning "amazon-instance" builder. >>>>>> here is the relevant part of the config - >>>>>> >>>>>> { >>>>>> "variables": { >>>>>> "aws_access_key": "", >>>>>> "aws_secret_key": "", >>>>>> "root_device_name": "/dev/sda1", >>>>>> "root_volume_size": "10", >>>>>> "root_volume_type": "gp2", >>>>>> "root_delete_on_termination": "true", >>>>>> "data_device_name": "/dev/sdb", >>>>>> "data_volume_size": "10", >>>>>> "data_volume_type": "standard", >>>>>> "data_delete_on_termination": "true" >>>>>> }, >>>>>> "provisioners": [ >>>>>> >>>>>> ], >>>>>> "builders": [ >>>>>> { >>>>>> "type": "amazon-instance", >>>>>> "access_key": "{{user `aws_access_key`}}", >>>>>> "secret_key": "{{user `aws_secret_key`}}", >>>>>> "region": "us-east-1", >>>>>> "source_ami": "ami-02eac2c0129f6376b", >>>>>> "vpc_id": "<vpc_id>", >>>>>> "ssh_username": "centos", >>>>>> "instance_type": "t2.micro", >>>>>> "ami_name": "{timestamp}}", >>>>>> "account_id": "<account>", >>>>>> "s3_bucket": "<my_bucket>", >>>>>> "x509_cert_path": "<cert>", >>>>>> "x509_key_path": "<key>", >>>>>> "launch_block_device_mappings": [ >>>>>> { >>>>>> "device_name": "{{user `root_device_name`}}", >>>>>> "volume_size": "{{user `root_volume_size`}}", >>>>>> "volume_type": "{{user `root_volume_type`}}", >>>>>> "delete_on_termination": "{{user >>>>>> `root_delete_on_termination`}}" >>>>>> } >>>>>> ], >>>>>> "ami_block_device_mappings": [ >>>>>> { >>>>>> "device_name": "{{user `data_device_name`}}", >>>>>> "volume_size": "{{user `data_volume_size`}}", >>>>>> "volume_type": "{{user `data_volume_type`}}", >>>>>> "delete_on_termination": "{{user >>>>>> `data_delete_on_termination`}}" >>>>>> } >>>>>> ] >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> And here is the full traceback - >>>>>> >>>>>> amazon-instance: /mnt/img-mnt >>>>>> ==> amazon-instance: >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/platform/linux/image.rb:793:in >>>>>> `execute': Failed to execute: 'mount -t xfs /dev/mapper/hda1 >>>>>> /mnt/img-mnt' >>>>>> (FatalError) >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/platform/linux/image.rb:670:in >>>>>> `mount_image' >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/platform/linux/image.rb:174:in >>>>>> `make' >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/amitools/bundlevol.rb:184:in >>>>>> `bundle_vol' >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/amitools/bundlevol.rb:231:in >>>>>> `main' >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/amitools/tool_base.rb:201:in >>>>>> `run' >>>>>> ==> amazon-instance: from >>>>>> /usr/local/ec2/ec2-ami-tools-1.5.7/lib/ec2/amitools/bundlevol.rb:239:in >>>>>> `<main>' >>>>>> ==> amazon-instance: Volume bundling failed. Please see the output >>>>>> above for more >>>>>> ==> amazon-instance: details on what went wrong. >>>>>> ==> amazon-instance: >>>>>> ==> amazon-instance: One common cause for this error is >>>>>> ec2-bundle-vol not being >>>>>> ==> amazon-instance: available on the target instance. >>>>>> >>>>>> what am I missing? >>>>>> Thanks!!! >>>>>> >>>>>> -- >>>>>> This mailing list is governed under the HashiCorp Community >>>>>> Guidelines - https://www.hashicorp.com/community-guidelines.html. >>>>>> Behavior in violation of those guidelines may result in your removal from >>>>>> this mailing list. >>>>>> >>>>>> GitHub Issues: https://github.com/mitchellh/packer/issues >>>>>> IRC: #packer-tool on Freenode >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Packer" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to packe...@googlegroups.com. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/packer-tool/f9218a8f-3f0e-46be-90ca-dd3a1d32f396%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/packer-tool/f9218a8f-3f0e-46be-90ca-dd3a1d32f396%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> This mailing list is governed under the HashiCorp Community Guidelines >>>> - https://www.hashicorp.com/community-guidelines.html. Behavior in >>>> violation of those guidelines may result in your removal from this mailing >>>> list. >>>> >>>> GitHub Issues: https://github.com/mitchellh/packer/issues >>>> IRC: #packer-tool on Freenode >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Packer" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to packe...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/packer-tool/838ece41-776e-4462-ba84-73998c8dbf14%40googlegroups.com >>>> <https://groups.google.com/d/msgid/packer-tool/838ece41-776e-4462-ba84-73998c8dbf14%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > This mailing list is governed under the HashiCorp Community Guidelines - > https://www.hashicorp.com/community-guidelines.html. Behavior in > violation of those guidelines may result in your removal from this mailing > list. > > GitHub Issues: https://github.com/mitchellh/packer/issues > IRC: #packer-tool on Freenode > --- > You received this message because you are subscribed to the Google Groups > "Packer" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to packer-tool+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/packer-tool/3185d39e-c631-4d1d-9dbc-a449c58c1a83%40googlegroups.com > <https://groups.google.com/d/msgid/packer-tool/3185d39e-c631-4d1d-9dbc-a449c58c1a83%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list. GitHub Issues: https://github.com/mitchellh/packer/issues IRC: #packer-tool on Freenode --- You received this message because you are subscribed to the Google Groups "Packer" group. To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CALz9Rt-%3DozRkuw9M2-v24P%3DQdYOqAJJQ24LHhbUbtcsqx%3Dt3Sg%40mail.gmail.com.
