Re: Meeting today: useful?

[Garrett Holmstrom]

On 4/1/2010 11:09, Greg DeKoenigsberg wrote:
> We've got the meeting time reserved, but I don't want to squander anyone's
> time.  If there are issues that anyone wants to discuss, I'm happy to have
> the meeting -- if even one person sends agenda items, I'm willing to
> convene a meeting.  But if not, I'm also happy not to convene a meeting.
> :)

All I can think of is simply that the latest euca2ools update needs 
testers.  It still has 0 karma, so it remains in updates-testing.

Chances are I won't be around during the usual meeting time today 
anyway, but I will read the meeting log if there is one.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Meeting today: useful?

[Garrett Holmstrom]

On 4/2/2010 0:49, Marek Goldmann wrote:
> On 2010-04-02, at 01:26, graziano obertelli wrote:
>> this is so cool that euca2ools are making into fedora. We just added them
>> to our Q&A so that whenver we test Eucalyptus on fedora (we are working
>> into making nightlies of rpm for fedora) we use your package. So far they
>> passed all the tests! Let me know if we can help in any other way.
>
> If they're good, please vote on tested releases:
>
>   https://admin.fedoraproject.org/updates/search/euca2ools
>
> Create an account if you don't have one here: 
> https://admin.fedoraproject.org/accounts/user/new

This.  Packages in updates-testing need, well, testing before they can 
be safely pushed to stable.  But they aren't considered "tested" until 
enough people try them out and give them comments and +1/-1 votes in the 
updates system.

You could even try it right now:  install euca2ools-1.2-2 from 
updates-testing, see if it works (and hopefully fixes some bugs), then 
add feedback to the update.

On 2010-04-02, at 01:26, graziano obertelli wrote:
 > So far they passed all the tests! Let me know if we can help in any
 > other way.

It looks like you could use some more test coverage, then. 
Specifically, I don't think you're covering block device mapping at all, 
given https://launchpad.net/bugs/544706.

Perhaps you folks could take a look at the bug I'm stuck on 
(https://bugzilla.redhat.com/show_bug.cgi?id=575258) and if you have any 
ideas comment on the bug report or just let me know.  If you want I 
could file it in Eucalyptus's tracker as well; would that help?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

euca2ools development snapshots?

[Garrett Holmstrom]

Hi folks,

Since Eucalyptus upstream doesn't support block device mapping in the euca2ools 
1.2 release, from what I gather it's pretty much worthless for building Fedora 
images.  Their development builds, however, claim to support block device 
mapping, so would it be worth updating Fedora's packages to development 
versions so we can hopefully use ec2-api-tools from RPM Fusion as little as 
possible?  (Or at least get feedback to upstream so we can start weaning 
ourselves off of it over time)

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud SIG meeting minutes - 2010/07/01

[Garrett Holmstrom]

On 7/1/2010 17:05, Robyn Bergeron wrote:
>* jforbes to ping gholms about eucatools<-->  kernel<-->  pvgrub image

Eucalyptus upstream claims to support block device mapping in 
development versions of euca2ools, but unfortunately that also requires 
a somewhat recent svn snapshot of python-boto.  I presume pvgrub images 
still need that capability?  I can upload a package with a recent 
euca2ools snapshot, but python-boto's maintainer prefers to stick with 
stable releases.  Where do you suppose we should go from here?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Pre-release euca2ools available for testing

[Garrett Holmstrom]

Good news, everyone!  Almost immediately after I commented about the 
lengthy time since their last release the python-boto people published 
an alpha of their latest code.  So I built it [0] and the latest 
euca2ools snapshot [1] and threw them up on the web for you folks to 
test* if you wish.  The source packages stand alongside the noarch 
packages should you need to rebuild them.  Since they can't yet go in 
the official repos, just let me or upstream know if you encounter any bugs.

I also quit putting off building euca2ools 1.2 for el6, so it should 
appear in the epel repos after the next push.

Happy hacking,
GH

[0] 
http://www.physics.umn.edu/~holms/repo/custom/13/python-boto-2.0-0.1.a2.fc13.noarch.rpm
[1] 
http://www.physics.umn.edu/~holms/repo/custom/13/euca2ools-1.2-3.20100701bzr293.fc13.noarch.rpm

* Standard disclaimer:  Pre-release code may opt to ruin your potted 
plants or maim your cat instead of run correctly, so please take proper 
precautions when testing.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: OpenStack

[Garrett Holmstrom]

Major Hayden wrote:
> You can find people on Freenode in #openstack if you have more questions.  If 
> you are looking for specific details, the voiced folks in that channel should 
> be able to answer your questions.
> 
> If the group would prefer a more formal meeting with some of the devs to 
> discuss it, just let me know and I'll get the ball rolling.

I asked a few people from #openstack to stop by #fedora-meeting to 
answer any questions that might pop up during today's SIG meeting.  We 
can always follow up via email or something more formal if necessary.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: adding some sort of [subject tag] for fedora cloud list

[Garrett Holmstrom]

Jesse Keating wrote:
> On 07/30/2010 08:50 AM, Robyn Bergeron wrote:
>> This has been requested by one of our community members.  Is this
>> kosher with everyone?
>>
>> I was just going to use [fedora-cloud].
> 
> I loath to see subject blocks like that.  This is what list headers and
> filtering is for.  :)

Same here.  Sorry, anonymous community member.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: KVM/Qemu on CloudSigma

[Garrett Holmstrom]

On 8/27/2010 8:33 AM, Renich Bon Ciric wrote:
> I know that, you guys, are focused on Amazon's EC2.
>
> Anyway, please, take a look on CloudSigma: http://cloudsigma.com/

One of this SIG's goals is to help get current Fedora images onto a 
variety of cloud providers.  As we are just starting out, EC2 happens to 
be the top priority, but we definitely encourage other providers to work 
with us to help us help them help us all provide the best Fedora 
experience possible.

Man, I sound like a politician...

-- 
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Eucalyptus 2.0 and Fedora

[Garrett Holmstrom]

On 31-Aug-10 20:37, graziano obertelli wrote:
> we would love to be able to coordinate with Fedora about Fedora's and our
> releases and web page (where to file bugs and where to find packages): are
> you the right person to coordinate with?

I maintain Fedora's euca2ools package, so I ultimately see all of the 
bug reports raised against it on Red Hat's Bugzilla [0].  When people 
file bug reports against Fedora's package I usually forward it upstream 
and then try to backport the resulting fix into Fedora's package.  While 
I typically wait for stable upstream releases before looking into adding 
features, I can try to accommodate RFEs as well.  If you need something 
fixed that is the best place to send communications.  For other 
cloud-related things this list is probably better.

Where possible, please direct people to their systems' native package 
managers (e.g. yum or PackageKit in Fedora) instead of offering binaries 
via the Eucalyptus web site.  This reduces confusion for me as a package 
maintainer since I don't have to worry about where someone's software 
came from, and for the Eucalyptus people since it's one less 
distribution to build for.

At one point someone looked into packaging Eucalyptus, though it cannot 
make it into Fedora until all of its dependencies are broken out into 
separate packages and it doesn't bundle any JAR files.  Since there is 
currently no Fedora package for Eucalyptus, this mailing list is 
probably the best point of contact for work on getting it packaged.

> We already have one engineer looking into the euca2ools packages. He
> already reached out to the euca2ools packages and we hope to have only one
> package soon.

I assume you are referring to Mitch Garnatt here, who I contacted 
independently a few days ago.  ;-)  The notion of "only one package" is 
a bad idea for binaries since different distributions have different 
conventions and software versions.  Having one spec file that supports 
multiple distributions is definitely possible, though.  I discussed this 
a bit with Mitch.  For licensing reasons I cannot simply provide you 
with a spec file that does what you need, though I am happy to provide 
feedback.

Ideally, Eucalyptus will not provide any binaries at all, but instead 
have packages included in distributions' package repositories.  This 
would render upstream-provided spec files and packages meaningless.  For 
instance, as a Fedora packager I cannot use the euca2ools spec file 
included with its source because I must maintain my own spec file in 
Fedora's version control system.

> How do you think we should proceed for Eucalyptus? Is it possible to have
> a 'mentor' to help us get Eucalyptus into Fedora?

I would first work on breaking the dependencies into separate packages 
and make sure it is possible to build Eucalyptus in its entirety from 
source without the need for JAR files.  I personally don't have much 
experience with Java packaging, but you can also direct questions to 
either Fedora's packaging list [1] or this list.  There are plenty of 
people who can help.

[0] https://bugzilla.redhat.com/
[1] packag...@lists.fedoraproject.org

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

RFC: EC2 mirror infrastructure proposal

[Garrett Holmstrom]

I would like to ask for feedback on a proposal I wrote [0] for how we 
could run a package mirror system on S3.  It would be useful if everyone 
read it before the meeting so we can discuss it.  This is a draft, so I 
would appreciate constructive feedback on the mailing list and/or IRC 
meetings.

[0] https://fedoraproject.org/wiki/User:Gholms/EC2_Mirror_Proposal
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: EC2 mirror infrastructure proposal

[Garrett Holmstrom]

On 10/3/2010 23:20, matt_dom...@dell.com wrote:
> are we thinking of one mirror per region, or one mirror per availability zone 
> (multiple per region) ?  Data transfer within a zone is free, within a region 
> is $0.01/GB unless we can get that waived too.

One per region.  Transfer between EC2 instances is free within a given 
zone, but transfer from S3 to EC2 is free within a given region.  Sorry 
if the plan wasn't clear enough on that.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Free Amazon AWS

[Garrett Holmstrom]

Peter Robinson wrote:
> For those that are interested in playing with the Amazon cloud
> (presumably with the new Fedora 14 ami ;-) ) Amazon is introducing an
> AWS Free Usage Tier from Nov 1. Details here
> 
> http://aws.amazon.com/free/

The fine print sez:

"Only accounts created after October 20, 2010 are eligible for the 
Offer. The Offer does not apply to any use of the AWS services prior to 
November 1, 2010."

So it's of no use to some of us (myself included), but it ought to help 
others jump in and test!
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Hey Cloud SIG, let's start talking about F15.

[Garrett Holmstrom]

Robyn Bergeron wrote:
> * BoxGrinder - Marek Goldmann is working on packaging.
> http://www.jboss.org/boxgrinder

We should try getting this working sooner rather than later so we can 
use it for building images in a manner that's easy to include in the 
release process.  The word on the street is that Koji can spin appliance 
images of some sort; perhaps we could leverage that somehow.

> * Eucalyptus - obino has been looking for some mentorship as far as
> packaging goes. If there are folks around to help out with this, I
> think it would be awesome to have as a feature.  Is anyone willing to
> help out here? http://open.eucalyptus.com/

It's a long and obnoxious process to do it the "right" way.  Thankfully 
Eucalyptus is building a dedicated release engineering team whose job 
is, among other things, to make packaging less of a pain.

> Additionally, we probably have further enhancements we could make on
> the EC2 front - namely, having really awesome documentation, which
> Sparks has been poking the list on since he'd love to help us out
> there. What other EC2-related enhancements are there for us to tackle?

I think we should port cloud-init to Fedora so users can configure new 
instances using a popular, existing method that we don't have to build 
from the ground up.  I'm looking into that at the moment; does anyone 
else want to help?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Hey Cloud SIG, let's start talking about F15.

[Garrett Holmstrom]

On 10/29/2010 12:50, Brian LaMere wrote:
> * Eucalyptus - obino has been looking for some mentorship as far as
> packaging goes. If there are folks around to help out with this, I
> think it would be awesome to have as a feature.  Is anyone willing to
> help out here? http://open.eucalyptus.com/
>
>
> I'm really excited to see their interest in Fedora.
>
> Unfortunately, I can't mentor, since I'm not a packager myself ;)  I
> will go help review the two packages mentioned in the meeting, for my
> journey down that road.  I can't atm justify working on things too far
> out of stuff that is relevant for my daily work; my employer is very
> supportive of OSS, but not if it means they're paying me to work on
> something completely irrelevant to my job ;)  So I've had a problem
> finding things to review.  I have a couple packages I could look at
> trying to get in that are smaller, and am willing to help them out on
> this...just not as a mentor.  Eucalyptus is something I wanted to use in
> the past, but didn't because it wasn't immediately Fedora-happy; if a
> time-saving tool doesn't save time, then I can't use it.  I *can* help
> develop it (on my company's timeclock) to be Fedora happy though, if
> it's something that they're doing - so I'm very on board with helping there.

The Eucalyptus people are very interested in getting help with that; 
their biggest problem has been a lack of manpower.  They're even hiring 
me to, among other things, do just that.  ;)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Hey Cloud SIG, let's start talking about F15.

[Garrett Holmstrom]

On 11/1/2010 14:14, Pete Zaitcev wrote:
> On Fri, 29 Oct 2010 10:50:13 -0700
> Brian LaMere  wrote:
> I do not quite understand what the advantage of integrating yum
> with S3 would be, but we can do it in Fedora today if we want.

We wouldn't be integrating yum itself with S3; we would be putting 
copies of Fedora's mirrors on S3 so instances can pull from there 
instead of public mirrors.  The EC2 mirror proposal page [1] has lots of 
details.

[1] https://fedoraproject.org/wiki/User:Gholms/EC2_Mirror_Proposal
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Converting the default instance store Fedora 14 to EBS

[Garrett Holmstrom]

Jan Pazdziora wrote:
> On Wed, Nov 03, 2010 at 08:16:03AM -0500, Justin M. Forbes wrote:
>> They are, and they should be working.  If I had to guess, I would say you
>> are trying to log in as root?  You cannot do that.  Log in as ec2-user and
>> you have full sudo access.  This goes with the Amazon documentation as
>> well.  The full list of official images is being maintained at:
>>
>> https://fedoraproject.org/wiki/Cloud_SIG/EC2_Images
>>
>> Unfortunately it takes a bit of time for Amazon to make the public image
>> links available, which will give us a bit more visibility as well.
> 
> I got the ami-e291668b running.
> 
> As the EBS AMIs are not yet available, I'd like to convert this
> instance to EBS backed instance, following roughly the steps in
> http://gnuyoga.wordpress.com/2009/12/28/ec2-instance-store-to-ebs-boot/
> 
> However, when I try to attach the newly created volume to my instance
> I either get
> 
>   Client.InvalidParameterValue: Value (/dev/xvdd) for parameter device is 
> invalid. /dev/xvdd is not a valid EBS device name.
> 
> when I try to use some /dev/xvd* device name, or I use -d /dev/sda
> and the ec2-attach-volume finishes fine but the device /dev/sda does
> not come to life.

euca2ools aims to be an open source drop-in replacement for most of 
ec2-*-tools.  While you folks are working with your images, please try 
installing the euca2ools package and running euca-attach-volume instead 
of ec2-attach-volume, euca-run-instances instead of ec2-run-instances, 
and so forth so we can get things working without needing to install 
proprietary bits from Amazon.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud SIG Meeting Reminder: 2010/11/04, 2100 UTC, in #fedora-meeting

[Garrett Holmstrom]

On 11/4/2010 13:27, Robyn Bergeron wrote:
> Meeting Details: Thursday, 2010/11/04 @2100 UTC (In NA - East coast,
> 5pm; West coast, 2pm.)
> Meeting channel: #fedora-meeting, irc.freenode.net
>
> Other details are available at https://fedoraproject.org/wiki/Cloud_SIG.

I will likely miss today's meeting, so could you folks please discuss 
what you think of shipping images with...

a) the @core yum group (jforbes might be working on this)

b) SElinux enabled like it is on default Fedora installs?

SElinux works on EC2, but you have to have the @core group installed. 
If you have an image with SElinux disabled you might first have to 
relabel everything.

Thanks!
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: "yum check" error with ami-669f680f

[Garrett Holmstrom]

On 11/4/2010 14:29, Eric Smith wrote:
> I'm not sure if this is the correct place to report this. If not, please
> let me know a more appropriate place.
>
> If I fire up an instance of ami-669f680f
> (fedora-images-us-east-1/fedora-14-i386-S3.ec2.manifest.xml) and the
> first command I run is "yum check", I get:
>
> $ yum check
> ec2-ami-tools-1.3-57676.noarch has missing requires of rsync
> ec2-ami-tools-1.3-57676.noarch has missing requires of ruby
> Error: check all
> $

Call it a hunch, but something tells me FESCo would be less than pleased 
to hear about the closed-source ec2-ami-tools appearing on 
Fedora-provided AMIs.  Why install it (or even euca2ools) at all when it 
is completely unnecessary for running an instance?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: fedora-14-x86_64-S3.ec2: AMIs with instance-store root device not supported for 't1.micro' ???

[Garrett Holmstrom]

On 11/7/2010 16:12, sean darcy wrote:
> Well it appears that the 32 bit image doesn't work either as a micro
> instance! Is this a result of the images being "instance-store" rather
> than EBS backed images? Whatever that means.

Yes.  You will need to create or wait for a EBS-backed image first.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: euca2ools

[Garrett Holmstrom]

On 11/11/2010 1:50, Jan Pazdziora wrote:
> On Wed, Nov 03, 2010 at 10:36:19AM -0500, Garrett Holmstrom wrote:
>>
>> euca2ools aims to be an open source drop-in replacement for most of
>> ec2-*-tools.  While you folks are working with your images, please try
>> installing the euca2ools package and running euca-attach-volume instead
>> of ec2-attach-volume, euca-run-instances instead of ec2-run-instances,
>> and so forth so we can get things working without needing to install
>> proprietary bits from Amazon.
>
> Thank you for pointing this out. I've tried
>
>   $ euca-describe-instances
>   EC2_ACCESS_KEY environment variable must be set.
>   Connection failed
>
> while having EC2_CERT and EC2_PRIVATE_KEY setup and in shell where
> ec2-describe-instances works. The man euca-describe-instances(1) says
> that
>
> Euca2ools  will  use the environment variables EC2_URL, 
> EC2_ACCESS_KEY,
> EC2_SECRET_KEY, EC2_CERT, EC2_PRIVATE_KEY, S3_URL,  EUCALYPTUS_CERT  
> by
> default.
>
> Is there anything else I need to have setup beyond EC2_CERT and
> EC2_PRIVATE_KEY?

Euca2ools programs don't hard-code things like URIs, so you typically 
need to specify more in the environment to get them to work with AWS.

I recommend setting everything of importance up in your shell's rc file. 
  For instance, my .zshrc file contains something similar to this:

export S3_URL=https://s3.amazonaws.com:443
export EC2_URL=https://ec2.amazonaws.com:443
export EC2_PRIVATE_KEY=~/ec2/euca2-gholms--pk.pem
export EC2_CERT=~/ec2/euca2-gholms--cert.pem
export EC2_ACCESS_KEY=''
export EC2_SECRET_KEY=''

...where the '0's are replaced with account-specific numbers and things.

At the moment you need all six of these variables, though the *_KEY 
variables may not be necessary in future releases.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Broken 32-bit F14 EC2 images?

[Garrett Holmstrom]

On 11/16/2010 12:56, Brian LaMere wrote:
> ec2-ami-tools shouldn't have been installed if I recall correctly, as it
> is not FOSS.  I (perhaps incorrectly?) remember someone saying it would
> be removed from the next image version.  I've had Life(tm) happening as
> of late so I haven't been at many of the recent meetings or tried to
> stay in the loop much, so I may be incorrect about that.
>
> Either way, you can remove it easily enough, without worry.  Unless
> you'll be allocating new AWS recources from the instance, it's not going
> to be of use - and if you do want to do such a thing, euca2ools are
> better, anyway.

jforbes told me that the next set of images he posts won't include it. 
As of yet I have no idea what the state of said images is, though.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud projects inside of RedHat

[Garrett Holmstrom]

On 12/9/2010 12:53, Chris Lalancette wrote:
> On 12/09/10 - 01:02:19PM, Jared K. Smith wrote:
>> On Thu, Dec 9, 2010 at 12:39 PM, Chris Lalancette  
>> wrote:
>>>  There are a group of us inside of RedHat working on a number of cloud
>>> related projects.  I'd like to get a bit more visibility to these projects, 
>>> so
>>> I'm going to make a few announcements on this list.  The idea is to keep up
>>> with these announcements and get more participation.
>>
>> Sounds great!  Thanks for sharing more about your project with us,
>> Chris. I encourage anybody else with cloud-related projects (whether
>> you're outside or inside of Red Hat) to join with the Cloud SIG and
>> help make Fedora the premier tech incubator for virt- and
>> cloud-related technologies.
>
> Yes, we plan to package this stuff up and submit it to Fedora.  Oz itself has
> minimal external dependencies, and they already exist in Fedora, so it is a
> no-brainer to submit that.

It might be worth your while to stop by #fedora-meeting on freenode 
today at 1600 EST for the Cloud SIG meeting.  ;)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: mirror to S3 task

[Garrett Holmstrom]

On 12/9/2010 16:39, Brian LaMere wrote:
> Re-looking at this again; I noted in the meeting that a new multipart
> upload feature was created for S3, which means anything that hasn't been
> changed to use it (it's been out just a couple weeks) is now sorely
> outdated; that feature is too big an improvement to ignore.
>   Additionally, most of the tools I've used simply can't handle large
> amounts of data very well, and the S3 mirror is indeed a decent amount
> of data.  I'll re-review s3cmd
>
> That said, was anything new added or removed from the spec of what the
> tool needs to do since whenever that was?

There hasn't really been any more discussion.  AFAICT, what we really 
need at this point is a prototype (or several) to get an idea of how 
things will work and what will work best.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Kickstart a Fedora (Anaconda) Install on EC2

[Garrett Holmstrom]

On Wed, Feb 2, 2011 at 10:19 AM, Brian LaMere
 wrote:
>> Right, it's entirely ephemeral AMIs at this point.  Unfortunately,
>> doing more is somewhere between tough and impossible in a sane way as
>> there isn't really a good API for uploading an EBS backed AMI.  It's
>> all "dd onto a block device, snapshot, foo".
>
> yeah, that's why although my silly script is ugly it still produces the
> right result; an AMI that can be used for EBS-backed stores, of an image
> that has never run and is free of such taint.  How one gets to there is
> important, but considering it's not something that gets repeated often (once
> you have an AMI, the AMI is done...) the real issue is what the end result
> ended up being.

At FUDCon I heard that rawhide's version of anaconda can install
directly to things like disk images and block devices, IIRC.  Maybe
that would be useful.

Maybe I'm missing something:  why would you ever want an instance to
kickstart at boot time?  You should create an image for every role you
care about and then boot the appropriate one for every instance you
need.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Testers needed for euca2ools on el5

[Garrett Holmstrom]

Hi, folks!

Are you sick of being stuck with a closed-source CLI for Amazon AWS or 
Eucalyptus because you use RHEL 5?  Have you been rolling your own 
euca2ools packages because no one else has?  *cough*BoxGrinder*cough*

No longer!  Now that all of its dependencies are available in EPEL 5, 
you, too, can experience the convenience of being able to install 
euca2ools from a repository you already use!  Just install it from 
epel-testing, give it a try, and give it +1 if it works.  I appreciate 
your feedback.

https://admin.fedoraproject.org/updates/euca2ools-1.3.1-7.el5
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Testers needed for euca2ools on el5

[Garrett Holmstrom]

On Mon, Mar 7, 2011 at 2:41 AM, Marek Goldmann  wrote:
> My tests indicate that proposed package doesn't like python26-boto-2.0. I've 
> added a comment to Koji with more info.
>
> Btw, thanks for pushing euca2ools to EPEL!

It looks like I forgot to backport a compatibility patch or two.  >_>
Does it work if you simply remove the line that reads ``from boto.s3
import Connection'' from /usr/bin/euca-*-bundle?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Spins issue

[Garrett Holmstrom]

On Tue, Mar 1, 2011 at 5:47 AM, Paul W. Frields  wrote:
> As I was going through some email this morning I saw that the Spins
> SIG is concerned that the Cloud SIG hasn't followed the Spins process
> to get their "ec2" labeled Spin included in the spins-kickstart
> collection in git:
>
> http://lists.fedoraproject.org/pipermail/spins/2011-March/001734.html
>
> If the Cloud group needs to see this Spin included in the F15 release,
> then my recommendation is to start talking with the Spins SIG about
> the process.  I believe there's a deadline that's passed, although you
> can request an exception and still make it into the release.
>
> To the Spins SIG, I would also recommend keeping lines of
> communication open if something goes wrong.  A Cc: to the cloud@ list,
> for example, would help raise the issue to more people in a way that
> would help reach a solution faster.

At today's Spins SIG meeting the EC2 image came into question again.
Whether it is a true spin or not, they would still like to see it go
through the parts of the spin process [1] that are applicable to it so
we can at least get on their radar and start figuring out what is and
is not important to do.  In particular, they asked for a wiki page
that is marked as ready for the spins wrangler some time this week.
Any thoughts?  Volunteers?

[1] https://fedoraproject.org/wiki/Spins_Process
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Testers needed for euca2ools on el5

[Garrett Holmstrom]

On Mon, Mar 7, 2011 at 12:04 PM, Garrett Holmstrom
 wrote:
> On Mon, Mar 7, 2011 at 2:41 AM, Marek Goldmann  wrote:
>> My tests indicate that proposed package doesn't like python26-boto-2.0. I've 
>> added a comment to Koji with more info.
>>
>> Btw, thanks for pushing euca2ools to EPEL!
>
> It looks like I forgot to backport a compatibility patch or two.  >_>
> Does it work if you simply remove the line that reads ``from boto.s3
> import Connection'' from /usr/bin/euca-*-bundle?

Here is an updated build that you folks can try:
http://koji.fedoraproject.org/koji/taskinfo?taskID=2891991 .  Thanks
to mgoldmann for helping test it.  :)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Testers needed for euca2ools on el5

[Garrett Holmstrom]

On Tue, Mar 8, 2011 at 12:46 AM, Marek Goldmann  wrote:
> Unfortunately I hit another issue :(
>
> Traceback and proposed fix:
>
>        https://gist.github.com/860040

Thanks to a bunch of testing from Marek, a new euca2ools package is on
its way to epel-testing.  I encourage everyone interested to give it a
try and vote it +1 or -1.  Thanks!

https://admin.fedoraproject.org/updates/euca2ools-1.3.1-8.el5
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: AMI - Fedora 14 - ami-e291668b

[Garrett Holmstrom]

On 4/13/2011 13:43, Kurt Seifried wrote:
> If you use the command line EC2 tools from Amazon you'll run into a
> few snags during install, I've documented getting them running on
> Fedora:
>
> http://kurt.seifried.org/2010/12/08/getting-amazone-aws-ec2-api-tools-working-properly/

If you install euca2ools from the Fedora or EPEL repositories you should 
be able to do the entire process without having to mess with either Java 
or Amazon's proprietary tools.  Just replace each command's leading 
"ec2" with "euca" (e.g. "ec2-register" becomes "euca-register").
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: F15 EC2 Test Day Rescheduling

[Garrett Holmstrom]

On 2011-07-27 20:27, Max Spevack wrote:
> On Wed, 27 Jul 2011, Jay Greguske wrote:
>> Koji builds the images fine, we're stuck at uploading and registering
>> them as EBS-backed AMIs currently.
>
> Can you give more information about this bug?  What is the process by
> which issues like this are being debugged and investigated?

While I can't say for sure what the current hangup is, I can tell you 
what makes the process so difficult:

Uploading an EBS-backed image is somewhat more convoluted than uploading 
a regular image because you have to boot up a virtual machine inside 
EC2, create a virtual disk to install Fedora onto, copy the contents of 
the entire image over the Internet onto that disk, and then create a 
snapshot of the disk.  Registering it is rather quick and painless, but 
until euca2ools-1.3.1-12 (which is on its way to updates-testing for all 
releases and EPEL) one had to write a custom script to do it.

The fact that BoxGrinder automates this entire process is the reason so 
many people recommend it.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Fedora S3 Backed AMIs

[Garrett Holmstrom]

On 2011-07-28 8:36, Tim Flink wrote:
> As I'm learning more about EC2 and AMIs, I'm wondering what the plans
> are for Fedora S3 backed AMIs.
>
> If I'm understanding correctly, an S3 backed AMI has no persistent
> storage - once the instance is rebooted, all changes are effectively
> reverted. This would mean that applying updates to an S3 backed AMI
> would be a questionable process and any updates requiring a reboot
> would be impossible.

Instance storage of this sort does not go away when instances are merely 
rebooted.  It is only destroyed at instance termination time.

> Are we planning to release S3 AMIs on a regular basis or are we
> expecting users to use the S3 AMIs to build their own images and take
> on the responsibility of updating those images on their own? Or are we
> focusing on the EBS backed AMIs that do have persistent storage?

Creating Fedora respins midway through a release consumes time and 
resources that, given our difficulty with the F15 images alone, we 
appear to lack.

Building updates into images also calls GPL compliance into question, as 
updates that are superseded by newer updates get removed from mirrors 
and eventually garbage-collected by koji.  This is not the case for 
packages in the release repository, which stick around forever, or the 
Fedora Unity respins, which provide source media alongside the binary media.

I wouldn't be too concerned about supplying people with updated cloud 
images; they are really only a baseline that people can respin for 
themselves and others as they wish.  People who wish to use EC2 merely 
as a pay-per-hour VPS provider can use either type of instance and keep 
it up to date via the usual channels like they are used to doing 
elsewhere.  People who are running cloud applications have designed 
their applications such that individual instances are unimportant and 
replaceable.  In those cases it is frequently easier to simply terminate 
out-of-date instances and replace them with new instances from fresh 
system images that cloud application writers already create as their 
applications evolve.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Fedora S3 Backed AMIs

[Garrett Holmstrom]

On Thu, Jul 28, 2011 at 11:16 AM, Hugh Brock  wrote:
> Amazon has deprecated S3 backed instances at this point.

Where did you learn this?  (Not doubting you, but wondering where you heard it)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud infrastructure package group

[Garrett Holmstrom]

On 2011-08-23 14:25, Peter Robinson wrote:
> On Tue, Aug 23, 2011 at 10:09 PM, Stephen John Smoogen  
> wrote:
>> On Mon, Aug 22, 2011 at 12:52, David Nalley  wrote:
>>> On Mon, Aug 22, 2011 at 2:00 PM, Bill Nottingham  wrote:
 https://bugzilla.redhat.com/show_bug.cgi?id=731712

 The HekaFS maintainers were looking for a appropriate group for their
 package. I was thinking that perhaps having a 'cloud infrastructure'
 or 'cloud support' group might be the best place, but we don't have
 one of those, and I'm not sure what all packages should be in it.

 Would someone fom the Cloud SIG like to take a stab at it?

 Thanks,
 Bill

>>>
>>>
>>> While I am happy to do this, haven't we already hit string freeze for
>>> F16 (August 2nd per the schedule)? So we are talking about
>>> comps-f17.xml.in?
>>>
>>> If I were to do so I think I'd put the following in the group:
>>>
>>> eucatools
>>> aeolus
>>> deltacloud
>>> sheepdog
>>> ceph
>>> glusterfs
>>> hekafs
>>> boxgrinder
>>
>> I am guessing that there will also be a need to have what is optional
>> and required...
>
> I would possibly suggest that they're all optional, there's lots of
> different cloud technologies there a lot of which are completely
> standalone separate products that aren't required to interoperate. By
> having them all optional there's a menu with the list there and people
> can select the particular type of cloud technologies they wish to use.

+1; marking them all as optional would be enough to get them into the 
appropriate menus without adding irrelevant packages to installs. 
(Basically what everyone else has already written)

In the interest of preventing a mis-typed package name in the yum group, 
note that the Eucalyptus/EC2 CLI tools are called "euca2ools", not 
"eucatools".  Upstream seems to enjoy word-play.  ;)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Beta AMI announcement?

[Garrett Holmstrom]

On 2011-10-04 12:06, Robyn Bergeron wrote:
>
> On Oct 4, 2011 12:59 PM, "Max Spevack"  > wrote:
>  >
>  > Hi folks,
>  >
>  > I know that there was work afoot to release the F16 Beta AMi on the same
>  > day as the rest of F16 Beta.
>  >
>  > Is that something that is good to go?  And if it's not, what was the
>  > thing that blocked or held up the release of the Beta?  The last AMI
>  > that I can see owned by a...@fedoraproject.org
>  is
>  > 125523088429/Fedora-16-Beta-ec2-20110923-x86_64-sda -- I played around
>  > with that AMI and while I think there were a few cloud-init issues, it
>  > worked well enough to get the authorized_keys in the right place.
>  >
>  > I'd also recommend that it become part of the schedule and workflow to
>  > update the get.fedoraproject.org  page
> with a link to the AMI IDs, as
>  > well as the EC2 Images wiki page itself, with those AMI IDs.  Not only
>  > for Alpha/Beta, but definitely for GA.
>  >
>  > Maybe even add a blurb to the release announcement when it talks about
>  > the different ways to consume Fedora.
>  >
>  > To me, those are the sorts of actions that begin to put Fedora on EC2 at
>  > a similar level to some of the other architectures, etc.
>  >
>  > Good think Fedora's PM and RelEng are on this list :)
>
> And the person who writes the release announcements, right? Heh.
>
> I have no idea what the status is. Dennis mentioned the other day that
> the latest thing he had wasn't working (while we were in milan) - beyond
> that, without a ticket or etc. i have no idea.

But I do.  :)  cloud-init is currently doing things that make SElinux 
unhappy in such a way that one cannot log in.  Dennis and I are working 
on the problem.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Beta AMI announcement?

[Garrett Holmstrom]

On 2011-10-04 20:14, Garrett Holmstrom wrote:
> On 2011-10-04 12:06, Robyn Bergeron wrote:
>> I have no idea what the status is. Dennis mentioned the other day that
>> the latest thing he had wasn't working (while we were in milan) - beyond
>> that, without a ticket or etc. i have no idea.
>
> But I do. :) cloud-init is currently doing things that make SElinux
> unhappy in such a way that one cannot log in. Dennis and I are working
> on the problem.

[Insert commentary about replying to oneself here]  8^)

I just tested a small fix by hand that appears to resolve the login 
issue.  When I get a chance I will spin a new cloud-init package that 
contains it in the hopes that we can get a working image out of koji.

An unrelated issue I noticed is that the images that koji generates seem 
to contain no man pages, and even installing packages that are supposed 
to contain them does not make them appear on the system.  Any ideas?

root@i-8d7dd3ca ~ # find /usr/share/man -type f | wc -l
0
root@i-8d7dd3ca ~ # yum -y install man-pages >/dev/null
root@i-8d7dd3ca ~ # find /usr/share/man -type f | wc -l
0
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: EC2 re-imbursement for the cloud test day?

[Garrett Holmstrom]

On 2011-10-10 4:33, Mo Morsi wrote:
> Alternatively, would it be possible for us to setup / share a single
> cloud account for the various testers to use? We could activate it for
> that day only and thus people would not have to sign up for the cloud
> providers on their own to test the software. Would just lower the
> boilerplate to testing the cloud w/ Fedora all that more.

+1; this is exactly the sort of thing that AWS's IAM service excels at. 
  You can create a user for each tester, use ACLs to pick what they are 
allowed to do, and remove the users when the test day is over.

It makes billing a lot easier, too.  ;)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: EC2 re-imbursement for the cloud test day?

[Garrett Holmstrom]

On 2011-10-10 8:40, Robyn Bergeron wrote:
> On Mon, Oct 10, 2011 at 8:25 AM, Garrett Holmstrom
>   wrote:
>> On 2011-10-10 4:33, Mo Morsi wrote:
>>> Alternatively, would it be possible for us to setup / share a single
>>> cloud account for the various testers to use? We could activate it for
>>> that day only and thus people would not have to sign up for the cloud
>>> providers on their own to test the software. Would just lower the
>>> boilerplate to testing the cloud w/ Fedora all that more.
>>
>> +1; this is exactly the sort of thing that AWS's IAM service excels at.
>>   You can create a user for each tester, use ACLs to pick what they are
>> allowed to do, and remove the users when the test day is over.
>
> Since you sound all expertish :) ... If I make a new account for that
> type of purpose, can I either get you to do the user creation
> magic/ACLs, or can you give me the walk-through? :D

Sure.  Doing it myself wouldn't teach anyone anything, so when I get a 
chance I will write up a couple wiki pages.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud infrastructure package group

[Garrett Holmstrom]

On 2011-10-16 13:21, Robyn Bergeron wrote:
>  > On Tue, Aug 30, 2011 at 01:49:22PM -0500, Robyn Bergeron wrote:
>  > > > Bill Nottingham (nott...@redhat.com )
> said:
>  > > >> https://bugzilla.redhat.com/show_bug.cgi?id=731712
>  > > >>
>  > > >> The HekaFS maintainers were looking for a appropriate group for
> their
>  > > >> package. I was thinking that perhaps having a 'cloud infrastructure'
>  > > >> or 'cloud support' group might be the best place, but we don't have
>  > > >> one of those, and I'm not sure what all packages should be in it.
>  > > >>
>  > > >> Would someone fom the Cloud SIG like to take a stab at it?
>  > > > Thanks for the suggestions - the initial group has been added in
>  > > > comps for Fedora 17. Feel free to update it.
>
> Also - thoughts on cloud-init?

Cloud-init is for VM images, not cloud infrastructure.  You usually need 
to have a very specific reason to want it on a system.  Then again I 
have no idea what group, if any, would be most appropriate for it.

-- Garrett "Just my two cents" Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: EC2 re-imbursement for the cloud test day?

[Garrett Holmstrom]

On 2011-10-11 8:03, Garrett Holmstrom wrote:
> On 2011-10-10 8:40, Robyn Bergeron wrote:
>> Since you sound all expertish :) ... If I make a new account for that
>> type of purpose, can I either get you to do the user creation
>> magic/ACLs, or can you give me the walk-through? :D
>
> Sure. Doing it myself wouldn't teach anyone anything, so when I get a
> chance I will write up a couple wiki pages.

Here you go:

https://fedoraproject.org/wiki/User:Gholms/EC2_Primer
https://fedoraproject.org/wiki/User:Gholms/IAM_Primer
https://fedoraproject.org/wiki/User:Gholms/IAM-Sponsored_EC2_Test_Days

I suggest reading them in order.  Feedback is welcome.

-- Garrett "The more you know" Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Cloud SIG meeting today

[Garrett Holmstrom]

Hi, everyone!

Today's meeting is quickly approaching, so come join us in
#fedora-meeting at 1900 UTC (2 PM EST; 11 AM PST).  Note the time
change for everyone who left daylight saving time last weekend.

-- gholms
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: fedora 16 AMI on EC2

[Garrett Holmstrom]

On 2011-12-19 10:05, Jared K. Smith wrote:

On Mon, Dec 19, 2011 at 1:00 PM, Sandeep Dixit  wrote:

Yes  - the instance is associated with the "default" security group
and the "decault" security group has a rule 0.0.0.0 - port 80 / http


I think Steve was referring to the iptables firewall on Linux itself.


It is my understanding that EC2 images generally have their inbuilt 
firewalls turned off because people are expected to use security groups 
instead.  Is this correct?  If so, how can I help that happen for the 
next Fedora release's EC2 images?

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: fedora 16 AMI on EC2

[Garrett Holmstrom]

On 2011-12-19 22:24, Dennis Gilmore wrote:

El Mon, 19 Dec 2011 20:07:31 -0800
Garrett Holmstrom  escribió:

On 2011-12-19 10:05, Jared K. Smith wrote:

On Mon, Dec 19, 2011 at 1:00 PM, Sandeep
Dixit   wrote:

Yes  - the instance is associated with the "default" security group
and the "decault" security group has a rule 0.0.0.0 - port 80 /
http


I think Steve was referring to the iptables firewall on Linux
itself.


It is my understanding that EC2 images generally have their inbuilt
firewalls turned off because people are expected to use security
groups instead.  Is this correct?  If so, how can I help that happen
for the next Fedora release's EC2 images?


correct me if im wrong but i thought that the amazon security groups
only protects you from whats outside of amazons network. the hosts
firewall protects you from being attacked from inside of ec2.


Hosts within a security group can communicate freely with one another. 
Communication between hosts in different security groups is subject to 
security group rules in the same way as that with the Internet at large. 
 Since security groups cannot be shared all instances in a given 
security group belong to the same account.

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: instance store-backed AMIs

[Garrett Holmstrom]

On Jan 23, 2012 7:02 AM, "Simon Smith"  wrote:
>
> Is there a particular reason why there are not any instance
> store-backed AMIs for Fedora?  I noticed that starting with Fedora 16,
> there are only EBS-backed EC2 images listed at:
>
> https://fedoraproject.org/wiki/Cloud_images
>
> I know I could create my own, but at this stage of our project I would
> much rather use a ready-to-go image.  I like the instance store-backed
> AMIs because we have been relying on the extra storage provided (>100
> GB), deploying multiple machines to provide redundancy.

Provided you aren't using t1.micro instances, you can still use instance
storage from an EBS instance. You just have to ask for it by supplying an
appropriate block device mapping (with -b if you are using
euca-run-instances) when you run the instances.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Fedora Torrent seeder plans and future

[Garrett Holmstrom]

On 2012-01-29 15:35, Robyn Bergeron wrote:

On 01/28/2012 08:45 PM, Max Spevack wrote:

On Sat, 28 Jan 2012, Matt Domsch wrote:

Responding here, but assuming that future messages will probably move
to the cloud@ mailing list.

Ding ding ding. :) I'm cc'ing the illustrious, world-famous cloud sig
mailing list (https://admin.fedoraproject.org/mailman/listinfo/cloud for
those of you who want to get in on the action). Whoever responds there
first: You're tasked with removing advisory-board from your reply. :)

I'd love to see this mini- (or perhaps mega-, I'm not sure how enormous
the scope is here) project move forward - I wonder if this is something
we can pull together before we do F17 alpha or beta, so we can get it
tested and make sure our Beefy Miracle is kosher on AWS/S3/EC2 for
final. :) Some of that presumably depends on the bandwidth of mdomsch
and dgilmore, but there are probably bits and pieces where others can
pitch in.


What is the purpose of putting package mirrors in S3?  Network traffic 
inbound to EC2 is free, so users have little to gain.  It will also cost 
the Fedora Project non-trivial amounts of time and money to run entire 
mirrors there and keep them up to date.


The original purpose of this thread was to discuss torrent seeds for 
Fedora disk images.  S3 seems like a reasonable way to accomplish this 
with a minimum amount of work, but please don't conflate a solution to 
media distribution and an entire package mirror that will take much more 
resources to run.

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init and F16

[Garrett Holmstrom]

On 2012-02-21 16:16, Aaron Bento wrote:

I'm looking to provide some feedback on cloud-init and the F16 AMI's.
I'm not sure the proper place to file the bug against. I'm attempting to
place a simple shell script in user data to perform post-boot
configuration. This fails since I'm unable to execute anything within
user data. (I'm hoping eventually to use cloud-config syntax)


Please file cloud-init bugs against cloud-init.  :-)  Since it got into 
F16 as a last-minute effort only the minimum amount functionality 
necessary to make an EC2 image start actually works.  So the more bugs 
we can root out on Fedora, the better.



The cloud-init startup process (sometime after placing the ssh keys) at
one point runs: /usr/bin/cloud-init-cfg all final During this stage,
run-parts gets run to actually execute the downloaded user data. It
fails with the following:

CalledProcessError: Command '['run-parts', '--regex', '.*',
'/var/lib/cloud/instance/scripts']

The --regex flag is not accepted by the Fedora version of run-parts.
Debian/Ubuntu both have a compiled binary of run-parts that accepts this
flag.


I spoke with cloud-init's upstream maintainer about this a few days ago. 
 AFAICT the ideal way to fix it is to stop using run-parts altogether 
since it is only there as a holdover from when cloud-init was just a 
shell script.  If this turns out to be impossible to fix in the near 
future we will just have to carry a patch.

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init and F16

[Garrett Holmstrom]

On Feb 22, 2012 5:32 PM, "Aaron Bento"  wrote:
>
> On Tuesday, February 21, 2012 at 10:06 PM, Garrett Holmstrom wrote:
>> Please file cloud-init bugs against cloud-init. :-)
>
> Thanks for the help! This was the sort of guidance I was looking for. And
thanks for filing the patch/bug. I'll use this as an example the next time
I run across such an issue.

There is also a cloud-init component in bugzilla, so feel free to take your
pick.  I will see it either way.
On Feb 22, 2012 5:32 PM, "Aaron Bento"  wrote:

> On Tuesday, February 21, 2012 at 10:06 PM, Garrett Holmstrom wrote:
>
>
> Please file cloud-init bugs against cloud-init. :-)
>
>
> Thanks for the help! This was the sort of guidance I was looking for. And
> thanks for filing the patch/bug. I'll use this as an example the next time
> I run across such an issue.
>
> --
> Aaron Bento
>
> ___
> cloud mailing list
> cloud@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/cloud
>
>
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Launching a Fedora 16 EC2 Instance

[Garrett Holmstrom]

On Mar 5, 2012 10:11 AM, "Juan Rodriguez"  wrote:
>
> Hey there,
>
> I'm new to Amazon EC2 / Cloud providers in general, but I tried to set up
using Fedora's F16 Images (Available here [1]) using ami-5f16d836. It
apparently passes all of Amazon's checks, however I can't ping the instance
once it's fully set up. I decided to check the System Log and noticed this:
>
> Determining IP information for eth0.../etc/init.d/functions: line 58:
/dev/stderr: Permission denied /etc/rc.d/init.d/functions: line 58:
/dev/stderr: Permission denied /etc/init.d/functions: line 58: /dev/stderr:
Permission denied /etc/rc.d/init.d/functions: line 58: /dev/stderr:
Permission denied

This always happens. I am equally clueless as to what it means, but AFAIK
it doesn't break anything.

> Complete logs for both the 32 bit and 64bit images can be seen here:
>
> 32 bit: http://pastebin.com/DEcRaaDn
> 64 bit: http://pastebin.com/tvhqjJ1S
>
> Anyone had better luck using Fedora's Amazon EC2 instances? Did I do
something wrong during setup?

Did you authorize SSH access to your instance's security group?

euca-authorize -p 22 default
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: help needed

[Garrett Holmstrom]

On Mar 28, 2012 9:06 PM, "Heherson Pagcaliwagan"  wrote:
>
> On Thu, Mar 29, 2012 at 12:22 AM, Dennis Gilmore  wrote:
> > i have uploaded a x86_64 image to us-east-1 in ec2 the ami is
> > ami-3fb16f56 for whatever reason that i can not yet figure out the
> > image is booting fine but ssh will not allow me to connect. ive stopped
> > the image and attached it to a f16 instance and examined the disk and
> > it all looks fine. with the ssh logs just saying that the client
> > disconnected.
> >
> > Id appreciate if some people could have a look and see if its working
> > for them or help diagnose what exactly is going on.
>
> Not sure if this is it, but I did not see a /root/.ssh/authorized_keys
file.

This is expected.  Look for it under /home/ec2-user instead.

Of course, if it isn't there either then there may be a problem.  ;-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: help needed

[Garrett Holmstrom]

On Mar 31, 2012 6:44 AM, "Andy Grimm"  wrote:
>
> SOLVED!
>
> From /usr/share/doc/cloud-init-0.6.3/ChangeLog :
>
> "read /etc/ssh/sshd_config for AuthorizedKeysFile rather than assuming
> ~/.ssh/authorized_keys (LP: #731849)"
>
> The problem is that this change in cloud-init does not properly handle
> relative paths, which are documented in the sshd_config manpage as
> being relative to the user's home directory.  So the quick fix was to
> change /etc/ssh/sshd_config from:
>
> AuthorizedKeysFile  .ssh/authorized_keys
>
> to:
>
> AuthorizedKeysFile  %h/.ssh/authorized_keys
>
> The more correct fix is in cloud-init, probably something like:
>
> --- a/cloudinit/SshUtil.py  2012-03-31 09:28:42.598996936 -0400
> +++ b/cloudinit/SshUtil.py  2012-03-31 09:40:47.758829938 -0400
> @@ -155,6 +155,8 @@
> akeys = ssh_cfg.get("AuthorizedKeysFile",
"%h/.ssh/authorized_keys")
> akeys = akeys.replace("%h", pwent.pw_dir)
> akeys = akeys.replace("%u", user)
> +if not akeys.startswith('/'):
> +akeys = os.path.join(pwent.pw_dir, akeys)
> authorized_keys = akeys
> except Exception:
> authorized_keys = '%s/.ssh/authorized_keys' % pwent.pw_dir
>
>
> How do you want to handle this?  Should I go ahead and file both RHBZ
> and LP issues for it?

If you're willing to, please do so. Otherwise I can forward a RHBZ bug to
Launchpad.

Thanks for figuring this out!
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Fedora 17 Beta RC images

[Garrett Holmstrom]

On Apr 5, 2012 2:34 PM, "Dennis Gilmore"  wrote:
>
> However you use them regularly. One thing I did see was iptables failed
to start. But did load fine when done manually.

Few EC2 images run their own firewalls, as security groups already perform
that function. Why should Fedora's?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: SSH Fingerprint

[Garrett Holmstrom]

On Apr 9, 2012 3:26 PM, "Alan Gutierrez"  wrote:
>
> Andy
>
> Thank you for responding. That is what I expected to find, the server's
ssh key printed to the console output. But, I don't see it there. Not when
using one of the Fedora 16 AMIs.
>
> http://fedoraproject.org/wiki/Cloud_images
>
> Here's a gist of the output of the California hosted Fedora 16 i386 AMI
ami-25e0bc60.
>
> https://gist.github.com/2345554
>
> I'm using a small instance.
>
> It this something that is a known issue? Or do people generally build
their own instances using BoxGrinder?

I suspect cloud-init prints the fingerprints and then systemd eats the
output as it blindly redirects all daemons' output to syslog. That can
probably be fixed in cloud-init's unit files if that is the case.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: [Openstack] centos 6 images

[Garrett Holmstrom]

On Tue, May 22, 2012 at 10:49 AM, Joshua Harlow  wrote:
> U might want to check out,
>
> https://github.com/yahoo/Openstack-Condense
>
> Its a stripped down/cleaned up/... version of cloud-init that I know works
> on RHEL6.
>
> I tried to improve the following:
>
> Code cleanliness (constants being uppercase, paths using os.path.join and
> so-on)
> Stripping out some of the odd handlers (byobu, right-scale and such)
> Improving logging by a lot (so that u can debug this thing)
> Making what handlers I left work on RH and ubuntu...
>
>
> Might be useful if u want to try it.
>
> I know just from doing the above work that the cloud-init for ubuntu,
> requires some work to get it to work on RH, but not tons, eventually I hope
> that I can merge this back, but for now its forked so that I could focus on
> getting it working and cleaned up, rather than pushing code through some
> review process via launchpad and such (ie the slow as molasses approach).

As Fedora's cloud-init maintainer I am happy to help you merge patches
upstream so everyone can benefit from your work.  I can also teach you
how to disable handlers without completely removing them from the
source tree.  ;-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: [Openstack] centos 6 images

[Garrett Holmstrom]

On 2012-06-04 2:13, Pádraig Brady wrote:

On 06/04/2012 07:19 AM, Sam Bashton wrote:

On Sun, Jun 03, 2012 at 11:17:09AM +0100, Pádraig Brady wrote:

Good find. Those SRPMs are here:
http://rpm.razorsedge.org/centos-6/bashton/www/SRPMS/
I've CC'd the rpm creator at bashton.

They seem to be based on the 0.5.15 version from amazon,
variants of which are also attached here:
https://bugs.launchpad.net/cloud-init/+bug/655837

So hopefully we can get something into EPEL soon,
between the above and my quick untested attempt of packaging the latest:
http://pbrady.fedorapeople.org/cloud-init-el6/

I'm willing to review or push packages through the EPEL process.


Happy to help out without whatever you need to get these into EPEL.  All
I did for these was to grab the Amazon Sources and fix the Requires,
which IIRC contained some Amazon Linux package names.


OK cool.

What we might do is continue preparing the latest 0.6.x version for EPEL,
while comparing against your built packages to ensure we've not broken anything,
and then push any required tweaks back upstream,
who are very receptive to changes supporting multiple distros.


Amazon's patchset is quite extensive and arguably over-engineered, but 
upstream is nonetheless interested in keeping things as compatible as 
possible.  You seem to have already found the upstream bug for doing 
just that.  :)


I'm the cloud-init maintainer for Fedora, so if you would like help or a 
co-maintainer I'm happy to give you a hand.

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

RFC: Cloud Interop FAD

[Garrett Holmstrom]

Hi, Everyone,

Now that a lot of cloud software is in or on its way into Fedora, I 
wonder how easy it is to actually make use of it.  We seem to have 
installation guides, but not much about how to actually employ the cloud 
to do something useful.  For instance, now that OpenShift is on the 
feature list, Fedora's Cloud Guide could use some recipes for OpenShift 
images on, say, OpenStack and Eucalyptus.


So in the interest of Getting Things Done, I suggest that we gather for 
a high-bandwidth Fedora Activity Day to work on recipes for making the 
various bits of cloud software in Fedora work in concert (i.e. for 
making $PaaS work on $IaaS), and fix whatever bugs we encounter during 
the process.


Which of you are interested in attending such an event?  What would you 
like to work on there?  Specific goals are the most important thing to 
have when planning a FAD to make the day the most likely to result in 
something useful.


What do you all think?

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: RFC: Cloud Interop FAD

[Garrett Holmstrom]

On 2012-06-18 9:30, Andy Grimm wrote:

Overall, I definitely like the idea of developing $PaaS on $IaaS
narratives for Fedora, as we so often see that people using a cloud
don't architect their apps for a cloud.  Anything we can do to promote
"proper" cloud apps is a good thing.

I don't know much about Openshift yet, but wonder if the FAD agenda
should narrow it even more to start with -- I know OpenShift is
designed to support multiple languages / frameworks, and perhaps it
makes sense to start with one.  In other words, the FAD goal is make
sure that Fedora has a full narrative for setting up $IaaS and getting
all the way to, say, a running Django app.  At that point,
substituting another framework is probably just a matter of pointing
to another OpenShift example in github or the OpenShift docs.


I simply used the first names that came to my mind out of the software 
that I know is in/on its way into Fedora; it doesn't have to be 
OpenShift specifically.  Sorry if it sounded like that.


You make a good point, but I'll take it one further:  perhaps getting 
everyone together to craft a single, very complete narrative at the FAD 
would provide everyone with a much-needed "whole story" that people can 
later tweak to work with different software stacks at all levels.  After 
this "initial" work is done it would make it that much easier for people 
to come up with good narratives on their own later.


How does that sound to you folks?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: RFC: Cloud Interop FAD

[Garrett Holmstrom]

On 2012-06-18 9:33, Mo Morsi wrote:

I suggest checking out Aeolus [1] as part of this as well. Its a
software suite that we're developing for Fedora to enable cross cloud
communication and compatibility.

I can help coordinate some hacking on and using Aeolus as part of this.
We can add support for new cloud providers, develop some tools to launch
instances against cloud providers, write up some docs, and more.


That's a good thought.  AFAIK the kinds of tasks that benefit the most 
from FADs tend to be those that:


1) benefit from high levels of communication between task forces that 
don't usually work on the same thing at once

2) get something intrinsically useful done in a day

Can you think of any Aeolus-related tasks that fit the bill?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: [Openstack] centos 6 images

[Garrett Holmstrom]

On Jun 27, 2012 3:43 PM, "Pádraig Brady"  wrote:
> I've been informed that there is a significant cloud-init rework
> in progress, being seriously considered for merging,
> and with a central motivation of RH / Fedora consume-ability.
>
> It's 374 commits starting at:
> http://bazaar.launchpad.net/~cloud-init/cloud-init/rework/changes/579

Nice! I will certainly take a look at it.

Which parts still need work?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: python-boto-2.5.1-1.el6 breaks cloud-init

[Garrett Holmstrom]

On 2012-07-06 23:15, Jan van Eldik wrote:

Just to say that python-boto-2.5.2-1.el6 (*) addresses the
issue.

  thanks, cheers, Jan

(*) http://koji.fedoraproject.org/koji/buildinfo?buildID=329388


Thanks for testing the fix!  If anyone else is interested in testing the 
update you can install it from epel-testing and +1 it here:


https://admin.fedoraproject.org/updates/FEDORA-EPEL-2012-6341

F17 has a similar bug report and an identical bugfix in updates-testing. 
 Testers for that would also be appreciated.  :-)


https://bugzilla.redhat.com/show_bug.cgi?id=837661
https://admin.fedoraproject.org/updates/FEDORA-2012-10373
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Cloud-init on RHEL5

[Garrett Holmstrom]

On Fri, Jul 13, 2012 at 4:37 AM, Pádraig Brady  wrote:
> On 07/13/2012 09:28 AM, Jan van Eldik wrote:
>> Do you know it the cloud-init rework will include support for RHEL5? We 
>> would like to contribute to that, but we are not sure what the best way 
>> forward is.
>>
>> It seems there are different ways to make cloud-init work under RHEL5:
>>
>>   - make it work w/ the system python24.
>>   - make it work w/ python26 from EPEL5
>>
>> We have a preference for the second option (as it is more forward-looking), 
>> but there may be constraints we have not considered.
>>
>> How do people feel about this?
>
> I didn't notice anything in the rework specific to python 2.4
> I also notice that Amazon's variant specifies /usr/bin/python2.6
> So I think at this stage python 2.6 is the earliest considered.

As of the last time I looked, a patch set that makes cloud-init work
with python 2.4 would be quite pervasive, not to mention that the last
version of python-boto to work with 2.4 is quite old.  EPEL already
has a python 2.6 stack, so I agree with the rest of you:  we might as
well use it.  ;-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Plan for adding RHEVm and vSphere support to Cloud-Init

[Garrett Holmstrom]

On Tue, Jul 17, 2012 at 12:03 PM, jvlcek  wrote:
> I just wanted to let folks know what is happening regarding cloud-init
> support for RHEVm and vSphere and what OSs.
>
> Currently cloud-init only supports Ec2. A version is available
> for Fedora and EPEL.
>
> I am working on updating upstream to also support sourcing
> user data for RHEVm and vSphere, using the same technique we do
> with Audrey from a Delta-Cloud launch.
>
> Once I have this pushed upstream I plan to push it to Fedora and then
> out to RHEL targeting release 6.4

Do you mean in RHEL itself or in EPEL?  If cloud-init is going to make
it into a RHEL channel then it would be useful if we could plan for
that in advance.

> See: BZ 838659 - https://bugzilla.redhat.com/show_bug.cgi?id=838659

Are we supposed to be able to see that bug?  :-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Plan for adding RHEVm and vSphere support to Cloud-Init

[Garrett Holmstrom]

On 2012-07-18 6:27, jvlcek wrote:

On 07/17/2012 04:49 PM, Garrett Holmstrom wrote:

On Tue, Jul 17, 2012 at 12:03 PM, jvlcek  wrote:

Once I have this pushed upstream I plan to push it to Fedora and then
out to RHEL targeting release 6.4

Do you mean in RHEL itself or in EPEL?  If cloud-init is going to make
it into a RHEL channel then it would be useful if we could plan for
that in advance.


Yes the plan is to make it into the RHEL channel. We are just starting
the planning now. It would be helpful to me if you could please pass
along any input for the planning phase that you have.


When porting things to new guests and clouds and whatnot it might be 
worth taking a look at the new extensions that the EC2 people made to 
cloud-config files in case anything new that people have to write can 
employ the same methods of configuration.


I will ask around for other input.  I sincerely appreciate your 
mentioning this beforehand so we all have a chance to help out.  :-)



See: BZ 838659 - https://bugzilla.redhat.com/show_bug.cgi?id=838659

Are we supposed to be able to see that bug?  :-)


Sorry I hadn't realized that is internal.

That BZ is for planning for getting this into RHEL.

I'll check into making it public.


Thanks!


Also I know you are the fedora maintainer. Do you have any plans
yet for updating the Fedora bits to newer upstream? I'd be glad to
help with that.


Absolutely!  Now that boto works again I can work on the bugfixes I was 
putting off.  8^)

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Plan for adding RHEVm and vSphere support to Cloud-Init

[Garrett Holmstrom]

On 2012-07-22 3:31, Itamar Heim wrote:

On 07/19/2012 09:54 AM, Garrett Holmstrom wrote:

When porting things to new guests and clouds and whatnot it might be
worth taking a look at the new extensions that the EC2 people made to
cloud-config files in case anything new that people have to write can
employ the same methods of configuration.


can you please provide some link / examples on these extensions?
are they all based on userdata, or something else?


Most of them are, yes.  Amazon's documentation for it is in the EC2 user 
guide [1].  Unfortunately, it isn't as thorough as I thought it would 
be, but Scott *did* manage to fetch their source package and attach it 
to a Launchpad bug about integrating changes from Amazon's fork [2]. 
Most of their customizations revolve around package and repo management. 
 The last time I looked, their code didn't really take advantage of 
cloud-init's existing module mechanism and configuration file, but 
regardless, it could be useful as inspiration.


[1] 
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AmazonLinuxAMIBasics.html#CloudInit

[2] https://bugs.launchpad.net/cloud-init/+bug/655837
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: potential meeting times

[Garrett Holmstrom]

On 2012-09-25 7:24, Matthew Miller wrote:

-1 is free except for on Wednesday at 10pm UTC. But in general, I think the
earlier timeslots are better for most people anyway, except for the West
Coast US. And there's an open slot in #fedora-meeting for 2pm UTC Tuesday.


As someone on the US's West coast, I would be okay with a later 
"catch-up" meeting as long as people also communicate on the mailing 
list.  My greatest concern is communication with release engineering, 
who produce Fedora's cloud images, as historically the cloud SIG 
meetings have been crucial for that.

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

cloud-init update

[Garrett Holmstrom]

Hi, everyone,

I just pushed the final (i.e. non-snapshot) version of cloud-init
0.7.0 to F18 and rawhide, which fixes a great deal of compatibility
bugs, makes most of the plugins work properly, and allows one to
configure the users on the system using instance user-data a lot
better than previous version.  There is one fairly important change to
be aware of relative to F17's version:  this version allows you to
change the user that it creates at startup time (and in fact add even
more), so it is recommended to *not* pre-create a user when you build
the image.  What do we need to do to remove that from Fedora's cloud
image kickstart before we start building test images?

I'd like to push this to F16 and F17 as well to replace the horribly
broken versions that those operating systems shipped with.  Any
objections?

How about EPEL?  This update fixes so many issues that I believe it's worth it.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: ec2/fedora-18-i386-ec2.ks ec2/fedora-18-x86_64-ec2.ks

[Garrett Holmstrom]

On Thu, Oct 11, 2012 at 1:23 PM, Itamar Reis Peixoto
 wrote:
> why fedora ec2-image doesn't have a swap ?

The standard in EC2 is to use ephemeral storage for swap.
Unfortunately, different instance types put their ephemeral storage in
different places, and some have none at all, so I suggest using a
script to manage swap space as the instance starts.  You can do this
by uploading a script as the instance's user data:

euca-run-instances ami-2ea50247 ... --user-data-file=some-shell-script.sh

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: ec2/fedora-18-i386-ec2.ks ec2/fedora-18-x86_64-ec2.ks

[Garrett Holmstrom]

On 2012-10-11 13:54, Matthew Miller wrote:

On Thu, Oct 11, 2012 at 01:48:29PM -0700, Garrett Holmstrom wrote:

why fedora ec2-image doesn't have a swap ?

The standard in EC2 is to use ephemeral storage for swap.
Unfortunately, different instance types put their ephemeral storage in
different places, and some have none at all, so I suggest using a
script to manage swap space as the instance starts.  You can do this
by uploading a script as the instance's user data:
euca-run-instances ami-2ea50247 ... --user-data-file=some-shell-script.sh


Hmmm. Can we include that to make things nicer?


Sure, if someone can figure out how to do that reliably.  We can 
certainly try to guess, but the location of the ephemeral storage is 
going to change based on both the instance type and where the user asks 
EC2 to put it.  t1.micro instances aren't entitled to any ephemeral 
storage at all, and other EBS instances don't get any either unless the 
user explicitly asks EC2 for it.  And that's just Amazon's cloud.



(I was just noticing that the kickstart puts /dev/xvda3 as swap in fstab on
i386 only, so we're already halfway trying to do something conditional.)


Using /dev/xvda3 for swap space used to be the standard on 32-bit 
m1.small instances when those instances were backed by instance storage. 
 It probably still is, too, but we don't publish those images any more 
anyway.


I'm not saying that what you're suggesting can't be done; it's just not 
as simple to make it "just work" everywhere as it first appears.  Anyone 
up to the challenge?  ;-)


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init update

[Garrett Holmstrom]

On 2012-10-17 13:35, Tomas Karasek wrote:

just tried to test cloud-init-0.7.0-1 from Alan's build from
http://koji.fedoraproject.org/koji/taskinfo?taskID=4582433
on quite vanilla image of SLC6 which is a derivate of EL6.

it seems they use argparse, which was added in Python in 2.7. On EL6 (and 5 as 
well I guess) there is python-argparse in separate package so it should go to 
dependencies of cloud-init 0.7.

Starting cloud-init: Traceback (most recent call last):^M^M
File "/usr/bin/cloud-init", line 24, in ^M^M
import argparse^M^M
ImportError: No module named argparse^M^M


Sounds like the el6 build should depend on python-argparse, then. 
That's already in EPEL.


I'm still trying to reproduce Vitaly's login problem and not having much 
luck.  :-\  Is anyone else able to do so and post logs? 
(/var/log/cloud-init.log)

___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: do we want `less` and `man` in the default cloud image?

[Garrett Holmstrom]

On Fri, Oct 19, 2012 at 8:58 AM, Matthew Miller
 wrote:
> Continuing a discussion with gholms on IRC. There's a general question of
> how minimal we want the cloud image to be overall. I'd like to make it
> reasonably small (where reasonable is some hand-wavy value), but we're not
> ever going to be a stripped down TinyLinux or Damn Small Linux.
>
> Garrett makes the point that the Fedora images should "feel like Fedora". We
> can probably discuss what exactly that means, but, okay, I can see that as a
> basic principle to weigh against "as tiny as possible". He also mentioned
> earlier that he has gotten feedback that people like to have man pages
> available. Right now, we're shipping about 6.2MB (on-disk) of man pages,
> but, um, no man browser. Adding man would add about 8.8M on-disk, as it
> pulls in groff and less. (Less is relatively tiny -- groff is the big
> thing.) *
>
>
> I really like having less around. I pretty much miss it on every system
> where it isn't installed. And installing all those man pages with no way to
> look at them seems silly.
>
> Thoughts? Non-binding votes?

The groff dependency is a bit unfortunate, but now that there's a bug
for it I'm less concerned about it.  Man pages aren't particularly
useful without a reader.  I suspect people have come to expect pagers,
too.  So I'm in favor of that.  ;-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init update

[Garrett Holmstrom]

On Thu, Oct 18, 2012 at 12:28 AM, Vitaly Kuznetsov  wrote:
> On 10/18/2012 05:47 AM, Garrett Holmstrom wrote:
>> I'm still trying to reproduce Vitaly's login problem and not having much
>> luck.  :-\  Is anyone else able to do so and post logs?
>> (/var/log/cloud-init.log)
>
> Hi Garrett,
>
> I can provide you with my exact sequence, maybe it is buggy itself? :)
>
> 1) Create F17 instance (ami-2ea50247, us-east-1 for example)
> 2) Login, update cloud-init:
> sudo yum -y install
> http://kojipkgs.fedoraproject.org//packages/cloud-init/0.7.0/1.fc18/noarch/cloud-init-0.7.0-1.fc18.noarch.rpm
> 3) Change user in /etc/clouf/cloud.cfg:
>  users:
>  - ec2-user2
> 4) remove old instance data:
> sudo rm -rf /var/lib/cloud/instance /var/lib//cloud/instances/i-*
> /var/log/cloud-init.log
> 5) reboot the instance
> 6) try to login using new 'ec2-user2' user and fail :(
> 7) login with old ec2-user and examine /var/log/cloud-init.log
> 8) ec2-user2 is created, but it has no /home/ec2-user2/.ssh/authorized_keys
> file

Ah, now I see what's going on!  Thanks for the info.  It seems that
the ssh plugin simply ignores everyone that isn't the default user.
:-\  Perhaps it should grab the first user when the default one isn't
there.  I'll check with upstream and try to come up with a fix.

The stock ec2-user user works when you delete its account before you
create the image, right?

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Please review / comment on / help with AWS Marketplace listing

[Garrett Holmstrom]

On Fri, Oct 26, 2012 at 5:55 AM, Matthew Miller
 wrote:
> On Thu, Oct 25, 2012 at 10:07:10AM -0400, Matthew Miller wrote:
>> Highlight1:
>>   Free and Open Source: built by a collaborative community using entirely
>>   free software.
>> Highlight2:
>>   Core Image Ready to Build Upon: use yum to add any of tens of thousands of
>>   software packages, or drop in your own code.
>> Highlight3:
>>   NEED SOMETHING HERE
>>   [Security? Fedora-ness? Values? Highlight some particlar awesomness?]
>
> Ideas, anyone?

We covered "freedom" with highlight 1.  We covered "friends" with the
note about community support.  "Features" is covered by highlight 2.

Guess what's left?  ;-)

https://fedoraproject.org/wiki/Foundations

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init package dependencies

[Garrett Holmstrom]

On Fri, Nov 9, 2012 at 2:34 PM, Florian La Roche
 wrote:
>> All told, cloud-init and dependencies add 50MB to the on-disk size of the
>> current F18 cloud image I'm working on.

Ow.  That's a lot more than I expected.  :-\

It's precisely the fact that it's built into the image that makes
cloud-init so useful.  You only get one user-data file when you run an
instance, so if we remove cloud-init from the image and make using it
a multi-step bootstrapping process then there isn't much point in
offering it at all.  Maybe there's a way to make it more modular.

> There should be a minimal version of it, whatever the packaging solution then
> looks like...

Right now it relies on a config file to list what plugins it's
supposed to run since said list of plugins can differ by distro, but I
bet one could make it try to compute the list automatically by looking
at all of the plugins.  Once it can do that, making it run with the
additional plugins should actually be quite straightforward.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init package dependencies

[Garrett Holmstrom]

On Tue, Nov 13, 2012 at 3:59 PM, Matthew Miller
 wrote:
> On Tue, Nov 13, 2012 at 02:59:50PM -0800, Garrett Holmstrom wrote:
>> It's precisely the fact that it's built into the image that makes
>> cloud-init so useful.  You only get one user-data file when you run an
>> instance, so if we remove cloud-init from the image and make using it
>> a multi-step bootstrapping process then there isn't much point in
>> offering it at all.  Maybe there's a way to make it more modular.
>
> You can ask for that user-data file more than once, though. It'd be kind of
> nice if *that* listed the plugins needed.

You might actually be able to do that now, but cloud-init doesn't yet
have a way to install plugins that are missing.  If that gets
implemented then this becomes easy!  :-)
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init package dependencies

[Garrett Holmstrom]

On Tue, Nov 13, 2012 at 4:38 PM, Matthew Miller
 wrote:
> On Tue, Nov 13, 2012 at 04:27:45PM -0800, Garrett Holmstrom wrote:
>> > You can ask for that user-data file more than once, though. It'd be kind of
>> > nice if *that* listed the plugins needed.
>> You might actually be able to do that now, but cloud-init doesn't yet
>> have a way to install plugins that are missing.  If that gets
>> implemented then this becomes easy!  :-)
>
> What about a service that runs before cloud-init itself, scans the user-data
> for needed plugins, and then yum-installs them if they're missing?
>
> It woudn't have to be that smart -- if it hit something complicated, it
> could just throw up its hands and install the whole shebang.
>
> The primary downside I can see is that it's probably slower to do that than
> to just have it all there.

That isn't going to work for every case, but it sounds like a great
"90%" starting point.  Maybe a little script could read user-data, and
if it contains cloud-config data, the script reads that and takes
action on the bits that it has to.

What do you folks think?  Anyone itching to give it a try?
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud-init update

[Garrett Holmstrom]

On Thu, Nov 15, 2012 at 2:59 AM, Tomas Karasek  wrote:
> Hey Alan, I tested 0.7.1 rpm build on SLC6 (EL6) built from
> https://github.com/apevec/cloud-init-el6.git
> and it's working alright. It's finally downloading the public keys.

Nice!  I will try this in Fedora as well.
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud and firewalld

[Garrett Holmstrom]

On 2012-12-12 7:27, Matthew Miller wrote:

This may be of interest to people using Fedora as a cloud solution, for
several reasons.

First, on _host_ systems providing virtualization services, the firewall
daemon provides an interface for tracking dynamic rules. (Libvirt already
has code to use it, for example.)

On cloud _guest_ systems, it's probably less desirable: the firewall is
unlikely to have dynamic changes, and resources will be more constrained.
Having an extra python-based daemon running all the time with literally
nothing to do probably isn't what we're looking for, and it also happens
that the code pulls in a large list of dependencies.


How much memory does firewalld actually use on F18 when it has nothing 
to do?  At what point should we become concerned about how much memory a 
process is using?



The FirewallD feature page proposes that both options should be available
for at least the next few Fedora releases (just as we have the legacy
network scripts). But right now, the appliance building tools and anaconda
both rely on the new firewalld commands. I suggested putting that back to
the old way for now, but that's going to take some work and testing.


Does the "no firewall" case still work, at least?  EC2 recommends images 
with *no* default firewall since they use security groups to control 
traffic, and adding a second, guest-level firewall tends to confuse people.


Should the F18 release image explicitly target clouds other than EC2? 
*Can* it?


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: tmp on tmpfs in the cloud images: good or bad?

[Garrett Holmstrom]

On 2012-12-12 16:42, Matthew Miller wrote:

The /tmp filesystem is on tmpfs by default in F18:
http://fedoraproject.org/wiki/Features/tmp-on-tmpfs (and affirmed by FESCO
here: https://fedorahosted.org/fesco/ticket/940#comment:45).

I'm agnostic about this as a default for Fedora overall, but I can see it as
problematic on cloud/virt guests where RAM is usually more constrained. (In
fact, there's a bugzilla report to that effect here
https://bugzilla.redhat.com/show_bug.cgi?id=858265).

The feature page and release notes explicitly call out tmpfs as
"Administrators can override this" so I don't think we would be going too
far off the path if we disable it by default in the cloud images.

What do you think?


I can think of two trains of thought here:  swap requirements and the 
size of /tmp itself.


The cost of putting /tmp on tmpfs in the worst-case scenario (the system 
needs all of its RAM *and* /tmp is full) is half the amount of RAM's 
worth of swap space.  At least in my experience, once I'm forced to 
create *any* swap space it doesn't really matter how large it is -- it's 
trivial to bump the amount of swap a little higher, and the instance is 
already likely to end up thrashing at some point anyway.  So especially 
on an I/O-constrained cloud, for me it really boils down to, "Am I going 
to have to provision swap space or not?"


For example, there's already one case of this happening even without 
/tmp on tmpfs:  a yum update on a t1.micro instance in EC2 may not 
finish if the update happens to involve the system recompiling its 
SELinux policy -- there simply isn't enough RAM to do everything.  In 
this case I don't really mind having /tmp on tmpfs since the system is 
going to be unusable during that process anyway and I only have to add 
around 300M of extra swap to compensate for the change.  The pivotal 
question for a given cloud and workload, then, is, "To how many 
instances will tmp-on-tmpfs force me to add swap space?"


The other case is, of course, the obvious one:  how big does /tmp 
actually have to be to run a reasonable Fedora server?  Do we know what 
is likely to break when /tmp is significantly smaller than usual?  Most 
of the distro's testing occurs on desktop/laptop machines with at least 
1 or 2 GB of memory, after all.


There are enough cloud- and workload-specific variables here that I'm 
not particularly enthusiastic about putting /tmp on tmpfs, so I'd 
support masking that by default.  Since doing so would be deviating from 
Fedora's new default we should make sure to document that fact and tell 
people how to turn it back on if we go that route, though.


Those are my thoughts, at least.  What do the rest of you think?

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: 6 commits - generic/fedora-18-x86_64-cloud.ks generic/fedora-18-x86_64.ks generic/fedora-18-x86_64-minimal.ks

[Garrett Holmstrom]

On 2012-12-13 14:09, Matthew Miller wrote:

+# Remove firewalld; was supposed to be optional in F18, but is required to
+# be present for install/image building.
+echo "Removing firewalld and dependencies"
+yum -C -y remove firewalld
+# These are all pulled in by firewalld (libselinux-python is too, but
+# is also required by cloud-init).
+yum -C -y remove cairo dbus-glib dbus-python ebtables fontconfig 
fontpackages-filesystem gobject-introspection js libdrm libpciaccess libpng 
libwayland-client libwayland-server libX11 libX11-common libXau libxcb 
libXdamage libXext libXfixes libXrender libXxf86vm mesa-libEGL mesa-libgbm 
mesa-libGL mesa-libglapi pixman polkit pycairo pygobject2 pygobject3 
python-decorator python-slip python-slip-dbus


We should keep a careful eye on this one; pygobject3 is getting 
refactored to trim its dependencies somewhat.



+# Non-firewalld-firewall
+echo -n "Writing static firewall"
+cat < /etc/sysconfig/iptables
+# Simple static firewall loaded by iptables.service. Replace
+# this with your own custom rules, run lokkit, or switch to
+# shorewall or firewalld as your needs dictate.
+*filter
+:INPUT ACCEPT [0:0]
+:FORWARD ACCEPT [0:0]
+:OUTPUT ACCEPT [0:0]
+-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
+-A INPUT -p icmp -j ACCEPT
+-A INPUT -i lo -j ACCEPT
+-A INPUT -m conntrack --ctstate NEW -m tcp -p tcp --dport 22 -j ACCEPT
+-A INPUT -m conntrack --ctstate NEW -m tcp -p tcp --dport 80 -j ACCEPT
+-A INPUT -m conntrack --ctstate NEW -m tcp -p tcp --dport 443 -j ACCEPT
+-A INPUT -j REJECT --reject-with icmp-host-prohibited
+-A FORWARD -j REJECT --reject-with icmp-host-prohibited
+COMMIT
+EOF


What do I need to file a bug against to get the EC2 image's firewall 
removed?


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: yum-cron in the cloud images?

[Garrett Holmstrom]

On Fri, Dec 14, 2012 at 6:17 AM, Matthew Miller
 wrote:
> On Fri, Dec 14, 2012 at 08:05:06AM -0600, Troy Dawson wrote:
>> It's frustrating enough on a local machine when an update goes wrong.
>> But on an Amazon machine, you have no console during bootup.  So if
>> something is going wrong during bootup ... you're out of luck.
>
> Yeah. On the other hand, with Fedora's fast pace, leading edge nature, and
> huge package selection, I feel like we're doing our users a disservice by
> not providing some level of protection.

This is a case for publishing new images periodically, not causing
churn on instances that already exist.  If we were talking about a VPS
provider then this might be different, but on a cloud instances are
ephemeral.  If I want to make a fleet of instances get the latest
stuff then I'll create a new image (or sometimes a new user-data
script) and replace all of the old instances with instances of the new
image.

> What if we did security updates only, by default? (Right now, yum-cron
> doesn't support this, so it'd be a F19 feature.)

I'd be okay with this (or even an all-kinds-of-updates version) if and
only if it becomes the default for the whole distribution.  A cloud
image is even less well-suited to automatic updates than a desktop or
server, and if we can't justify it for those cases, it definitely
doesn't belong here either.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: to ec2-user or not to ec2-user?

[Garrett Holmstrom]

On Fri, Dec 14, 2012 at 1:00 PM, Jay Greguske  wrote:
> On 12/14/2012 03:12 PM, Matthew Miller wrote:
>> Amazon recommends using ec2-user (with passwordless sudo) for EC2 images.
>> That's what Fedora has been doing. Do we want to continue this? Arguments:
>>
>>
>> A. It doesn't really provide any added security, but does add complication.
>>Additionally, normal "don't run as root" advice is less important since
>>cloud instances should be ephemeral and recreatable.
>>
>> B. But, consistency.
>
> Fedora can of course do its own thing, but Ubuntu, Amazon Linux, future
> RHELs, and other distros use ec2-user. This lines up with the EC2
> documentation as well. I'd discourage changing it just because we can.

Some historical info:  since our first cloud image targeted EC2, we
looked at the EC2 documentation and other distros, most of which
tended toward ec2-user, so we went with that.

>> What's our SIG consensus here?
>>
>> Other points:
>>
>>  - We're making images for EC2 and for other cloud systems as well.
>>'ec2-user' seems particularly silly on, say, OpenStack.
>>  - We could use ec2-user and something else (including just root) on the
>>generic images.
>
> Fair points.

If we end up with One Image to Rule Them All at some point, I think
using something more generic is fair.  We could probably get pretty
close with some fine-tuning.  Just not for F18; I suspect we're a
little late for that kind of churn.

>>  - We should decide this really fast because it's already past the last
>>    minute; default is to just stay with ec2-user for F18 and revisit for
>>F19.
>>
>
> +1

+1.  This is an excellent time to discuss plans for F19 images, not so
much F18 images.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: image formats

[Garrett Holmstrom]

On 2012-12-17 10:48, Bill Nottingham wrote:

Matthew Miller (mat...@fedoraproject.org) said:

On Fri, Dec 14, 2012 at 04:14:01PM -0500, David Nalley wrote:

(Eucalyptus can't digest qcow2.) And I'm afraid they will have to be
compressed because there's no reasonable way to make them a downloadable
yet useful size.

VHD and VMDK might also be useful formats.


Yeah. As I understand it, CloudStack uses different formats depending on the
hypervisor used, right?


Is it best to produce one image and scripts for post-processing it into the
appropriate formats, or to simply produce them all?


Unless the various formats are fundamentally different, if we're looking 
for portability then I'd be inclined to teach people how to package a 
common format like a raw filesystem image into something cloud-specific, 
then publish the common format.  If there's enough space/time/demand 
then it could also be worth publishing multiple formats to make things 
more convenient.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: yum-cron in the cloud images?

[Garrett Holmstrom]

On 2012-12-16 10:21, Matthew Miller wrote:

On Fri, Dec 14, 2012 at 06:02:01PM -0800, Garrett Holmstrom wrote:

What if we did security updates only, by default? (Right now, yum-cron
doesn't support this, so it'd be a F19 feature.)

I'd be okay with this (or even an all-kinds-of-updates version) if and
only if it becomes the default for the whole distribution.  A cloud
image is even less well-suited to automatic updates than a desktop or
server, and if we can't justify it for those cases, it definitely
doesn't belong here either.


Well, with the desktop, there's theoretically someone who will see GUI-based
"updates available" notifications and apply them. I'm happy to make the case
for security updates for the whole distro, though. (For F19+.)


Sure, but a cloud instance isn't a desktop.  It isn't really even a 
virtual server.  Cloud instances are disposable -- you service them by 
replacing them, not by updating or changing them.  This is different 
from the other kinds of systems that the distribution targets, and 
automatically updating instances breaks that model.


Simply put, if we're building a cloud image then I would like to see 
that image's default behavior match the way cloud applications generally 
work.  Since a Fedora image should still be Fedora, I can certainly live 
with automatic updates if the rest of the community disagrees with me, 
but when we target a new platform like the cloud I believe we ought to 
encourage habits that are appropriate for it rather than encouraging old 
workflows that can make managing stuff in the cloud more difficult.


Those are my two cents, anyway.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: yum-cron in the cloud images?

[Garrett Holmstrom]

On 2012-12-18 9:54, Matthew Miller wrote:

On Tue, Dec 18, 2012 at 09:26:25AM -0500, Andy Grimm wrote:

Since a Fedora image should still be Fedora, I can certainly live with
automatic updates if the rest of the community disagrees with me, but when
we target a new platform like the cloud I believe we ought to encourage
habits that are appropriate for it rather than encouraging old workflows
that can make managing stuff in the cloud more difficult.

+1


I think you guys are right for general package updates and bug fixes. But I
don't think cloud is anything particularly new in giving the luxury to avoid
patching vulnerabilities.

I'm not going to change anything now, but I think we need to think about how
to do this. Amazon Linux automatically applies critical security fixes, and
notifies on login of important ones. I'm not so keen on the "on login"
approach, because I think _that's_ the "old workflow".


It's harder to sell that idea for Fedora than it is for operating 
systems with less churn, because security updates quickly end up getting 
conflated with enhancements and other changes.  That doesn't negate your 
point, though.  Does anyone have any useful thoughts/experiences with that?


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: idle=halt on grub command line in cloud image?

[Garrett Holmstrom]

On 2012-12-19 11:47, Matthew Miller wrote:

I just want to double-check that this is still the right thing to do in
EC2 (and in our other virt platforms).

I notice that Amazon does not have this in their grub.conf for Amazon Linux
(even for 32-bit) and I don't want to be cargo-cult carrying it forward.


IIRC, it was added because of this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=739499

Since that seems to have been fixed, I think it's worth giving an image 
without idle=halt a shot.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: cloud and local firewall at all (sig consensus?)

[Garrett Holmstrom]

On 2012-12-20 12:49, Matthew Miller wrote:

On Wed, Dec 12, 2012 at 09:58:04PM -0800, Garrett Holmstrom wrote:

EC2 recommends images with *no* default firewall since they use security
groups to control traffic, and adding a second, guest-level firewall tends
to confuse people.


I'd like to get a group consensus on this. Dennis Gilmore has expressed
concern about leaving the local firewall off -- having it on may be
redundant, but it protects against configuration errors or security bugs in
EC2 itself.

Options for the out-of-the-box config are:

  A) no local firewall (Garrett, do you have a reference to an EC2
 recommendation for this configuration?)


Not any more.  The only reference to instance-specific firewalls that I 
can find in today's documentation [1] is, "In addition to these 
examples, you can maintain your own firewall on any of your instances. 
This can be useful if you have specific requirements not met by the 
Amazon EC2 distributed firewall."



  B) firewall allowing ssh in by default (normal Fedora default)

  C) firewall allowing in ssh + http/https (since cloud systems are often
 web servers)

I'm lightly in favor of C, since I like the concept of defense-in-depth, and
this seems like a decent compromise. But I really don't have a very strong
opinion. What are your thoughts?


There seem to be enough people here who are okay with defaulting to dual 
firewalls to narrow it down to B and C.  To be honest, I'd choose B. 
It's Fedora's default, it makes fewer assumptions, and since we're 
already considering an exploit in EC2 itself to be in scope, we might as 
well block off a couple a couple more ports out of the box.


I don't feel incredibly strongly about that, though.  I just think it 
makes more sense.


[1] 
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using-network-security.html


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: f19 feature in progress; First-Class Cloud Images

[Garrett Holmstrom]

On 2013-01-23 8:05, Matthew Miller wrote:

We worked on this at FUDCon.

<https://fedoraproject.org/wiki/Features/FirstClassCloudImages>

Summary:

   This feature expands Fedora's current cloud image deliverables beyond just
   EC2, and overhauls how they are produced. The goal is to produce cloud
   images for EC2 and other cloud deployments for the Alpha, Beta, and Final
   compose process and distribute them on the mirror network. There will also
   be nightly or weekly image builds for Rawhide to assist with early
   development. All images should be constructed using a newer generation of
   tools.

Comments and input *very* welcome.


Thanks for writing this up!

What constitutes an EC2-specific customization?  Eucalyptus functions 
very similarly to EC2, so it will likely need the same tweaks as well.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: AWS tools for Fedora

[Garrett Holmstrom]

On 2013-02-16 1:21, Danishka Navin wrote:

i have installed following rpm but there are only few ec2 commands

s3.amazonaws.com/ec2-downloads/ec2-api-tools.noarch.rpm
<http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.noarch.rpm>

ec2-ami-tools-version  ec2-delete-bundle  ec2-migrate-manifest
ec2-bundle-image   ec2-download-bundleec2-unbundle
ec2-bundle-vol ec2-migrate-bundle ec2-upload-bundle


That URL might point to what you meant to install, but that isn't what 
you actually installed.  What you installed are the AMI tools that are 
used for image management.  If you're looking for command line tools for 
the EC2 web service, I recommend either using euca2ools, which you can 
get directly from Fedora, or ec2-api-tools from AWS.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Pushing cloud-utils to stable

[Garrett Holmstrom]

On 2013-03-06 1:46, Sandro "red" Mathys wrote:

cloud-init is very useful even without growroot support. So I'd agree
with Padraig to push it now.


That slip-up is amusing, in a way.  Cloud-init just gained growroot 
support yesterday.  8^)


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Broken dependencies: cloud-utils

[Garrett Holmstrom]

On 2013-04-03 1:54, Juerg Haefliger wrote:


On Mon, Mar 18, 2013 at 9:59 AM, Juerg Haefliger mailto:jue...@gmail.com>> wrote:

On Mon, Mar 18, 2013 at 9:40 AM, Sandro "red" Mathys
mailto:r...@fedoraproject.org>> wrote:
 > On Mon, Mar 18, 2013 at 9:21 AM, Juerg Haefliger
mailto:jue...@gmail.com>> wrote:
 >>
 >> > cloud-utils has broken dependencies in the epel-6 tree:
 >> > On i386:
 >> > cloud-utils-0.27-0.2.bzr216.el6.noarch requires qemu-img
 >> > On ppc64:
 >> > cloud-utils-0.27-0.2.bzr216.el6.noarch requires qemu-img
 >> > Please resolve this as soon as possible.
 >>
 >> Is this something that I need to fix or can I ignore it?
 >
 >
 > You should fix this. But unless cloud-utils can somehow work without
 > qemu-img, the solution will simply be to stop building for i386
and ppc64.
 > RHEL only provides qemu-img for x86_64 and attempts to bring it
to the other
 > architectures through EPEL were met with issues (see
 > https://bugzilla.redhat.com/show_bug.cgi?id=877259).
 >
 > The line you need for EL6 builds would be:
 > ExclusiveArch: x86_64


Hmm... doesn't seem to work for a 'noarch' package. The build fails. How
can I tell koji to only build a noarch package on x86_64?


You can't.  You simply have to keep retrying the build until it lands on 
a machine with a compatible arch.  :-(


The bug for that is here:  https://fedorahosted.org/koji/ticket/210

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: F18 cloud-init proposed rebase update

[Garrett Holmstrom]

On 2013-05-02 9:04, Pádraig Brady wrote:

On 05/02/2013 04:49 PM, Steven Hardy wrote:

Hi,

I've been working through debugging a number of issues with cloud-init
0.7.1 as currently packaged for F18.

The most major problem is that it no longer seems possible to define a user
via the user-defined cloud-config data and have that user get an SSH
authorized_keys correctly installed, when the SSH key comes from the Ec2
datasource.  This is a regression from the version shipped in F17 AFAICS,
and it's fixed upstream[1].

There are a number of other F18 related fixes upstream which would be
valuable, including:
- Improved hostname handling (doesn't truncate hostnames containing ".")[2]
- Fedora locale, hostname and tz related fixes[3]
- Fedora systemd, blkid and sysconfig fixes (rev 809)[3]

So I'm proposing a rebase to bzr rev 809, which will include fixes for the
issues above, and probably many more by the looks of the changelogs.

I've created a test package:
http://people.redhat.com/~shardy/cloud-init-test/cloud-init-0.7.2-0.1.bzr809.fc18.noarch.rpm
http://people.redhat.com/~shardy/cloud-init-test/cloud-init-0.7.2-0.1.bzr809.fc18.src.rpm

I'd like to know if this update can go into update-testing for F18 - I've
done some basic testing of the package above, and so far all looks OK.

Thanks!

Steve

[1] https://bugs.launchpad.net/cloud-init/+bug/1100920
[2] https://bugs.launchpad.net/heat/+bug/1164400
[3] http://bazaar.launchpad.net/~gpadgett/cloud-init/ovirt/revision/802


Thanks for all the detail.
That all seems sensible to me.

You've not changed cloud-init-fedora.cfg and just updated the logic
to fix known issues.

I'll apply this to rawhide, f18, f19 in your name later this evening.


Sounds reasonable to me.  That's the only issue I found as well.

Thanks for handling the update so quickly, Pádraig.  8^)

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: F18 cloud-init proposed update to 0.7.2

[Garrett Holmstrom]

On Fri, May 17, 2013 at 9:21 AM, Matthew Miller
 wrote:
> On Fri, May 17, 2013 at 05:07:14PM +0100, Steven Hardy wrote:
>> I've built and lightly tested a proposed update to cloud-init, which aligns
>> us with the recently released upstream 0.7.2 cloud-init release.
>> This includes all of the Fedora-related fixes mentioned in my justification
>> for the cloud-init-0.7.2-0.1.bzr809 update, but aligns us with the upstream
>> release, which includes a few additional fixes.
>
>
> Have you tested this on F19, by any chance? I'm having some issues with the
> set-hostname service.

For posterity's sake, here is the bug for that:
https://bugzilla.redhat.com/show_bug.cgi?id=964006

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Default cloud user name

[Garrett Holmstrom]

On 2013-05-24 7:57, seth vidal wrote:

On Fri, 24 May 2013 16:32:15 +0200
Juerg Haefliger  wrote:


Hi all,

Per Matt's request, I'm starting a new thread about the default user
name for Fedora cloud images. Currently it's 'ec2-user' which I don't
really like. OK, coming from the OpenStack-side of the cloud I might
be a little biased :-) Nevertheless, I think we want to achieve an end
goal of a single image that can be used in different cloud
environments rather than having different images for the different
environments. As such, the user name needs to be cloud/service
provider independent. Following the lead of Ubuntu and Debian I
propose to use 'fedora' as the default user name for F19 and going
forward.

Let the popularity contest begin...



How about we do-away with the 'faux user which is and is not root even
though they  are a trivial unpassworded sudo away' security theater that
amazon and ubuntu have been peddling for years now.

I mean seriously - it's meaningless - let's stop pretending.


Yup.  We went with ec2-user because that's what EC2 wanted people to 
standardize on at the time and that was the only cloud that mattered. 
If we're going to change it to something else today that's fine, but we 
should really just go with root.  The name is well-understood and 
non-political, we aren't enabling password auth, it's trivial to make an 
instance add a non-root user at boot time when one is really needed, and 
frankly, I'm tired of playing a reversed version of "sudo says" every 
time I want to log into an instance.  Just cut to the chase and use root 
already.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: Default cloud user name

[Garrett Holmstrom]

On 2013-05-26 18:57, Steven Dake wrote:

On 05/25/2013 01:09 PM, Steven Hardy wrote:

On Fri, May 24, 2013 at 04:32:15PM +0200, Juerg Haefliger wrote:

Hi all,

Per Matt's request, I'm starting a new thread about the default user
name for Fedora cloud images. Currently it's 'ec2-user' which I don't
really like. OK, coming from the OpenStack-side of the cloud I might
be a little biased :-) Nevertheless, I think we want to achieve an end
goal of a single image that can be used in different cloud
environments rather than having different images for the different
environments. As such, the user name needs to be cloud/service
provider independent. Following the lead of Ubuntu and Debian I
propose to use 'fedora' as the default user name for F19 and going
forward.

If we have to have a default user configured in the package, then
"fedora",
or "fedora-user" gets my +1.

I also agree that just using root would be easier & less confusing, since
the paswordless sudo amounts to that anyway.

Steve,

Applications run as the user (fedora-user) and would need a more
complicated attack vector to escalate privileges via sudo then a root
run daemon running inside the instance would (No remote execution of
sudo plus other commands would be required).  For example, a network
daemon running only as root could be attacked by reading files via the
network via a non-remote-execution attack (think web app reading and
displaying mysql passwords from the filesystem).  This mysql leak could
then be used as a different attack, which would not have been possible
if the app was running without non-privileged capabilities.

Further complicating things, many applications will not run when root
capabilities are present in the process (they self-check and complain
don't run as root).


I take it we should assume that people will run their daemons and other 
applications as whatever user is there by default and not bother 
creating their own, then?


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud

Re: please help test beta test candidate cloud images

[Garrett Holmstrom]

On 2013-10-17 9:15, Matthew Miller wrote:
> On Thu, Oct 17, 2013 at 11:09:39AM -0500, Joe Brockmeier wrote:
>> 1) You get a complaint that delta RPM isn't enabled when installing
>> packages. Not a major problem, but talking to Matthew we wondered
>> whether we should include delta RPM by default or suppress the warning.
>
> In the past, I've found deltarpm to actually be slower when bandwidth is
> plentiful. The package itself is small, so I'll do some tests here.

When I did some testing around this on F17 I found that deltarpm not
only increased update time by a non-trivial amount, but also generated
extra I/O.  It would be interesting to see how it behaves nowadays.

>> 2) Firewall on by default is not a good thing. It took me a minute to
>> realize *why* I couldn't hit the default Apache page after I'd fixed the
>
> Yes, as mentioned, this is a frequent complaint. In the past the majority
> opinion was that a basic packet filter was the best choice, but given the
> constant feedback I get to the contrary, I'm inclined to revisit.
>
> Especially as we make the cloud image a) more visibile and b) more
> differentiated, I think this makes a lot of sense.

Indeed.  This sounds like a differentiator for the "server" and "cloud"
products you proposed.  8^)

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: please help test beta test candidate cloud images

[Garrett Holmstrom]

On Thu, Oct 17, 2013 at 12:26 PM, Sandro "red" Mathys
 wrote:
> Crazy idea: disable the Firewall by default but allow users to enable
> it through cloud-init (i.e. user-data)? Just in case there are people
> who prefer the instance's firewall over security groups and open the
> latter up a lot / completely. Probably a stupid thought, but I thought
> I'd share it for the lulz.

If nothing else, if you pass a script as user-data cloud-init will
execute it late in the system's startup process.  It looks for "#!".

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Roles and responsibilities

[Garrett Holmstrom]

On Tue, Nov 12, 2013 at 10:52 AM, Chris A. Roberts
 wrote:
>> > Do we need a spot on the wiki setup for the Cloud group? I know duffy
>> > created
>> > one for the server group and since I have been fixing and cleaning the
>> > wiki
>> > up, I thought I would ask and create the space if needed. That could be
>> > part
>> > of my role :)
>>
>> > We've already got a trac instance - https://fedorahosted.org/cloud/.
>> > There's definitely a need for some documentation, though, so  >we should
>> > start a list of things to start documenting more thoroughly.
>> >
>> >
>
>  >Excuse me if I am wrong but I think what he meant here was in terms of the
> landing pages and all that.
>  >At least that is how I understood it .
>
> -- Frankie, that is what I was talking about, to have a place for
> documentation and for us to collaborate.
> Here is the link to the server groups wiki page and was hoping to setup
> something like this for the Cloud group. [1]
>
> [1] http://fedoraproject.org/wiki/Server

Like these?  :-)

https://fedoraproject.org/wiki/Cloud_SIG
https://fedoraproject.org/wiki/Cloud

They could use a little more info now that we have it, but as far as
starting points go we're probably already there.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: live images import the rpm gpg key -- should we do this too?

[Garrett Holmstrom]

On 2013-12-05 10:54, Matthew Miller wrote:

This bug came to my attention from the go/no-go meeting

   <https://bugzilla.redhat.com/show_bug.cgi?id=1023178>

https://git.fedorahosted.org/cgit/spin-kickstarts.git/commit/?id=32e066e039d130698396c2435f15db7232e47e06

Do we want to do this too?

(I kind of think so.)


Might as well if all the other spins are doing it.

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: flipping the console order?

[Garrett Holmstrom]

On 2013-12-10 10:58, Lars Kellogg-Stedman wrote:

On Tue, Dec 10, 2013 at 01:47:25PM -0500, Matthew Miller wrote:

We currently have this the other way around, and indeed, in OpenStack as
we're doing it currently, this means that, for example, cloud-init output
goes to the vnc console rather than the log. Should we flip it?


Having console output on the virtual serial port is great because it
becomes accessible via the api (or via "nova console-log"), whereas on
the VNC console it's basically discarded.


Yep, that's how it works in eucalyptus as well.  We're already 
special-casing EC2 anyway, so I say go for it.  8^)


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Fedora @ Google Cloud

[Garrett Holmstrom]

On 2014-06-18 16:46, Filipe Brandenburger wrote:

On Wed, Jun 18, 2014 at 3:52 PM, Renich Bon Ciric
 wrote:

On Wed, Jun 18, 2014 at 3:49 PM, Dennis Gilmore  wrote:

The issue is we are trying to build one image to run in all clouds, so
cloud provider specific tools are out. extend the functionality of
cloud-init if you feel its useful. Ideally its something that can work
in all providers.


Oh, a single image for all. That will be a hard one.


Agreed, that would be really hard to accomplish on a first moment.

Can we try to target a GCE-specific Fedora image to start and look at
convergence on a second stage? At the very least, that would allow us
to figure out what works and what doesn't... Also, it would help
figure out what extensions we'll need in cloud-init to make it do the
same things.


I believe cloud-init treats "user-data" attributes the same way it 
treats user-data in EC2, so if that works then that would give us a 
fairly painless way to install or configure whatever is holding testing 
back.  Perhaps that would be a suitable next step to try for.


Since it runs at boot time and goes away cloud-init could be made to 
replace the startup scripts if it isn't already sufficient.  I doubt it 
can replace the daemons in a reasonable manner without a decent amount 
of work, though -- the way GCE expects instances to work with metadata 
is somewhat unusual, and cloud-init simply isn't built to handle that 
(yet).  Assuming there is enough interest, though, I might be interested 
in working on something relatively portable that can do things like 
mounting disks based on metadata since that would be useful to me on EC2 
as well.  With any luck that sort of thing could potentially even go on 
the stock Fedora image, too.


--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: [PATCH] Don't enable by default

[Garrett Holmstrom]

On 2014-07-14 14:09, Colin Walters wrote:

[ Posting here for increased visibility ]

Right now, cloud-init is only installed in the cloud images, so it's OK
if it enables itself by default.

However, for future Atomic work, we'll likely only have one tree
that's also running on bare metal, and that will include cloud-init.

This change should be safe because the Fedora Cloud kickstart in
spin-kickstarts uses services --enabled=cloud-init.
---
  cloud-init.spec | 22 ++
  1 file changed, 2 insertions(+), 20 deletions(-)


This would be a great thing to add to the bug report for this issue [1] 
as well.


If the kickstart explicitly enables cloud-init then yes, let's drop the 
self-enabling bits.  We should really be using per-product service 
presets for that instead; I suppose I should work on a policy for that 
[2] now that it is finally clear which packages those belong in.  8^)


Please remove .atomic from the release tag.  The spec file will also 
need to use %systemd_post and %systemd_postun in addition to 
%systemd_preun if we're to follow the packaging guidelines [3].


[1] https://bugzilla.redhat.com/show_bug.cgi?id=850058
[2] https://fedorahosted.org/fesco/ticket/945
[3] https://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Systemd

--
Garrett Holmstrom
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct