Re: Puppet, Chef or CFEngine?
We currently use puppet. We have used it for quite some time, and just
revisited our configuration management system, to see if it was still the
right way to go.
In looking at Chef, CFEngine and Puppet, we decided to stick with Puppet.
The cost of changing over a number of extremely complex systems to a new
management service was simply too high, for minimal (if any) gain.
On the topic of user management, while a shell script may be easier /
faster in the short term, over time (and once an environment is
sufficiently large) it can result in an inconsistent environment. Servers
can be down, unresponsive, have some random failure, and if not immediately
and manually remediated, you end up with users on servers that shouldn't
be, missing users on others, and old passwords on yet others.
Using Puppet, you can either maintain /etc/{passwd|group|shadow} (wouldn't
personally do this, but it is an option, so included here in the interest
of being complete), or you can use the 'user' and 'group' resource type (
http://docs.puppetlabs.com/references/stable/type.html#user) to maintain
users across the environment. This is if you need / want to continue using
local users. Personally, I'm with Bryan, and prefer a central
authentication method, as it resolves many of the problems you would have
with local users, and provides for an easier method of auditing user
accounts.
-- Dan.
On Tue, Nov 8, 2011 at 7:43 PM, Lisa Kachold lisakach...@obnosis.comwrote:
Thanks to all who responded.
I believe this is an excellent subject for a blog after about 10,000 lab
testing package comparison hours!
Laugh!
On Tue, Nov 8, 2011 at 9:34 AM, Bryan O'Neal
bryan.on...@theonealandassociates.com wrote:
Personal opinion - for large scale use with many people maintaining
different sections puppet is one of the best - however it is really
only good for file management. Since nearly everything on a linux
system is a file, this should not be a problem. As for user management
- I am still under the opinion on that (unless you are a pure Linux
environment) this should be solved by using Active Directory for
authentication and pam for access mismanagement. (if you don't want to
integrate your services with pam they probably have a simple
configuration file that controls access management that could be
handled by puppet just as easily)
Chef is more extensible with access to a full ruby stack - however
unless you have a very small group of well coordinated developers who
insist on adhering to standards you will rapidly find your
provisioning code will become unwieldy and almost useless as you
inheritances start overriding key portions without notice as to why or
what section did what. In the rite hands the flexibly is an asset that
may help solve key problems. In the wrong hands it will propagate
problems whose effect compound over time until the entire system is
scraped.
Disclaimer - I know very little regarding this compared to others. I
use puppet, write manifests, build systems, etc. I am not responsible
for the engineering.
On Sun, Nov 6, 2011 at 3:56 PM, Ed p...@0x1b.com wrote:
On Sat, Nov 5, 2011 at 4:59 PM, James Mcphee jmc...@gmail.com wrote:
I am also looking at implementing one of these at some point in the
near
future. The standard scripts over ssh is simple and relatively well
controlled, but teaching new people how to use them and maintaining
them in
a sane fashion is troublesome. I've used a few HP, Dell, Sun, and IBM
config products in the past and they were all bad enough I went back to
scripts in no time.
On Nov 5, 2011 11:33 AM, Lisa Kachold lisakach...@obnosis.com
wrote:
Can anyone chime in on using enterprise mass systems configuration and
management tools?
What are you using? Chef, Puppet or CFEngine and why?
I like CFengine - the task based focus is on promises and the
install is painless. The only ruff spot I could point to is with
application updates - the interface to yum is less polished than some
- updates work if you work on them as groups vs particular apps. There
are many promises online and in the maillists for particular tasks. I
think there is even a starter pack on github somewhere. CFengine fits
well into ITIL and managing IT - lots of IT - and it has it's own
directory in /var too! ;)
The RH world has worked with Cobbler plus Puppet - this is getting
tighter with Puppet plus TheForman and Pulp - if I remember the
roadmap.
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Is it possible to extract the root password from the file system?
Can you SSH as the hammerhead user? When you FTP as the hammerhead user, can you move the script.php file to the htdocs directory? It has 777 permissions, so should be able to open it / drop a file there. If you can get a PHP file uploaded and able to execute properly, perhaps a PHP based shell could help? -- Dan. On Mon, Jul 18, 2011 at 9:20 PM, Lisa Kachold lisakach...@obnosis.comwrote: I believe the script.php has to be moved the webroot directory and given permissions there I believe, but well if you can't get a login via ssh... -- how to do it? On Sun, Jul 17, 2011 at 8:58 AM, Mark Phillips m...@phillipsmarketing.biz wrote: On Sun, Jul 17, 2011 at 3:54 AM, Lisa Kachold lisakach...@obnosis.comwrote: There are alot of password files and dictionary lists on various sites. Backtrack5 contains a good number. But I imagine that it's either not allowing root via ssh or you have the wrong username. It turns out the box is smarter than a fifth grader.after a few hydra attacks, it started rejecting all the hydra attempts to ssh in via root. Once I stopped hydra (after running all night), it took a couple of hours before it would respond to ssh attempts from root. It now will ask for the root password, but I still have no idea what it is. Or it's a truely random string. It could bethe password for the zip file to unzip the file system is YvSInIQopeipx66t_DCdfEvfP47qeVPhNhAuSYmA4 . Someone retrieved it using a disassembler on the file system. I did some more reading, and one person was able to use php to allow ssh login. The box allows one to create a web space, and it comes with php installed. One can edit the php.ini file, and I can upload via ftp a php script. The script they suggested is: ?php $file = '../../../../etc/pam.d/sshd'; $fh=fopen($file, 'w') or die(can't open file); $stringData = account required pam_unix.so\n; fwrite($fh, $stringData); $stringData = session required pam_unix.so\n; fwrite($fh, $stringData); $stringData = auth required pam_permit.so\n; fwrite($fh, $stringData); fclose($fh); ? I uploaded the script, but I get a 404 File not Found when I access the page. I thought it might be a file permission error since the file is only rw. I tried chmod 777 at the ftp prompt, and got the error message File not Found, but ls shows it is there. ftp ls 200 PORT command successful 150 Opening ASCII mode data connection for file list drwxrwxrwx 2 apache apache 6 Jul 17 08:23 cgi-bin drwxrwxrwx 2 apache apache 22 Jul 17 08:23 htdocs drwxrwxrwx 2 apache apache 39 Jul 17 08:23 log -rw-rw-rw- 1 hammerhead hdusers 335 Jul 17 08:49 script.php 226 Transfer complete ftp chmod 777 script.php 550 CHMOD 777 script.php: No such file or directory ftp Is there anything I can change in the php.ini file to make this script execute? Or, am I missing something else? BTW, I cannot ftp as root, but I can ftp as a user I created, hammerhead. Thanks, Mark On Fri, Jul 15, 2011 at 10:33 PM, Mark Phillips m...@phillipsmarketing.biz wrote: Since this is a drive buffalo, I might try ettercap ssh downgrade attack: http://openmaniak.com/ettercap_filter.php ttp://sites.google.com/site/clickdeathsquad/Home/cds-ssh-mitmdowngrade Not sure how a man in the middle attack will work, since I don't know the password to begin with... Or Hydra: Hydra Instructions: http://www.youtube.com/watch?v=7CP-JB4QARo Hydra is promising. I tried it with the common passwords list from openwall. No luck. Do you have any better password lists? Thanks, Mark --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss -- (602) 791-8002 Android (623) 239-3392 Skype (623) 688-3392 Google Voice ** HomeSmartInternational.com http://www.homesmartinternational.com --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss -- (602) 791-8002 Android (623) 239-3392 Skype (623) 688-3392 Google Voice ** HomeSmartInternational.com http://www.homesmartinternational.com --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Job Board
Hi all, I seem to be having a few problems on the main site for the group. I'm am sure these are likely just me having my guy eyes on, and overlooking the obvious, but am hoping I can get someone to point me in the right direction. 1) I'm not able to find where to go to create an account to login. I can join the groups no problem, just can't create an account. 2) How does one add a posting to the Job Board? My place of employment would like to add a listing here, and is not able to find where to properly do so. Any help in this regard would be much appreciated. Thanks! -- Dan. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
OT: Teaching at a CC
So, I know we have a few instructors at a community college here on the list. Just wondering what the requirements are to obtain said job. Do you need to teach a certain number of classes? What level of degree is required? It's something I'd like to do eventually, and would like to know what is needed to get there. Thanks! Dan. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Verify if an email address exists.
Sort of depends on what you are looking for. There are some domains that have a catch all, so may not go to the specific address you mail to. What are you wanting to accomplish with this? If you want to know if you can send mail to a specific domain, you can open a socket to the domain on port 25 (assuming you have outbound port 25 available to you from your location). If you want to know if a specific email address is available, you can try to send an email to the address, and note the response code after sending it. Using php, you can use the mail() function, however, the error message it returns is binary in nature, either it sends or not, without a real reason why it failed to send. -- Dan. On Tue, Sep 7, 2010 at 6:09 PM, AZ RUNE arizona.r...@gmail.com wrote: For quicky help dirtsearch.org, .com something like that Brian On Tue, Sep 7, 2010 at 6:07 PM, keith smith klsmith2...@yahoo.com wrote: I would like to verify email addresses to verify if the actually exist. I seem to recall doing a reverse lookup. Is that possible? I would like to do this in PHP if possible. Google returns no real help. Thanks in advance. Keith Smith --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss -- Brian Fields arizona.r...@gmail.com --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Locked Directories
Should create them as the user that is going to use them. There are times it is appropriate to create them as root, other times not so much. It is very much application specific, though usually you want to create it as the user that will need to use the file or directory. -- Dan. On Mon, Aug 16, 2010 at 12:04 PM, Eric - A ericall...@juno.com wrote: So I should create all directories as normal user then? /*SNIP*/ --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Tarballs
Contents of the .bin file: These are the commands I'm following: - Tar jxfv konversation-1.3.1.tar.bz2 ls cd konversation-1.3.1 ./configure make make install vi INSTALL - and nothing happens. Are you actually capitalizing the Tar command? Or was that just your editor that did that? That said, I went and grabbed a copy of the tarball. The commands listed above wouldn't work, since it doesn't have a configure script, and instead uses cmake. -- Dan. On Sat, Aug 14, 2010 at 9:50 AM, Eric - A ericall...@juno.com wrote: Hi all, I'm trying to learn how to use tarballs but I'm running into a dead-end with the commands I found on a couple of how-to pages. The commands I executed are attached and show the commands I used. Any help will do. . Eric - A . Notre Dame Certificates 100% Online Programs in Negotiation Leadership and Mgmt. Enroll Today! http://thirdpartyoffers.juno.com/TGL3141/4c66c9882a780756834st02vuc --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Determin which FTP server is running and turn off non secure FTP
rpm -qa | grep ftp ? also chkconfig --list at the bottom of the list will be the xinetd based services. Find the FTP daemon in there, and: chkconfig [daemon_name] off On Wed, Jun 23, 2010 at 10:56 PM, walter tocalini curo...@gmail.com wrote: Keith let asume that you use yum to install pure-ftp and you use---and you did not use nothing else to configure pure-ftp than chkconfig, -yum install pure-ftpd now after that you create the system startup links chkconfig --levels xx pure-ftpd on (xx =whatever level you use) or chkconfig pure-ftpd on then or just reboot OS /etc/init.d/pure-ftpd start now check if is listed chkconfig --list pure-ftpd then you command will be /etc/init.d/pure-ftpd stop now using chkconf you will remove the startup link chkconfig --levels xx pure-ftpd off or chkconfig pure-ftpd off now that should do it, good luck WT --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Determin which FTP server is running and turn off non secure FTP
You could check /etc/init.d for something that looks like an FTP server You could check: lsof -i :21 which should provide you with the pid / process that is listening on port 21. Once you have that, you should be able to kill off the process. Don't forget to chkconfig off the process, so it doesn't come up if / when you reboot the server. -- Dan. On Wed, Jun 23, 2010 at 3:50 PM, keith smith klsmith2...@yahoo.com wrote: Hi, I have a CentOS server that I am working on. Got SFTP working and now I want to turn off plain FTP. It is unclear to me how to do so and I can't find a reference that shows me how to determine which FTP server we are running. Thanks a bunch! Keith Smith --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Determin which FTP server is running and turn off non secure FTP
FTP control channel is on port 21, data is on 20 (for active ftp). SFTP uses the SSH daemon, so runs on port 22. It has been my experience that the pure-ftpd init script is far from graceful, as Eric pointed out, the error that was given likely means that the service wasn't running. That, or it just couldn't find the pid file. lsof -i :21 will tell you the process (with pid) that is listening on port 21. You can then kill that process. Provided you have also used chkconfig to disable the service on startup, it will then effectively be stopped from running. The rpm -e or yum remove commands listed above will make doubly sure that the service won't be started up again on the server. Additionally, you could use iptables to disable any connection to port 21 on the server : /sbin/iptables -A INPUT -p tcp --dport 21 -j REJECT I think that should stop incoming connections on the port. -- Dan. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: CloudLinux
I've been playing around with CloudLinux a bit and haven't had a chance to properly reply to this thread until now. That said, CloudLinux itself is not a virtualization layer in the sense of Xen, VMWare or the like. It instead is a way to limit system resources based on the user (uid) instead of by process. While CloudLinux does include some security features (a grsecurity patch, utilization of fcgid / suphp for running cgi processes as the VirtualHost user, instead of as the apache user), it's primary benefit is in the limitation of CPU, I/O, and process count per user. In a hosting environment, being able to prevent one user from completely tanking a server (either intentionally, as a result of a Digg or Slashdot article, or some attack aimed at the site), this is a huge benefit. IMO the name itself (CloudLinux) is somewhat misleading, as it does not employ any Cloud features (no real abstraction of services from the hardware). @R P Herrold I agree that the basic support is useless. I do not know the pricing of licenses / support as of yet, however, I suspect that Basic plan is simply there to make the others look more attractive for whatever the price is (see ATT's .5GB data package vs 2GB data package for a similar concept). In the OT part of this thread, I would also agree that many of the security risks of Cloud computing are much arm waving, and can largely be resolved by proper encryption of data, in addition to using the Cloud properly. There are parts of it that can be useful (non-critical data storage, inexpensive off site backups, etc), and parts that you may not want to keep out there (unencrypted user SSN's, CC#'s, etc), more as part of good practice than any real security concerns. @Lynn I also agree that any way we can make the bad guys keep scratching their heads, gives us time to implement better policies and procedures to further complicate their lives. However, while we should try to keep our methods of thwarting them quiet, it could also be beneficial to inform others who have noticed (and been open) about the same attacks know what you did to resolve it, even if done so out of band via a simple phone call, etc. -- Dan --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: power supply
I've gone with the Enermax modular PSUs for the past couple I purchased, and have been quite pleased. That said, I'd go with Eric's advice, and check out the Newegg reviews prior to purchasing. -- Dan. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
CloudLinux
Hey all, Just wondering if any of you have played around with CloudLinux ( http://cloudlinux.com) and what your thoughts and experiences with it may have been. -- Dan. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: PLUG Eastside Meeting - Thursday
What time is it to start? Sorry, new to the list / group, so not sure if this is available elsewhere or not. -- Dan. On Wed, Jun 9, 2010 at 10:44 AM, Dennis Kibbe denn...@sdf.lonestar.orgwrote: Join us Thursday evening at our new meeting location at Mesa Community College's Kirk Student Center on the Southern Dobson campus for pizza and a presentation. Scott Gwin will give a presentation on Linux and the Android. Please see the PLUG website for meeting details. Did I mention PIZZA?! :-) dennisk -- Free Software, Free Society Free Software Foundation --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: What syntax for global 'chown' fix?
For the record, I'm with Dale here. Generally speaking, running a chown /
chmod against an entire system is bad. There are system files that have
setuid / setgid set, and for good reason (/bin/su comes to mind). There
could be uses for specific directories however, and with that in mind:
From the find man page:
-P Never follow symbolic links. This is the default behaviour.
When find examines or prints information a file, and the file is a
symbolic link, the information used shall be taken from the properties of
the symbolic link itself.
So to shorten Kaia's command, just add -P to it, and it will ignore
symlinks. Mind you, this is being explicit in the command, as the default
is to not follow symlinks. If you want it to follow symlinks, use the -L
switch.
This would modify the command to:
find -P [dir] -exec chown user:group {} \;
On Tue, Jun 8, 2010 at 11:57 AM, j...@actionline.com wrote:
Thanks Dale, Kaia, and Eric ...
Sincerely appreciate all of your answers. Each one helped.
I fully realize that entirely too often, I have no idea what I am doing;
but I just blindly muddle along anyway and somehow, by the grace of God
and the guidance of so many excellent plug friends, I manage to sort
things out and happily survive. ;)
In this case, I learned a bit more from each answer, some of which I
understand, and some of which I still do not understand. However, I got
the result that I needed. I just test various commands on a small sample
and once I eventually get something to work, I apply it further.
Joe
I originally wrote:
While the example commands below work to change permission for either a
complete system or for a complete directory and all sub-directories,
what would the syntax be for a similar command to 'chown' (change the
owner) globally or for a designated directory and and the files and
subdirectories below it?
find . -type f -print0 | xargs -0 chmod 644
find . -type d -print0 | xargs -0 chmod 755
find dir -type f -print0 | xargs -0 chmod 644
find dir -type d -print0 | xargs -0 chmod 744
Dale wrote:
Joe, before answering your question, I feel the need to warn you.
If you understood what the above commands do, the answer would be obvious
and you wouldn't have asked the question. Further, IMHO, unless you know
what each part of the above commands do, you shouldn't use them. Each
line has three commands, each of which is readily understandable with
some effort.
(BTW, none of the above commands change permissions for a complete
system.
They only do it recursively for files in a directory or for a directory
and its subdirectories.)
To change the owner and group for a directory and recursively to its
files and subdirectories, do:
(1) chown -R owner:group dir
To change just the owner:
(2) chown -R owner dir
To change just the group:
(3) chown -R :group dir
An alternative way (just less efficient) to accomplish (1) is:
(4) find dir -print0 | xargs -0 chown owner:group
-Dale
= Previous replies:
A: Eric wrote: to globally change owner:
find dir -type f -exec chmod user:group {} \;
find dir -type d -exec chmod user:group {} \;
(J: I discovered that 'chmod' in the Eric's example
should apparently have been 'chown' ... I think.)
A: kaia.tay...@schwab.com -- To avoid any surprises with links,
if there are any, do this first:
find [dir] -type l | more
Then:
find [dir] -exec chown user:group {} \;
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: An actual Linux problem/solution (slicehost)
If you are running into issues with too many httpd processes spinning up,
would it make more sense to limit the number of child processes it can spin
up? You could run into similar constraints if you hit MaxClients (assuming
apache here?), however, you could monitor for that to trigger your restart
instead.Is the cause of the load from the web server spinning up a bunch
of processes all of a sudden? Or from the code it is attempting to parse /
deliver? Increasing the number of servers that are started up could also
help in preventing a spike of load if it is the former. If the latter is
the issue, perhaps some other optimizations could be made. I've had some
pretty decent success with memcache in helping reduce mysql traffic and
server load.
While the script you have here could keep the server from having to be
rebooted, it seems that having that 30 second sleep in the middle of a bunch
of traffic, would go against the (supposed) goal of driving traffic in the
first place.
-- Dan.
On Wed, May 19, 2010 at 12:12 PM, Matt Graham danceswithcr...@usa.netwrote:
I have a slicehost running Gentoo. Because I'm cheap, I got the smallest
slice available (256M RAM, 10G disk, 100G bandwidth, small amount of CPU).
This usually worked fine since I'm not running a whole lot of stuff or
getting
high traffic.
However, if the slice got a large amount of traffic in a short period of
time,
the load on it would go through the roof. Then it'd run out of RAM, then
usually run out of swap. This usually led to an unresponsive slice that
couldn't be recovered without a reboot. This was *really* inconvenient,
especially if it happened while I was sleeping or busy.
So after screwing around for a while and tweaking some parameters, I came
up
with a small Perl script, below in pseudocode:
daemonize();
$count=0;
while(1){
$load=check_load();
$date=`date`;
if($load 10.0 and $count == 0){ # seems about right
writelog($date load $load WHOA! force-killing everything.\n);
force_kill(httpd,mysqld);
sleep(30);
restart(httpd,mysqld);
$count=2; # seems OK
}
elsif($load 3.0 and $count == 0){ # seems about right
writelog($date load $load restarting everything\n);
restart(httpd,mysqld);
$count=2; # seems about right
}
sleep(60); # maybe 45? This works though.
if($count 0){ $count--; }
}
# end
with this in place, things have been working for over a week with no
manual reboots required. Restarts of one stripe or another get triggered
about once or twice a day, at random times, and there's not any correlation
that I could see to what's in the apache logs or the slow query logs.
Later today, if I don't get beaten down with 5 tons of other junk, I'll
post
the real code. Maybe some other slicehost user will find it useful.
Comments/criticism/stuff RELATED TO THIS HERE MESSAGE are welcome. cat
Offtopic stuff offtopic_thread , please. Thank you, and keep reaching
for the stairs.
--
Matt G / Dances With Crows
The Crow202 Blog: http://crow202.org/wordpress/
There is no Darkness in Eternity/But only Light too dim for us to see
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
---
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: load balanced configuration
The question I have, are you trying to actually load balance things? Or just have a remote location that you can fire up with live data at a moments notice? Basically, are you wanting an active/active configuration, or active/passive? active/active across DC's can get kind of hairy depending on what the network looks like. active/passive won't give you any performance gains, but can simplify the configuration, while providing the HA you seem to be after. As Kaia pointed out, what the traffic looks like (reads vs writes) is a consideration. If it is something that users don't write to, and data doesn't have to be replicated across DCs frequently, this further simplifies things. Ultimately, the configuration will depend on what the application and network looks like currently, and what level of redundancy you want / need. -- Dan. On Wed, May 19, 2010 at 1:40 PM, Matt Iavarone matt.iavar...@gmail.comwrote: I think the original question was around stateless load balancing, not clustering. Cross DC clustering is a headache, but HA web sites aren't exactly terchnical challenges these days. On May 19, 2010 4:33 PM, Alex Dean a...@crackpot.org wrote: On May 19, 2010, at 2:47 PM, keith smith wrote: Hi Plug, I am considering combining the ... You're entering a world of pain. :) HA is cool, but is no panacea. If you haven't actually experienced downtime due to your server crashing or your datacenter losing connectivity, I recommend thinking long and hard about it. Don't solve a problem you don't have. The downtime created from unneeded failovers will likely exceed the actual/real downtime caused by either a server or datacenter being offline. Managing the cluster itself (as distinct from the services provided by the cluster) needs to be accounted for as an expense/responsibility. I don't want to sound overly pessimistic. I've set up quite a few HA clusters, and actually enjoy it most of the time. But it WILL cause you headaches in the middle of the night which you wouldn't have had if you only had a single server. Leave yourself lots of time to set up a development/test cluster, and abuse it in many ways. Pull out network cables, kill the switch, yank out power cables, etc. Do this with real hardware, not VMs. When the cluster nodes lose contact with each other, both will decide to become primary. This is a split brain. This can happen when the switch in-between them gets busy and starts dropping pings. Now, you can always recover from such things. I'm just recommending you become very familiar with these issues before going live with this setup. http://clusterlabs.org/wiki/Main_Page http://people.linbit.com/~florian/heartbeat-users-guide/ Let me/us know if you have specific questions once you start setting things up. Good luck! alex --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Coding a program to do mass downloading help
wget? If there are simply links on the page to get, you can use the recursive option: -r --recursive Turn on recursive retrieving. If you have a list of the URLs for the files to get: -i file --input-file=file Read URLs from file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.) If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. The file need not be an HTML document (but no harm if it is)---it is enough if the URLs are just listed sequentially. However, if you specify --force-html, the document will be regarded as html. In that case you may have problems with relative links, which you can solve either by adding base href=url to the documents or by specifying --base=url on the command line. On Wed, May 19, 2010 at 1:44 PM, AZ RUNE arizona.r...@gmail.com wrote: I have a friend that does DJ work with a subscription to a closed music repository. In the repository there are 4 categories of music he wants to download with 4,000+ songs per category Is there a program that will do that automated over http if given the url? Or would it have to be custom built? Any ideas? -- Brian Fields arizona.r...@gmail.com --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Coding a program to do mass downloading help
--user=user --password=password Specify the username user and password password for both FTP and HTTP file retrieval. These parameters can be overridden using the --ftp-user and --ftp-password options for FTP connections and the --http-user and --http-password options for HTTP connections. This assumes a basic authentication scheme I believe. There is also support for cookie handling, and SSL support (if compiled in). -- Dan. On Wed, May 19, 2010 at 2:15 PM, Taylor, Kaia kaia.tay...@schwab.comwrote: Cool instructions, thanks for the URL, James! I wonder, though, about the *closed* music repository part of the request. Will the DJ need to handle authentication/https/cookies? -- *From:* plug-discuss-boun...@lists.plug.phoenix.az.us [mailto: plug-discuss-boun...@lists.plug.phoenix.az.us] *On Behalf Of *James Finstrom *Sent:* Wednesday, May 19, 2010 1:57 PM *To:* Main PLUG discussion list *Subject:* Re: Coding a program to do mass downloading help http://www.editcorp.com/Personal/Lars_Appel/wget/v1/wget_7.html --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: load balanced configuration
On Wed, May 19, 2010 at 7:57 PM, keith smith klsmith2...@yahoo.com wrote: This is kind of fizzy to me. I'm glad you brought it up. I did experience this 6 to 9 month ago when the data center chanced the NIC card. I think they had to flush some buffers in their routers so the new MAC address could be found and cached if I recall correctly. They likely had to clear the ARP cache. We are in a data center and use their DNS. So I'm thinking the request goes to the root server then to the data center's DNS and it tells the client what the IP address is. So if the Data Center's DNS is changed to point to a new IP for our domain then that would be instantaneous or would the client and everyone along the way cache the IP? The A Record for your site can still be cached for up to the TTL time by local dns services. Thus, the following can occur: 1) Client visits site 2) ISP cache's IP address after it does the initial look up 3) Something bad happens to the data center 4) IP address is updated 5) Client goes to visit site again 6) Client's ISP realizes they have visited that site recently, and provides the cached answer for the DNS response. 7) Clients browser / application is forwarded to the old IP address, and encounters an error. Keith Smith --- On *Wed, 5/19/10, Ed Knapp catber...@hotmail.com* wrote: From: Ed Knapp catber...@hotmail.com Subject: Re: load balanced configuration To: Main PLUG discussion list plug-discuss@lists.plug.phoenix.az.us Date: Wednesday, May 19, 2010, 6:09 PM One thing struck me here with your description... “and a change to the DNS and we are off and running” While your DNS records might be changed relatively quickly during an incident, the change Itself can take quite a while to trickle down to the end users/clients out in the cloud. Any client’s DNS resolution that has not expired in the cache nor manually refreshed will still fail to properly resolve/connect. It doesn’t usually, but I tell clients to plan for 48 hours Estimated time for the change to completely propagate. I would hate for you to get blindsided with a person hovering over you asking how much longer It is going to take before the site is back up and operational. It is frustrating when you have Fixed the issue [ problem :-) ] but have to just sit and wait for it to complete. There are certainly strategies to mitigate this risk and I do not know if you maintain your Own DNS servers or do you work through a hosting provider/domain registrar. I hope this helps a bit. Ed On 5/19/10 2:07 PM, keith smith klsmith2...@yahoo.com wrote: Currently we have two servers in our main data center. One serves our shopping cart. The other contains quite a bit of content that is data driven (reads). The content site is very active. The orders on the shopping cart are spread apart by one or two minutes during the busiest part of the day. We store a lot of data with each order so most of this is writing. The shopping cart is backed up to the server in the other data center. Supposedly if there is a problem, a few things need to be done to the backup server in preparation to make it live, and a change to the DNS and we are off and running. The problem I am trying to solve is the other server (content site) is not currently backed up automatically. Another layer of this is these are managed servers. We have an excellent relationship with the data center owner and have 24/7 access to him and his staff. He manages all three servers and has always done a good job. I am the one tasked with keeping our sites online 24/7. I was hoping by configuring two servers, each in a different location, that, in the event of one of the data centers being completely severed from the Internet that the other server would automatically, without any human intervention, take over the full load of the other server and those visiting either of our sites would not know there had been an issue. In a nutshell I am trying to create an automated backup that is a automated fail over solution. I appreciate all your feedback! Keith Smith --- On *Wed, 5/19/10, Dan Dubovik dand...@gmail.com* wrote: From: Dan Dubovik dand...@gmail.com Subject: Re: load balanced configuration To: Main PLUG discussion list plug-discuss@lists.plug.phoenix.az.us Date: Wednesday, May 19, 2010, 1:45 PM The question I have, are you trying to actually load balance things? Or just have a remote location that you can fire up with live data at a moments notice? Basically, are you wanting an active/active configuration, or active/passive? active/active across DC's can get kind of hairy depending on what the network looks like. active/passive won't give you any performance gains, but can simplify the configuration, while providing the HA you seem to be after. As Kaia pointed out, what the traffic looks like (reads vs
Re: Command Line Question
rm -i *.txt | yes may work for you? [r...@localhost]# yes --help Usage: yes [STRING]... or: yes OPTION Repeatedly output a line with all specified STRING(s), or `y'. --help display this help and exit --version output version information and exit On Mon, May 10, 2010 at 9:35 AM, Eric Cope eric.c...@gmail.com wrote: Good morning all, I have a question that I don't know how to google. When I issue a command like rm -i *.txt, it responds with a question that I type 'y' for every line. I recall a way to issue a 'y' to every question. I thought it was 'y!' but that doesn't seem to work. Does anyone know this trick. Thanks, Eric p.s. yes, I know I can use \rm *.txt or rm -f *.txt for this specific trite case, but there are other situations where this would be helpful. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: Command Line Question
yes | rm -i *txt seems to be the proper syntax. [r...@localhost yestest]# rm -i file.* yes -bash: yes: No such file or directory Dan On Mon, May 10, 2010 at 10:11 AM, Eric Cope eric.c...@gmail.com wrote: Thanks, the yes command was what I was looking for. Thanks everyone. Eric On Mon, May 10, 2010 at 9:52 AM, tjones...@cox.net wrote: #unalias rm Sent from my Verizon Wireless BlackBerry -Original Message- From: Eric Cope eric.c...@gmail.com Date: Mon, 10 May 2010 09:35:15 To: Main PLUG discussion listplug-discuss@lists.plug.phoenix.az.us Subject: Command Line Question --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
