Planet CentOS

April 11, 2013

Karanbir Singh

Vault.centos.org changes

Hi,

In the last 45 days, Vault.centos.org's 2 public facing machines delivered just under 66 TiB of data. So while we try and spread this load a bit ( its growing at 25 - 35% month on month ), we've had to make a few changes.

Firstly, isos are no longer directly downloadable from vault.centos.org, you will need to go the torrent route if you want older, deprecated release isos

Secondly, we've turned off multi range requests ( httpd will still accept upto 5 range's, and then block after that )

Over the next few days, we are going to recycle some of the larger disk mirror.centos.org nodes into vault.centos.org; If someone wants to contribute to this effort, please come find us on irc.freenode.net in channel #centos-devel or #centos-mirror or tweet us @centos or email us the address mentioned at http://wiki.centos.org/Donate - but keep in mind that need machines with more than 1 TiB of usable space, and more than 300mbps of network capacity, and since we will consume that bandwidth high density hosting facilities with high contention on the links wont work.

- KB

by Karanbir Singh at April 11, 2013 11:08 PM

April 01, 2013

Jim Perrin

Big changes within CentOS and Xen

As most of you know I've been rather disconnected from the CentOS project for a few years. For both personal and professional reasons I had stepped away to get some things straight for myself. About 2 months ago Karanbir and Johnny Hughes reached out to bring me back into the community. As part of the return, we have been working together to collaborate on some truly significant changes within

by Jim Perrin (noreply@blogger.com) at April 01, 2013 06:38 AM

March 30, 2013

Fabian Arrotin

Automatic laptop backup with NetworkManager (and correct selinux policies …)

Those days, almost everyone uses a laptop as his primary (work)station : I don't remember when I was using something else than a laptop for both work and home usage. I admit that I'm using what I'll describe in the following sentences for quite some time, but it seems some people I spoke to don't know what can be done around NetworkManager, and because I encountered a (small) issue with that process (because of updated selinux policies), I thought it would be a good time to speak about it.

Let me first discuss a (little) bit about NetworkManager : almost everyone (using CentOS/Fedora or other distributions) knows what it's all about : helping you to quickly switch from one network to another, that network being a wired one, a Wifi hotpot, or even a 3G connection through your 3G usb modem or your smartphone being used as a modem, etc, etc .... That's the "visible" part of NetworkManager.  While some people don't seem to like it, I admit myself that I really appreciate it and I use it on a daily basis for $work and $home usage (switching from wired to wireless, and so on). A quick read in the NetworkManager man page shows that you can "script" events based on the actual status of your network interface : basically all executables scripts found by NetworkManager under /etc/NetworkManager/dispatcher.d/ will be executed on network change. When I discovered that (was quite some time ago now ...), I decided that it would be good to launch backup script for my laptop, depending on the network my laptop is connected, and using different profiles. For example, (the "head" of ) a simple script can look like :

#!/bin/bash
IF=$1
STATUS=$2

if [[ "$IF" = "eth0" && "$STATUS" = "up" ]] ; then

NET=$(/sbin/ip -4 route show dev eth0|awk '{print $1}'|grep -v default)
if [ "$NET" = "192.168.2.0/24" ] ; then # and now the rest up to you ....

You've got the idea, so it's now just a matter of writing the whole script. One thing that I like when writing some small scripts is the fact that I can be notified on my laptop when something happens (or doesn't, because of errors). I use also quite often notify-send for that, but because all scripts under dispatcher.d are executed under root, I prefer from there "jumping" to my user account with a "su - $my_user_name -c $my_backup_script.sh".

Of course, my script needs several things to "interact" with my desktop session : the DISPLAY to use and also the dbus-session I currently use (because I also have to use gvfs-mount to automatically mount in my gnome session some remote folders, like , (yeah, don't shoot me for that, not my idea) CIFS shares for $work).

So that backup script needs some variables like this :

export DISPLAY=":0"
export DBUS_SESSION_BUS_ADDRESS=$(cat /proc/$(pidof nautilus)/environ|tr '\0' '\n'|grep DBUS_SESSION_BUS|cut -f2- -d '=')

If I started that blog post, it's not to speak about NetworkManager at first (well, I still thought that some people would benefit of those unknown/unused dispatcher.d scripts ....) but because I encountered an issue with the recent updates to CentOS 6.4 (and to be precise, newer selinux-policy-3.7.19-195.el6_4.3.noarch package). So it was time to dive into that issue , and *yes*, i run selinux everywhere, including on my laptop ...

Long story short : because I use rsync for my backup scripts (why having to reinvent the wheel ? ), I had to enable two selinux booleans :

setsebool -P rsync_client 1
setsebool -P rsync_export_all_ro 1

But that was still not enough. sealert/audit.log/audit2allow to the rescue (read the Selinux page on the CentOS wiki) and finally I created a custom policy that suits my needs. Here it is :

module rsync-client.pol 1.0;

require {
type initrc_tmp_t;

type user_home_t;
type rsync_t;
class sock_file getattr;
class file write;
}

#============= rsync_t ==============
allow rsync_t initrc_tmp_t:file write;

allow rsync_t user_home_t:sock_file getattr;

Now, everytime I connect my laptop to a (recognized) network, my laptop auto-backups itself :

by fabian.arrotin at March 30, 2013 03:49 PM

March 20, 2013

Karanbir Singh

Announcing the CentOS Dojo at Antwerp 2013

Hi,

The first ever CentOS Dojo, a one day training and socalising day dedicated to CentOS and how people use it, will be held at Antwerp, Belgium on the 8th of Apr.

You can see the great speaker lineup on the events page at : http://wiki.centos.org/Events/Dojo/Antwerp2013 - we have tried to cover all the major conversation areas around CentOS these days. Ranging from provisioning, management, app deployments, system and virtualisation tuning, virtual infrastructure and more.

Its going to be a great day, register up, and see you all there. And remember, there is an exclusive CentOS Dojo Tshirt for everyone who attends ( plus, there might be more goddies too ).

Jump directly to the registration page : http://centosdojoantwerp2013.eventbrite.com/

- KB

by Karanbir Singh at March 20, 2013 11:18 AM

March 15, 2013

Karanbir Singh

qaweb.dev.centos.org is now gone

It started off by being a place that everyone could chat and talk about things that were happening in the QA cycles inside CentOS. But things have changed quite a lot - our QA cycles are a lot shorter, there is a lot more automation and there is almost no real security exposure to users.

And I think we can do this better. We can create a better end user experience that gives them direct access, easily, to the state of play within the testing. And we should be able to automate more to get better coverage.

To that aim, qaweb.dev.centos.org is now going away. And we are working on some alternatives. Starting with having a nightly QA cycle, that considers point releases and all updates upto that point. And adding more external tests as well, like the ltp content ( http://ltp.sourceforge.net/ ). If you wish to join in that effort, drop in on the centos-devel list ( http://lists.centos.org/ ) and jump right in. Ref threads: http://lists.centos.org/pipermail/centos-devel/2013-March/009098.html and http://lists.centos.org/pipermail/centos-devel/2013-March/009099.html

Here is a link to the official announcement that just went out : http://lists.centos.org/pipermail/centos-announce/2013-March/019649.html

See you there,

- KB

by Karanbir Singh at March 15, 2013 01:28 PM

March 12, 2013

Karanbir Singh

nazar.karan.org in maint mode

Hi,

https://nazar.karan.org/ services are going to be partially down as I migrate services over to a faster, more memory, lesser power consuming, many more cores machine. Everything should be back to production by midday Mar 12th, 2013. Services impacted include:

  • git repos
  • Reimzul's irc interface
  • Alt.Bsys triggers

There is a backup instance running, so if anyone needs to get to some specific data in a rush, ping me on irc and we can get access setup.

- KB

by Karanbir Singh at March 12, 2013 01:15 AM

February 08, 2013

Karanbir Singh

Introducing Raindrops

Introducing ProjectRaindrops, a service that will build disk images for you ready to be used in your favourite cloud or virtualised environment.

One of the key barriers to entry into a cloud or virtualised environment : setting up and maintaining a piece of infrastructure that builds disk images. Its also a colossal waste of time and involves needing a complete instance of the environment one is going to deploy the image in. Wouldn't it be nice if there was a service that allowed you to drop in a kickstart file, write up a config to go with that kickstart and just build the image for you ?

Now there is. Its called ProjectRaindrops and its live, in beta mode, at http://projectraindrops.net/ ; As an initial kick off we are doing builds for HVM ( ie. any fully virtualised environment, be it KVM, Xen, VirtualBox or VMware ); With more HyperVisors and more disk formats coming soon.

Getting started is easy, goto the Raindrops website; sign in using either github or twitter credentials ( we don't store any personal details, but if your account does not have an email address, we won't be able to send out email notifications ). There are two key components:

  • A config file: that contains metadata about what you want the build to do
  • A kickstart file: that has the actual details on what the build should contain

Example pre-populated templates are available, just click on the new button and the template will be injected in. And there is some validation rules that track the config and kickstart file, so if you make mistakes or lose format validation, you will find out right away. And we have versioning for each file built in too. Finally, there is no real correlation between a config file and a kickstart file, when you create a new job you can pick an arbitrary kickstart and any config file in your account to match it with.

So, now that you have a config and a kickstart file, click on new job, give it a name, select which config you want to use and what kickstart file, click on 'Submit' and in a few minutes your build should be done. You can even track the job as it works its way through various stages.

Lots of interesting things in planning and development stages, stay tuned for more news in the coming weeks. For now, go ahead and drop in on http://projectraindrops.net/ and give it a shot. Just consider it to be a Beta release, so send us lots of feedback.

- KB

by Karanbir Singh at February 08, 2013 04:57 PM

February 02, 2013

Jim Perrin

Copy nearly every file with bash

I found an interesting trick in bash today that may help a few other folks as well. Occasionally I find that need to copy almost every file in a directory, except for one or two. Usually I'd copy everything and then delete the stragglers I didn't want from the destination directory. There had to be a better way, but as I said I'm lazy. Turns out I found the better way today. [jperrin@ferrata ~]$

by Jim Perrin (noreply@blogger.com) at February 02, 2013 11:42 AM

January 22, 2013

Karanbir Singh

Building a development cloud on budget

I'm looking to put together a development cloud - a full featured one at that, on a budget. So here is what I'm thinking about compute nodes :

  • AMD FX-6 6100 AM3+ cpu
  • Motherboard to go with it
  • 32 GB of ram
  • 180 watt PSU
  • 32GB SSD for local storage

I can get one of those 'sets' for just under £190.00 ; For Network, a HP ProCurve J9028A / their 1800-24G should do, and available cheaply off ebay. For Storage, I am thinking of repurposing my HP MicroServer with 4x500GB SATA's.

So four 'compute nodes' + switch + cables and disks for the MicroServer should clock in at £1,000.00 still need some sort of a case or rails ( intend to host this at home ).

What am I missing ? What might I be better off with ? £1k for 24 cores, 128gb of ram and 4 physical nodes seems like a good deal to me, but could I do better ?

- KB

by Karanbir Singh at January 22, 2013 02:57 PM

November 16, 2012

Jim Perrin

Really Gnome?

I really don't want to turn this blog into an anti-gnome3 themed thing, but they seem to insist on terrible things. Being the type of person I am, if I find terrible things, I'm going to share terrible things. So in that spirit, here's your terrible thing: http://www.slideshare.net/juanjosanchezpenas/brightfuture-gnome This slide deck starts off like any other. A bit of backstory, a bit of

by Jim Perrin (noreply@blogger.com) at November 16, 2012 07:43 AM

November 10, 2012

Karanbir Singh

Off to another DC in the morning

Off to another DC tomorrow morning, hoping to swap out some seriously old hardware ( ~ 7'ish years old ) with something slightly newer ( only 4 years old! )

by Karanbir Singh at November 10, 2012 09:51 PM

November 08, 2012

Jim Perrin

Gnome is just not getting the message

While I've bounced around to various desktop environments, I have found that I always end up coming back to gnome. That is, until now. Gnome3 has already been widely regarded as a step in the wrong direction, however the developers appear to be largely ignorant of what the users want. The arrogance, and ignorance coming from the gnome community has finally pushed me to the breaking point. It's

by Jim Perrin (noreply@blogger.com) at November 08, 2012 11:06 AM

November 04, 2012

Karanbir Singh

Off to the DC in the morning

There was a time when I spent lots of time in the Data Center. I didnt enjoy them very much then, too much noise, too many people telling me what to do and way too many unpaid overtime hours clocked up. The last of that was about 12 years ago. These days I goto the DC maybe three or four times in a year, and its mostly related to CentOS infrastructure or my own personal machines hosted around London. And unlike things 12 years ago, I quite enjoy my trips into the vault like colo rooms now.

Apart from a bunch of routine things that I need to sort out tomorrow, I'm hoping to bring online a hardware rng adapter, and a Tilera Server. If you havent seen this platform before, I recommend you do : specially if lots-of-cores is something you are keen on, or have a problem domain overlap with.

- KB

by Karanbir Singh at November 04, 2012 09:36 PM

November 01, 2012

Jim Perrin

Best Networking Cheat Sheets.

This guy has an absolutely fantastic set of cheat-sheets. If you're doing anything network related, these are definitely good to have on-hand.  http://packetlife.net/library/cheat-sheets/

by Jim Perrin (noreply@blogger.com) at November 01, 2012 09:11 AM

October 30, 2012

Karanbir Singh

A few hours of Mail and List outage for centos.org

The CentOS.org infra is pretty well spread out, but we got caught out by the Sandy storm. Specially our mail and list services, which run from an Internap facility in New york.

Outage lasted from just after 03:00 hrs UTC October 30th 2012 to 07:12 hrs UTC October 30th 2012; and we dont seem to have lost any communcation. If you had bounce backs during that timeperiod, please do retry / resend the emails.

We dont have a machine that can be used as a hot-standby, but we do have fairly good backups. So should the machine go offline completely or suffer major damage, we would be able to bring services back on a different machine with no real loss of data.

- KB

by Karanbir Singh at October 30, 2012 07:09 PM

Fabian Arrotin

Using Openssh as transport for Ansible instead of default paramiko

You've probably read that Ansible uses by default paramiko for the SSH connections to the host(s) you want to manage. But since 0.5 (quite some ago now ...) Ansible can use plain openssh binary as a transport. Why ? simple reasons : you sometimes have complex scenario and you can for example declare a ProxyCommand in your ~/.ssh/config if you need to use a JumpHost to reach the real host you want to connect to. That's fine and I was using that for some of the hosts i have to managed (specifying -c ssh when calling ansible, but having switched to a bash alias containing that string and also -i /path/to/my/inventory for those hosts).

It's great but it can lead to strange results if you don't have a full look at what's happening in the background. Here is the situation I just had yesterday : one of the remote hosts is reachable, but not a standard port (aka tcp/22) so an entry in my ~/.ssh/config was containing both HostName (for the known FQDN of the host I had to point to, not the host i wanted to reach) and Port.

Host myremotehost
HostName my.public.name.or.the.one.from.the.bastion.with.iptables.rule
Port 2222

With such entry, I was able to just "ssh user@myremotehost" and was directly on the remote box. "ansible -c ssh  -m ping myremotehost" was happy, but in fact was not reaching the host I was thinking : running "ansible -c ssh -m setup myremotehost -vvv" showed me that ansible_fqdn (one of the ansible facts) wasn't the correct one but instead the host in front of that machine (the one declared with HostName in ~/.ssh/config). The verbose mode showed me that even if you specify the Port in your ~/.ssh/config, ansible will *always* use port 22 :

<myremotehost> EXEC ['ssh', '-tt', '-q', '-o', 'AddressFamily=inet', '-o', 'ControlMaster=auto', '-o', 'ControlPath=/tmp/ansible-ssh-%h-%p-%r', '-o', 'StrictHostKeyChecking=no', '-o', 'Port=22', '-o', 'User=root', 'myremotehost', 'mkdir -p /var/tmp/ansible-1351603527.81-16435744643257 && echo /var/tmp/ansible-1351603527.81-16435744643257']

Hmm, quickly resolved : a quick discussion with people hanging in the #ansible IRC channel (on irc.freenode.net) explained the issue to me : Port is *never* being looked at in your ~/.ssh/config, even when using -c ssh. Solution is to specify the port in your inventory file, as a variable for that host :

myremotehost ansible_ssh_port=9999

In the same vein, you can also use ansible_ssh_host , this one corresponding to the HostName of your  ~/.ssh/config.

Hope that it can save you time, if you encounter the same "issue" one day ...

by fabian.arrotin at October 30, 2012 01:35 PM

Jim Perrin

Add self-signed certs to Chrome

While Chrome is an excellent browser, there isn't a quick and easy method to convince it to stop freaking out over self-signed or custom ssl certificates. For the majority of users this is probably a good thing, however for sys-admins or developer types there has to be a better way. This bash script takes the hassle out of importing certificates to make Chrome be quiet. The first option for the

by Jim Perrin (noreply@blogger.com) at October 30, 2012 12:26 PM

October 26, 2012

Fabian Arrotin

Ansible as an alternative to puppet/chef/cfengine and others …

I already know that i'll be criticized for this post, but i don't care :-) . Strangely my last blog post (which is *very* old ...) was about a puppet dashboard, so why speaking about another tool ? Well, first i got a new job and some prerequisites have changed. I still like puppet (and I'd even want to be able to use puppet but that's another story ...) but I was faced to some constraints when being in front of a new project. For that specific project,  I had to configure a bunch of new Virtual Machines (RHEL6) coming as OVF files. Problem number one was that I can't alter or modify the base image so i can't push packages (from the distro or third-party repositories). Second issue is that I can't install nor have a daemon/agent running on those machines. I had a look at the different config tools available but they all require either a daemon to be started, or at least having extra packages to be installed on each managed node. (so not possible to have puppetd nor puppetrun or invoke puppet directly through ssh , as puppet can't even be installed, same for saltstack). That's why i decided to give Ansible a try. It was already on my "TO-test" list for a long time but it seems it was really fitting the bill for that specific project and constraints : using the 'already-in-place' ssh authorization, no packages to be installed on the managed nodes, and last-but-no-least, a learning curve that is really thin (compared to puppet and others, but that's my personal opinion/experience).

The other good thing with Ansible is that you can start very easily and then slowly add 'complexity' to your playbooks/tasks. I'm still using for example a flat inventory file, but already organized to reflect what we can do in the future (hostnames included in groups, themselves included in parents groups - aka nested groups). Same for the variables inheritance : at the group level and down to the host level, host variables overwriting those defined at the group level , etc ...)

The Yaml syntax is really easy to understand so you can have quickly your first playbook being played on a bunch of machines simultaneously (thanks to paramiko/parallel ssh). The number of modules is less than the puppet resources, but is quickly growing. I also just tested to tie the execution of ansible playbook with Jenkins so that people not having access to the ansible inventory/playbooks/tasks (stored in a vcs, subversion in my case) can use it from a gui.. More to come on Ansible in the future

by fabian.arrotin at October 26, 2012 02:02 PM

October 25, 2012

Jim Perrin

Playing with Pulp's REST API

Pulp gives you a very powerful admin cli utility in pulp-admin, however that power comes with a price. The command string can sometimes get a bit lengthy. Issuing iterative commands (bulk operations to all repositories for example) often require a bit of grep or awk piping in order to get things done. Fortunately pulp has implemented a very nice rest api that allows you create your own

by Jim Perrin (noreply@blogger.com) at October 25, 2012 11:14 AM

Welcome to Pulp

TL;DR: Keep an eye on this project. It's going to save you EONS time because it handles the boring work for you. Testing and applying operating system updates is one of the more mundane tasks for sys-admins, and yet it can quickly become complex when you're dealing with vendor support or multiple versions of applications. Testing updates to determine if they're safe to push to production, while

by Jim Perrin (noreply@blogger.com) at October 25, 2012 09:53 AM

October 03, 2012

Karanbir Singh

90 days of IPv6

Most people, me included, still consider IPv6 usage to be something not worth worrying about. This comes from the fact that most services are quite happy chugging along with just IPv4 access at both the service provider end and the service consumer end. However, what happens when an IPv6 option shows up ? Here are some numbers, many will find interesting, from the CentOS Mirrorlist service.


1st June 2012 : 0 hits on IPv6
11th June 2012 : We launched IPv6 access for mirrorlist.centos.org
+ 90 days ( midnight 13th Aug ):
--- 12,831,352 Hits to mirrorlist.centos.org were over ipv6
--- 711,014 unique IPv6's
--- with a usage average of 1.4 hits/sec
--- peak at 350 hits/sec

There are two, sometimes three, but always atleast two machines that respond to mirrorlist.centos.org requests over ipv6. Looking at the stats for one of these machines that has always been around, right since day one we get :

Total hits : 6,324,980
Of this 5,089,935 were CentOS-5 and 1,176,602 were CentOS-6 requests, the rest were invalid requests ( could be either CentOS-4 or for repos that dont exist )
Of the CentOS-5 hits, 61.24% of requests were to x86_64 repos
and from CentOS-6 hits, 27.05% of the reqests were to i386 repos

Interesting stuff, fairly large numbers.

Also worth noting here is that the numbers represent quite a skewed sample set, IPv6 is only really usable in some specific setups and in some specific environments / data centers. It does not represent an overall state of CentOS userbase, so please dont use these numbers to signify that.

- KB

by Karanbir Singh at October 03, 2012 12:22 PM

July 28, 2012

Jim Perrin

Gnome isn't dead, it just needs brains.

Not a full posting, but this definitely struck me as worth linking to and/or resposting. http://blogs.gnome.org/otte/2012/07/27/staring-into-the-abyss/

by Jim Perrin (noreply@blogger.com) at July 28, 2012 09:25 PM

July 10, 2012

Johnny Hughes

CentOS-6.3 Release

July 9th, 2012 - Today the CentOS Project has released CentOS-6.3.  This release came 18 days after the upstream release of EL 6.3.  The major issues that we had with getting this release out were getting the i386 distribution to boot properly and adjusting the content of DVD-1 and DVD-2 of each architecture to allow for the most common install groups to come from DVD1 and not require DVD2.

Some of the major changes for this release include a move from OpenOffice 3.2.1 to LibreOffice 3.4.5 and the addition of Virtual to Virtual and Physical to Virtual (virt-v2v and virt-p2v) server migration tools to KVM Virtual Machines.  Please see the CentOS-6.3 release notes for more information on these and other features.  The Open H Website has a very good article on what's new in EL6.

We were pretty much ready to release the distribution by Friday, July 6th, but we had to then copy our trees to the several dozen internal mirror servers in the CentOS Project's infrastructure and then open those mirrors up to the more than 500 external mirrors that serve CentOS in more than 75 countries world wide.  While the CentOS Project is doing a point release, you can see what is happen by visiting our QA Development website and looking at the blog entries on the dashboard,

We hope you enjoy CentOS-6.3 !

by Johnny Hughes (noreply@blogger.com) at July 10, 2012 06:14 AM

DRBD 8.3.12 for CentOS-5 in testing

The ELRepo Project has DRBD packages for CentOS-5 and CentOS-6, named drbd83-utils or drbd84-utils.  The CentOS Project does not want to maintain extra packages that exist in other places unless we need to change them ... so we are not going to create DRBD packages for CentOS-6.

Since CentOS-4 is being EOL'ed in less than a month, we are also not going to publish updates for the DRBD in CentOS-4.

This leaves the DRBD for CentOS-5 that are part of CentOS Extras.  Since these have been released for CentOS-5, we will continue to maintain the DRBD version 8.3.x  tree (drbd83) in CentOS Extras.

A new version of DRBD 8.3 (drbd83-8.3.12) has been released to the testing repository for CentOS-5.  You can see the details here:

DRBD 8.3.12 for CentOS-5

If you want to use DRBD 8.4.x for CentOS-5, rather than releasing it separately, the CentOS Project recommends that you use drbd84-utils from ELRepo (linked above).

For users who want to use the drbd83-8.3.12 version ... please test the version that is currently in CentOS Testing and provide feedback.  With enough feedback I will move the packages from testing to CentOS Extras.

NOTE:  These packages have now been moved to CentOS Extras and can be installed normally with yum.

by Johnny Hughes (noreply@blogger.com) at July 10, 2012 06:09 AM

June 15, 2012

Jim Perrin

Making Gnome3 useful on Fedora 17

There are some very nice feature improvements in gnome3, but if your used to the older gnome releases, odds are pretty good that you hate gnome3.  Don't let that stop you from using it though. With a few minor tweaks, you can have the best of both worlds.     To start, you'll need a few additional packages that you probably didn't install from the beginning, so lets grab them now. [root@

by Jim Perrin (noreply@blogger.com) at June 15, 2012 12:31 PM

June 07, 2012

Karanbir Singh

Getting in touch

I've never listed my mobile number anywhere on the internet, and as far as I remember I've only ever shared it with friends and family. On the other hand, I've had the same number for years and its possible that its 'leaked'; But I still find it quite odd that people around the world manage to get their hands on the number, with no real effort. And that means I get calls.

Calls from people in Argentina at 4am UK time, wanting to know when php-5.4 is going to be released into CentOS-5. Calls from people in the UK, at 8am wanting to know if the httpd update released last night had a fix for CVE-XXX. Calls from people in India at 10pm UK time wanting to find out if the sound card on the motherboard they bought a few hours back, is supported on CentOS. A disgruntled passenger trying to check-in to their flight the next day, and the system throwing up a 'Apache on CentOS' page.

Some are a bit more alarming. eg. a call from people at a Large Defence Contractor in the USA asking who their 'CentOS Technical Account Manager' was and if I knew what the SLA terms were. Or the time when I got a call from a hosting company's Data Center saying there was a fire in the DC and they wanted to know if their CentOS backups were intact.

It's not something new, I've had these calls over the years from maybe 2008 or so. At one point, when it was really hectic with almost 10 to 12 calls a week, in 2010 I was seriously considering changing my number. Just doing the 'ignore if the number isnt in the address book' wasent scaling for me. But I didnt, the process of changing my number with everyone I knew was too much hassle, so I started giving people an alternative number and mostly started ignoring the 'popular' old mobile number. The number of such calls has now drastically reduced. I get maybe 1 or 2 in a week and in many cases I answer them and have had the odd interesting conversation. But realistacally, I think the time has come to change that number.

What I will, however, do is offer up a Voip line : +44-207-0999389 ; This terminates at a phone that I have on my desk. And I will try to make sure its turned on whenever I am doing CentOS stuff, or am in 'Open Source' mode. Go ahead, use that number - give me a call and if I am around, would love to have a chat. But please stop calling me on my mobile.

btw, I have tried to find my own number and failed to do so - even entering parts of the numbers into the various search engines does not bring up my mobile number. So, I have no idea where all these people suceed in finding it ?

- KB

by Karanbir Singh at June 07, 2012 08:52 PM

Johnny Hughes

CentOS Project Release Times

One of the issues we have had in the CentOS Project is that at point release time, we were lagging behind a bit on getting the releases out.  Recently, the CentOS Project has addressed this issue in 3 major ways:

1.  We created the CR repository where we can, if there are delays, push out multiple updates that work together while we take a care of problems with packages that don't build, if we have issues getting the ISOs to properly build, etc. during the point release process.  This gives us an avenue to release pieces of the point release without having to release the whole thing.  We have actually not had to use the CR repository on the last 2 release cycles (CentOS-5.8 and CentOS-6.2) as we got each of them out within 10 days... however, this is an option that we have if we need to get out pieces of the release (like critical updates) if there is any kind of delay in the main process.

2.  We now have corporate sponsors who sponsor 2 CentOS Developers to work on the CentOS Project full time.  That means that we now have 80 paid hours per week of CentOS Project time where we get do nothing but CentOS Project related work.  The sponsors do not ask for anything in return, just faster CentOS updates by the current CentOS developers who get to make the CentOS Project their daily work priority.  This should be huge in preventing future delays.

3.  We have given better tools to the CentOS QA team.  They now see the packages that we are building in the QA IRC channel, the status of the builds on the QA mailing list and we have a scratch build area setup where they can do test builds to help us develop patches for issues, etc.  We have also been using trello for collaboration during point release time to get updates out faster.

The end result of all of these efforts have shown that we can get packages out in a timely manner.  Since January 2012, there have been no updates (outside of point releases) that have taken more than 3 days to push.  In fact, only one package during the entire time has taken more than one day to get released.  As I mentioned before, the last two point releases were also completed very quickly.  Here is some press on that:

5.8 Release

6.2 Release

The bottom line is that the CentOS Project has expended great effort to ensure that point releases of current products will be released in a timely fashion.  With the changes we have put in place and with the ability to do CentOS as our main daily work focus the future looks quite good for timely CentOS updates.

by Johnny Hughes (noreply@blogger.com) at June 07, 2012 05:27 AM

April 26, 2012

Jeff Sheltren

Memcached and PECL memcache on CentOS and Fedora

At Tag1 Consulting we do a lot of work on increasing web site performance, especially around Drupal sites. One of the common tools we use is memcached combined with the Drupal Memcache module. In Drupal, there are a number of different caches which are stored in the (typically MySQL) database by default. This is good for performance as it cuts down on potentially large/slow SQL queries and PHP execution needed to display content on a site. The Drupal Memcache module allows you to configure some or all of those caches to be stored in memcached instead of MySQL, typically these cache gets/puts in memcache are much faster than they would be in MySQL, and at the same time it decreases work load on the database server. This is all great for performance, but it involves setting up an additional service (memcached) as well as adding a PHP extension in order to communicate with memcached. I've seen a number of guides on how to install these things on Fedora or CentOS, but so many of them are out-dated or give instructions which I wouldn't suggest such as building things from source, installing with the 'pecl' command (not great on a package based system), or using various external yum repositories (some of which don't mix well with the standard repos). What follows is my suggested method for installing these needed dependencies in order to use memcached with Drupal, though the same process should be valid for any other PHP script using memcache.

PECL Packages

For the Drupal Memcache module, either the PECL memcache or PECL memcached (note the 'd'!) extensions can be used. While PECL memcached is newer and has some additional features, PECL memcache (no 'd'!) tends to be better tested and supported, at least for the Drupal Memcache module. Yes, the PECL extension names are HORRIBLE and very confusing to newcomers! I almost always use the PECL memcache extension because I've had some strange behavior in the past using the memcached extension; likely those problems are fixed now, but it's become a habit and personal preference to use the memcache extension.

Installing and Configuring memcached

The first step is to get memcached installed and configured. CentOS 5 and 6 both include memcached in the base package repo, as do all recent Fedora releases. To install memcached is simply a matter of:
# yum install memcached

Generally, unless you really know what you're doing, the only configuration option you'll need to change is the amount of memory to allocate to memcached. The default is 64MB. That may be enough for small sites, but for larger sites you will likely be using multiple gigabytes. It's hard to recommend a standard size to use as it will vary by a large amount based on the site. If you have a "big" site, I'd say start at 512MB or 1GB; if you have a smaller site you might leave the default, or just bump it to 512MB anyway if you have plenty of RAM on the server. Once it's running, you can watch the memory usage and look for evictions (removal of a cache item once the cache is full) to see if you might want to increase the memory allocation.

On all Fedora / CentOS memcached packages, the configuration file is stored in /etc/sysconfig/memcached. By default, it looks like this:

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=""

To increase the memory allocation, adjust the CACHESIZE setting to the number of MB you want memcached to use.

If you are running memcached locally on your web server (and only have one web server), then I strongly recommend you also add an option for memcached to listen only on your loopback interface (localhost). Whether or not you make that change, please consider locking down the memcached port(s) with a firewall. In order to listen only on the 127.0.0.1 interface, you can change the OPTIONS line to the following:

OPTIONS="-l 127.0.0.1"

See the memcached man page for more info on that or any other settings.

Once you have installed memcached and updated the configuration, you can start it up and configure it to start on boot:

# service memcached start
# chkconfig memcached on

CentOS / RHEL PECL Module Install

Fedora

If you are on Fedora and using PHP from the base repo in the distribution, then installation of the PECL extension is easy. Just use yum to install whichever PECL extension you choose:

# yum install php-pecl-memcache

Or

# yum install php-pecl-memcached

CentOS 5 / RHEL 5

CentOS and RHEL can be a bit more complicated, especially on EL5 which ships with PHP 5.1.x, which is too old for most people. Here are the options I'd suggest for EL5:

  • If you are OK using the PHP provided with EL5, then you can get the PECL extensions from EPEL. Once you've enabled the EPEL repository (instructions), you can install either PECL extension by using the same yum commands outlined above in the Fedora section.
  • If you want to use PHP 5.2 or PHP 5.3 with EL5, I suggest using the IUS repositories (IUS repo instructions). Note that IUS provides the PECL memcache extension, but not the PECL memcached extension. Based on which PHP version you decide to use, you can install the PECL memcache extension with either:

    # yum install php52-pecl-memcache

    Or

    # yum install php53u-pecl-memcache

CentOS 6 / RHEL 6

EL6 ships with PHP 5.3, though it is an older version than is available for EL6 at IUS. If you are using the OS-provided PHP package, then you can install the PECL memcache extension from the base OS repo. If you want the PECL memcached extension, it is not in the base OS repo, but is available in EPEL. See the instructions linked from the CentOS 5 section above if you need to enable the EPEL repo.

# yum install php-pecl-memcache

Or, enable EPEL and then run:

# yum install php-pecl-memcached

As with EL5, some people running EL6 will also want the latest PHP packages and can get them from the IUS repositories. If you are running PHP from IUS under EL6, then you can install the PECL memcache extension with:

# yum install php53u-pecl-memcache

Similar to EL5, the IUS repo for EL6 does not include the PECL memcached module.

PECL Memcache Configuration

If you are using PECL memcache extension and will be using the clustering option of the Drupal Memcache module which utilizes multiple memcached instances, then it is important to set the hash strategy to "consistent" in the memcache extension configuration. Edit /etc/php.d/memcache.ini and set (or un-comment) the following line:

memcache.hash_strategy=consistent

If you are using the PECL memcached module, this configuration is done at the application level (e.g. in your Drupal settings.php).

Once you've installed the PECL memcache (or memcached) extension, you will need to reload httpd in order for PHP to see the new extension. You'll also need to reload httpd whenever you change the memcache.ini configuration file.

# service httpd reload

SELinux

If you have SELinux enabled (you should!), I have an older blog post with instructions on configuring SELinux for Drupal.

That's it, you're now good to go with PHP and memcache!

by jeff at April 26, 2012 06:02 PM

cfengine 3.3.0 packages for Fedora / CentOS / RHEL

As I've used cfengine less and less recently the packages in Fedora and EPEL have been a bit neglected. At one point someone stepped up to update them, but then nothing ever came of it. I've finally updated the packages to the latest upstream version as of this writing (3.3.0) in Fedora 16, Fedora 17, Fedora Devel, and EPEL 6. They should be pushed to the updates-testing repos for each of those releases soon if not already there. There are some package changes since the last 3.x release, so any testing people can do would be appreciated.

I've uploaded EL6 and F17 RPMs here for reference: http://sheltren.com/downloads/cfengine/testing/

Note that these are quite different from the upstream-provided RPMs which simply dump everything in /var/cfengine. The good news here is I've actually provided a source RPM for those that need to tweak the build. Also, I hit some configure errors when attempting to build on EL5 which I haven't worked out yet -- looks like an upstream bug with the configure script to me, so there are no EL5 packages at the moment.

If anyone is willing to co-maintain these in Fedora and/or EPEL with me, please let me know.

by jeff at April 26, 2012 04:06 PM

March 16, 2012

Karanbir Singh

Into Web Hosting .. Deploying CentOS images in a Cloud ?

If you are a web hosting company and deploying CentOS 5 or 6 images in a Cloud, I want to speak with you. Please get in touch via one of the mechanisms listed at : http://www.karan.org/contact.html

Alternatively, if you use services from a hosting company that uses CentOS images in a cloud : drop me a line and let me know about them, I will try and reach out directly.

Thanks in advance and look forward to getting in touch with some of you guys.

- KB

by Karanbir Singh at March 16, 2012 08:08 PM