Tuesday, January 15, 2013

Live Partition Table Updates in RHEL 6

A typical process for us is sending a request to the ESX team and requesting the expansion of a disk.  Once we get it back, we rescan the SCSI bus, add a new LVM partition, probe the partition table with partprobe to update /proc/partitions, and add the new partition as a physical LVM volume.

This was ideal for us on our RHEL 5 guests, since it allowed our ESX team to manage one or two large disks per guest rather than dozens of small ones.

Unfortunately, it seems that if a block device has partitions mounted, the ability to probe it for new partitions with partprobe is no longer present on RHEL 6.  There's a great post summing up the reasons for that here, as well as a workaround with the partx command.  However, the post -- ostensibly wisely -- suggests that partx is a dangerous workaround and that the best option is a reboot.

I'm prepared to bite the bullet and suggest that the ESX team start provisioning a new disk for every storage expansion request, but the fact remains that we had a production guest that already had a disk expanded and we wanted to avoid a reboot if possible.

It turns out that partx -a /dev/sdb and partprobe /dev/sdb run exactly the same thing.  From libparted/disk.c:

 * Tell the operating system kernel about the partition table layout
 * of \p disk.
 * This is rather loosely defined: for example, on old versions of Linux,
 * it simply calls the BLKRRPART ioctl, which tells the kernel to
 * reread the partition table. On newer versions (2.4.x), it will
 * use the new blkpg interface to tell Linux where each partition
 * starts/ends, etc. In this case, Linux does not need to have support for
 * a specific type of partition table.
 * \return 0 on failure, 1 otherwise.

ped_disk_commit_to_os (PedDisk* disk)

So it looks like it makes an ioctl() call with BLKRRPART for pre-2.4 kernels and BLKPG on more recent kernels.

Let's see what partx does, from util-linux-ng/addpart.c:partx_add_partition():

return ioctl(fd, BLKPG, &a);

The partx command makes the same ioctl() call.  Dangerous?  Possibly, but since it's doing the same thing that we've done on RHEL 5 for ages, I wasn't particularly worried about it.  I went ahead and ran partx -a /dev/sdb and voila!  The kernel updated its partition table and I was able to go ahead and add the disk as an LVM physical volume.

In the future, we'll be adding new disks instead of expanding old ones, but if you find yourself in the same position of having to add a new partition on a live disk, hopefully this will help.

Thursday, January 3, 2013

Errybody's Talkin' 'Bout Notifications

DMNotify- The Latest Iteration of Irssi Notifications

It seems like everybody has their own notification solution for IRC.  I've gone through a few over the years, including GMNotify , hodgepodges of Perl and Python scripts unprofessionally cobbled together to leverage Growl notifications, and no notifications at all!

DMNotify  is the latest solution, and the best thus far.  It uses the Twitter API via Net::Twitter to send DMs on nick highlights and private messages, and you can respond to the last person who notified you.  The full README at the GitHub link provides a bit more information.