Expanding Disk Capacity

Expanding Disk Capacity

Introduction

The lack of available disk storage frequently plagues Linux systems administrators. The most common reasons for this are expanding databases, increasing numbers of users, and the larger number of tasks your Linux server is expected to perform until a replacement is found.

This chapter explores how to add a disk to a Linux system in two ways. The first is by moving directories from a full partition to an empty one made available by the new disk and then linking the directory structures of the two disks together. The second is by merging the partitions together to create a combined partition using the Linux Logical Volume Manager (LVM).

Adding Disks To Linux

At some stage you'll be faced with the task of installing an additional hard drive into your Linux server. Perhaps an existing device failed, or maybe you ran out of available space. To provide more space, this section will cover adding a hard disk with only one partition and will then explain how to migrate data from the full disk to the new one.

Scenario

Things are getting crowded on bigboy: Even after you removed all unwanted data, the /var partition is full. You need to add a new hard drive to the system. You can verify this situation with the df -k command's output, which also shows that the other partitions are too full to accept any more data.

[root@bigboy tmp]# df -k

Filesystem  1K-blocks Used Available Use% Mounted on

/dev/hda3   505636 118224 361307 25% /

/dev/hda1  101089 14281 81589 15% /boot

none  63028 0 63028 0% /dev/shm

/dev/hda5  248895 6613 229432 3% /tmp

/dev/hda7  3304768 2720332 416560 87% /usr

/dev/hda2  3304768 3300536 4232 99% /var

[root@bigboy tmp]#

A new hard disk was added according to the manufacturer's instructions, but you now need to know how to proceed.

Determining The Disk Types

Linux stores the names of all known disk partitions in the /proc/partitions file. The entire hard disk is represented by an entry with a minor number of 0, and all the partitions on the drive are sequentially numbered after that. In the example, the system has two hard disks; disk /dev/hda has been partitioned, but the new disk (/dev/hdb) needs to be prepared to accept data.

[root@bigboy tmp]# cat /proc/partitions

major minor #blocks name

3 0 7334145 hda

3 1 104391 hda1

3 2 1052257 hda2

3 3 2040255 hda3

3 4 1 hda4

3 5 3582463 hda5

3 6 554211 hda6

22 0 78150744 hdb

[root@bigboy tmp]#

Note: Linux hard disk device names follow a specific standard. SCSI disks all start with sd and IDE disks with hd. After this comes a letter that identifies the unit number of the disk, so for example, the first disk would be a, the second would be b, the third would be c, and so on. Finally, a two-digit number defines the partition number. Using this convention the fifth partition on the fourth IDE drive would be /dev/hdd5.

Preparing Partitions on New Disks

Linux partition preparation is very similar to that in a Windows environment, because both operating systems share the fdisk partitioning utility. The steps are:

1) The first Linux step in adding a new disk is to partition it in preparation of adding a filesystem to it. Type the fdisk command followed by the name of the disk. You want to run fdisk on the /dev/hdb disk, so the command is:

[root@bigboy tmp]# fdisk /dev/hdb

The number of cylinders for this disk is set to 9729.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help):

2) Just to make sure you're on the correct device, issue the p command to print all the known partitions on the disk. In this case, there are none which is good.

Command (m for help): p

Disk /dev/hdb: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help):

3) The fdisk m command prints a small help manual of valid commands. You will see that n is the command to add a new partition. Add a new primary partition, number 1, and use the defaults to make the partition occupy the entire disk.

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 1

First cylinder (1-9729, default 1):<RETURN>

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-9729, default 9729):

4) Run the print (p) command to confirm that you successfully created the partition partition.

Command (m for help): p

Disk /dev/hdb: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/hdb1  1 9726 78148161 83 Linux

Command (m for help):

Tip: If you make a mistake, you can use the d command to delete the partition and start over. The t command enables you to change partition type from the default of 83 for regular Linux partitions to something else, such as 82 for swap space. In most cases, this won't be necessary, the default value is sufficient.

Note: When you created the new partition, you may have noticed that fdisk queried you as to whether it was going to be a primary or secondary partition. Linux allows only four primary partitions; if you need more, you can convert one of the primary ones into an extended one. Here is an example of a partition table that includes an extended partition followed by two regular partitions within it.

Command (m for help): p

Disk /dev/hda: 7510 MB, 7510164480 bytes

255 heads, 63 sectors/track, 913 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/hda1 * 1 13 104391 83 Linux

/dev/hda2  14 144 1052257+ 83 Linux

/dev/hda3  145 398 2040255 82 Linux swap

/dev/hda4  399 913 4136737+ 5 Extended

/dev/hda5  399 844 3582463+ 83 Linux

/dev/hda6  845 913 554211 83 Linux

Command (m for help):

Adding more partitions is just a question of repeating the previous steps the required number of times, while remembering that at some stage, you may need to add an extended partition.

5) Changes won't be made to the disk's partition table until you use the w command to write, or save, the changes. Do that now, and, when finished, exit with the q command.

Command (m for help): w

Command (m for help): q

After this is complete you'll need to verify your work and start migrating your data to the new disk. These steps will be covered next.

Verifying Your New Partition

You can take a look at the /proc/partitions file or use the fdisk -l command to see the changes to the disk partition structure of your system:

[root@bigboy tmp]# cat /proc/partitions

major minor #blocks name

22 0 78150744 hdb

22 1 78150744 hdb1

[root@bigboy tmp]#

[root@bigboy tmp]# fdisk -l

Disk /dev/hdb: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/hdb1  1 9729 76051710 83 Linux

[root@bigboy tmp]#

Putting A Directory Structure On Your New Partition

You now need to format the partition, giving it a new directory structure by using the mkfs command. The Fedora installation procedure defaults to an ext3 type, which is what you should use here.

[root@bigboy tmp]# mkfs -t ext3 /dev/hdb1

Next, you must create special mount point directory, to which the new partition will be attached. Create directory /mnt/hdb1 for this purpose.

[root@bigboy tmp]# mkdir /mnt/hdb1

When Linux boots, it searches the /etc/fstab file for a list of all partitions and their mounting characteristics, and then it mounts the partitions automatically. You'll have to add an entry for your new partition that looks like this:

# File: /etc/fstab

/dev/hdb1 /mnt/hdb1 ext3 defaults 1 2

The first entry is the name of the partition followed by the mount point directory and the filesystem type. The fourth entry defines the mounting options, which need be only default for most scenarios. The fifth entry governs whether the dump filesystem backup command can be used for the filesystem. A value of 0 means no, and 1 means yes. The final entry defines the order in which a filesystem check is done at boot time. The check is done twice. The root (or master) filesystem has a value of 1 and is checked on the first pass, all other filesystems should have a value of 2. If you are not familiar with the /etc/fstab file use the man fstab command to get a full explanation of its various options.

You don't have to wait for a reboot to mount your partition. You can use the mount command with the -a option to read the /etc/fstab file for new entries.

[root@bigboy tmp]# mount -a

You are now able to access your new partition as device /mnt/hdb1.

Migrating Data Over To your New Partition

As you remember from investigating with the df -k command, the /var partition is almost full.

[root@bigboy tmp]# df -k

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/hda3  505636 118224 361307 25% /

/dev/hda1  101089 14281 81589 15% /boot

none  63028 0 63028 0% /dev/shm

/dev/hda5  248895 6613 229432 3% /tmp

/dev/hda7  3304768 2720332 416560 87% /usr

/dev/hda2  3304768 3300536 4232 99% /var

[root@bigboy tmp]#

The du -sk * command shows the disk usage of all subdirectories in a directory. You can recursively use the command by using the cd command to step down through all the subdirectories until you discover the one with the greatest file usage. In this case, you only had to go to the /var directory to see that the /var/transactions directory was the culprit.

[root@bigboy tmp]# cd /var

[root@bigboy var]# du -sk *

2036 cache

db

empty

133784 transactions

[root@bigboy var]#

As a solution, the /var partition will be expanded to the new /dev/hdb1 partition mounted on the /mnt/hdb1 directory mount point. To migrate the data, use these steps:

1) Back up the data on the partition you are about to work on.

2) Use the who command to see who's logged in. If other users are present, send a message with the wall command informing them that the system is about to shutdown:

[root@bigboy tmp]# who

root pts/0 Nov 6 14:46 (192-168-1-242.my-site.com)

bob pts/0 Nov 6 12:01 (192-168-1-248.my-site.com)

bunny pts/0 Nov 6 16:25 (192-168-1-250.my-site.com)

[root@bigboy tmp]# wall The system is shutting down now!

Broadcast message from root (pts/0) (Sun Nov 7 15:04:27 2004):

The system is shutting down now!

[root@bigboy tmp]#

3) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

4) Rename the /var/transactions directory /var/transactions-save to make sure you have an easy to restore backup of the data, not just the tapes.

sh-2.05b# mv /var/transactions /var/transactions-save

5) Create a new, empty /var/transactions directory; this will later act as a mount point.

sh-2.05b# mkdir /var/transactions

6) Copy the contents of the /var/transactions-save directory to the root directory of /dev/hdb1, which is actually /mnt/hdb1.

sh-2.05b# cp -a /var/transactions-save/* /mnt/hdb1

7) Unmount the new /dev/hdb1 partition.

sh-2.05b# umount /mnt/hdb1

8) Edit the /etc/fstab file, removing our previous entry for /dev/hdb1 replacing it with one using the new mount point.

# File: /etc/fstab

#/dev/hdb1 /mnt/hdb1 ext3 defaults 1 2

/dev/hdb1 /var/transactions ext3 defaults 1 2

9) Remount /dev/hdb1 on the new mount point using the mount -a command, which reads /etc/fstab and automatically mounts any entries that are not mounted already.

sh-2.05b# mount -a

10) Test to make sure that the contents of the new /var/transactions directory is identical to /var/transactions-save.

11) Return to multi-user mode by typing exit. The system will return to its default runlevel.

sh-2.05b# exit

12) Make sure your applications are working correctly and delete both the /var/transactions-save directory and the /mnt/hdb1 mount point directory at some later date.

This exercise showed you how to migrate the entire contents of a subdirectory to a new disk. Linux also allows you to merge partitions together, to create a larger combined one. The reasons and steps for doing so will be explained next.

Expanding Partitions With LVM

The Logical Volume Manager (LVM) enables you to resize your partitions without having to modify the partition tables on your hard disk. This is most useful when you find yourself running out of space on a filesystem and want to expand into a new disk partition versus migrating all or a part of the filesystem to a new disk.

Physical Volume: A physical volume (PV) is another name for a regular physical disk partition that is used or will be used by LVM.

Volume Group: Any number of physical volumes (PVs) on different disk drives can be lumped together into a volume group (VG). Under LVM, volume groups are analogous to a virtual disk drive.

Logical Volumes: Volume groups must then be subdivided into logical volumes. Each logical volume can be individually formatted as if it were a regular Linux partition. A logical volume is, therefore, like a virtual partition on your virtual disk drive.

This may seem complicated, but it allows you to create new virtual partitions with sizes you can change from groups of real disk partitions whose sizes you probably cannot change. Another advantage of LVM is that this can all be done without disturbing other partitions on your hard disks.

Physical Extent: Real disk partitions are divided into chunks of data called physical extents (PEs) when you add them to a logical volume. PEs are important as you usually have to specify the size of your volume group not in gigabytes, but as a number of physical extents.

Make sure you understand these terms fully as they will be used repeatedly in many of the following sections. Lets go!

Configuring LVM Devices

It is probably best to learn about the features of LVM through a scenario. Suppose a small company needs to expand disk capacity, but there isn't the budget to purchase an adequately sized hard drive. The /home filesystem, which resides on /dev/hde5, has become too full. You just added a new hard drive /dev/hdf with 50% of the capacity of /dev/hde5 into which you want to expand /home. The device /dev/hdf has a single partition named /dev/hdf1 into which /dev/hde5 will be merged. Take a look at the required steps.

Backup Your Data

Use the tar command or some other method to backup your data in /home. The LVM process will destroy the data on all physical volumes.

Unmount your /home filesystem

As /home stores most users' data, you'll need to do some preparatory work before unmounting the filesystem.

1) Use the who command to see who's logged in. If other users are present, send a message with the wall command informing them that the system is about to shutdown.

[root@bigboy tmp]# who

root pts/0 Nov 6 14:46 (192-168-1-242.my-site.com)

bob pts/0 Nov 6 12:01 (192-168-1-248.my-site.com)

bunny pts/0 Nov 6 16:25 (192-168-1-250.my-site.com)

[root@bigboy tmp]# wall The system is shutting down now!

Broadcast message from root (pts/0) (Sun Nov 7 15:04:27 2004):

The system is shutting down now!

[root@bigboy tmp]#

2) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

3) Unmount the filesystem.

sh-2.05b# umount /home

Now we're ready to start modifying the partitions which is covered next.

Determine The Partition Types

You have to change each LVM partition used to be of type 8e (Linux LVM). You can test this with the fdisk -l command. Here is an example using /dev/hde that shows your target partitions are of the incorrect type.

sh-2.05b# fdisk -l /dev/hde

Disk /dev/hde: 4311 MB, 4311982080 bytes

16 heads, 63 sectors/track, 8355 cylinders

Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System

/dev/hde1  1 4088 2060320+ fd Linux raid autodetect

/dev/hde2  4089 5713 819000 83 Linux

/dev/hde3  5714 6607 450576 83 Linux

/dev/hde4  6608 8355 880992 5 Extended

/dev/hde5  6608 7500 450040+ 83 Linux

sh-2.05b#

Start FDISK

You can change the partition type using fdisk with the disk name as its argument. Use it to modify both partitions /dev/hde5 and /dev/hdf1. The fdisk examples that follow are for /dev/hde5; repeat them for /dev/hdf1.

sh-2.05b# fdisk /dev/hde

The number of cylinders for this disk is set to 8355.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help):

Set The ID Type To 8e

You now need to set the partition types to the LVM value of 8e. Partitions /dev/hde5 and /dev/hdf1 are the fifth and sixth partitions on disk /dev/hde. Modify their type using the t command, and then specify the partition number and type code. You can also use the L command to get a full listing of ID types in case you forget.

Command (m for help): t

Partition number (1-6): 5

Hex code (type L to list codes): 8e

Changed system type of partition 5 to 8e (Linux LVM)

Command (m for help): t

Partition number (1-6): 6

Hex code (type L to list codes): 8e

Changed system type of partition 6 to 8e (Linux LVM)

Command (m for help):

Make Sure The Change Occurred

Use the p command to get the new proposed partition table.

Command (m for help): p

Disk /dev/hde: 4311 MB, 4311982080 bytes

16 heads, 63 sectors/track, 8355 cylinders

Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System

/dev/hde1  1 4088 2060320+ fd Linux raid autodetect

/dev/hde2  4089 5713 819000 83 Linux

/dev/hde3  5714 6607 450576 83 Linux

/dev/hde4  6608 8355 880992 5 Extended

/dev/hde5  6608 7500 450040+ 8e Linux LVM

Command (m for help):

Save The Partition Changes

Use the w command to permanently save the changes to disk /dev/hde.

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.

The kernel still uses the old table.

The new table will be used at the next reboot.

Syncing disks.

sh-2.05b#

The error above will occur if any of the other partitions on the disk is mounted. This shouldn't be grave as you are already in single user mode in which most of the system's processes that would be accessing the partition have been shutdown.

Define Each Physical Volume

After modifying the partition tables of /dev/hde and /dev/hdf, initialize the target partitions with the pvcreate command. This wipes out all the data on them in preparation for the next step. If you haven't backed up your data yet, do it now!

sh-2.05b# pvcreate /dev/hde5

pvcreate -- physical volume '/dev/hde5' successfully created

sh-2.05b# pvcreate /dev/hdf1

pvcreate -- physical volume '/dev/hdf1' successfully created

sh-2.05b#

Run VGscan

The next step is to make Linux scan for any new LVM disk partitions and automatically create the LVM configuration files in the /etc directory. To do this, use the vgscan command.

sh-2.05b# vgscan

vgscan -- reading all physical volumes (this may take a while)

sh-2.05b#

Create A Volume Group For the PVs

Use the vgcreate command to combine the two physical volumes into a single unit called a volume group. The LVM software effectively tricks the operating system into thinking the volume group is a new hard disk. In the example, the volume group is called lvm-hde.

sh-2.05b# vgcreate lvm-hde /dev/hdf1 /dev/hde5

Volume group 'lvm-hde' successfully created

sh-2.05b#

Therefore, the vgcreate syntax uses the name of the volume group as the first argument followed by the partitions that it will be comprised of as all subsequent arguments.

Create A Logical Volume From The Volume Group

Now you're ready to partition the volume group into logical volumes with the lvcreate command. Like hard disks, which are divided into blocks of data, logical volumes are divided into units called physical extents (PEs).

You'll have to know the number of available PEs before creating the logical volume. This is done with the vgdisplay command using the new lvm-hde volume group as the argument.

sh-2.05b# vgdisplay lvm-hde

--- Volume group ---

VG Name  lvm-hde

VG Access read/write

VG Status available/resizable

VG #  0

MAX LV  256

Cur LV  0

Open LV  0

MAX LV Size  255.99 GB

Max PV  256

Cur PV  2

Act PV  2

VG Size  848 MB

PE Size  4 MB

Total PE 212

Alloc PE / Size 0 / 0

Free PE / Size 212 / 848 MB

VG UUID  W7bgLB-lAFW-wtKi-wZET-jDJF-8VYD-snUaSZ

sh-2.05b#

As you can see, 212 PEs are available as free. You can now use all 212 of them to create a logical volume named lvm0 from volume group lvm-hde.

sh-2.05b# lvcreate -l 212 lvm-hde -n lvm0

Logical volume 'lvm0' created

sh-2.05b#

Note: You can also define percentages of the volume group to be used. The first example defines the use of 100% of the volume group's free space and the second example specifies using 50% of the total volume group.

sh-2.05b# lvcreate -l 100%FREE -n lvm0 lvm-hde

sh-2.05b# lvcreate -l 50%VG -n lvm0 lvm-hde

Format The Logical Volume

After the logical volume is created, you can format it as if it were a regular partition. In this case, use the -t switch to specify to the mkfs formatting program that you want a type ext3 partition.

sh-2.05b# mkfs -t ext3 /dev/lvm-hde/lvm0

mke2fs 1.32 (09-Nov-2002)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

108640 inodes, 217088 blocks

10854 blocks (5.00%) reserved for the super user

First data block=0

7 block groups

32768 blocks per group, 32768 fragments per group

15520 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840

Writing inode tables: done

Creating journal (4096 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

sh-2.05b#

Create A Mount Point

When you formatted the /dev/hde5 partition, you lost the /home directory. Now you have to recreate /home on which you'll later mount your new logical volume.

sh-2.05b# mkdir /home

Update The /etc/fstab File

The /etc/fstab file lists all the partitions that need to be automatically mounted when the system boots. This snippet configures the newly labeled partition to be mounted on the /home mount point.

/dev/lvm-hde/lvm0 /home ext3 defaults 1 2

The /dev/hde5 and /dev/hdf1 partitions are replaced by the combined /lvm0 logical volume. You, therefore, don't want the old partitions to be mounted again. Make sure that any reference to them in this file has either been commented a # character at the beginning of each line or deleted entirely.

#/dev/hde5 /data1 ext3 defaults 1 2

#/dev/hdf1 /data2 ext3 defaults 1 2

Mount The Volume

The mount -a command reads the /etc/fstab file and mounts all the devices that haven't been mounted already. After mounting, test the volume by listing its directory contents. It should just contain the lost+found directory

sh-2.05b# mount -a

sh-2.05b# ls /home

lost+found

sh-2.05b#

Restore Your Data

You can now restore your backed up data to /home.

Get Out Of Single User Mode

Return to your original run state by using either the init 3 or init 5 commands. The exit command will make you return to your default runlevel.

Conclusion

The demise of the hard disk has been predicted for many years. Faster, denser memory chips were supposed to eliminate their need, but hard disk technology has evolved, dramatically increasing their speed and capacity too. They will be around for a long time to come.

It seems as if when drives get bigger, so does the data they are intended to store. Expanding the existing disk capacity of your server may become an everyday occurrence and the tools described in this chapter should make the task easier.