[CENTER]
…
If you wish to link to this tutorial from another forum,
blog or website please use this URL:
note to self: fill in LVMOnRaid - Ubuntu Wiki when i have finished this tutorial
…
[/CENTER]
In this tutorial i will walk you through creating a Level 5 RAID with LVM, which will allow you to add more disks to increase the size of your RAID at a later date, and then how to mount it as your home directory (and copy all your documents from your old home directory onto it). Anyone should be able to follow this tutorial assuming that they are able to read it, the only thing i won’t cover is how to open a console! [INDENT][B]Disclaimer: RAID is NOT a backup solution, it will help protect your data from hardware failure but does nothing towards preventing file system corruption. Always backup non-replaceable documents!
This tutorial was compiled in good faith, I cannot and will not be held responsible for anything that goes wrong (including but not limited to data loss) even if you follow this tutorial to the letter, i am not an expert on this subject. You should backup everything before you start and double check every command before you execute it (read the manual for each command). This is how i created my RAID, not necessarily how you should create a RAID.
[/B][/INDENT]To make things simple this guide has been split up using ‘spoiler’ tag (click show to show that part of the tutorial, click hide to hide it!), commands to be executed have been put in ‘format’ boxes and the output from commands has been put into indented ‘code’ boxes, example below (also contains more information on the style / formatting of this tutorial)
[spoiler=This is a example spoiler box for tutorial sections.]
[formatbox=Commands to execute]This is format box for commands to execute.[/formatbox][INDENT]
This is an indented code box for example outputs of executed commands.[/INDENT]More information:
Underlined text is a hyperlink pointing to more information (example, the words RAID, “Level 5” and LVM at the start of this post all point to their wikipedia pages).
Red text means its very important you read it properly (example, the disclaimer above… you did read it, didn’t you? :rolleyes:).
Bold text are points which i believe are important or that i wish to draw attention too.
[/spoiler]
If you have any problems following this tutorial or have any improvements, spelling & grammar corrections, suggestions on how to make it easier to follow or read if English is not your first language please don’t hesitate to post them here, if you do not have an account on this forum click here to create one.
[SIZE=4]Lets start![/SIZE]
Why would i want to do this? Storing data on level 5 raid helps prevent data loss, if one of your hard drives dies then you will not lose anything as long as you replace the drive before a second drive dies (if this happens, you will lose everything on the RAID)
To create a Level 5 RAID you will need at least 3 blank disks that are all the same size*. If you are buying new hard disks to do this you will probably want to buy 750GB hard disks as they seem the best value at the moment (28th July 2008) depending on where you get them! For a list of reputable online stores, click here.
*technically they can be different sizes, but they will all be recognized as the size of the smallest disk.
I wrote this tutorial using 3 x 500GB SATA hard disks, to make things simple i will refer to them as /dev/sdx , /dev/sdy and /dev/sdz. you will need to change the commands so point to your hard disks
REMEMBER: Half of the commands in this tutorial will cause data loss if executed on the wrong hard drives, MAKE SURE YOU SPECIFY THE CORRECT HARD DRIVES!
this section is incomplete, it needs to be expanded & images need to be added to better explain it.
use hot glue to hold sata power & data cables to hard drive, use cable ties to hold cables to prevent movement.
This section covers partitioning your disks, creating one primary partition on each disk using ‘fdisk’, to start run the command in the code box below (change sdx to one of the hard drives you wish you use in your raid)
[formatbox=Command to execute]
sudo fdisk /dev/sdx
[/formatbox][INDENT][code]The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
- software that runs at boot time (e.g., old versions of LILO)
- booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help):[/code]press p and hit enter to list the partitions on the hard drive, make sure you have the right drive!
[code]
Command (m for help): p
Disk /dev/sde: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x3142aa5b
Device Boot Start End Blocks Id System
Command (m for help):[/code][spoiler=read this if the above was not a blank line]
if the output from fdisk listed ANY partitions then the first thing you should do is check that you entered the correct hard drive.
If you are 100% certain that you entered the correct hard drive, and you know why the drive is still partitioned (ie it has been used before) then do the following to delete the partition(s). After you have done this, you can return to creating the new partitions .
you will lose all data stored on the hard disk you do this too, again, make sure this is the right hard disk and that there is nothing stored on it that you want to keep.
the output from fdisk will look something like this, with one or more entries (i have 2 partitions in this example)
[code]
Command (m for help): p
Disk /dev/sde: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x3142aa5b
Device Boot Start End Blocks Id System
/dev/sde1 1 30400 244187968+ 83 Linux
/dev/sde2 30401 60801 244196032+ 83 Linux
Command (m for help):
[/code]press d and hit enter to delete a partition
Command (m for help): d
Partition number (1-4):
enter the number of the last partition on the disk listed by fdisk, in this example 2, and press enter.
[code]
Partition number (1-4): 2
Command (m for help):
[/code]Repeat the above for each partition, unfortunately fdisk does not tell you it has deleted the partition but you can check the remaining partitions by pushing p and then enter.
When you push d and enter to delete the last partition on the hard disk fdisk will automatically select and delete the last partition
[code]
Command (m for help): d
Selected partition 1
Command (m for help):
[/code]you can now continue with creating the new partition.[/spoiler]
press n and hit enter to create a new primary partition
Command (m for help): n
Command action
e extended
p primary partition (1-4)
press p and hit enter to select primary partition
p
Partition number (1-4):press 1 and hit enter to make it the first partition on the disk
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Accept the default starting cylinder by pressing enter
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-60801, default 60801):
Accept the default last cylinder by pressing enter
[code]
Using default value 60801
Command (m for help):
[/code]press w and hit enter to write the new partition table to the disk
[code]
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[/code][/INDENT]You will now have been dropped back down to a command line, repeat the above for the other hard disks disks you are going to use in your RAID.
This section covers creating the level 5 RAID, but before we do that we are going to double check that you have partitioned your disks correctly! Remember to change the drive letters to your own hard disks! you must list all the hard drives you plan to use in your raid (i am only using 3 but if you are using 4 or more just add them in with the rest of the drives in the same fashion)
[formatbox=Commands to execute]
sudo fdisk -l /dev/sdx /dev/sdy /dev/sdz | grep Linux
[/formatbox][indent]
/dev/sdx1 1 60801 488384001 83 Linux
/dev/sdy1 1 60801 488384001 83 Linux
/dev/sdz1 1 60801 488384001 83 Linux
[/indent]
If your partitions are setup correctly then all the lines should all say exactly the same, apart from the drive letters and there should only be one line per disk (if you specified 3 hard drives, then there should only be 3 lines!). If for some reason this is not true then go back to partitioning and try again, if your still having trouble after that make a post and ask for help!
To do the following you must have mdadm installed
To install mdadm run the command for your distribution
[formatbox=Debain / Ubuntu]
sudo apt-get install mdadm
[/formatbox]
[formatbox=Arch Linux]
pacman -S mdadm
[/formatbox]
If instructions for your distribution are not listed here, please post them when you find out and i will add them
If your partitions are setup correctly, then it is time to create the RAID!
remember to change the drive letters to point to the correct hard drives and partitions (ie /dev/sdx1 not /dev/sdx), if you are using more than 3 hard drives then you will also need to change --raid-devices=3 to however many hard disks you are using.
[formatbox=Command to execute]
mdadm --create /dev/md0 --level=5 --raid-devices=3 --run /dev/sdx1 /dev/sdy1 /dev/sdz1
[/formatbox]
mdadm: array /dev/md0 started.
You can monitor the progress of mdadm with the following command
[formatbox=Command to execute]
cat /proc/mdstat
[/formatbox]
[indent]mdstat before mdadm has finished building the raid[code]Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sde1[3] sdd1[1] sdc1[0]
976767872 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
[>…] recovery = 0.1% (507020/488383936) finish=96.2min speed=84503K/sec
unused devices: [/code]
mdstat after mdadm has finished building the raid[code]Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdc1[0] sde1[2] sdd1[1]
976767872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
unused devices: [/code][/indent][/spoiler]
[spoiler=Creating & formatting the LVM]
[formatbox=Command to execute]pvcreate /dev/md0[/formatbox]
[indent] Physical volume "/dev/md0" successfully created[/indent]
[formatbox=Command to execute]vgcreate -s 32M raid-lvm-home /dev/md0[/formatbox]
[indent] Volume group "raid-lvm-home" successfully created[/indent]
[formatbox=Command to execute]vgdisplay raid-lvm-home[/formatbox]
[indent] --- Volume group ---
VG Name raid-lvm-home
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 931.50 GB
PE Size 32.00 MB
Total PE 29808
Alloc PE / Size 0 / 0
Free PE / Size 29808 / 931.50 GB
VG UUID X1T3tt-k6LJ-c5p9-7C3P-BIll-Owyw-y1zyf2[/indent]
[formatbox=Command to execute]lvcreate -l 29808 raid-lvm-home -n lvm0[/formatbox]
[indent] Logical volume "lvm0" created[/indent]
[formatbox=Command to execute]
mkfs.ext3 /dev/raid-lvm-home/lvm0
[/formatbox]
[indent][CODE]mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61046784 inodes, 244187136 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
7452 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: 304/7452[/CODE][/indent]
[indent][code]mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61046784 inodes, 244187136 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
7452 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[/code][/indent]
In this section i will cover how to move all your documents and settings stored in your home directory onto the raid and then mounting it as your home directory.
[formatbox=Command to execute]
mkdir /mnt/lvm0
ls -a /mnt/lvm0/
[/formatbox]
[indent]. ..[/indent]
[formatbox=Command to execute]
mount /dev/raid-lvm-home/lvm0 /mnt/lvm0
ls -a /mnt/lvm0
[/formatbox]
[indent]. .. lost+found[/indent]
[formatbox=Command to execute]
rsync -avS --progress /home/* /mnt/lvm0/
[/formatbox]
[indent]building file list ...
XYZ files to consider
sent XYZ bytes received XYZ bytes XYZ bytes/sec
total size is XYZ speedup is XYZ[/indent]