LVM: Logical Volume Management
tldp.org/HOWTO/LVM-HOWTO/index.html
What is it?Normally /dev/hda is a drive, and /dev/hda1 is a partition of that drive. In LVM, those are called Physical Volumes (PV). You put this PVs into a Volume Group (VG). From that Volume Group, you carve out Logical Volumes (LV). LVs act like partitions. so it looks like this: hda1 hdc1 (PV:s on partitions or whole disks)
Requirementsthe package lvm10 and support built into the kernel (under 'Multi-device support (RAID and LVM)'). Check to see if /proc/lvm exists. If not, load the module:root> modprobe lvm-mod
root> vgscan
Setting it upNOTE: if you are using devfs, then you can't use the aliased device paths, you must use the real thing: eg /dev/ide/host0/bus0/target0/lun0/part1 instead of /dev/hda1. If unsure, do an "ls -l" on the device, it will show where it is linked to. Create Physical VolumeFor a partition:The partition must have type "Linux LVM" (8E), set with cfdisk root> cfdisk /dev/hdb
root> pvcreate /dev/hdb
Create Volume Grouproot> vgcreate testvg /dev/hda1 /dev/hdb1
root> vgdisplay testvg
Create Logical VolumesTo create a 1500MB linear LV named 'testlv':root> lvcreate -L1500 -ntestlv testvg
root> lvscan -v
root> mke2fs /dev/testvg/testlv
root> mount /dev/testvg/testlv /mnt
Using itReducing sizeto reduce spool by 1 gig:umount /dev/sympavg01/spool
Increasing sizelvextend -L +1G /dev/sympavg01/archives
removing a volumelets say you have /dev/hda and /dev/hdb that are a part of a volume and you want to take out /dev/hda if you have enough space on /dev/hdb you can issue a command which will move all data off of /dev/hda (pvmove /dev/hda) that will move everything from /dev/hda onto the rest of the disks in the volumegroup (in this example /dev/hdb)... it isn't a fast thing as it is a block copy, but you can see what is going on with a -v... then once it is removed you can remove the disk from the volume group with a vgreduce then you can put a new disk in and redistribute the moved data to the new diskcopying over an existing partitionok, this is not exactly an lvm issue, but it comes up a lot: for example, cp /usr to new lvm mounted as /mnt(which will become /usr later): root> cd /usr
FAQThis is a list of questions that I've seen come across the LVM mailing list that deal with everything from how do I restore this LVM partition that failed because one of the underlying disks died, to how to recover from failures etc. It is mostly a random assortment of things that I've pulled together, and is still a work in progress.How to use a LVM devices on a different machine: If you have LVM setup on disks on machine A, and it goes down and you
want to take the drives out and put them in machine B, what do you
need to do to prep machine B beforehand to deal with this?
You don't need to do anything apart from vgscan/vgchange -ay when you need to use the volumes, then mount them. vgscan/vgchange reads the metadata off the disk and creates the necessary volume groups and device files on the system. In otherwords, /dev/lvg1/lvol1 will be created on machine B based on the metadata. Do not re-do vgcreate or lvcreate because this will mangle the meta data for the other node. A disk has died, how do I get the data off? Try dd_rescue www.garloff.de/kurt/linux/ddrescue/ (either to
another disk of the same size, or to a file which can be burned to a
CD or mounted in a loopback)
How do I build a mirrored LVM system with Debian? 1. Start Debian install
2. Create 20MB /boot filesystem/partition at the start of the disk
3. Then create a "temporary" root filesystem/partition the size
that you want swap to eventually be (minimum 200MB or so to get
minimal debian install onto it)
4. Then create one partition for all remaining space
5. Do not specify to mount /boot partition during installation so
that /boot files are put on "temporary" root filesystem
6. Do minimal Debian install (no to tasksel and dselect) into 250MB
partition, specify no swap partition
7. When all done installing truly minimal install, put second disk
8. Use sfdisk to duplicate partition table to second disk
9. Create raidtab mapping partition(s) 1, 2 and 3 to md0, md1 and
md2 respectively
10. mkraid /dev/md0
11. Mount /dev/md0 on /mnt and copy /boot to it, remove /boot/* and
mount /dev/md0 on /boot
12. mkraid /dev/md2
13. pvcreate /dev/md2
14. vgcreate /dev/md2
15. lvcreate root, mkfs ... /dev/.../root
16. lvcreate usr, mkfs ... /dev/.../root
17. lvcreate ...
18. mount /dev/.../root /mnt, /dev/.../usr /usr, ...
19. copy -rax / /mnt, cp -rax /usr /mnt, ...
20. Create initrd to reflect above -- I use Adrian Bunk's
initrd-tools
21. Edit /etc/lilo.conf to reflect /dev/md0 is boot partition and
lvm lv as root partition
22. edit fstab to reflect above changes
23. reboot
24. mkraid /dev/md1
25. add /dev/md1 to fstab as swap partition and swapon -a
My system can't find the volume groups!vgscan tells me no volume groups are foundtry lvscan lvscan -- ERROR: "/etc/lvmtab" doesn't exist; please run vgscan vgscan -- ERROR "creating lvmtab directory" creating lvmtab vgscan -- no volume groups found mount -o remount,rw / then check for /etc/lvmtab and /etc/lvmtab.d ls: lvmtab.d: Input/output error boot the machine with a boot disk and fsck root! Or do touch /forcefsk and reboot single-user. Fsck says its clean? Try fsck -f do pvscan inactive pv hda10 is associated to unknown volumegroup sympavg01 (run vgscan)
run vgscan, this will create lvmtab.d then run lvscan volumegroup is not active try lvscan -D, do a vgchange -a y (instead of lvscan -D because lvscan -D will only scan, not activate) When setting up a volume group, half of the space is gone, whats upwith that? * When you do the vgcreate command it says how large the physical
extent sizes are, and there is always one extent that is reserved at
the beginning of the partition for the LVM metadata. So if your
physical extent size is 32MB, then you will lose 32MB for the
metadata. Also, LVM volumes have to be multiples of the extent size
(32MB, 64MB, 96MB, etc.), so if you make a volume that is 93megabytes,
that is 3MB too little for a 3 extent volume, which leaves you with a
2 extent volume totalling 2 * 32 MB = 64MB.
What is the maximum size of logical volumes? It depends on which version of LVM you are using. If you are
using LVM1, then there is a 255G limit on LV sizes (this is based on
PE size). If you use the
LVM2 metadata format, then you are limited only by disk space and the maximum size that your kernel supports. If you create PVs in LVM1 format and then migrate to LVM2, then you will still be limited to 255G, unless you remake your VGs and LVs with the LVM2 tools. If you made a VG with LVM1, or if you created it with the -M1 flag to vgcreate in LVM2, then you have an upper limit which is imposed by the format1 metadata. What filesystem is best with LVM? It all depends on what you need. All the different filesystems
work under LVM, it would be best to do some research into the current
technologies that each offer, and make an assessment based on what
your needs are. There are a number of benchmarks out there, and the
filesystems are changing rapidly, so these benchmarks will get out of
date and need to be updated.
Here are some benchmarks: otn.oracle.com/oramag/webcolumns/2002/techarticles/scalzo_linux02.html oregonstate.edu/~kveton/fs/ fsbench.netnation.com/ Ext2: The most stable filesystem out there for Linux is ext2, think twice about not using it, it actually is fast and has what most people need. Seriously consider the reasons why you want to use something else, because most journaling filesystems are still very much in a development stage and are risky. Stick with ext2 if all you are wanting is the ability to never fsck again, because this is not something you will gain with a journaling filesystem. Disorderly shutdowns, even with full journaling on your filesystems, can lead to dataloss of all data that is stored in the buffercache, however filesystem consistancy will probably be maintained. Ext3: This is one of the most stable journaling filesystems out as of this writing. It has been in the kernel for some time, and is mostly just a hack ontop of ext2. It is simple for converting your ext2 systems to ext3 and back, but it requires some kernel patches if you want to do online resizing. If you are using Redhat, probably should stick to ext2/3, Redhat does that well and doesn't do others so well, so you are fighting if you do anything else (in fact the ext3 online extend code is included in the latest RedHat Rawhide kernel). Reiserfs: They say reiserfs has much better performance in dealing with lots of small files, but it doesn't have any bad block management capabilities. If your drive gets bad blocks, you can very quickly be rebuiling your entire filesystem from backup. Reiser currently performs rather poorly on really large files (ie. databases). Oracle doesn't support the tuning of a system if it is running reiserfs, and ranks the different filesystems as follows: ext2, ibm-jfs, ext3, reiser. Reiser works great for maildir spools and usenet feeds. Because resier3 does not do true data journaling (only metadata), it is able to bring filesystem integrity back to normal after a bad shutdown, but some data loss is possible. Reiser4 supposedly will fix this (as well as add plug-ins). It seems a lot of people avoid Reiser on Redhat (something having to do with redhat's kernel interoperability causing problems, they support ext3 well, so I would recommend sticking with ext3 if you are using Redhat), but a lot of SuSE people use it, as it is well integrated. I personally am a fan of Reiser for maildir spools and list archives that are parceled out in very large individual files, it has been pretty stable for us so far, although the website graphics freak me out and the advertising you get when you create a filesystem is always a bit of a shock (and "secret" sources make me nervous too). On that note, if you are using reiserfs, it balks badly at being a readonly filesystem. Even if you mount it readonly, it still wants write access to the journal. We have a situation in our shop where linux can not access the disks in read/write mode. For those filesystems that were bootable, we cannot use reiserfs. JFS: You can dynamically grow this filesystem without intervention, which is kinda neat, but scary. I have yet to see people consider JFS stable in linux XFS: ? In terms of stability, I've had significant problems with it in the past, but it could've been a particular kernel patch/version. XFS only made it into the official kernel in 2.4.23, so be aware... pvmove gives "device-mapper: error adding target to table" errors Do you have the device-mapper mirror target installed? Not
having it available is the most common cause of this problem. Some 2.6
kernels don't include this.
I cannot mount my snapshot volume, gives "wrong fs type, bad option,bad superblock on /dev/etc or too many mounted filesystems" * Try mounting with the parameters "-onouuid,ro"
My logical volumes are inactive and unusable! Run "vgscan" and then "vgchange -ay" to activate all the VGs and the LVs.
My logical volumes are not active or usable on restart! Check that you have your startup scripts available that start your
logical volumes. You should have one that does a "vgscan" and then a
"vgchange -ay" somewhere at bootup, without this your VGs and LVs wont
be available on bootup.
The system is not recognizing my boot LVM partitions and kernel panics. You need to run "lvmcreate_initrd", which will create an
lvm-initrd file in /boot. Then edit grub.conf to use the new initrd.
This will make your lvm tools available at boot and your VGs will come
online. To do this with a rescue disk do the following (change
/mnt/sysimage to whatever your actual root is mounted as):
mount -t proc proc /mnt/sysimage/proc
chroot /mnt/sysimage /bin/bash
source /etc/profile
lvmcreate_initrd
exit
umount /mnt/sysimage/proc
sync
shutdown -r now
How do I see if I have LVM support available? LVM1: cat /proc/devices and see if you have "lvm" listed, see if
you have the vgcreate command.
LVM2: check to see if you have the "lvm" base command by doing "lvm version" - this should give you complete output and confirm that device-mapper is available in the live kernel and that /dev/mapper/control is setup correctly (see How do I go from LVM1 to LVM2? If you have a LVM1 setup on your Debian machine you just need to
switch to LVM2 by installing the lvm2 packages. lvm-common is the
Debian package that switches between LVM versions on the same system
You will get this package as a dependency when you install lvm10 or
lvm2. The Debian LVM binaries get installed libraries in
/lib/lvm-How do I go from a 2.4 kernel to a 2.6 kernel? You need to make sure you have CONFIG_DM_IOCTL_V4 support in your
kernel for device-mapper. You need to select CONFIG_BLK_DEV_DM so that
you can see the option for IOCTL_V4
How do you get device mapper in the 2.4 kernels?Follow the directections for device mapper at evms.sourceforge.net/install/kernel.htmlGlossaryLVM1 = Userspace tools + kernel ioctls included in marcelo's 2.4 tree
device-mapper kernel patches - needed only for 2.4;
userspace libdevmapper and dmsetup - needed with all dm kernels.
The LVM2 tarball contains the LVM2 command line tools.
Development code can be found via:
people.sistina.com/~thornber/ (for kernel patches)
www.sistina.com/products_CVS.htm (for userspace code)
Device-mapper mailing list:
|
