# Agenda: ex-lvm-scripted-install.htm

File ex-lvm-scripted-install.htm, 68.1 KB (added by admin, 4 years ago)
Line
1<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2<html xmlns="http://www.w3.org/1999/xhtml">
4  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
5  <meta http-equiv="Content-Style-Type" content="text/css" />
6  <meta name="generator" content="pandoc" />
7  <title>VM image management: Scripted installs and LVM</title>
8  <style type="text/css">code{white-space: pre;}</style>
11<body>
13<h1 class="title">VM image management: Scripted installs and LVM</h1>
14</div>
15<div id="TOC">
16<ul>
17<li><a href="#scripted-installs"><span class="toc-section-number">1</span> Scripted installs</a><ul>
18<li><a href="#install-grml-debootstrap"><span class="toc-section-number">1.1</span> Install grml-debootstrap</a></li>
19</ul></li>
20<li><a href="#build-a-vm-image-file"><span class="toc-section-number">2</span> Build a VM image file</a><ul>
21<li><a href="#check-the-image"><span class="toc-section-number">2.1</span> Check the image</a></li>
22<li><a href="#convert-the-image"><span class="toc-section-number">2.2</span> Convert the image</a></li>
23<li><a href="#start-a-vm"><span class="toc-section-number">2.3</span> Start a VM</a></li>
24</ul></li>
25<li><a href="#lvm"><span class="toc-section-number">3</span> LVM</a></li>
26<li><a href="#build-a-vm-filesystem-in-a-logical-volume"><span class="toc-section-number">4</span> Build a VM filesystem in a logical volume</a><ul>
27<li><a href="#examine-the-image"><span class="toc-section-number">4.1</span> Examine the image</a></li>
28<li><a href="#booting-the-image"><span class="toc-section-number">4.2</span> Booting the image</a></li>
29</ul></li>
30<li><a href="#optional-exercises"><span class="toc-section-number">5</span> Optional exercises</a><ul>
31<li><a href="#cloning-virtual-machines"><span class="toc-section-number">5.1</span> Cloning virtual machines</a><ul>
32<li><a href="#duplicate-the-image-file"><span class="toc-section-number">5.1.1</span> Duplicate the image file</a></li>
33<li><a href="#create-a-new-vm-definition"><span class="toc-section-number">5.1.2</span> Create a new VM definition</a></li>
34<li><a href="#patch-up-the-vm"><span class="toc-section-number">5.1.3</span> Patch up the VM</a></li>
35</ul></li>
36<li><a href="#examining-and-modifying-image-files"><span class="toc-section-number">5.2</span> Examining and modifying image files</a><ul>
37<li><a href="#raw-images"><span class="toc-section-number">5.2.1</span> Raw images</a></li>
38<li><a href="#qcow2-images"><span class="toc-section-number">5.2.2</span> qcow2 images</a></li>
39<li><a href="#guestfish-libguestfs"><span class="toc-section-number">5.2.3</span> guestfish / libguestfs</a></li>
40</ul></li>
41</ul></li>
42</ul>
43</div>
44<p>Objectives:</p>
45<ul>
46<li>perform a scripted install for a fully-automatic OS installation</li>
47<li>use LVM commands to create and manage logical volumes</li>
48</ul>
49<p>Most of the actions in this exercise are done as &quot;root&quot;, so if you are not root already type:</p>
50<pre><code>$sudo -s 51#</code></pre> 52<p>Any set of actions/commands that starts with <code>#</code> is to be run as root and any that starts with a <code>$</code> should be run as the normal class user.</p>
53<h1 id="scripted-installs"><a href="#scripted-installs"><span class="header-section-number">1</span> Scripted installs</a></h1>
54<p>Scripted installs can be used to automate the creation of new VM images, and avoid most of the interactive questions asked during a CD install.</p>
55<p>Under Ubuntu, the tool &quot;python-vm-builder&quot; can be used for this; under Debian the equivalent tool is &quot;grml-debootstrap&quot;, and this is what you will use in this exercise.</p>
56<h2 id="install-grml-debootstrap"><a href="#install-grml-debootstrap"><span class="header-section-number">1.1</span> Install grml-debootstrap</a></h2>
57<pre><code># apt-get install grml-debootstrap</code></pre>
58<p>The configuration settings are available by reading <code>man grml-debootstrap</code> and by looking in the file <code>/etc/debootstrap/config</code></p>
59<h1 id="build-a-vm-image-file"><a href="#build-a-vm-image-file"><span class="header-section-number">2</span> Build a VM image file</a></h1>
60<p><code>grml-debootstrap</code> creates a familiar disk image file, containing an MBR and a partition table, so it can be booted directly. This is the direct equivalent of what we did earlier using virt-manager to install Ubuntu inside our debian hosts.</p>
61<pre><code># grml-debootstrap --vmfile --target /var/lib/libvirt/images/debian1.img \
62    --hostname debian1 -v \
63    --mirror http://apt.ws.nsrc.org:3142/cdn.debian.net/debian</code></pre>
64<p>This will take a few minutes. It builds a raw image, default size 2GB. If you watch carefully, you will see messages about loopback devices like <code>/dev/mapper/loop0p1</code>. This is explained in the optional exercises at the end.</p>
65<p>Near the end of the process you will be prompted for a root password for your new VM.</p>
66<h2 id="check-the-image"><a href="#check-the-image"><span class="header-section-number">2.1</span> Check the image</a></h2>
67<p>This file is &quot;sparse&quot; - the host operating system has only allocated space for blocks which have been written to. You can see this by comparing the size of the file with the disk space allocated to the file.</p>
68<pre><code># ls -lh /var/lib/libvirt/images/debian1.img      # size of file = 2GB
69# ls -sh /var/lib/libvirt/images/debian1.img      # space used &lt; 2GB
70# du -h /var/lib/libvirt/images/debian1.img       # another way to get the same info</code></pre>
71<p>From our terminology presentation this is the sparse disk image that is supported by some filesystems. We had it illustrated thusly:</p>
72<div class="figure">
74</div>
75<p>However, such a file may lose its &quot;sparseness&quot; when copied: that is, it could grow to take 2GB of disk space, with all the unused blocks being filled with zeros.</p>
76<h2 id="convert-the-image"><a href="#convert-the-image"><span class="header-section-number">2.2</span> Convert the image</a></h2>
77<p>If you want the image in a different format, e.g. qcow2, then use <code>qemu-img convert</code> to convert it. This will take a little while as it has to read and rewrite the entire image.</p>
78<pre><code># qemu-img convert -f raw -O qcow2 /var/lib/libvirt/images/debian1.img /var/lib/libvirt/images/debian1.qcow2</code></pre>
79<p>qemu-img is part of the <code>qemu-utils</code> package. <code>-f raw</code> selects the input format and <code>-O qcow2</code> selects the desired output format.</p>
80<p>Once it is finished, compare the file sizes:</p>
81<pre><code># du -sh /var/lib/libvirt/images/debian1.*</code></pre>
82<p>The two image file sizes may be slightly different.</p>
83<p>This qcow2 image is what we illustrated as a growable image file in our terminology presentation:</p>
84<div class="figure">
86</div>
87<h2 id="start-a-vm"><a href="#start-a-vm"><span class="header-section-number">2.3</span> Start a VM</a></h2>
88<p>Finally, to boot this as a VM using libvirt, you need to create a VM inside libvirt and attach the image.</p>
89<p>Rather than write the libvirt XML for this all by hand, you could use the virt-manager GUI, but this time you will use the command line tool &quot;virt-install&quot; to create the machine. You pass a number of parameters telling it the parameters of the machine to build.</p>
90<blockquote>
91<p>Hint: if you are inside the X11 graphical environment then it's better to run this as a regular user, not root, so that it can open the graphical console window for you.</p>
92</blockquote>
93<p>Note: the password below is &quot;xyzzy&quot; when prompted for this to connect to the VM once it is started.</p>
94<pre><code>$virt-install --name debian1 --ram 512 \ 95 --graphics vnc,listen=0.0.0.0,password=xyzzy \ 96 --import --disk /var/lib/libvirt/images/debian1.qcow2,format=qcow2 \ 97 --network=bridge:br-lan</code></pre> 98<p>This image has a partition table and Grub so it's very easy to boot. However, since it has a partition table inside the image growing it beyond the initial 2GB specified at creation time is not that easy to do.</p> 99<p>When your VM is running, move onto the next section about LVM.</p> 100<h1 id="lvm"><a href="#lvm"><span class="header-section-number">3</span> LVM</a></h1> 101<p>The Logical Volume Manager gives an approach to disk space management which is much more flexible than partitioning.</p> 102<p>First, try each of the following commands to list the logical volumes you have:</p> 103<pre><code># lvscan 104# lvs 105# lvdisplay</code></pre> 106<p>Compare the output - it has differing levels of detail but should show the same logical volumes.</p> 107<p>Have a look at the device nodes:</p> 108<pre><code># ls -l /dev/mapper 109# ls -l /dev/ganeti</code></pre> 110<p>Now create a new logical volume called &quot;foo&quot; (or some other name you choose) inside the &quot;ganeti&quot; volume group:</p> 111<pre><code># lvcreate --size 1G --name foo ganeti</code></pre> 112<p>Use these commands to check that the device node exists, and to see the size of the block device you have created:</p> 113<pre><code># ls -l /dev/ganeti 114# blockdev --getsize64 /dev/ganeti/foo</code></pre> 115<p>Now we can create a filesystem within it, and mount it. MAKE SURE YOU DO THESE OPERATIONS ON THE CORRECT DEVICE NODE - if you write to your root, var or swap volumes you will break your system!</p> 116<pre><code># mkfs.ext4 /dev/ganeti/foo 117# mkdir /opt # if /opt already exists, ignore the error 118# mount /dev/ganeti/foo /opt 119# df -h /opt</code></pre> 120<p>You should now be able to create files in /opt if you wish, and they will be stored in this logical volume.</p> 121<pre><code># echo &quot;testing this&quot; &gt; /opt/file1.txt 122# cp /etc/motd /opt 123# mkdir /opt/bin 124# cp -p /bin/ls /opt/bin/. 125# ls -l /opt/*</code></pre> 126<p>Now let's grow the filesystem. This requires two steps: firstly add some extents to the logical volume.</p> 127<pre><code># lvextend --size +1G /dev/ganeti/foo 128# blockdev --getsize64 /dev/ganeti/foo 129# df -h /opt</code></pre> 130<p>You'll see that although the logical volume is bigger, the filesystem is still the same size. You need to resize the filesystem to fill the larger space available. Fortunately, with ext4 you can do this even while the filesystem is mounted:</p> 131<pre><code># resize2fs /dev/ganeti/foo 132# df -h /opt</code></pre> 133<p>Now let's undo everything. Unmount the filesystem and delete the logical volume.</p> 134<pre><code># umount /opt 135# lvremove /dev/ganeti/foo 136# lvscan</code></pre> 137<p>As you can see, creating and destroying logical volumes is a straightforward process. There is plenty of additional LVM documentation online that you can read.</p> 138<p>From the point of view of a virtual machine, a logical volume behaves much like a raw disk image file, but avoids the overhead of going through the host filesystem.</p> 139<h1 id="build-a-vm-filesystem-in-a-logical-volume"><a href="#build-a-vm-filesystem-in-a-logical-volume"><span class="header-section-number">4</span> Build a VM filesystem in a logical volume</a></h1> 140<p>Since you now know how to create logical volumes, you will create one for this image. We will then install debian into it directly treating the volume as a single straight partition.</p> 141<pre><code># lvcreate --size 4G --name debian2 ganeti 142# grml-debootstrap --target /dev/ganeti/debian2 --hostname debian2 -v \ 143 --mirror http://apt.ws.nsrc.org:3142/cdn.debian.net/debian</code></pre> 144<p>You will be asked to confirm the settings. If it looks ok, answer <code>y</code>.</p> 145<p>This will take a few minutes. You will see it installing the packages into the filesystem. After the initial confirmation to proceed, the only questions you are asked are to set a root password.</p> 146<p>Once finished, you should see:</p> 147<pre><code> * Finished execution of grml-debootstrap. Enjoy your Debian system.</code></pre> 148<h2 id="examine-the-image"><a href="#examine-the-image"><span class="header-section-number">4.1</span> Examine the image</a></h2> 149<p>When built this way, the logical volume contains a single filesystem (with no partition table), and it is easy to mount and examine, which is very useful.</p> 150<pre><code># mount /dev/ganeti/debian2 /mnt 151# ls /mnt</code></pre> 152<p>Let's make a small change to the filesystem: we will enable the serial console. For a Debian image it's done like this:</p> 153<pre><code># vi /mnt/etc/inittab 154 155---- Uncomment this line (remove the '#' in front of it) ---- 156T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100</code></pre> 157<p>Save the change. Finally, unmount the fileystem:</p> 158<pre><code># umount /mnt</code></pre> 159<p>NOTE: to avoid filesystem corruption, it's VERY IMPORTANT that the same block device is not mounted in two places at once. So NEVER mount a volume like this while it is attached to a running VM; and always remember to unmount it before you start a VM which uses it.</p> 160<h2 id="booting-the-image"><a href="#booting-the-image"><span class="header-section-number">4.2</span> Booting the image</a></h2> 161<p>Now we want to boot the image as a VM.</p> 162<p>However there is a problem. Because this image is a straight filesystem, without a partition table, it does not contain a Master Boot Record (MBR) and doesn't have the GRUB boot loader. So it won't boot in the way a normal PC does.</p> 163<p>What we can do instead is to pass KVM two files (a kernel and init ramdisk) stored on the host. KVM will then do the work grub would have done and boot the kernel for us. We will use the same kernel and initrd as is used to boot the Debian host you have built. You will use virt-install as before to set this up.</p> 164<pre><code>$ virt-install --name debian2 --ram 512 \
165  --graphics vnc,listen=0.0.0.0,password=xyzzy \
166  --import --disk /dev/ganeti/debian2 --network=bridge:br-lan \
167  --boot kernel=/vmlinuz,initrd=/initrd.img,kernel_args=&quot;root=/dev/sda&quot;</code></pre>
168<blockquote>
169<p>Hint: If you don't get a console, use your laptop's VNC client to connect. <code>virsh vncdisplay debian2</code> will show you which VNC screen to connect to.</p>
170</blockquote>
171<blockquote>
172<p>Hint: Note the password option above. Password to connect is &quot;xyzzy&quot;</p>
173</blockquote>
174<p><code>kernel_args=&quot;root=/dev/sda&quot;</code> tells the kernel which device to mount as its root filesystem.</p>
175<p>You should now have a running VM using the image you built. If not, ask for help. Remember to do some basic debugging first:</p>
176<pre><code># virsh list
177# virsh dumpxml debian2
178# virsh console debian2</code></pre>
179<h1 id="optional-exercises"><a href="#optional-exercises"><span class="header-section-number">5</span> Optional exercises</a></h1>
180<h2 id="cloning-virtual-machines"><a href="#cloning-virtual-machines"><span class="header-section-number">5.1</span> Cloning virtual machines</a></h2>
181<p>Another approach to creating VMs is to clone one you have already installed. This can be much faster than doing an installation from scratch, but some work may be needed to update the clone before it can be used.</p>
182<p>You need to duplicate the disk image or logical volume, create a new VM definition, start it, and then patch up the new VM.</p>
183<h3 id="duplicate-the-image-file"><a href="#duplicate-the-image-file"><span class="header-section-number">5.1.1</span> Duplicate the image file</a></h3>
184<p>This part is easy! Let's say you have a VM backed by &quot;debian1.qcow2&quot;, and we want to create &quot;debian3.qcow2&quot;; you simply copy it. Make sure the original VM is shutdown first, so its filesystem is in a consistent state.</p>
185<pre><code># virsh shutdown debian1
186# cd /var/lib/libvirt/images
187# cp debian1.qcow2 debian3.qcow2
188# ls -l</code></pre>
189<h3 id="create-a-new-vm-definition"><a href="#create-a-new-vm-definition"><span class="header-section-number">5.1.2</span> Create a new VM definition</a></h3>
190<p>We will take the libvirt XML definition of debian1, and make some minimal changes to make a machine called debian3 with the same parameters.</p>
191<pre><code># cd
192# virsh dumpxml debian1 &gt;debian3.xml
193# vi debian3.xml
194
195-- CHANGE the name
196&lt;name&gt;debian3&lt;/name&gt;
197
198-- DELETE the &lt;uuid&gt;...&lt;/uuid&gt; line
199
200-- CHANGE the source file line
201&lt;source file='/var/lib/libvirt/images/debian3.qcow2'/&gt;
202
203-- DELETE the &lt;mac address='52:54:xx:xx:xx:xx'/&gt; line</code></pre>
204<p>Save and exit from the file.</p>
205<p>Now define the new VM from this file (the missing UUID and MAC address should be assigned automatically).</p>
206<pre><code># virsh define debian3.xml</code></pre>
207<p>If that gives no error, then start your new VM:</p>
208<pre><code># virsh start debian3</code></pre>
209<h3 id="patch-up-the-vm"><a href="#patch-up-the-vm"><span class="header-section-number">5.1.3</span> Patch up the VM</a></h3>
210<p>Get a console onto your new VM (using VNC), and you'll find there are some changes you would need to make.</p>
211<p>Firstly, this host still thinks its name is &quot;debian1&quot;. So you need to edit <code>/etc/hostname</code>, <code>/etc/hosts</code>, <code>/etc/mailname</code> and maybe others.</p>
212<p>Secondly, this host may still be configured with the IP address of debian1. In our case we're using DHCP so it's not a problem, but with static addressing you would have to be very careful to avoid conflicts.</p>
213<p>Thirdly, this host may still remember the MAC address which used to be on <code>eth0</code>, and so call its primary interface something else (like eth1), because it doesn't recognize the MAC address assigned when we did a virsh define - and thinkg it's a new interface.</p>
214<p>If the VM is Ubuntu, check <code>/etc/udev/rules.d/70-persistent-net.rules</code>. If it is CentOS, check for MAC addresses in <code>/etc/sysconfig/network-scripts/ifcfg-eth*</code></p>
215<p>For operating systems like Windows, there may be other things to do such as changing the machine ID in the registry. We do not have time in this workshop to go into details.</p>
216<p>In many cases you will have a local cloning script or utility that takes care of these details for you.</p>
217<h2 id="examining-and-modifying-image-files"><a href="#examining-and-modifying-image-files"><span class="header-section-number">5.2</span> Examining and modifying image files</a></h2>
218<h3 id="raw-images"><a href="#raw-images"><span class="header-section-number">5.2.1</span> Raw images</a></h3>
219<p>It would be very useful if we could look inside an image <em>file</em> to see its contents. The &quot;loop&quot; device lets us attach a file as if it were a block device. However it's slightly awkward because the image we have is not just a filesystem - it's an image of a disk containing a partition table <em>and</em> a filesystem within a partition, which is offset from the beginning of the disk.</p>
220<p>Try running the <code>file</code> command on the <code>debian1.img</code> file:</p>
221<pre><code># file /var/lib/libvirt/images/debian1.img
222
223/var/lib/libvirt/images/debian1.img: x86 boot sector; partition 1: ID=0x83, starthead 0, startsector 4096, 4190208 sectors, code offset 0x31</code></pre>
224<p>This shows it's a PC disk image with a boot sector and partition table.</p>
225<p>(Note: you would not mount the disk image of a running VM to avoid corruption)</p>
226<p>The 'kpartx' utility is able to set up separate loopback devices for each partition contained within the image:</p>
227<pre><code># kpartx -av /var/lib/libvirt/images/debian1.img
228# ls /dev/mapper/loop*</code></pre>
229<p>You should now see you have a device <code>/dev/mapper/loop0p1</code> which is a block device to access the data in partition 1 of the image, and now you can mount the filesystem and look inside:</p>
230<pre><code># mount /dev/mapper/loop0p1 /mnt
231# ls /mnt
232... do whatever you need inside the /mnt directory</code></pre>
233<p><em>IMPORTANT!!</em> As before, once you have finished using the mount, it's vital to unmount it and deconfigure the loopback device - otherwise you risk corruption if you start running a VM on this image at the same time as the host system has it mounted.</p>
234<pre><code># umount /mnt
235# kpartx -d /var/lib/libvirt/images/debian1.img</code></pre>
236<p>If you want to learn more about loopback devices, read the documentation for the <code>losetup</code> tool.</p>
237<h3 id="qcow2-images"><a href="#qcow2-images"><span class="header-section-number">5.2.2</span> qcow2 images</a></h3>
238<p>qcow2 images need to be mounted in a different way, since they are a custom format with a header and data, and the tool to do this is qemu-nbd (network block device). Be sure you stop your debian1 virtual image before mounting its filesystem:</p>
239<pre><code># virsh list --all</code></pre>
240<p>Verify if debian1 is running. If it is, then:</p>
241<pre><code># virsh shutdown debian1</code></pre>
242<p>And, now use the ndb module and qemu-nbd:</p>
243<pre><code># modprobe nbd max_part=4
244# qemu-nbd -c /dev/nbd0 /var/lib/libvirt/images/debian1.qcow2
245# mount /dev/nbd0p1 /mnt
246# ls /mnt
247... do work here
248# umount /mnt
249# qemu-nbd -d /dev/nbd0</code></pre>
250<h3 id="guestfish-libguestfs"><a href="#guestfish-libguestfs"><span class="header-section-number">5.2.3</span> guestfish / libguestfs</a></h3>
251<p>Another tool to access the data inside a VM image is 'guestfish' - the shell frontend to <a href="http://libguestfs.org/">libguestfs</a>. Install it, read the documentation and try it out.</p>
252<pre><code># apt-get install guestfish    # say &quot;Yes&quot; to &quot;Install supermin appliance now&quot;
253# man guestfish</code></pre>
254<p>It works by building a tiny VM (the &quot;supermin appliance&quot;) and running it with the disk image attached. It's especially helpful in complex scenarios, such as when the guest disk is a logical volume but the guest is also using LVM inside that.</p>
255<p>The related package 'libguestfs-tools' includes further tools such as <a href="http://libguestfs.org/virt-resize.1.html">virt-resize</a></p>
256</body>
257</html>