Editing FreeBSD Reference (section)

= MD/VN Devices and Mounting Customer Filesystems =

Ok, so we have discussed how to dump and restore filesystems - a fairly easy concept.  But, what to do with these dumped filesystems, once they are dumped and exist in the form of a file ?

We use a special device- md or memory disk- to attach these dump files to a md device- this creates a md (or vn)-backed filesystem, also known as a file-backed filesystem. Once configured, we can use this like we would a physical disk- we can fsck it, mount it, etc.

On older 4.x systems we used a similar technology called vn- the difference being md devices are created on the fly (as needed) whereas vn* devices are pre-created when we create the new system.

To create these from scratch, we start with an empty file. The process of configuring the file will assign it a particular size- the size of a customer's jail.
That means, a 20 GB md-backed filesystem will actually be a 20 GB regular file inside which a full filesystem exists. That regular file is configured with mdconfig (or vnconfig), and then newfs'd and then mounted, like any other filesystem.

When we create a new md/vn we start off with a simple, empty file. After we md/vnconfig the file with a particular size (say 20 GB) this creates a truncated file- allowing us to oversell disk space. See below for how truncation works.

Here is the procedure for creating a new mdfile-backed filesystem:

Make & configure the mdfile:
 touch /path/to/mdfile 
 mdconfig -a -t vnode -s <GB>g -f /path/to/mdfile -u <#>

Ex: 
 touch /mnt/data2/69.55.232.68-col02144
 mdconfig -a -t vnode -s 20g -f /mnt/data2/69.55.232.68-col02144 -u 11

-s specifies how big to make the filesystem/file<br>
-f specifies which file to use for the mdfile (just make sure the is not in use, and if it doesn't exist- touch it first)<br>
-u specifies which md device number you may use. Any unused md # is ok, you can see what's '''already in use''' (and therefore what NOT to use) by running [[VPS_Management#js|js]] or <tt>mdconfig -l</tt>

Format the filesystem:
 newfs /dev/md<#>
Ex:  <tt>newfs /dev/md11</tt>

Create a mountpoint:
 mkdir /path/to/dir

Ex:  <tt>mkdir /mnt/data2/69.55.232.68-col02144-DIR</tt>

Mount it:
 mount /dev/md<#> /path/to/dir

Ex: mount /dev/md11 /mnt/data2/69.55.232.68-col02144-DIR

Confirm it's properly mounted at the right mount point and has the correct size:
<pre>cd /path/to/dir
df -h .

Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/md<#>   <size>    0      <size>    0%    /path/to/dir</pre>

Ex: <pre>cd /mnt/data2/69.55.232.68-col02144-DIR
    df -h .

Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/md11     20G       0M     20G     0%    /mnt/data2/69.55.232.68-col02144-DIR</pre>

Look to make sure the mount point is right (<tt>/mnt/data2/69.55.232.68-col02144-DIR</tt>) the device is right (<tt>/dev/md11</tt>) and the size is right (<tt>20G</tt>).



Here is the procedure for creating a new vnfile-backed filesystem:

Configure the vnfile:
 vnconfig –T –S <#>g -s labels -c /dev/vn<#> /path/to/vnfile 

Ex:  <tt>vnconfig –T –S 20g -s labels -c /dev/vn12 /mnt/data1/69.55.230.44-col00137</tt> 

-S specifies how big to make the filesystem/file<br>
-c specifies which vn device number you may use. Any unused vn # is ok, you can see what's already in use (and therefore what NOT to use) by running <tt>js</tt><br>


Create a partition on the device:
 disklabel -r -w vn<#> auto

Ex:  <tt>disklabel -r -w vn12 auto</tt>

Format the filesystem:
 newfs /dev/vn<#>c

Ex:  <tt>newfs /dev/vn12c</tt>

Create a mountpoint:
 mkdir /path/to/dir

Ex:  <tt>mkdir /mnt/data1/69.55.230.44-col00137-DIR</tt>

Mount it:
 mount /dev/vn<#>c /path/to/dir

Ex: <tt>mount /dev/vn11c /mnt/data1/69.55.230.44-col00137-DIR</tt>

Confirm it's properly mounted at the right mount point and has the correct size:
<pre>cd /path/to/dir
df -h .

Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/md<#>c  <size>    0      <size>    0%   /path/to/dir</pre>


Ex:  <pre>cd /mnt/data1/69.55.230.44-col00137-DIR
     df -h .

Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/vn11c     20G      0M     20G     0%    /mnt/data1/69.55.230.44-col00137-DIR</pre>

Look to make sure the mount point is right (<tt>/mnt/data1/69.55.230.44-col00137-DIR</tt>) the device is right (<tt>/dev/vn12c</tt>) and the size is right (<tt>20G</tt>).


At this point, there is now a 20 GB filesystem mounted on the system, with a mount point of  "<tt>/path/to/dir</tt>".  If you cd to <tt>/path/to/dir</tt>  and place 1 GB of files there, the disk space used on the mount point on which the md or vnfile are located (i.e. /mnt/data1) and the md/vn device mount point, will go up by 1 GB ... but the size of the md/vn file will remain unchanged at 20 GB.

To illustrate how this works, let's see how we determine whether a file is truncated and to what extent. Compare an (existing customers) md/vn file size reported by ls with what du reports:

 jail9 /mnt/data1# ll -h 69.55.232.70-col01963
 -rw-r--r--  1 root  wheel    30G Apr  6 14:45 69.55.232.70-col01963

The md file is 30 GB

 jail9 /mnt/data1# du -sh 69.55.232.70-col01963
 4.7G    69.55.232.70-col01963

However, since it was created as a truncated file, it's only occuping 4.7 GB of disk space- because the customer has written (up-to*) 4.7 GB of data in their filesystem, at some point.

In other words, if you create a md/vnconfig 100 GB device & file, then immediately run a df of the underlying filesystem (i.e. /mnt/data1) you will not see any change in the space free space available. If after you mount that md/vnfile and write 100 GB into the mounted device, 100 GB will disappear from the underlying filesystem (i.e. /mnt/data1) and from the md/vn filesystem.


 jail9 /mnt/data1# df -h 69.55.232.70-col01963
 Filesystem    Size    Used   Avail Capacity  Mounted on
 /dev/md24      30G    1.6G     27G     9%    /mnt/data1/69.55.232.70-col01963

<tt>*</tt>They are currently only using 1.6 GB in their filesystem. Why doesn't du report 1.6 GB as well? This is because once the mdfile is written to ("expanded") even if you delete files, the md/vnfile doesn't contract down again- usage always increases. So for instance, someone could use all their disk space, but then delete all files, the result will be their md/vnfile will expand to occupy the full size of their filesystem (30 GB in this example)...so, to reclaim the unused space you'd need to create a new md/vn file and dump their filesystem into that.


A few details:

1. Please note the specific syntax of the commands we used to make the filesystem - specifically, how we refer to /dev/vn12.  In the vnconfig command, we refer to "/dev/vn12", however in the disklabel command we refer to just "vn12", and then in the newfs and mount commands, we refer to "/dev/vn12c".  These must always be just like this.  You cannot run the disklabel command with "/dev/vn12", and you cannot run the vnconfig command with "/dev/vn12c" - the syntax needs to be exactly as it is shown above. In general, avoid using md0-2 and vn0-2 since we may use those for file-backed swap on some systems.

2. The disklabel and newfs commands are, of course, only needed when first creating the new filesystem - if you were to unmount this new filesystem, and use <tt>mdconfig -u <#> or vnconfig -u /dev/vn<#></tt> to disassociate the md/vnfile from the md/vn device, you would then be left with just a plain old file sitting on your disk.  

The process of unmounting and unconfig'ing md/vn devices, and then re-configing and remounting is as follows:

For md: 

Take it down:

 umount /path/to/dir
	or
 umount /dev/md<#>

(note, your pwd must not be inside the mount point)
Ex:  <tt>umount /mnt/data2/69.55.232.68-col02144-DIR</tt>
		or
    <tt>umount /dev/md11</tt>


 mdconfig -d -u <#>

Ex:  <tt>mdconfig -d -u 11</tt>

Then to bring it back up:

 mdconfig -a -t vnode -f /path/to/mdfile -u <#>

Ex:  <tt>mdconfig -a -t vnode -f /mnt/data2/69.55.232.68-col02144 -u 11</tt>

 mount /dev/md<#> /path/to/dir

Ex: <tt>mount /dev/md11 /mnt/data2/69.55.232.68-col02144-DIR</tt>

So, first we umount the filesystem, like we would any other filesystem. Then we use mdconfig -d -u # to disassociate it from the md device.

Now, when we reconfigure it against md11, note that the mdconfig command is much simpler than the original mdconfig command we used to create this filesystem - you do not need the '-s #GB' argument.

Then we mount the filesystem again.


For vn:

Take it down:

 umount /path/to/dir
	or
 umount /dev/vn<#>c

(note, '''your pwd must not be inside the mount point''')
Ex:  <tt>umount /mnt/data1/69.55.230.44-col00137-DIR</tt>
		or
    <tt>umount /dev/vn12c</tt>



 vnconfig -u /dev/vn<#>

Ex:  <tt>vnconfig -u /dev/12</tt> 

Then to bring it back up:

 vnconfig /dev/vn<#> /path/to/vnfile

Ex:  <tt>vnconfig /dev/vn12 /mnt/data1/69.55.230.44-col00137</tt>


 mount /dev/vn<#>c /path/to/dir

Ex: <tt>mount /dev/vn12c /mnt/data1/69.55.230.44-col00137-DIR</tt>

So, first we umount the filesystem, like we would any other filesystem. Then we use <tt>vnconfig -u /dev/vn#</tt> to disassociate it from the vn device.

Now, when we reconfigure it against vn12, note that the vnconfig command is much simpler than the original vnconfig command we used to create this filesystem - you do not need the `<tt>-s labels -c</tt>` arguments.

Then we mount the filesystem again.


3. A particular md/vnfile is not married to the device it was created with. In the above example we created the device on /dev/md11 and /dev/vn12, and then mounted it with that - but, if we were to unmount it, unconfig it from /dev/md11 or /dev/vn12 (at this point we have nothing but a plain old  file), we could then mdconfig or vnconfig it against md25 or vn14 and mount /dev/md25 or /dev/vn14c instead - it would work just fine.

The only caveats, of course, is that /dev/md25 must not already be in use, and /dev/vn14 must exist, and vn14 must not already be in use by some other filesystem.

4. A particular md/vnfile is also not wedded to its machine, ot is mount point. If you umount a md/vn-backed filesystem, and then unconfig it, you can then copy that file to some other system, md/vnconfig it to a different md/vn device number, and mount it on some totally differently named mount point. While technically possible, this is not a good idea. When using truncated files, the moment you move that file to a different filesystem (even on the same machine: /mnt/data1 -> /mnt/data2), as the file is being transferred off the source filesystem, empty/unused/truncated parts of the file will be filled with 0's by the sending process. So while you started off with a 20 GB file (as reported by ls) and 1 GB usage (as reported by du) after the move it will be reported as 20 GB by both ls and du. Therefore, any time we move a file off a filesystem we use dump and restore. See [[VPS_Management#dumpremoterestore|dumpremoterestore]] and [[FreeBSD_Reference#Dump_and_Restore|dump and restore]].

Note: if you un-mn/vnconfig before an unmount, re-md/vnconfig then unmount then un-md/vnconfig