Wednesday, November 20, 2013

Using VAAI UNMAP on vSphere 5.5 & NetApp Storage

Welcome: To stay updated with all my Blog posts follow me on Twitter @arunpande !!


In my previous blog vStorage APIs for Array Integration (VAAI) & NetApp – How to set it right? I have shared steps to use VAAI. In this blog I will cover the steps required to use the VAAI UNMAP primitive in vSphere 5.5. The UNMAP primitive is used by the ESXi host to update the Storage Array about the storage blocks that has to be reclaimed after deleting a VM or migrating it to another datastore using Storage vMotion. In vSphere 5.5 # esxcli storage vmfs unmap command is used whereas in the earlier version vmkfstools –y command was used. You can now specify the number of blocks to be reclaimed using -n option whereas with vmkfstools –y command you had to specify the percentage of blocks that you want to reclaim. It is advised to perform this step after business hours or when there is no active I/O on the datastore, however I have not tested


In this scenario I am using a thin provisioned LUN from NetApp Storage and to demonstrate space reclamation I will create two scenarios (i) deleting the thick disk  (ii) migrating VMs using Storage vMotion. I will also share the storage capacity from NetApp Virtual Storage Console (VSC) which will give a view about the available space not only on the VMFS datastore but also the underlying LUN/Volume/Aggregate.


Scenario 1 – Deleting a thick disk from the virtual machine
Here is an overview about the Capacity of Datastore/LUN/Volume/Aggregate as per VSC.

             


Capacity of the datastore as per ESXi Shell


# du -h /vmfs/volumes/iscsi_2/
1.0M    /vmfs/volumes/iscsi_2/.sdd.sf
8.0K    /vmfs/volumes/iscsi_2/ntap_rcu1374646447227
8.0K    /vmfs/volumes/iscsi_2/ntap_rcu1374459789333
8.0K    /vmfs/volumes/iscsi_2/.naa.600a09802d6474573924384a79717958
194.1G  /vmfs/volumes/iscsi_2/Win2k8-1
194.9G  /vmfs/volumes/iscsi_2/


This indicates that the total used capacity on the datastore is 194.9 GB
We will now delete the 150 GB Eager Zeroed Thick Disk. After deleting this virtual disk the ESXi shell reports the following capacity.


# du -h
1.0M    ./.sdd.sf
8.0K    ./ntap_rcu1374646447227
8.0K    ./ntap_rcu1374459789333
8.0K    ./.naa.600a09802d6474573924384a79717958
44.1G   ./Win2k8-1
44.9G   .


The free space on the datastore is now 205 GB and the used space is approximately 44.9 GB.  However NetApp Storage does not detect this free space on the LUN, here is the output of the lun show command that is executed from the Clustered Data ONTAP CLI.


clus-1::> lun show -v /vol/iscsi_2/iscsi_2
              Vserver Name: vmwaretest
                  LUN Path: /vol/iscsi_2/iscsi_2
               Volume Name: iscsi_2
                Qtree Name: ""
                  LUN Name: iscsi_2
                  LUN Size: 250.3GB
                   OS Type: vmware
         Space Reservation: disabled
             Serial Number: -dtW9$8JyqyX
                   Comment: The Provisioning and Cloning capability created this lun at the request of Administrator
Space Reservations Honored: false
          Space Allocation: enabled
                     State: online
                  LUN UUID: 7fe6d24a-f782-476d-827e-a4d20f371abb
                    Mapped: mapped
                Block Size: 512
          Device Legacy ID: -
          Device Binary ID: -
            Device Text ID: -
                 Read Only: false
Inaccessible Due to Restore: false
                 Used Size: 237.9GB
       Maximum Resize Size: 2.50TB
             Creation Time: 12/16/2010 03:27:26
                     Class: regular
                     Clone: false
  Clone Autodelete Enabled: false
          QoS Policy Group: -


VSC also reports the same capacity for this LUN.

            

We will now use the UNMAP primitive from the ESXi shell using the command
# esxcli storage vmfs unmap -l iscsi_2


NOTE: You can also specify the number of blocks that you want to reclaim using –n option. If you specify 500 then 500 x 1MB (i.e. default block size in VMFS 5) blocks would be reclaimed.


On monitoring the esxtop I have observed that the DELETE statistics has increased to 52527.


VSC now reports the following capacity, where we see that the free space is now updated for LUN & Volume.



Scenario 2 – Test UNMAP after relocating VMs using Storage vMotion.


NetApp VSC reports the following storage usage.

            


Datastore Usage according to ESXi Shell


~ # df -h
Filesystem   Size   Used Available Use% Mounted on
VMFS-5       1.0T 881.5G    143.0G  86% /vmfs/volumes/FC-Infra


Datastore Usage per VM is given below


~ # du -h /vmfs/volumes/FC-Infra /
74.5G   /vmfs/volumes/FC-Infra/VC
78.3G   /vmfs/volumes/FC-Infra/DB
15.4G   /vmfs/volumes/FC-Infra/Oncommand-Proxy
8.0K    /vmfs/volumes/FC-Infra/.vSphere-HA
1.3M    /vmfs/volumes/FC-Infra/.dvsData/7a 4c 23 50 26 82 38 5d-d9 e5 e2 78 4f 7d af 26
32.0K   /vmfs/volumes/FC-Infra/.dvsData/3e 55 23 50 21 27 03 84-e3 f4 4a 7f de 48 08 32
1.3M    /vmfs/volumes/FC-Infra/.dvsData
29.4G   /vmfs/volumes/FC-Infra/AD
64.1G   /vmfs/volumes/FC-Infra/VASA
23.5G   /vmfs/volumes/FC-Infra/VSI Launcher-9
23.5G   /vmfs/volumes/FC-Infra/VSI Launcher-7
12.0G   /vmfs/volumes/FC-Infra/OnCommand Balance
32.7G   /vmfs/volumes/FC-Infra/ViewComposer
63.3G   /vmfs/volumes/FC-Infra/View Connection Server
19.5G   /vmfs/volumes/FC-Infra/VSIShare
19.5G   /vmfs/volumes/FC-Infra/VSI Launcher-10
21.4G   /vmfs/volumes/FC-Infra/UM-6.0
20.6G   /vmfs/volumes/FC-Infra/VSI Launcher
15.3G   /vmfs/volumes/FC-Infra/VSI Launcher-Template
24.4G   /vmfs/volumes/FC-Infra/VSI Launcher-2
23.5G   /vmfs/volumes/FC-Infra/VSI Launcher-4
24.7G   /vmfs/volumes/FC-Infra/VSI Launcher-3
23.5G   /vmfs/volumes/FC-Infra/VSI Launcher-5
25.5G   /vmfs/volumes/FC-Infra/VSI Launcher-6
25.5G   /vmfs/volumes/FC-Infra/VSI Launcher-8
34.4G   /vmfs/volumes/FC-Infra/UI VM
181.5G  /vmfs/volumes/FC-Infra/Analytics VM
1.5G    /vmfs/volumes/FC-Infra/vmkdump
881.3G  /vmfs/volumes/FC-Infra/


To make some free space on the Storage I have Storage vMotioned the following VMs to another datastore.
29.4G   /vmfs/volumes/FC-Infra/AD
78.3G   /vmfs/volumes/FC-Infra/DB


After the above VMs were migrated to other datastores the following datastore usage was reported:


From the Filer, notice that the LUN Used Size remains the same.


veo-f3270::> lun show -v /vol/infra_services/infra


             Vserver Name: Infra_Vserver
                 LUN Path: /vol/infra_services/infra
              Volume Name: infra_services
               Qtree Name: ""
                 LUN Name: infra
                 LUN Size: 1TB
                  OS Type: vmware
        Space Reservation: disabled
            Serial Number: 7T-iK+3/2TGu
                  Comment:
Space Reservations Honored: false
         Space Allocation: disabled
                    State: online
                 LUN UUID: ceaf5e6e-5a6a-11dc-8751-123478563412
                   Mapped: mapped
               Block Size: 512B
         Device Legacy ID: -
         Device Binary ID: -
           Device Text ID: -
                Read Only: false
                Used Size: 848.9GB
            Creation Time: 9/3/2007 18:12:49


NetApp VSC does not report any changes in LUN Usage either.

            


ESXi Shell reports the updated free space.
~ # df -h
Filesystem   Size   Used Available Use% Mounted on
VMFS-5       1.0T 773.8G    250.7G  76% /vmfs/volumes/FC-Infra


I have now performed the reclaim operation from the ESXi Shell using the below command
# esxcli storage vmfs unmap -l FC-Infra


NOTE: You can also specify the number of blocks that you want to reclaim using –n option. If you specify 500 then 500 x 1MB (i.e. default block size in VMFS 5) blocks would be reclaimed.

VSC now reports free space in the LUN Usage.

            



The filer also reports the updated Storage Capacity.
veo-f3270::> lun show -v /vol/infra_services/infra


             Vserver Name: Infra_Vserver
                 LUN Path: /vol/infra_services/infra
              Volume Name: infra_services
               Qtree Name: ""
                 LUN Name: infra
                 LUN Size: 1TB
                  OS Type: vmware
        Space Reservation: disabled
            Serial Number: 7T-iK+3/2TGu
                  Comment:
Space Reservations Honored: false
         Space Allocation: disabled
                    State: online
                 LUN UUID: ceaf5e6e-5a6a-11dc-8751-123478563412
                   Mapped: mapped
               Block Size: 512B
         Device Legacy ID: -
         Device Binary ID: -
           Device Text ID: -
                Read Only: false
                Used Size: 742.6GB
            Creation Time: 9/3/2007 18:12:49

1 comment:

  1. Hi. What about NFS datastores. We have noticed that when we storage vmotion vm's from one NFS datastore to another, the expected free space in the source NFS datastore has not gone down. I'd expect that since WAFL owns the NFS filesystem, there's no need for any UNMAP functinality. Shouldn't free space be automatically updated for NFS datastores? If so, what might be happening with my environment. Would NetApp snapshots have any bearing on what I'm seeing?

    ReplyDelete