How to Give Easy, Peasy Lemon Squeezy Developer Access to Production Cloud Data

How to Give Easy, Peasy Lemon Squeezy Developer Access to Production Cloud Data

Posted by: E Lewis


I have been religiously snapping my critical VMs on my Datrium DVX storage. I know I can restore a VM from a snapshot if the VM gets damaged. I know I can clone a VM from a snapshot if I want an exact copy of a whole VM.  But what else can I do? Well, my developers have been clamoring for access to production database data so that they can test their queries in a realistic context.  Seems reasonable. I just need to figure out the best way to give the developers access to a clone of this data. You don’t think I should give them access to live production data do you?!? It turns out DVX makes this super simple because it is not only aware of what constitutes a VM, it is also aware of what constitutes a virtual disk (e.g. my G: drive).


The many ways to clone VM data

As simple as my goal sounds, there are multiple ways to proceed.  Here’s a quick—and not necessarily complete—summary.

Clone a production VM. This is a reasonable place to start. Now I just give my developers access to this VM. Done! Not quite. Developers can’t even log into the (cloned) production DB VMs. In addition, these VMs contain sensitive data to which developers are not allowed access. Finally, even without the above problems, the (cloned) production VMs do not contain the necessary development tools. I can’t ask them to install this stuff whenever they get fresh production data. I need another approach.

Clone a production VM—Take 2. The developers only need access to the disk volume containing the DB data. I suppose I could clone the production VM, delete all but the VMware .vmdk files representing the volume they need, and then give them access to these files so that they can attach them to their development VMs. I’ll bet this would work, but I don’t know how to figure out what VMware .vmdk files constitute which virtual disk. Even if I did, life’s too short to waste time managing VMware files! I’ve gotta do better than this.

Clone the guest virtual disk and hot add it to the developer VM. Now we’re talking. What I really want to do is manipulate (clone/attach) a virtual disk that is part of a VM snapshot, not VMware files. Perfect. I’ll walk through the steps to achieve this, below.

Are there other use cases for managing virtual disks? There sure are. When my users want to pull data out of a snapshot, I have a couple options.  I can restore the VM from a snapshot, but that replaces the whole VM; often my users just want a couple files. I can clone the VM and give my user access to it so that they can copy out the files of interest. This works.  I’ve done it. But it’s awkward. Now with DVX, I simply clone the virtual disk containing the data of interest and attach it to my user’s VM, giving them direct access to the data in the snapshot from within the same VM from which the snapshot was taken. Easy for me.  Easy for my users.

What does it mean to clone a guest virtual disk? VMware represents VMs as a collection of files residing in a DVX datastore, and each virtual disk device is represented with at least two cryptically-named files, often more. Naturally, I certainly don’t want to manage these files individually, because I don’t understand them. You probably already know that the DVX is VM-aware so I can, for example, ask for a particular VM to be snapped, and the DVX will automatically snapshot all the files that constitute this VM. Similarly I can clone a VM and the DVX both (i) determines what files constitute the VM and clones them and (ii) transforms the metadata associated with these files so that together they represent a valid VM (e.g., don’t have IDs or MAC addresses that conflict with the VM from which they were cloned). In a similar manner the DVX is also guest-virtual-disk aware, i.e., it not only knows what files constitute each VM, it also knows what files constitute each virtual disk device within each virtual machine. Furthermore, when cloning a guest virtual disk, the DVX ensures that the metadata within the cloned virtual disk is ready to be attached to a VM (e.g., ensures VMware disk IDs won’t conflict with the source virtual disk of the clone operation).

Enough talk. Let’s walk through the process step by step.


How to Clone/Attach a Guest Virtual Disk

The production VM I’m interested in is called “oracle-customer,” and it is running Windows 7. This VM has a bunch of hard disks. The data I want is on the Disk 3/G: (“DB Volume”). Here’s what I see in the Disk Management utility within the VM.

Step 1: Datrium GUI (Clone Virtual Disk)

In order to clone the guest virtual disk, I navigate to the VM snapshot view. This is achieve as follows: Click on the “Protection” tab; click on the protection group containing the snapshot; click on the snapshot. Now I see the following list of VMs contained within the snapshot.

I select “oracle-customer” and click on “Show VM files in snapshot” in the pull-down menu. This shows me all the VMware files that constitute this virtual machine.

Here, I select a file that is part of the guest virtual disk of interest. This is the hard part. I’ll use the file size to find the right file; I want the 4GB disk, not the 2TB disk. Recall that each virtual disk is represented by multiple files. Selecting any file within a single virtual disk has the same effect. See the end of this post for additional hints on selecting the right file.

Next, I’m asked where to place the files. Although the text in the dialog below is asking for a path and filename, when cloning a virtual disk we are providing the path to the directory to which we want to store all the files representing the virtual disk of interest. In this case, I am placing the files in a directory called “/oracle-elewis-dev00/db-disk-clone.” Click “Clone.”


Step 2: Virtual Center UI (Hot Add Cloned Disk)

I have all the files representing the clone of the G: drive, so now I can attach it to the developer VM, called “oracle-elewis-dev00.” I am going to “hot add” the disk, because I don’t want to have to power off the VM.

In the Virtual Center UI I navigate to the “oracle-elewis-dev00” VM and select “Edit Settings” under “Actions”. In the “Edit Settings” window I add a new disk device.  By “New device:” I select “Existing Hard Disk” and click “Add”.

In the Edit Settings window I add a new disk device.  By “New device:” I select “Existing Hard Disk” and click “Add”.

Next, I’m asked to select the file to use for the virtual disk. I browse to the “/oracle-elewis-dev00/db-disk-clone” dir, select “db-disk-clone_1.vmdk”, and click “OK.” Note that Virtual Center hides some of the VM files in this browser so don’t be surprised if you see fewer files than were actually cloned.

Click “OK” in “Edit Settings” window to complete the disk-add operation.


Step 3: Pat yourself on the back

The cloned virtual disk has been attached to my “oracle-elewis-dev00” virtual machine and Windows has automatically brought it online.

Step 4: Cleanup

When the developer is done with this data, I delete it in vCenter like any other virtual disk.

Okay there you have it, the instructions on how to use virtual disk cloning safely and simply to provide developers access to production data. Easy, peasy, lemon squeezy—thanks DVX!


Interested in additional details?

If you are already a Datrium DVX customer and are interested in additional details, please refer to the “Virtual Disk Clones” section of Datrium DVX System Management.

y:inline;"> line;">