Short tutorial: ESX 3.0 cluster in-a-box with RDM

Hi guys,I decided to write a small tutorial on clustering (in a box) with RDM on ESX 3.0.
What you need:

• 1 ESX 3 server
• 2 virtual nodes with the same config:
  • 2 virtual NICs, 1 to an external switch, 1 to an internal switch with no external pNICs
  • min. 2 virtual hard drives, on a separate virtual LSI Logic SCSI controller
• A small LUN to use as the Quorum disk (100 Mb)

Each VM node should be configured with a separate LSI logic controller, 2 extra Hard disks (pointing to a Raw LUN, quorum and resource) and an extra “heartbeat” NIC. The SCSI controller should be set to “Virtual” when using a cluster-in-a-box (on 1 ESX server).

The extra harddisks are mapped directly to a SAN/iSCSI LUN via RDM in virtual compatibility mode (non‐pass‐through RDM).

For the first node, use the add hardware wizard to add the extra disk and select “Mapped SAN LUN” to make a .vmdk disk that points directly to the RAW LUN (instead off a new/existing .vmdk disk on a VMFS datastore).

On the compatibility mode, select “Virtual”.

Make sure that you choose SCSI (1:0) and NOT (0:0)!

For the second node, follow the same procedure, but choose “Use an existing virtual disk” and point it to the .vmdk file created for the other node (should be something like “[iscsilun4-DS] Sr-Node1/Sr-Node1_1.vmdk”). Use the exact same config (SCSI (1:0) for the quorum disk, etc.).

That’s about it! The rest of the configuration is a default cluster setup.

For extra info, consult this PDF on clustering

Installing Virtual Center 2.5 & ESX 3.5

Anyway, I first put the VC-VM in snapshot mode, then connected the vc25 iso and started installing. Real men don’t do backups ! (At least not untill they lose some realy important data ). When creating the snapshot through VC, the ESX host suddenly got disconnected, but fortunately just for a second or two. Snapshot succesful, so I connected the vc25 iso to the VC VM and started the installation.

After a few next, next, actions I got the question which database I would like to use. I did have MSDE installed and pointed the DSN file to this installation. Unfortunately, I then got a message explaining that this MSDE version can no longer be used and I should upgrade my database to SQL Express or another supported database. Because I was not sure how to upgrade my MSDE to SQL Express, I started installing SQL Express. The installation files can be found on the VC25 cdrom at \redist\SQLEXPR32. Again a few next next next clicks and then a question prompted me to enter the name of the SQL instance I would like to use, plus a button called “Installed Instances”. There I could select my current MSDE instance. Next I received a report telling me that the instance found could be upgraded, so I continued. Looking very good here

I now switched back to the VC 2.5 installation, which was still waiting at the “select database” screen. I again used the DSN name I used before and now I received a message telling me that the database was about to be upgraded and can no longer be used by older products. Yes, please do so  Oeps, now something went wrong. I seemed to be using the “SQL Server” ODBC driver but VC wants the “SQL Native Client” driver. I tried to edit the DSN connection, but this was no longer possible. I received connection errors. I found that enabling tcp/ip on the “SQL Server 2005 network configuration” plus restarting the MSDE_VC instance helped getting connection again. I then created a new DSN but this time with the correct SQL Native driver and pointed it to the same database (the older vc 2.0.2 database). Now installation could continue.

Next question was the location of my license server and which Virtual Center edition I wanted to use. I could choose between Virtual Center Management Server and Virtual Center Foundation Management Server. I choose the first and pointed to my already running license server. Next I was asked for the VC credentials and IP address. I entered my local admin account, because my XP virtual center is not part of a domain. I’m not using any users or groups in VC, other then the local admin. This was the last question before realy starting the install. I pressed finish and the installation started. I leaned back and watch InstallShield do its work.

After a short while there was the “VMware Virtual Center Database Upgrade Wizard”. I pressed next and a short analysis was done by the wizard. It asked me which data I want to preserve during upgrade. I could choose to “Preserve events and taks” and “Preserve performance data”. I left it at default settings, which is to preserve both. Then again pressed next. I’m quite good at pressing next  The wizard finished without problems and installation continued. Finaly I received the “Installation succesful” message. I pressed finish and the VI Client was started automatically and I logged on to the new VC server.

After ignoring a certificate error, I could finaly look at the new VC 2.5 interface. Very nice, very nice indeed. When looking at the already running tasks, I noticed that my first ESX host had been disconnected. It said that the VC agant could not be installed on this host. The second host didn’t have any problems. When looking at the version numbers, I noticed that the first host was at 3.0.2 build 61618 and the second at 3.0.2 build 62488. Could that be the difference? I searched the VC 2.5 CD for any docs and discovered in a readme.txt that even ESX 2.5.4 would be supported, which ofcourse makes sense. So I checked if the esx host was still running fine.

Using ssh I connected to the console and stopped the vpxa service (service vmware-vpxa stop). Then tried to reconnect the host, but VC still reported an error. Next I removed the vpxa rpm from the ESX host. I first check which version was installed using “rpm -qa | grep vpx” and noticed that it was probably VC 2.5 already: VMware-vpxa-2.5.0-64192. Still I decided to remove it, using. In VC I tried to reconnect again, but this still gave me errors. So I now removed the disconnected host from VC and then added it as a new host. Unfortunately still unable to connect the host.

I also noticed that I couldn’t run any vmware-cmd commands anylonger and a direct connection to the esx host using the VI Client was also not possible. Well, lets play tough then…. I planned to just upgrade the host to esx 3.5. Since I couldn’t use vmware-cmd to stop the VMs, all that was left, was to rdp to each VM and shut it down. Two VMs didn’t have a rdp option, so I just let them running. Then I shutdown the host, put in the esx 3.5 CD and booted from it.

Upgrade went fast and smooth. I pressed the Next button, choose the upgrade path and installation started. After install it ejected the CD and I was able to reboot into ESX 3.5. Smooth as a baby. After a few minutes I could shh into the cos and see that my most important VMs had already been started automatically as they did in ESX 3.0.2. Pfffff… lucky me Next I made a RDP session into the VC server and added the ESX host. This time all went fine. I quickly ran through all the new options and was pleased to see that some nice improvements have been made. But I’m gonna talk about those tomorrow. Its 2:45AM right now and I think its better to get some sleep

Using VMWARE

Basic System Administration -Part 04

If you remove users from the VirtualCenter domain, they lose permissions to all objects in the VMware Infrastructure and will not be able to log on again. Users who are currently logged on and are removed from the domain retain their Vmware Infrastructure permissions only until the next validation period (the default is every 24 hours)

Vmware doesn’t not explicitly restrict users from with same login and password from accessing and taking action within the VC.

If you rename user domain account, it becomes invalid in VC and same applies to group but before that(for groups only) you need to restart virtucal center.

Following activities can be scheduled as Tasks

• Change the power state of a VM
• Create a VM template
• Move a VM with Vmotion
• Create a VM
• Make snapshot of VM
• Customize VM
• Add a Host

This was the last part of the series from basic administration task,this pdf is in more details and end in 364 pages. There should be more information which be might be useful for VCP. I might add more to this series soon.

Basic System Administration -Part 3

With ESX 3.0 you can migrate suspended VMs under cold migration process. You can move files of VMs only when they are power off and suspended.

When you modify User’s permissions, Users do not need to log off and log on into Virtual Center for changes to take effect. All changes take effect immediately.

You can define permission on

• VM
• Folders
• Datacenters
• Resource pools
• Templates
• Host
• Clusters

You cannot directy define permission on

1. Networks
2. Datastores

In virtual center you can assign permission to

• Folder
• Datacenter
• Host
• VM
• Templates
• Cluster
• Resource Pools
• In ESX server you can assign permission to
• Resource Pool
• VMs
• Hosts

There are two types of roles

System (cannot be modified): administrator/read-only/No-access

Sample (can be modified):

Detail Description is available in Excel Sheet HERE

Basic System Administration -Part 2

When you install VMWARE Tools it installs

1. Vmware tool service
2. Set of Vmware device drivers

• SVGA Display drivers

• Vmxnet networking drivers

• Bus Logic SCSI drivers

• Memory control driver

• Vmware mouse driver

• I/O consolidate backup to quiesce I/O

3. set of scripts that let’s you automate guest OS
4. component that supports copying between the guest and managed host

If you don’t have vmware tools installed you don’t have option to restart or shutdown option. You will’ve to gracefully shutdown OS within VMs console. In order to upgrade VMWare tools, you need to shutdown VMs.

When you suspend VMs, a file is created with a. vmss extension is created, which contains the entire state of VMs. The remove from inventory cmd removes the VM only from VI client and not from from it’s datastore, however delete from disk removes it from datastore. Similiarly .vmtx is extension for template and remove from inventory and delete from datastores applies here as well. File with NVRAM extensions contains BIOS Settings.

In many cases, you can get past the problem by temporarily disabling acceleration in the virtual machine, but the applications stablize deselect “disable acceleration”

Choose Hyperthreading Sharing Mode from the pull-down menu. The options are:

Any – (default) The virtual CPUs of this virtual machine can freely share cores with other virtual CPUs of this or other virtual machines.

None – The virtual CPUs of this virtual machine have exclusive use of a processor core whenever they are scheduled to it. The other hyperthread of the core is halted while this virtual machine is using the core.

Internal – On a virtual machine with exactly two virtual processors, the two virtual processors are allowed to share one physical core (at the discretion ofthe ESX Server scheduler), but this virtual machine never shares a core with any other virtual machine. If this virtual machine has any other number of processors other than two, this setting is the same as the none setting.

Audio is available only for Windows XP and not for Window2000/windows 2003

If you wish to edit template, you’ll need to convert the template into VM.

Customization of guest OS is saved in xml file. Saved customization files are unique to each VirtualCenter Server and to each version of VirtualCenter due to encryption. You must recreate the customization files for each VirtualCenter Server. Also if you uninstall VirtualCenter and later do a fresh installation, the ability to decrypt passwords from the earlier installation is lost.

Basic System Administration

Bullet points from Basic system Administration Guide -Part01

• The text is sent as a notice message to all active session users and to new users when they log in. The MIB definition files can be found at C:\Program Files\VMware\Vmware VirtualCenter\MIBS when the default installation directory is used.

• You can perform migration between datacenters, the root folder is set as a default for every virtualcenter server, you can change the name but not add or remove it.

• ESX SERVER SUPPORTS 5 DIRECT, CONCURRENT VI CLIENT CONNECTIONS

• When you add multiple ESX servers to Virtual Center, VC will identify any naming conflicts that exist between virtual machines and alerts system administrator, who can rename virtual machines as necessary. The name can be 80 character long and may contact underscore,hypen.

• Disconnecting managed hosts differs from removing it, when you disconnect it; virtual center monitoring activities are temporarily suspended. When you remove it, VC stops are monitoring and managing functions of that managed host. Hosts and all VMs on the host are removed from the inventory but historical data is not removed from VC database.

• When adding or removing hosts, make sure NFS mounts are active, if NFS are unresponsive, the operation fails.

• Systems with dual-core processor (two CPU cores in each processor) must use ESX server 2.5.2. VC licenses are issused by pairs of processor packages not by processor cores. Host can be removed from the cluster only if all of the virtual machines on it are powered off and host is placed into maintenance mode.

• The Virtual CPUs page does not appear if the host is single-processor or the guest operating system does not support SMP (for example, NetWare and Windows NT 4.0)

• When you map a LUN to a VMFS volume, virtual center creates a file that point to the LUN.Encapsulating disk information to the file allows VC to lock the LUN so that only one Virtual Machine can write to it.

VMFS -VMWARE

CONSIDERATION WHEN CREATING VMFS

You should always have one VMFS volume per LUN, however you can have multiple smaller or one larger VMFS volume. With ESX Server you can create 1.2 GB Minimum and 256 VMFS volume per system. You can connect upto 32 ESX servers to single volume.

Environment where you should go for Larger VMFS Volume:

When you need more flexibility in creating VMs, more flexibility for resizing VMDKs,snapshots

Few Volumes better management

If you go for smaller VMFS Volume you following Advantages:

• Less wasted storage space
• Less contention on each VMFS due to locking and SCSI reservation issues
• More flexibility, as the multipathing policy and disk shares are set per LUN
• Use of MSCS requires each cluster disk resources has its own LUN

NB: Best practise would be configure few servers with Larger VMFS vols and few with smaller VMFS vols

• Maximum VMDK file size: 2 TB
• Maximum file size: 2TB
• Block size: 1 MB to 8 MB

When you add datastore, name must be unique within the current Virtual Infrastructure instances. Before creating a new datastore on a FC device, rescan a fibre channel adapater to discover any newly added LUNs.

UPGRADING VMFS 2.0 TO VMFS 3.0

When upgrading to 3.0 ESX server file-lock mechanism ensures that no remote ESX Server or local process is accessing the VMFS volume being converted. ESX Server 3.0 supports VMFS 3. VMFS-3 is not backward compatible with earlier versions of ESX server

Before you carry out upgrade process make sure

• Commit or discard any changes to VMDK
• Backup the VMDK suppose to be upgraded
• No Power ON VM is using VMFS2.0
• No ESX Server is accessing VMFS2.0 or mounted on any ESX Server

STORAGE -02 VMWARE

ESX server does not typically perform I/O load balancing across paths for a given storage device.At any given time, only single path is used, which is called as Active Path. The ESX server host automatically sets multipathing policy according to the make and model of the array it detects. If the detected array is not supported, it is treated as active/active.

Manually changing MRU to Fixed is not recommended. If you are using Fixed Policy, you can see which path is the preferred path with an asterisk mark.

It is recommended to use fixed policy when SP are active/active and for MRU should be used when SP are active/passive mode

RDM is a special mapping file in VMFS volume that manages metadata for its mapped device. Mapping file has a .vmdk extension, but the file contains only disk information describing the mapping to the LUN on the ESX server system

Benefits of Raw Device Mapping (RDM)

• User-friendly persistent name
• Dynamic name resolution
• Distributed file locking: distributed locking on a RDM makes it safe to use a shared RAW SCSI devices without losing date when two VM are accessing the same LUN.
• File permissions
• File system operations
• Snapshots
• Vmotion

In RDM there are two modes physical modes and virtual modes

In Physical mode, VMKernel passess are SCSI commands to the device except REPORT LUN command is virtualized so the Vmkernel can isolate the Lun from the owning VM.

All mapped LUNs are uniquely identified by VMFS, RDM lets you give a permanent name to a device name, which is relative to the first visible LUN. so that any change in HBA,FC failure can change Vmhba because name includes initiator,HBA,SP,LUN. Dynamic resolution can compensate this.

Vmkfstools can used for managing RDM from SVC console, typical operations are querying mapping information, create mapping file and to import or export a virtual disk.

STORAGE-Advance Concepts

For preparing VCP you first need to read the Exam Blue print available on vmware site, after going through it you would realize that one should go through

1. Basic Administration Guide
2. Server configuration Guide
3. Resource Administration

All the above guides and additional guides are available at Vi3 Documents in PDF

Below are the contents from all three guide, they are actually few important concepts rather than entire text. This blog talks about storage.

STORAGE

TYPES OF STORAGE

• Local
• Fibre Channel (FC)
• ISCSI (Hardware iniatiated)
• ISCSI (software iniatiated)
• NFS (NFS client is built-in into ESX server)

iSCSI

With iSCSI, SCSI storage cmds are send by VM to its VMDKs & are converted into TCP/IP protocol packets and transmitted to a remote device or target, that stores the virtual disk. ISCSI initiators are responsible for transporting SCSI requests between ESX Server and the target storage device on the IP Network.

There are two types of ISCSI initiators

1. Software based

2. Hardware based

Software based iSCSI initiators have a code built into VMKernel which carries out the transporting job, using software initiators, the ESX server connects to a LAN through an existing NIC card using network stacks, in short you can implement iSCSI without purchasing specialized hardware. You also need to open a firewall port by enabling the iSCSI software client service.

Hardware based iSCSI initiators requires HBA cards which are specialized to transport iSCSI cmds over LAN to the target. Currently ESX Server supports only Qlogic QLA4010 iSCSI HBA.

NB: ESX 3.0 does not support both types of initiators on single system.

Naming requirements:

IQN (iSCSI qualified name)

e.g. iqn.1998-01.com.mycompany:myserver

Format Template: iqn.<year-mo>.<reversed_domain_name>:<unique_name>

Discovery methods

Initiator discovers iSCSI targets by sending a sendtargets requested for specific target address.

Static: Only available for Hardware based iSCSI initiators, you can manually add additional targets or remove unneeded targets. If you remove a dynamically discovery static target, the target can be returned to the list the next time a rescan happens, the HBA is reset, or the system is rebooted.

Dynamic: to use this method enter the address of the target device so that the initiator can established a discovery session with this target. The target device then responds by forwarding a list of additional targets that the initiator is allowed to access.

iSCSI Security

Since iSCSI communications between initiator and target happens over TCP/IP stack, it is necessary to ensure security of the connection. ESX server supports CHAP that iSCSI initiators can use for authentication purposes.

You can’t store VM on IDE or SATA, but on SCSI,NAS or FC storage only.

VMs communicate with datastore (where vmdk is placed) using SCSI commands, SCSI commands are encapsulated into various protocols e.g. FC,iSCSI, NFS depending type of physical storage.

HBA Naming convention vmhba1:1:3:1, Hba card 1, on Storage processor 1, using LUN3 and partition 1. First 2 numbers can change but last will remain unchanged

Select a large LUN if you plan to create multiple virtual machines on it., if more space is needed you can increase the VMS volume at any time –up to 64 TB.

VMWARE HA

Clustering in VMWare is based upon customer requirements.Cluster-in-a-Box: Both the Nodes in same Physical Hosts, this type of configuration is suitable in case there is possibility of data crashes or administrative errors, but there is no cover if ESX host fails on hardware front.

Cluster-across-Boxes: Both the nodes are placed on seperate ESX host, and this takes of ESX host’s hardware failure.

Physical-to-Virtual Cluster: Here Node A is actually physical box and Node B is Virtual Machine in ESX host, acting as standby host.

VMWARE HA solutions has some advantages which not very obvious. But we should any case apply VM HA for one simple reason, if the ESX host fails, all VM’s at least get started at other host. You don’t have to manually do that. Downtime will be Non-Zero

VMHA and VC 2.0 deals only with Host failures, for VM’s (Node failure) you monitor Heart Beat using AlarmPRE-REQUISITES VMHA:

• Each host must be able to poweron VM’s i.e. Each host must have access to VM’s files, in other words all VMotion requirements are met.
• ESX server is reachable when you type it’s fully qualified domain name

For VMHA heartbeats it is recommended to set

• Two service console port on different virtual switch
• One service console with NIC teaming enable at virtual switch level

VMHA is fully integrated with DRS, which means when your host fails and all VM’s are moved to different hosts, DRS takes care of resource management. VMHA is reactive solution, which means it will act only when one or more host fails but VMDRS is proactive solution, it is always best to implement both VMHA & VMDRS

Failover capacity: When you enable cluster, two important configurations you need to do and they are again dependant upon client’s requirement.

1. Number of host allowed failures allowed
   

   Maximum is 04 and Minimum is 01. This configuration help HA to determine if there are enough resources to power on VM in the cluster. But it is we who decided how much redudant capacity to be made available.

2. Admission Control
   1. Do not power ON VM if they violate availability constraints (Selected as default option)
   2. Allow virtual machines to be powered on if they violate availability constraints

Depending upon adminission control option you select, VM will be either powered ON or NOT. These values help VMHA to balance and calculated enough resource across hosts in case there is any host failures. Current failover capacity under Cluster’s summary tab informs how many hosts are available at that time to hold the VM’s

We only need to provide number of host, rest like resources required to power on VM’s across these host or only 1 host is alive, decision like this is taken by VMHA. If resources are not enough VMHA wouldn’t all VM’s to be powered ON(default option). You can force VMHA to start VM’s(when you like the constraints to be voilated), in this case Cluster will show RED sign, which means failover might not be guaranteed. It is not recommended that you work with red clusters. Also if you have 3 hosts and 2 fails cluster will turn RED.

So when you enable VMHA, you should design in such a way that hosts in ESX will be able to handle additional VM’s without any over utilization of resource.

For example: Two ESX Host having equal capacity handling 50 VM’s each. We should design in way that each Host should be able to handle 100 VM’s.

VMWARE HA

Clustering in VMWare is based upon customer requirements.Cluster-in-a-Box: Both the Nodes in same Physical Hosts, this type of configuration is suitable in case there is possibility of data crashes or administrative errors, but there is no cover if ESX host fails on hardware front.

Cluster-across-Boxes: Both the nodes are placed on seperate ESX host, and this takes of ESX host’s hardware failure.

Physical-to-Virtual Cluster: Here Node A is actually physical box and Node B is Virtual Machine in ESX host, acting as standby host.

VMWARE HA solutions has some advantages which not very obvious. But we should any case apply VM HA for one simple reason, if the ESX host fails, all VM’s at least get started at other host. You don’t have to manually do that. Downtime will be Non-Zero

VMHA and VC 2.0 deals only with Host failures, for VM’s (Node failure) you monitor Heart Beat using AlarmPRE-REQUISITES VMHA:

• Each host must be able to poweron VM’s i.e. Each host must have access to VM’s files, in other words all VMotion requirements are met.
• ESX server is reachable when you type it’s fully qualified domain name

For VMHA heartbeats it is recommended to set

• Two service console port on different virtual switch
• One service console with NIC teaming enable at virtual switch level

VMHA is fully integrated with DRS, which means when your host fails and all VM’s are moved to different hosts, DRS takes care of resource management. VMHA is reactive solution, which means it will act only when one or more host fails but VMDRS is proactive solution, it is always best to implement both VMHA & VMDRS

Failover capacity: When you enable cluster, two important configurations you need to do and they are again dependant upon client’s requirement.

1. Number of host allowed failures allowed
   

   Maximum is 04 and Minimum is 01. This configuration help HA to determine if there are enough resources to power on VM in the cluster. But it is we who decided how much redudant capacity to be made available.

2. Admission Control
   1. Do not power ON VM if they violate availability constraints
   2. Allow virtual machines to be powered on if they violate availability constraints

Depending upon adminission control option you select, VM will be either powered ON or NOT. These values help VMHA to balance and calculated enough resource across hosts in case there is any host failures. Current failover capacity under Cluster’s summary tab informs how many hosts are available at that time to hold the VM’s

We only need to provide number of host, rest like resources required to power on VM’s across these host or only 1 host is alive, decision like this is taken by VMHA. If resources are not enough VMHA wouldn’t all VM’s to be powered ON. You can force VMHA to start VM’s, in this case Cluster will show RED sign, which means failover might not be guaranteed. It is not recommended that you work with red clusters. Also if you have 3 hosts and 2 fails cluster will turn RED.

VCB -Backup Strategies

There are two things when you think of backup of virtual machines

• Application back which is called File Level Backup
• Entire VM back Image Level Backup, ( which is quite easy, cause you just need to back VMDK)

File Level Backup: It is recommended that you put all your data in Non-System Disk, it brings it’s own advantages.

Backup Proxy Server is required for carrying out this task, this has been implemented especially to remove backup overheads from ESX/VM’s. This would be Windows 2003 server, with Backup software installed(for example netbackup) which has VCB plugin to carry out the task.

COMPONENTS involved in VCB back process are:

• Hostd: On ESX Server and interacts with Virtual center
• VM to be backup:
• Backup Proxy server with 3rd party software installed on it.
• VCB Framework which consists of
  • vcbMounter
  • vLUN driver
  • Integration module
    • Pre & Post-backup scripts which ties with
  • Backup application (for example Netbackup)

---

VCB WORKFLOW:

• Backup application starts backup job as per pre-schedule time
• Pre-backup script is intiated by backup software 
  • Quiesces NTFS/FAT (only in case of MS Guest OS),this ensure no write operations are pending
  • Puts VM in snapshot mode
  • Snapshot is taken and put’s VM into normal opertion
  • Backup software mounts this Snapshot for File Level Backup, and selected files are copied. (Done by Backup client)
  • For Image level back, entire disk is export to Backup proxy server.(Done by backup client)
• Post-backup script is called
  • which unmounts VM snapshot from backup proxy
  • Takes VM out of snapshot mode, commits any changes made to the disk during the snapshot mode.

Restoring backups done using VCB approach

Restoring file/Images taken via backup is not straight forward. There are three approaches for this

• Self-Service restore : Backup agent Installed on each and every VM
• Per-group restore : Select VM’s which will do restore work (i.e. install backup agent only these VM’s) and then get someone to restore files of those specific VM’s
• Centralized restore: Backup agent is installed only on Backup Proxy and restore file/Image on backup proxy. After you can uses windows share to copy data over the location

These approachs differs from each other at one level i.e. present of backup agents. Backup agents here are only doing restore work.

For Image level backup you can use VCBMounter to backup entire virtual machine in the service console. VCBMounter quiesce the snapshot of the VM and export the setfiles which can be later on used to restore using VCBRestore. For file level back you have to use third party backup software. This can be done only from Service Console.

Shares,Reservation,Limits -CPU/Memory Resource settings

Before we talk about Resource Pool we need to know what are resources in VMware terminology. Resources are

• CPU’s
• Memory
• Disk (Manages which proportional share mechanisim)
• Network (are controlled by using Network-traffic shaping)

Resources are provided by ESX host and they are consumed by VM’s. if ESX host are clustered, then Cluster is actual resource provider.

Every VM will be allocated Memory and CPU resources, there are 3 settings in resource pool which influence as to what memory and CPU any VM would get.

RESERVATION:Reservation specified for the resource pool or VM, if the reservation is set to Zero (which is by default), it means no reservation is set. VM will not start if its reservation is not met or guaranteed. When reservations are not utilized ESX host can assign them to other VM’s. Lets take example

You have VM’s A and B. A & B is configured for 1GHZ reservation. Now during some days A only goes as far a 0.5 GHz, under such case B can use 1.5 GHz. But if B is using 1.5GHz and A is poweroff, Once A is powerON B has to give away 0.5 GHz

LIMIT:Specified limit for the VM, default is unlimited. Server can allocate more memory/CPU than reservation but it cannot assign more than it’s limit. Applying limit will vary depending upon the circumstances or your design requirement.

It is recommended to apply limit, when you wish to manage few VM’s and you know Max Memory/CPU utilization of these VM’s applications. Because one you apply limit, even if the resource on one system is under utilize and other machine is going to require more CPU/Memory sometime, it won’t be able to get that idle resource and there are chance paging would start happening.

SHARES: Number of shares determine which VM will get resources when there is competition for resources among VM’s.Specifying shares makes sense only with regard to sibling virtual machines or resource pools, that is, virtual machines or resource pools with the same parent in the resource pool hierarchy.

The amount of resources represented by each share changes when a new virtual machine is powered on. This affects all virtual machines. Shares are typically specified as high, normal, or low. High, normal, and low specify share values with a 4:2:1 ratio.

For example:Two virtual machines run on a host with 8GHz. Both are set to Normal and get 4GHz each.A third virtual machine is powered on. It is set to High, which means it should have twice as many shares as the machines set to Normal. The new virtual machine receives 4GHz and the two other machines get only 2GHz each.

Share values default to:

• High — 2000 shares per virtual CPU, 20 shares per MB of virtual machine memory
• Normal — 1000 shares per virtual CPU, 10 shares per MB of virtual machine memory
• Low — 500 shares per virtual CPU, 5 shares per MB of virtual machine memory

By going by above defination, if we’ve VM with 2 CPU’s and 1 GB RAM and shares is defined as High, so this VM is going to get

• 2000 Shares x 2 CPU’s =Shares of CPU Power
• 20 Shares x 1024MB=Shares of Memory

Mind you, this is just shares and they will be useful in cases where VM is competing either for CPU or Memory resources.

Resource pool is used to distribute CPU’s and Memory across VM. resource pools can have child resource pool or VM or Both.Resource pool or VM which are at the same level of heirarchy, it is called sibling. Picture below gives very good information about the relation.

You can actually create resources pool as per the department’s requirement and can guarantee that particular resource pool will provide required processing power when the condition demands.

You don’t need to configure resources to each VM, in fact you apply common settings across group of VM using resource poo. In above figure RP-Marketing is a resource pool, all resources defined in it, will be automatically applied to VM’s under it, with configuring such setting for each VM.

In order to really leverage the power of resource pools, Group hosts into cluster. When you create cluster, resource are managed at the cluster level rather host level. In short resources on each host are combined into one. This one resource is total resource which can be allocated to all VM’s under that cluster. Figure below explains it briefly.

VirtualCenter Security Model

Now that we have installed virtual center, next steps would be assigning permission to all those people who are responsible for managing managing VMware Infra 3.0. In order to do that we need to understand how the permissioning works. There are two elements in this, First is ESX host and other is Virtual Center. Permissioning on both these element is seperate and cannot be mix with eachother. For simple reason, one is Linux and other is Windows.Security Model explained.

Let take user Greg, who works in first line support and need maximum rights to shutdown VM in case it hangs or user request.

Greg ——->Needs to Reset VM’s ——->To achieve this we need to assign permission

[ User ]                [ROLE ]                                      [Priviliges]

1. Needs to Reset VM’s = TASK [ROLE]
2. In order to do the TASK=Need to assign Permissions
3. USER

All three makes Permissions in VMWare and in all security model. However to little bit more to it, permission is also a combination of user account, Role,priviliges and position in the inventory to which the user/role applies.

Now Greg can be restricted to do Datacenter, VM. We can decide whether we need same permissions to flown across the datacenter or to specific folder. This is called as propogations of permissions. VMware has come with pre-defined roles, these roles are can been seen when you assign permission. You have the option of selecting the pre-defined roles or create one for yourself. But these pre-defined roles are again differ from ESX and Virtual Center perspective.

Predefined ESX Servers Roles:

1. No Access
2. Read-Only
3. Administrator

Predefined Virtual Center Roles:+ Predefined ESX Servers Roles

1. VM Administrator
2. Datacenter Administrator
3. Virtual Machine Power User
4. Virtual Machine User
5. Resource Pool Administrator

But customs roles can be created for both ESX aswell VC.

Virtual Center Security Model:

Virtual center security model includes accounts created in Windows which could be local or domain account. This account is again assigned role which is again decided at what heirarchy you apply this role. Default permission for VC is assigned to local Administrators groups of Windows 2003 server at the top level in the inventry.

ESX Security Model:

ESX security model includes user account created on ESX Server which is basically a linux user account. This account is again assigned role which is again decided at what heirarchy you apply this role. By default vpxuser and root are already created and assigned to administrator roles. Vpxuser is used for interacting ESX server. Root is admin account and performs task  assigned by virtual center.

Step-by-Step process of assigning permissions:

Select object on which you wish to apply permisison.

Expand the inventory

Right the click object, select add permission

Select role to be select from predefined list or select custom roles

Select if you wish to propogate the permission to child objects

Select user (Local/Domain) user

Add the user to users or group fields

 In order to create custom roles, go to the admin tab, right anywhere

Name the role and select priviliges you wish to give it to the role

There is lot in permissioning, I will update that later on

VMware -VM Management

You can move VM’s between ESX servers. Moving VM’s when they are power off it is called Cold Migration. So what would be Hot Migration, nothing but VMotion. We will deal with it in more details in later posts. Cold migration of VM gives you option to move the files to different datastores, but in most cases Datastores are shared and ESX shouldn’t have problem in accessing those files. In case situation where it is not possible to access these files, files are also moved along.So when you perform Cold Migration ?

• When you would like to move VM’ to local datastore of ESX server
• And when VM’s are to be moved to two different CPU families
• Or when are upgrading your ESX hosts

Adding Devices:

VM need to be Power Off for adding most of the devices, Except for Hard disk, which is called Hot Plug. In case you need to add additional NIC, you can do only when the VM is power off. In order add device, Power off VM. In the summary tab, click Edit settings, VM properties are displayed. Click on add hardware Wizard and select the device you would like to add.

Adding SCSI Device

Select Device, remember to tick Device Status 

Select Device to be mapped on VM

Summary to finish

More Information:

http://pubs.vmware.com/vi301/wwhelp/wwhimpl/js/html/wwhelp.htm and Search for add device

VM Provisioning -02

Cloning a VM is equivalent to deploying VM from template. Here as well you get option to customize guest OS. To clone VM, simply right click VM, say clone this VM. In order to enable Guest OS customization.In order to enable guest OS customization, you need to configure Virtual center. Download latest sysprep tool from Microsoft, this customization applies only for Windows,for Linux it is inbuilt. Launch the installer and extract the contents to C:\Documents and Settings\All Users\Application Data\Vmware\VMware VirtualCenter\sysprep1.1

Some folders as per OS configuration are already created under this folder, extract them into respective folders.

Guest OS customization would look like

1) Enter Name and Organization

2) Guest OS Name, you have various option to select from, Select as per your organization’s policy

3) Enter Product ID and License information

4) Enter Administrator Password, also in the same screen if you would like Admin to logon and how many times, you select.

5) Pick up appropriate time zone

6) Run once keep default

7) Workgroup or domain select as per your organization policy

Operating system options

Generate SID’s

Delete all user accounts

9) In the end you have option to save customization for later use and it finishes OS customization part.MORE INFORMATION:

http://pubs.vmware.com/vi301/wwhelp/wwhimpl/js/html/wwhelp.htm

VM Provisioning

Template is used for provisioning of VM’s in VMware. Provisioning is inbuilt feature in VMInfra 3.0. Technically Template is a VM which cannot be powered ON. Templates/VMDK can be stored in Monolithic form or sparse file format.

What is monolithic or spare file format ?

lets take a example:  If you create a monolithic file format of 16 GB, totally 16 GB is claimed  at one go, But if you create sparse file format 16GB will be consumed as and when it is utilized. Templates can be stored in NFS/SVC Console/VMFS

Templates can be created in two ways

• Clone to template – Original VM is retained.
• Convert to template – converts VM to template

DEPLOY VM FROM TEMPLATE

 Connect to Virtual center via VI client

Change the view to VM and Templates

Right click the template and select deploy this VM from this template.

Wizard will ask you VM’s Name and Host on which you want to put this VM

Next select resource pool

Last you get option to customize OS.

You can select YES /NO depending upon your choices.

1) Select Template

2) Select Datacenter and ESX Host

UPDATING TEMPLATES:

In case you need to include latest Hotfix/Patches into templates, you easily do it. In order to achieve this task. Select template and select convert to virtual machine. Once VM is powered ON, apply patches and etc. Convert the same VM back to template.

VMWare -VM Creation

The virtual Machine’s display name is used to name the files of the Virtual Machine itself. Therefore avoid using special characters, including spaces, in the VM’s display name. When you create virtual machine you have by default 6 PCI slot, one reserved for Video Adapater and therefore effectively you have 5 to work on.Virtual chipset is Intel 440X-Based motherboard with an NS338 SIO chip. This ensure wider range of OS compatibility.

RESOURCE ALLOCATION TO VM:

Maximum of 16 GB RAM and 4 CPU can be allocated to VM. But it is recommended against allocated more than 1 CPU to VM unless and until application on VM is going to make use of it. More CPU is allocated, more hard CPU scheduling has to work, which might cause overburden on Physical CPU.

VM Creation:

In order to create VM you need to have following details handy

• Location where to Store VM’s files. i.e. VMDK file
• Location where the ISO Image of the Guest OS is kept.
• VM’s Name, it Location in Datacenter
• Number of Processor,Memory Allocation,Disk Size
• NIC to connect to
• Virtual Device Node and Disk Mode

What is Disk Mode?

In simple word they way you wish VM to react to changes made to it. If you want to those change permanent select Persistent Mode or select Non-Persistent mode. In non-persistent mode, all changes made to VM is lost when you powercycle VM. Both Persistent and Non-Persistent mode comes under Independant category. There is other category called as Snapshots. Snapshots is selected by default and it allows to take snapshot of disk, you can restore the snapshot, in case you had undesirable result of changes you made.

Virtual Device node asks how you wish to connect VMDK to VM.

More detail Information is available at http://pubs.vmware.com/vi301/wwhelp/wwhimpl/js/html/wwhelp.htm

Step-By-Step Procedure for creation of VM ScreenShots

1) Virtual Machine Name and Select Datacenter where VM should reside

 

2) Select ESX server which will Host VM

 

3) Memory selection

4) Where your VM files, configuration files resides

5) Select OS version to Install

6) Select NICs and remember to select connect a power ON

7) Select disk Capacity

Select destination where you wish to store VMDK

9) Select Virtual Disk Node

More information in detail is availabe at http://pubs.vmware.com/vi301/wwhelp/wwhimpl/js/html/wwhelp.htm

VMWare -Virtual Center -03

Virtual center can be access using VI client or Web access(With Limited functionality), Web Access provides browser based interface for managing VM’s.Hostd and VPXA are two services which are running on ESX host responsible for performing tasks assigned by Virtual Center.

Virtual Center deployment:

Virtual Center with Minimum Hardware requirement can Handle

———>20 concurrent connections

   ——->50 Managed Hosts

     —–>1000 VM’s

With Dual CPU and 3Gig RAM

———>50 concurrent connections

   ——->1000 Managed Hosts

     —–>2000 VM’s

 BACKUP Strategy for VirtualCenter Server:

Virtual Center is recommended to be on Physical Box, as a DR strategy you can created one VM and leave it powered off. Use it only when Primary fails. When Primary fails, Power On DR server, Point it to Virtual Center database. Have System Admins point to DR server till you bring back the Primary.

Other recommended strategy is to use Cluster capabilities of SQL database.

Technorati tags: VMWare, Virtual Center

VMWare -Virtual Center -02

The VMware License Server and Virtual Center Server typical reside on the same system.

The inventory hierarchy is used to group your hosts and virtual machines in meaningful way. It also provides the natural structure upon which you apply permissions. Datacenter is aggregration of all the different types of objects needed to work in a virtual Infra,Hosts,VM,Networks and Datastores.

Datacenter can be divided on the basis of Geographical locations by creating folder inside it or as per your convenience.But make sure you design in way which will allow to delegate roles and responsibilities for Managing VMInfra.

You can Group them on basis on

• VMotion requirement
• To form single pool of resources
• Single Administrative control

 Typically a datacenter consists of Managed Objects Viz:

• Virtual Machines
• Hosts
• Virtual Machine Templates

Tasks such as Cloning VM’s,deploying VM’s from templates or Migrating VM’s can be only be performed with objects in the same datacenter.

VM,VMTemplates can be organized based on Functions and departments. CPU family,Application servers,Infra Servers. Below Image is very example of it.

In order to use feature like VMHA,VMDRS  we have to cluster servers.

Above view can change according to our needs. It is categorized as

1. Hosts and Clusters
2. VM and Templates
3. Networks
4. Datastores

First two are most commonly used. In above image it is Hosts and Templates. In above examples Hosts are group into folders viz

Racks

==>    Hosts

      ====>Server types (Messaging,SQL,IIS)

In order to add Host to Virtual Center, you need

1. FQDN of ESX Host
2. Root Password or equivalent user account

once you add ESX host, change the license type to Server based pointing it to License server

VM Networking

Virtual Switches uses software constructs implemented by VMKernel. VMKernel itself uses virtual switches to access iSCSI and NAS based storage and to implement VMotion.

NIC Team, which is simply a virtual switch connected to 2 or more Physical NIC’s. And NIC team provides automatic distribution of packets and failover.

Each Switch is internal LAN, implemented entirely in software by the VMKernel. Internal only switch is used for network isolation for testing purpose for example Anti-virus software and IDS and One Box Firewall enviornment.

Default number of ports on switch is 54, however one created during installation has 24 ports and maximum limit is 1024.

The simplest way to give virtual machine access to network is to make virtual switch and associated it with outbound Phyical Nic. High performance application can benefit from NIC teaming which offers higher bandwidth and provide automatic load balancing and network fail over.

There are three types of Network connections

• Service console:Managing ESX hosts
• VMKernel Port:For managing iSCSI and NAS devices
• Virtual Machine port group:For accessing VM Networks

More than one connection type can exist on a single virtual switch. Seperate IP Stacks are configured for the service console and the VMkernel, which means each port must be assigned with it’s own IP address.

When creating new virtual switch you have to specify connection type.

All virtual switches are known as vSwitch# (remember S is captial, since LINUX is case sensitive), Each Port or Port group has a network label

While Service console port are known as vSwif#

Virtual Switch Properties

General: Allows to configure number of ports

Network Policies: VLAN,Security,Traffic Shaping and NIC Teaming.

Network policies for virtual switch becomes default policies at port and port group level, which can be override at the respective level.

To change the speed of the NIC card,

Configuration Tab->Networking ->Properties->Network Adapters->Edit

 LittleBit about VLANS

VLAN are a network layer 2 concept (the same layer at which MAC address and Ethernet live, one layer below IP addressing and routing). Smart L2 switches that can keep track of which ports belongs to which VLAN. In order to extend VLAN across switches, a trunk link must interconnect switches.

ESX server provides VLAN support through virtual switch tagging, which is simply provided by giving port group VLAN ID. Then VMKernel takes care of all tagging and untagging as the packets pass through the virtual switches. VLAN ID is optional by default.

Security:There are three security policies exception; Promiscuous [Default -Reject];MAC Address Changes [Default -Accept];Forget Transmits; [Default -Accept]

Traffic Shaping:VM’s Network Bandwidth can be controlled by Traffic Shaping. Traffic shaper only controls outbound network traffic only.To control inbound traffic, use a load-balancing system, or turn on rate limiting features of your router. Network traffic shaping is off by default. That is, each VM can consume as much outbound traffic as its guest is configured for. VM subject to these controls may exceed its average bandwidth and spike up to its peak bandwidth -but only enough to transmit data defined in Burst Size. 

Port Group level: If you set average Bandwidth at 1000 kbps on a port group, then any VM connected to that port group can use an average bandwidth of 1000kbps

VMWare -License Information

Host-based licensing

Advantages:

One less piece of Infrastructure

sufficient for small organization

Disadvantages:

Licenses do not float

features which requires virtual center cannot be used.

ESX server features do not require Virtual Center nor a license server and are transferable.

There are two types of licenses, Starter and Standard

Starters License :

1. Only 4 Processors
2. 8 GB RAM
3. No fiber channel or iSCSI storage available only Local and NAS
4. VMFS (Virtual Machine File System) only on local storage
5. Virtual Center Agent

However you would be able to add few features using add-on license, seperate charges apply

Standard License :

1. No limitations on RAM and Processor
2. Local/NAS/SAN/iSCSI storage available
3. Virtual center agent
4. VMFS

Enterprize License :

Standard License +

1. VMotion
2. VMWare HA
3. VMWare DRS
4. VCB (VMware consolidated Backup) (This is also available as add-on cost license)

More information could be found at Doc ID: 5357713 under VMTN

VMWARE – Virtual Center

Virtual Center has

• Core Services

1. Mgmt of Resources
2. VM’s
3. Mgmt Alarms,Events
4. VM Provisioning
5. Host and VM Configuration

• Distributed Services

1. VMotion
2. VMware DRS
3. VMWare HA

Database Interface

Active directory Interface.

Order of Installation

1. Database Server (Create a connection to SQL or Oracle Database
2. License Server
3. Virtual Center
4. VI Client -in Windows World RDP software

Virtual center database contains

• configuration information
• Current Status and
• utilization data of the managed Hosts and Virtual Machines

If you are using VC then you must use License server for completetly using its features

If License server is not available then still VMWare Infra can survive for 14 days of grace period.

There are three software editions

• Starter
• Standard and
• Enterprize.

License based model is named similiarly

Starter and Standard. However standard license licenses both standard and Enterprize mode.

When you install License server, Following services is seen in Services.msc console.

VMWare License Server

Similiarly when you install Virtual Center Following services is seen in Services.msc console.

• => VMware Virtual Infrastructure Web Access
• => VMware Virtual Mount Manager Extended
• => VMware VirtualCenter Server

If Management Server must go through firewall Open Port 902

VMFS

VMFSVMWare file system is a file system optimized for storing VM’s. A virtual disk stored on a VMFS always appears to the virtual machine as mounted SCSI device. VMFS store is used to ISO Images,templates.

VMFS volumes are accessible in the service console underneath /vmfs/volumes directory

To create VMFS datastore

Configuration tab ->

- > Hardware

o Storage(SCSI,SAN and NFS)

§ Add Storage

Adding extends to datastore

Datastore can span upto 32 physical disks. You generally wish to add extend when VM’s need more space or you need to create more space.

To add one or more extend to the datastore

Configuration

Storage

Properties

Volume properties

Extends

Select the disk which you want to add as an extend and click next

If disk or partition you add was formatted previously, it will be reformatted and loose file systems and any data it contained.you have the option to decided the disk space to utilize.

To remove extends you will have to delete the entire VMFS, to remove VMFS, select VMFS and click remove. Make sure there no running VM’s on it. Removing datastore from the ESX server breaks the connection between system and storage device that holds the datastore and stops all functions of that storage device.

Managing Paths for Fibre Channel and iSCSI

ESX Server supports multipathing to maintain a constant connection between the server machine and the storage device in case of the failure of an HBA, switch, storage processor (SP), or cable. Multipathing support does not require specific failover drivers.

To support path switching, the server typically has two or more HBAs available, from which the storage array can be reached using one or more switches. Alternatively, the setup could include one HBA and two storage processors so that the HBA can use a different path to reach the disk array.

By default, ESX Server systems use only one path from the host to a given LUN at any given time. If the path being used by the ESX Server system fails, the server selects another of the available paths. The process of detecting a failed path and switching to another is called path failover. A path fails if any of the components—HBA, cable, switch port, or storage processor—along the path fails.

The process of one HBA taking over for another is called HBA failover. The process of 1 SP taking over SP2 is called SP failover. VMware ESX Server supports both HBA and SP failover with its multipathing capability.

Setting Multipathing policies for LUN’s

MRU: Most recently used: [Default] which means once failover occur, we do not automatically failover. Recommended under Active/Passive storage devices

Fixed: means ESX server will always try to use preferred path. Recommended under active/active storage devices

The ESX Server host automatically sets the multipathing policy according to the make and model of the array it detects. If the detected array is not supported, it is treated as active/active.

NAS and NFS

NAS is a specialised storage device that connects to a network and can provide file level access services to an ESX server. VMWare only support NFS for access file system over network.

NAS is low cost and less infrastructure investment required than FC. NFS volumes are treated just like VMFS volume, can hold ISO/Templates and VM’s. ESX server supports

- VMotion

- Create VM

- Boot virtual Machines

- Mount ISO files

- Create virtual machine snapshots on NFS mounted volumes. The snapshot feature lets you preserve the state of the virtual machine so you can return to the same state repeatedly.

NFS client built into ESX server lets us access NFS Server and use NFS volume for storing VM’s.

When ESX Server accesses a virtual machine disk file on an NFS-based datastore, a special .lck-XXX lock file is generated in the same directory where the disk file resides to prevent other ESX Server hosts from accessing this virtual disk file. Don’t remove the .lck-XXX lock file, otherwise the running virtual machine will not be able to access its virtual disk file.

NFS and Permission

ESX server must be configured with a VMKernel port defined on a virtual switch. VMkernel port must be access NFS server over the network.

/Etc/Exports defines the systems allowed to access the shared directory. The options used in this file are :

Name of the directory to be shared

Subnet allowed to access the share

The root squash feature maps root to a user with no significant privileges on the NFS server, limiting the root user’s abilities. This feature is commonly used to prevent unauthorized access to files on an NFS volume. If the NFS volume was exported with root squash enabled, the NFS server might refuse access to the ESX Server host. To ensure that you can create and manage virtual machines from your host, the NFS administrator must turn off the root squash feature or add the ESX Server host’s physical network adapter to the list of trusted servers

If the NFS administrator is unwilling to take either of these actions, you can change the delegate user to a different identity through experimental ESX Server functionality. This identity must match the owner of the directory on the NFS server otherwise the ESX Server host will be unable to perform file level operations. To set up a different identity for the delegate user, acquire the following information:

• User name of the directory owner

• User ID (UID) of the directory owner

• Group ID (GID) of the directory owner

The delegate user is configured globally, and the same identity is used to access to every volume.

Setting up the delegate user on an ESX Server host requires that you complete these activities:

• From the Users & Groups tab for a VI Client running directly on the ESX Server host, either:

• Edit the user named vimuser to add the correct UID and GID. vimuser is an ESX Server host user provided to you as a convenience for setting up delegate users. By default, vimuser has a UID of 12 and a GID of 20.

• Add a completely new user to the ESX Server host with the delegate user name, UID, and GID.

You must perform one of these steps regardless of whether you manage the host through a direct connection or through the VirtualCenter Server. Also, you need to make sure that the delegate user (vimuser or a delegate user you create) is identical across all ESX Server hosts that use the NFS datastore.

To change the virtual machine delegate

1 Log on to the VI Client through the ESX Server host.

2 Select the server from the inventory panel.

1. The hardware configuration page for this server appears with the Summary tab displayed.

3 Click Enter Maintenance Mode.

4 Click the Configuration tab and click Security Profile.

5 Click Virtual Machine Delegate > Edit to open the Virtual Machine Delegate dialog box.

2. Enter the user name for the delegate user.

3.

6 Click OK.

7 Reboot the ESX Server host.

After you reboot the host, the delegate user setting is visible in both VirtualCenter and the VI Client running directly on the ESX Server host.

Before you begin access NFS datastore you have to create VMKernel port manually.VMkernel port can be created on an existing virtual switch or as new connection on a new virtual switch

Best Practices of Resizing Windows Virtual Machine Disks on VMWARE ESX Server 3

Expand:

Workaround 1 : vmkfstools + diskpart ( additional VM need)

http://vmprofessional.com/index.php?content=2k3resize

Workaround 2: VMWARE  Converter (Best and Easy)

Workaround 3: vmkfstools + Parted LiveCD (for no VMWARE Converter users or VMWARE Converter doesn’t work)

Step 1) Power off Virtual Machine

Step 2) Expanding the VMDK

Log into the ESX server via ssh (use Putty) Find the .vmdk file you want to expand and use vmkfstools to expand it:

  

vmkfstools –X xxxxG /vmfs/volumes/file_system_label/dir/myDisk.vmdk

notes: chance xxxx with whatever size you want it to be

Sample :

vmkfstools -X 30g  /vmfs/volumes/ESX01_LOCAL01/WSOEx64/WSOEx64.vmdk

ls -lah WSOEx64.vmdk

Step 3) Power up the virtual machine

Step 4) Go to command prompt type “chkdsk” – This is critical – it may cause issues if you skip this step.

Step 5) Download the Parted LiveCD from  here (A linux based free GUI tool like Partition Magic
VM2008 Notes: 0.3.4-8 doesn’t work for VMWARE machine, but 0.3.4-7 works well

Step 6) Mount the downloaded ISO as the CD-Rom of the Virtual Machine. You can mount the ISO directly or burn it to CD then mount the CD-Rom to the Virtual Machine.

Step 7) Restart the virtual machine and boot from the CD. (You may have to hit ESC to bring up the boot menu.)

Step Choose your Language and Keyboard Layout

Step 9) Click the partition you want to expand then apply change

Step 10) Reboot, Power up the virtual machine

====================================================

Shrink:

Workaround 1: VMWARE  Converter (Best and Easy)

Workaround 2: Parted LiveCD + cloning utility (Ghost) (for no VMWARE Converter users or VMWARE Converter doesn’t work)

Step 1) Power off Virtual Machine

Step 2) Download the Parted LiveCD from  here (A linux based free GUI tool like Partition Magic
VM2008 Notes: 0.3.4-8 doesn’t work for VMWARE machine, but 0.3.4-7 works well

Step 3) Mount the downloaded ISO as the CD-Rom of the Virtual Machine. You can mount the ISO directly or burn it to CD then mount the CD-Rom to the Virtual Machine.

Step 4) Restart the virtual machine and boot from the CD. (You may have to hit ESC to bring up the boot menu.)

Step 5) Choose your Language and Keyboard Layout

Step 6) Click the partitions, just shrink the partition, no need to move partitions from right to left to release the free space near the right(end) of vmdk file.

 Step 7) Power off Virtual Machine

Step create a new vmdk file at the desired size
vmkfstools -a lsilogic -c 15G HA_TEST_small.vmdk
ls –lah HA_TEST_small.vmdk

Step 9)  mount the new vmdk in your VM, and use a disk imaging/cloning utility (ghost, trueimage,etc) to move the VM’s primary disk contents to the secondary disk

Step 10) detach the old vmdk from your VM, Power on the VM

VM2008 Notes:: Old solution : Parted LiveCD + vmkfstools  doesn’t work under ESX Server 3

Migratie Stappenplan ESX 2.5X naar VI3

Stappenplan

1. Inventarisatie maken van de huidige omgeving.
2. Een back-up maken van de oude VC 1.x Server.
3. De VC 1.x Server deïnstalleren.
4. VC 2.0.2 installeren en configureren.
5. Back-up maken van alle VM’s.
6. Server / SAN hardware bijwerken met de nieuwe firmware en bios updates.
7. Een ESX 3.0.1. server opbouwen die beschikking heeft over een VMFS3 lun.
8. Binnen alle Windows VM’s alle partities defragmenteren.
9. De VM’s clonen of via DMotion overzetten naar een ESX 3.0.1 server met een VMFS3 lun.
10. De Virtuele Hardware van de VM’s upgraden naar VI3.
11. VMware Tools upgraden op de bijgewerkte VM’s
12. De overige ESX Servers opnieuw installeren met ESX 3.0.1. (bijvoorbeeld via UDA).
13. De “nieuw” geïnstalleerde ESX Server configureren in VirtualCenter
14. De overige VMFS2 Luns upgraden naar VMFS3.
15. De configuratie van de ESX3 Servers nalopen aan de hand van je inventarisatie van stap 1.
16. De VMs terug clonen of via VMotion terug plaatsen.
17. De verschillende logfiles binnen de VI3 omgeving nakijken.

Het stappenplan zal ik nu nader beschrijven. Tevens vindt je zo nu een dan een verwijzing naar een andere blogpost.

Stap 1: Inventarisatie maken van de huidige ESX 2.5.x omgeving.

Wat heb je nodig in je nieuwe ESX 3.0.x omgeving?

• IP-Adres Service Console.
• IP-Adres VMotion.
• nieuw VMKernel IP-Adres. Dit adres heb je nodig als je gebruik wilt maken van iSCSI of NFS datastores.
• Virtuele Switches en dan gaat het vooral om de naamgeving.
• Bios en Firmware levels van de Servers. Storage, HBA’s en Switches.

Stap 2 & 3: Back-up en Deïnstalleren van VC 1.x.

Stap 2 & 3 spreken voor zich echter heb ik nog een tip. Verwijder eerst alle ESX Servers uit VC 1.x zodat de ESX Servers weten dat ze niet meer beheerd worden door een VC server. Doe je dit niet heb je kans dat je bij het toevoegen van de ESX Servers op de VC2 server de melding krijgt dat deze al beheerd wordt door een andere VC.

Maak vervolgens een back-up van je database. Zodra dit klaar is deïnstalleer je VC 1.x.

Stap 4: Installeren en Configureren van VC 2.0.x.

Het installeren van VMware VirtualCenter 2.0.x (de laatste versie is 2.0.2.) spreekt eigenlijk voor zich. Wil je wat meer informatie dan raad ik je aan de volgende blogpost te lezen: http://thelifecode.blogspot.com/.

Stap 5: Een Back-Up maken van alle VM’s.

Dit kun je op verschillende manieren doen:

1. vmbk.pl meer informatie vindt je hier: http://www.vmts.net/
2. esxranger meer informatie vindt je hier: http://www.vizioncore.com/
3. esxpress meer informatie vindt je hier: http://www.esxpress.com/

Zoals je misschien al wist gebruik ik vmbk.pl voor mijn back-up’s.

Stap 6: Updaten van Bios & Firmware van de Servers, HBA,s en SAN.

Dit is per hardware fabrikant verschillend. Weet je niet zeker waar je mee bezig bent besteed het dan uit. De kans op onherstelbare schade is groot.

Stap 7: Een tijdelijk ESX3 Server opbouwen.

Het doel van deze Server is het tijdelijk huisvesten van de VM’s zodat je de Virtuele Hardware en de VMware-Tools kunt upgraden. Ook dient deze constructie voor een extra Back-up.

Stap 8: Binnen de Windows VM’s alle partities defragmenteren.

Defragmenteer alle partities met Power_Defragmenter_GUI.

Stap 9: De VM’s overzetten naar een ESX 3.0.1 server.

Log in op de VC 2.0.x Server. Sluit de VM die wilt gaan verplaatsen “netjes” af. Vervolgens selecteer je de VM met de rechter muis knop en kies je voor de optie Clone of Migrate als je gebruik wilt maken van DMotion. In dit voorbeeld gaan we aan de slag met de optie Clone. Dit omdat de tijdelijke server die wij gebruiken alleen lokale storage heeft.

Kies een map om de VM in te plaatsen.

Kies de tijdelijke server.

Kies de datastore waar de VM naar toe gecloned dient te worden.

De VM hoeft niet aangepast te worden.

Let erop dat de VM niet automatisch opgestart wordt. Want na de clone operatie upgraden we gelijk de Virtuele Hardware.

Het clone proces is bezig.

Stap 9: De Virtuele Hardware Upgraden.

Kies de nieuwe VM die aangemaakt is na het clone proces en klik er weer met de rechter muis op. Vervolgens klik je op de optie Upgrade Virtual Hardware.

Deze actie eenmalig en onomkeerbaar. Klik op Yes om door te gaan.

Binnen enkele seconden is de hardware bijgewerkt.

Stap 10: VMware-Tools Upgraden.

Nadat de virtuele hardware is bijgewerkt naar laatste versie is het nu de beurt aan de VMware-Tools.

De versie die nu in gebruik is, namelijk build-38650 is verouderd en dient te worden bijgewerkt. Hieronder lees je hoe je dit doet voor Windows en Linux VM’s.

VMware-Tools installeren onder Windows

Open het console naar de desbetreffende VM. Klik op VM en daarna op de optie “Install VMware Tools“.

Klik op next en volg de stappen.

Nadat de installatie klaar is dien je de VM opnieuw op te starten.

Zoals je ziet zijn de VMware Tools bijgwerkt.

VMware-Tools installeren onder Linux.

Open het console naar de desbetreffende VM. Klik op VM en daarna op de optie “Install VMware Tools“.

Vervolgens moet je de CD-Rom mounten. Weet je niet welk device je CD-Rom speler heeft meegekregen, dan kun je dit achterhalen met het volgende commando: dmesg | grep “CD” als het goed is krijg je de onderstaande uitkomst:

hda: VMware Virtual IDE CDROM Drive, ATAPI CD/DVD-ROM drive
Uniform CD-ROM driver Revision: 3.20

Nu je het devicenaam weet (hda), kun je de CD-Rom mounten met de volgende commando’s:

mkdir -p /media/cdrom

mount /dev/hda /media/cdrom

Nu de CD-Rom gemount is kan het installatie bestand gekopieerd worden naar de /tmp directory

cp /media/cdrom/VMwareTools-x.x.x-xxxxx.tar.gz /tmp

Daarna pak je het tar.gz bestand uit

cd /tmp

tar zxvf VMwareTools-x.x.x-xxxxx.tar.gz

En als laatste stap installeren we de VMware tools

cd vmware-tools-distrib

./vmware-install.pl

Nadat je een paar keer enter hebt gegeven is de installatie klaar. Ik zou voor de zekerheid nog even de VM herstarten. Dit doe je met het volgende commando

reboot -n

Het kan voorkomen dat de ETH0 verdwijnd na het clonen van de Linux VM. De oplossing lees je in de volgende post.

Stap 11: De overige Servers Migreren naar ESX 3.0.1.

De ESX Servers worden geïnstalleerd met behulp van de Ultimate Deploiment Appliance (UDA). Meer informatie over UDA vindt je hier: http://www.rtfm-ed.co.uk/ en hier: http://www.ultimatedeployment.org.

Hieronder zie je een schema over hoe UDA bij ons is ingezet.

Stap 12: De “nieuw” geïnstalleerde ESX Servers configureren in VirtualCenter

Zodra je de ESX Servers opnieuw hebt geïnstalleerd en ze houden dezelfde namen dan zie je in VC dat de ESX Servers op not responding staan.

Dit los je als volgt op. Klik met de rechter muisknop op de ESX Server. Daarna kies je Disconnect.

Nadat de Disconnect actie klaar is doe je nogmaals hetzelfde alleen kies je niet voor Disconnect maar voor Connect.

Voer de username en password in en druk een aantal keer op next.

De ESX Server is weer beschikbaar binnen VC.

Stap 13: De overige VMFS2 Luns upgraden of formateren naar VMFS3.

Mocht je de oude LUN’s van het SAN behouden dan dien je de VMFS LUN’s bij te werken naar VMFS3 hoe dit in zijn werk gaat lees je onder het kopje upgraden. Mocht je gebruik maken van nieuwe LUN’s dan lees je onder het kopje Format / Add Storage hoe dit in zijn werk gaat.

Upgraden:

Voordat je de upgrade actie uit kunt voeren dient de ESX Server in Maintenance Mode te staan. Mocht je al VM’s actief hebben, dan moeten deze eerst afgesloten of via VMotion naar een andere ESX Server verplaatst worden.

Zoals je in het onderstaande plaatje ziet staat de ESX Server in maintenance mode.

Ga nu naar het tabblad configuration en klik op Storage (SCSI, SAN, and NFS).

Klik vervolgens op Upgrade to VMFS-3.

Nu krijg je de onderstaande melding. Voldoe je aan de eisen klik dan op Yes om door te gaan.

Nadat de actie klaar is verlaat je de maintenance mode door op Exit Maintenance Mode te klikken.

Format / Add Storage:

Mocht je gebruik maken van nieuwe storage zoals een SAN Lun of iSCSI Target.

Ga nu naar het tabblad configuration en klik op Storage (SCSI, SAN, and NFS).

Klik vervolgens op Add Storage….

De Add Storage wizard start. Klik op Disk/Lun en klik op next.

Klik op het Lun wat je wilt toevoegen. In dit voorbeeld wordt een iSCSI Lun toegevoegd.

Zoals je ziet is de hardisk blank.

Deze moet dus geformatteerd worden.

Stap 14: De configuratie van de ESX3 Servers nalopen aan de hand van je inventarisatie van stap 1.

Spreekt eigenlijk voor zich.

Stap 15: De VMs terug clonen of via VMotion terug plaatsen.

Dit gaat hetzelfde als stap 8.

Stap 16: De verschillende logfiles binnen de VI3 omgeving nakijken.

Eerder heb ik al een opsomming gemaakt in deze post over waar de verschillende logfiles staan. Hieronder vindt je een overzicht.

Vmkernel
/var/log/vmkernel
Houd de activiteiten bij van de virtual machines en de ESX Server.

Vmkernel Warnings
/var/log/vmkwarning
Houd de activiteiten bij van de virtual machines.

Vmkernel Summary
/var/log/vmksummary
Wordt gebruikt voor het bepalen van de uptime en availabiltiy statistieken van de ESX Server. Een logfile die “human readable” is kun je vinden in /var/log/vmksummary.txt

ESX Server host agent log
/var/log/vmware/hostd.log
Bevat informatie over de agent die voor de configuratie van de ESX Server en zijn virtual machines zorgt.

Service Console
/var/log/messages
Bevat alle generieke log messages voor het gerbuik tijdens het troubleshooten bij problemen met de virtual machines op de ESX Server.

Web Access
/var/log/vmware/webAccess
Houd alle informatie bij over de Web-based access naar de ESX Server.

Authentication log
/var/log/secure

Bevat data over de connecties die authenticatie vereisen. Bijvoorbeeld de VMware daemons en acties die gestart worden door de xinetd daemon.

VirtualCenter agent
/var/log/vmware/vpx

Bevat informatie over de agent die communiceerd met de VirtualCenter Server.

Virtual Machines
The same directory as the affected virtual machine’s configuration files; named vmware.log

Bevat informatie over een crash of abnormaal afgesloten virtual machine.

Conclusie:

Een migratie zoals deze behoeft nogal wat voorbereiding. Is de voorbereiding echter goed dan kan de migratie eigenlijk niet mis gaan.

Ports ESX

 

Links

Links over ESX 3.5 en VC 2.5

Compatibility & Version Info:
VI3 Key Features & Benefits Summary by Version – http://www.vmware.com/files/pdf/key_features_35.pdf
VMware Infrastructure Compatibility Matrixes – http://vmware.com/pdf//vi3_35/esx_3/r35/vi3_35_25_compat_matrix.pdf
Details of What’s New and Improved In VI3 Version 3.5 – http://www.vmware.com/support/vi3/doc/whatsnew_esx35_vc25.html

Must Read:
RTFM Upgrade guide for ESX 3.5 and VirtualCenter 2.5 – http://www.rtfm-ed.eu/docs/vmwdocs/Appendix_C_What’s_New_and_Different_in_Vi3-5.pdf

Release Notes/Install, Upgrade and Patch Guide:
ESX Server 3.5 and VirtualCenter 2.5 Release Notes – http://www.vmware.com/support/vi3/doc/vi3_esx35_vc25_rel_notes.html
ESX Server 3 Installation Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_installation_guide.pdf
Upgrade Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_upgrade_guide.pdf
ESX Server 3 Patch Management Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_esxupdate.pdf

Additional documentation:
Configuration Maximums for VMware Infrastructure 3 – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_config_max.pdf
Quick Start Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_quickstart.pdf
Basic System Administration – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_admin_guide.pdf
Virtual Infrastructure Web Access Administrator’s Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_web_access.pdf
ESX Server 3 Configuration Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_3_server_config.pdf
Resource Management Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_resource_mgmt.pdf
Fibre Channel SAN Configuration Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdf
iSCSI SAN Configuration Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_iscsi_san_cfg.pdf
Virtual Machine Backup Guide – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_vm_backup.pdf
VMware Infrastructure 3 Primer – http://vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_prim.pdf

Additional downloads:
Remote CLI Download – http://www.vmware.com/download/download.do?downloadGroup=VI-RCLI
License Server for ESX 3.5 – http://download3.vmware.com/software/vi/VMware-licenseserver-64192.exe
CPU Compatibility Tool for ESX 3.5 – http://download3.vmware.com/software/vi/CPU_Compatibility-64557.zip

Converter:
VMware Converter Enterprise for VirtualCenter 2.5 Admin Guide – http://vmware.com/pdf/vi3_vec_10_admin_guide.pdf
VMware Converter Enterprise for VirtualCenter 2.5 Release Notes – http://vmware.com/support/vi3/doc/vi3_vec_10_rel_notes.html

Update Manager:
VMware Update Manager Release Notes – http://vmware.com/support/vi3/doc/vi3_vum_10_rel_notes.html
VMware Update Manager Admin Guide – http://www.vmware.com/pdf/vi3_vum_10_admin_guide.pdf
VMware Update Manager Sizing Estimator – http://vmware.com/support/vi3/doc/vi3_vum_10_sizing_estimator.xls

Latest Hardware Compatibility Guides:
Systems – http://vmware.com/pdf/vi3_systems_guide.pdf
I/O – http://vmware.com/pdf/vi3_io_guide.pdf
Storage/SAN – http://vmware.com/pdf/vi3_san_guide.pdf
Backup Software – http://vmware.com/pdf/vi3_backup_guide.pdf

Blog & News Articles:
VMware ESX 3.5 goes live with key new features – http://searchservervirtualization.techtarget.com/originalContent/0,289142,sid94_gci1285629,00.html
What’s new in ESX 3.5 & VC 2.5? -
http://virtualgabe.wordpress.com/2007/12/08/what%e2%80%99s-new-in-esx-35-vc-25-part-2/
http://virtualgabe.wordpress.com/2007/12/08/what%e2%80%99s-new-in-esx-35-vc-25-part-3/
http://virtualgabe.wordpress.com/2007/12/08/what%e2%80%99s-new-in-esx-35-vc-25-part-4/
VMware VI Client 2.5 does not support 64-bit workstations – http://www.dabcc.com/article.aspx?id=6674

Relevant KB Articles:
Licensing:
Updates to your VMware VI3 Starter Licenses – http://kb.vmware.com/kb/1003299
Changes in licensing for VI3 Standard Edition When upgrading to VI 3.5 – http://kb.vmware.com/kb/1003301
Understanding VI 3.5 Licensing: Server and Host-based Licensing Models – http://kb.vmware.com/kb/1003295

ESX 3.5:
Installing ESX Server throws an “Anaconda Error” in the Partitioning Options screen – http://kb.vmware.com/kb/1003217
VMFS Partition cannot be created for “Typical” ESX Server Installation if Prior installation is detected – http://kb.vmware.com/kb/1003309
IBM System x3850 M2 and System x3950 M2 Servers fail to connect to 100Mbps Networks – http://kb.vmware.com/kb/1003226
Installing the Tivoli Storage Manager Client on the Service Console results in an error – http://kb.vmware.com/kb/1003142
Virtual Machine on a RDM Shared Storage becomes invalid after migration from ESX Server 2.5.x to ESX Server 3.5 or 3i – http://kb.vmware.com/kb/1003092
Vmotion is disabled after ESX Server upgrade – http://kb.vmware.com/kb/1003060
Certain Special Characters cause software iSCSI Initiator CHAP Configuration corruption – http://kb.vmware.com/kb/1003095
Connection to ESX Server host through VI Client is lost if you attempt to delete several VM’s at once from the Datastore Browser – http://kb.vmware.com/kb/1003250
Storage Devices connected to McData FC Switch through Qlogic adapters occasionally do not re-appear after reboot – http://kb.vmware.com/kb/1003250
Snapshot operations submitted directly to an ESX Server Host during Storage vMotion corrupts Virtual Machine data – http://kb.vmware.com/kb/1003114
Storage vMotion on a VM with I/O intensive workload may result incorrectly in a timeout error – http://kb.vmware.com/kb/1003276
Upgrading to ESX Server 3.5 when the Root Parition is nearly full might cause Incomplete System Configuration – http://kb.vmware.com/kb/1003311
Restarting Hostd (mgmt-vmware) on ESX Server hosts restarts Hosted Virtual Machines where VM Auto Startup/Shutdown is enabled – http://kb.vmware.com/kb/1003312
ESX Server becomes temporarily unresponsive under a Heavy I/O load – http://kb.vmware.com/kb/1003039
Consolidation of Large or Deeply Nested Snapshots using VirtualCenter, SDK or VCB might take longer on ESX Server 3.5 than on ESX Server 3.0.x – http://kb.vmware.com/kb/1003308

Consolidated Backup 1.1:
Upgrading Consolidated Backup version 1.0.x to 1.1 causes the installer to hang – http://kb.vmware.com/kb/1003045
Consolidated Backup cannot create Quiesced Snapshots of VM’s running Windows Vista – http://kb.vmware.com/kb/1003074
VCB 1.1 Command Line utility connection to port 902 causes an error message – http://kb.vmware.com/kb/1003088

VC 2.5:
When you install SQL Server Express on a System where SQL Native Client is present the installation might fail with error – http://kb.vmware.com/kb/1003076
VirtualCenter Server Fails to Start after your replace Default SSL Ceritifcates with Custom Certificates – http://kb.vmware.com/kb/1003070
Error Message During Installation: error 1603: error installing Windows installer engine – http://kb.vmware.com/kb/1003036
Administrative Credentials are Required for Oracle and SQL Database when Installing or Upgrading VirtualCenter – http://kb.vmware.com/kb/1003052
Client-side CD-ROM or Floppy can become disconnected – http://kb.vmware.com/kb/1003118
VirtualCenter Server does not detect changes in Host IP Address unless SSL Certificate Verification has been enabled – http://kb.vmware.com/kb/1003066
Permission problem if host had been in lockdown mode – http://kb.vmware.com/kb/1003117
Virtual Machines might lose Network Connectivity when moved to a different Port Group – http://kb.vmware.com/kb/1003061
Powering on Virtual Machines with multiple PCI Devices might fail – http://kb.vmware.com/kb/1003048
Incorrect Device Paths for LUNs displayed in Storage Summary – http://kb.vmware.com/kb/1003064
VirtualCenter Consolidation service Usernames and Passwords must use only ASCII characters – http://kb.vmware.com/kb/1003096
VI Client installation fails on Windows Vista Business Edition with enabled Anti-virus software – http://kb.vmware.com/kb/1003079
VirtualCenter Server might crash in a cluster with Manual or Partially Automatic DRS and Automatic DPM – http://kb.vmware.com/kb/1003027
Deleting Snapshots of VM’s with Heavy disk I/O might cause host to be Disconnected from VirtualCenter – http://kb.vmware.com/kb/1003024
Paravirtualization option is not Disabled for Unsupported Operating Systems – http://kb.vmware.com/kb/1003008
VirtualCenter Server installation fails or results in an error if your system does not have MDAC 2.8 SP1 or later installed – http://kb.vmware.com/kb/1003160
Installing Update Manager with Unified Installer might faile if Disparate Databases are used – http://kb.vmware.com/kb/1003277
Some Alarms may disappear after upgrading to VirtualCenter 2.5 – http://kb.vmware.com/selfservicekb/1003072
VirtualCenter Database upgrade fails with an exception when a Password that contains Apostrophes or Double Quotes is used – http://kb.vmware.com/kb/1003049
The VirtualCenter Server might Crash when using an older ODBC driver with Oracle 9i – http://kb.vmware.com/kb/1003049
Guest Operating System Standby feature removed in VirtualCenter Server 2.5 – http://kb.vmware.com/kb/1002414
Cannot specify Destination Folder on Non-default Datacenter when Cloning Virtual Machines – http://kb.vmware.com/kb/1003075
VirtualCenter Service will not start on a machine with non-Ascii characters in it’s Machine Name – http://kb.vmware.com/kb/1003075
vMotion from ESX Server 3.5 hosts to ESX Server 3.0.x hosts causes the console sessions of the migrated VM’s to become blank – http://kb.vmware.com/kb/1003038
Automatic VMware Tools upgrade does not upgrade to the latest version on VM’s with Insufficient space in the Root parition – http://kb.vmware.com/kb/1003051

Bron: http://communities.vmware.com/thread/117185?tstart=0

ESX Best Practises and tips

Vi3 Ports
———
Virtual center to ESX host – 902
Esx to iScsi – 3260
Esx to license server – 27000-27010

——————————————————————
ESX host services/Daemons
————————-
vpxa
hostd
VMkernel is like a service.Service console boot the vmkernal.

——————————————————————
Features
——–
IVT-Intel vertualization technology means you can create 64 bit Vms on 32 bit machines as long as if its BIOS supports.

——————————————————————-
NOTs
—-
Virtual Center 2.0 does not support VMware server (Formerly known as GSX)
VI3 does not support USB.Use usb over ip solutions.
With iSCSI and NFS clustering is not supported.
You can not boot ESX from a shared SAN.
You cannot boot ESX from NFS.
You cannot create VMFS on NFS.
You cannot RDM on NFS.
VCB not used on iSCSI and NFS.

——————————————————————-
GSX/VMware Server
—————–
GSX is now knows as vmware server.
You can install GSX/vmware server on windows XP.But you have to stop the default web site.
You can run Vmware server (GSX) on win2k,win2k3 and linux.

Every 20 mllisecond vmkernal looks for better cpu resource allocation.
Esx consider a Isolated Host as a failed host after receving no heart beat from it for 14 seconds.

To change number of CPUs you have to shut down the VM.
——————————-
Partition Information
———————-
Default size boot partition 100meg

Default size root partition 5gig

Default size vmkore partition 100meg

Root partition minimun size 2.5 gig.

Service console gets 272 meg.So swap is 544 MB
Maximum service console memory can be 800meg.That means max swap can be 1600 meg.

The ESX Server swap partition is not to be confused with virtual machine swap space.

virtual machine swap space/file maximum 16Gig

For better patch management/log files reasons its good to have /var rather than /var/log.

If esx crashes Purple screen of death (psod) happens.The stuffs on the memory will be dumbed on vmkcore partition.

Its better to run ESX on raid 1 and keep the vms on san.

Instead of upgrading esx 2.x to vi3,go for migration.Its easy and is recommended.

—————————————————————-
Commands
——–
esxcfg command can be used to bind virtual swith to physical nic.
What can be done to increase the size of a VMDK file? vmkfstools -X
vcbmounter.exe
vcbrestore.exe
vm-support

Build a VM with always 1 processor.If you give more it tend to use it more unnessesarily.

Built VM with 2 vmdks one for just os and other for applications.

VMs with volumekey license doest require activation.But others require activation when you do P2V.

VMWare converter is a new to which is the mixure of earlier vmware tools p2v assistant and vmware importer.

One of a diffrence between vmware converter starter edition and enterprise edition is:
1.To p2v 10 machine you got to install starter edition in all machines.But enterprise edition will help you to do it from a single machine.
2.Another diffrence is starter edition is a windows installer where as enterprise edition is a winpe based bootable CD.

ESX server is a frame work where Virtual Center is the brain.

ESX virtualization layer vmkernel (Hypervisor) and linux like service console are depends each other.

ESX VMware licensing is based on CPU slots. Not per core but by slot.
Quad core cpu require single processor license.Not 4.

Virtual center is not mandatory for VI3 infrasturture but recommended for good management practises if you got more the one ESX hostd.

VCP exams $175 and 80 questions. all objective. no simulations. no command line.Provides by vue.

DRS dynamic resource sheduler does auto vmotion based on resources availability.

File systems for windows ntfs-linux ext3-esx vmfs

VCB can not be installed on a VM.
VCB can not have advantage of having two HBAs.Powerpath/multipath wont work.
With VCB you can not do diffrential backup.
It can not take back up on tapes.You need 3rd party back up softwares.But not mandatory.

To implement Windows cluster shared storage should be RDM.Raw device mapping.

HA-If one ESX host is down other host takes care of all those VMs.
Not for individual VMs.If so go for windows cluster.
HA to work you should have DNS.

If virtual center is down DRS will not work.But HA will work.

AAM Heartbeat happen in service console.
Isolation response is after the heartbeat is gone(15 secs).Verify if esx host is there.

.vmdk Contains os/apps like a harddisk
.vmx configuration file information about virtual hardwares.
.lic license file
-flat.vmdk
vmware.log
.nvram
.vmsd
.vmxf
.vmtx(only on template)

Host based licensing does not support HA/DRS/Vmotion

VCP is are getting free workstation 6 license.

vpxuser is created by virtual center to access esx server.

Esx based tools
esxReplicator http://www.vizioncore.com/esxReplicator.html not visioncore
Platespin Power converter -to do V2P,P2P,V2V,P2V and all
Veeam FastSCP 2.0 for VMware ESX Server
Veeam Reporter 2.0 for VMware Infrastructure 3
Symantec Ghost
Microsoft data protection manager server.
microsoft setupcl.exe
sysprep exe.
diskpart
Acronis Disk Director Suite
datacore, microsoft string bean virtual iscsi san.
NAP network access protection
Hardware keylogger
win sep

Remote access tools are
Hp ILO
Dell Remote Access Control
RSA
Terminal server
Remote desktop
VNC

Resource management
limit-is the maximun resources allowed
reservation-default 0-is the guatenteed minimum amount of resource.If its not met Vm wont start.
Share-when there is a fight for resiurces between vms-is to control fight or to assign priority.

Per VM Swap file size is=limit-reservation size
for example if limit is 4gig and reservation is 0 then swap size is 4 gig

resource pools are only for CPU and memory.

expandable reservation onresource pool means it could get resources from its parent.

for vms shares can be chaged on the fly.reservations can not be changed on the fly.

vmotion conditions
both esx has access to storage
both connected to vmotion/production network
both on same subnet

service console is connected to a vswitch called vswif0

vms mac address start with 00:50:56:———–

Windows licensing
if you got datacenter edition-unlimited per host
if you got enterprise editiob-4vms per license
if you got standard edition-every vm needs a license

vmkernal takes care of vmotion,nfs and iscsi traffics.

Its is good to follow dns nmaing scheme whenyo create a vms display name.(no space etc)

2 physical processor smp is better than dual core.and its is beter than hyperthreading.

VCB would help to take vm snapshots.
persistent is like hardisk writable
non persistant readonly drive.

Virtual center
There is a virtual center agent on esx host.
ads is not mandatory.
license server can be installed in the same machine where virtual center is installed.
Virtual center will have apache tom cat installed.
database server should be diffrent.
If it is sql 2000 user should be member of dbo.
if it is sql 2005 change owner ship from dbo to the virtual center user.
sql 2005 sp1 is required.
Go for sql authenticvation.

If Virtual center and sql is on the same box go for windows authentication.but this scenario is not recommended.

After the license is expired you cannot start the Vms.Currently running will remain run.

Esx used ports
Virtual center to Esx management communication – 902 tcp
Esx to license server – 27000-27010
virtual center diag – 8083
http – 80
https – 443
heart beat -902 udp
apache tomcat – 8086
iscsi – 3260

A resource for a vm comes from a single host.

Two Vms cannot access the same raw harddisks until windows clustering is configured.

Services on virtual center box
vmware license server
vmware tools service
vmware vi we access
vmware virtual mount manager extended
vmware virtual center server

Iscsi
with iscsi and nas you cannot have vcb and windows clustering.
storage network should be dedicated and non routable.
hardware qlogic 4050 4052
port 3260
static manual
send target is auto

vmfs
to refarmat in to vmfs remove lun.that will delete files.
vmdk max file size 2tb

how to increase the size of vmdk?
vmkfstools -X 90G

where 90G is the total new size.
then you have to extend the c drive on vm.
can not shrink vmdk.multipath
mru-does fail over-but does not fail back-you have to manually fix it.
fixed-auto fail back-but you have to manually configure per lun-difficult.

service console can see smb
vmkernal cannot see smb

Nas is file level
fiber channel and iscsi is vlock level

networking
If no vlan in vmkernal create one vswitch per ipsubnet.or one vlan for for every ip subnet.

Traffic shaping is to controll outbound traffic.
Average bandwidth kbbs
Peak bandwidth kbbs
Burst size kb

nicteaming is good for network intensive servers like terminal server,ftp,sharepont etc.

service consloe – needs ipaddress – for management
vmkernal – need ipaddress – for vmotion.nfs,iscsi
vmportgroup – no need of ip – physical infra to accesss virtual infra

do not partition the lun-use as it is.
to add hba you have to shutdown.
to add lun no need to shutdown.
internal only vswitches- cannot do vmotion.

Starter edition
standard edition
enterprise editin

Good to have 4 nics – one nic is must
nic1 for service console
nic2 for vmotion
nic3 for vms
nic4 for vms

memory
500 meg that is guranteed or comitted
1 gig max that is over committed

service console only use cpu0 needs guranteed 8%

boot lun should be seen by only its coresponding esx server.Can be done by san masking.other luns can be seen,its good for vmotion.

mui management user interface

vi client is for host/vm management
mui w2eb client is for vm management

vmss

root password mustbe atleast 6 charecters

host based license wont float between esx servers

There is no licensing server requirement for host-based licensing.

two types of hostbased licensing
starter
standard

three types of serverbased licensing (needs licenseserver)
starter
standard
enterprise

How should the amount of RAM for the VMkernel be determined?
Sum the RAM maxima for all VMs, or sum the RAM minima for a VMs if
overcommitment is desired

What must be done in order to configure an ESX server to boot from a SAN-
based disk?
a. Server BIOS must designate an HBA as the boot device
b. Fibre Channel card must initiate a “primitive” connection to target boot
LUN
c. Fibre Channel card must be accessible to Service Console and VMkernel

Maximum scsi controllers per VM 4 – 15 devices/controller
Maximum scsi controllers per host 16 – 16 devices/controller

Virtual Center server Min req
2GHz-2Gig-560MB

Vi Client Min req
266MHz-256MB-150MB

ESX server Min req
1500MHz-1Gig-

Max VMFS3 volume size 64TB

LUN Masking – is configured on Storage processor or Host

Zoning – which servers can access which LUNs – on SAN-switch level
Soft zoning-wwn,storage processor
Hard zoning-swithport,FCfabric

Esx server roles
No Access
Read Only
Administrator

Virtual Switch
56 Ports by default
Max 1016 Ports

If there is no SAN , VMKCORE and VMFS volume has to be created on local drive.

/var and /swap is on local drive

iScsi maximum LUNs 256

Max vmfs3 volumes 256

Max vmfs3 file size 64TB

Min VMFS3 Volume size 600MB

You can create a maximum of 248 vSwitches on a single host.

sharemem 50
sharememtotal 200
tax 75
ctl 65
sample 60
balance 15

ESX Server Requirements
ESX Server requires a computer with the following specifications:
• At least two processors of one of the following types:
• 1500MHz Intel Xeon and later, or AMD Opteron (32-bit mode)
• 1500MHz Intel Viiv or AMD A64 x2 dual-core processors
• 1GB RAM minimum
• One or more Ethernet controllers
• A SCSI disk, Fibre Channel LUN, or RAID LUN with unpartitioned space

Virtual SCSI Devices
• Up to 4 virtual SCSI adapters per virtual machine, with up to 15 devices per adapter
• 9TB per virtual disk

110 previleges
13 objects
8 standard roles

service console memory 272 to max 800 mb

max nfs mounts per host 8

ESX server
16 CPUs
64 GB
16 SCSI adapter/15 devices

VM
4 CPUs
16 GB
4 SCSI adapter/15 devices

4 NIC

RDM ViZioncore

RDM/vRDM/Clustering Management

RDM (Raw Device Mapping)
A Raw Device Mapping (RDM) is a special file in a VMFS volume that acts as a proxy for a raw device. The RDM provides some of the advantages of a virtual disk in the VMFS file system while keeping some advantages of direct access to physical devices.

RDM might be required if you use Microsoft Cluster Service (MSCS) or if you run SAN Snapshot or other layered applications in the virtual machine. RDM’s better enable

Systems to use the hardware features inherent to SAN arrays Physical RDM When using Physical RDM’s the Virtual machine guest does not support snapshots. Physical RDM’s are not able to utilize the snapshots technology because this mode allows the virtual machine to manage its own snapshot or mirroring operations.

Unfortunately at this time since snapshots apply to the whole Virtual Guest, vRanger cannot backup the OS or any other drive when the VM has physical RDM attached.

***Ranger backups in regards to ESX 2.5 since you could add re-do’s to the specific disk (vmdk) that you wanted to backup. Since the adaptation of snapshots in VI3 this is not possible, and is an all or nothing snapshot scenario. Note: VMware’s Consolidated Backup does not allow for the mounting of the VMDK where a physical RDM is attached to the VM, since it also relies on snapshots. This is a limitation of VMware..

Virtual RDM

When using Virtual RDM’s the Virtual Machine DOES support snapshots and can be skipped if the OS or other data VMDK needs to be backed up with vRanger, VCB also works with vRDM at this time.

RDM vs. VMDK

An RDM is used when performance is an issue, in the need of clustering, or if you have a volume that is already formatted with NTFS or other file system. Virtual RDM performance is the same as a VMDK since they are both virtual containers for data. There is no performance gain with a vRDM. With a Physical RDM there is more of a performance gain in regards to Disk I/O, but once again snap shots are disabled to the entire VM when a RDM is attached.

Physical RDM Clustering
There are two ways to create clustered VM’s.
1. Physical RDM’s
2. A single VMDK shared between two VM’s

1. If you have a physical RDM’s attached to a VM you can not create snap shots. (VMware limitation)
2. VMDK’s or RDM that are shared between two VM’s, snap shots are disabled on the entire VM

To exclude an vRDM

If you want to exclude to virtual RDM from snapshots you have to define it as an “independent” disk for the VM. To do so, you need to switch it to “virtual” mode. This is quite simple; however you need to power down the VM to change a virtual RDM to “independent” mode.

A physical mode RDM can be changed to virtual mode; this is quite simple; however you need to power down the VM to change a physical RDM to virtual. Once this is done you will lose the performance gain of having a physical RDM by changing the physical RDM it to a virtual RDM.

MSCS

MSCS AND VMWARE

Few points to remember when you decide to built clustering inside VM which might be CIB or CAB.

Virtual Machine (Cluster Node) have forementioned boundaries

• Only LSI Logic virtual SCSI Card
• Only VMXnet
• Only 32-Bit VMs
• 2-Node Clustering only
• Nic teaming is not supported
• iSCSI clustering is not supported
• Boot from SAN is not supported
• VMs part of clustering cannot be part of VMHA & DRS
• Cannot VMotion on VMs using clustering software
• ESX 2.5 and ESX 3.0 is not supported
• Different HBA’s card manufacturer not supported
• When using N+I SCSIPort Miniport driver must be present on Physical Node and not Storport Miniport driver, also there must be no powerpath software installed on physical node.
  

If you clone VM’s with RDM enabled, RDM will be converted into vmdks You must zero-out the disk which you would like to shared disk, you can also use mapped SAN LUN, in this case you don’t need to use VMKFSTOOLS Disk must map to SAN LUN and it is recommended to have RDM set up in physical mode Upgrade of VMs, which are using MSCS, is supported only from ESX 2.5.2 to ESX 3.0.

UPGRADING RDM AND BOOT VOLUME VMFS ON DIFFERENT VOLUMES

Power off Virtual NodesUpgrade volumes from VI ClientPower on each node, in case you get error ‘Invalid argument, you have misconfigured cluster setup’, Virtual disk of ESX 2.x cannot be powered on ESX3.0. In this case you need to import this disk using VMKFSTOOLS utility

UPGRADING RDM AND BOOT VOLUME ON SAME VMFS VOLUME

Power off Virtual NodesUpgrade volumes from VI ClientUpgrade from 2 to VMFS3.0 relocates RDM and first Node VMx file, when you now upgrade the second node that unable to locate vmdk file, ignore it. In anyway it will uprade VMDK now manually edit second node’s vmx file and point to quorum and RDM file’s new location.

UPGRADING CLUSTER ACROSS BOX

Using shared pass-through RDM is similar to upgrading on same VMFS volume

Using files in shared VMFS-2 volumes

1. Change the mode of volumes from shared to public
2. Upgrade ESX server
3. Upgrade VMFS volume from VI client
4. Create LUN for each shared disk
5. For each shared disk , create RDM pointing to respective luns

e.g vmkfstool –i oldvmdk.vmdk newrdm.vmdk –d respectivelun

1. Finally modify VMx file for each node pointing to respective new RDMs