VMware ESX and ESXi 4.1 Comparison

Purpose

This article provides a detailed comparison of VMware ESX and ESXi 4.1. The article is separated into capabilities or features and compared at that level.

Resolution

Capability	VMware ESX	VMware ESXi
Service Console	Service Console is a standard Linux environment through which a user has privileged access to the VMware ESX kernel. This Linux-based privileged access allows you to manage your environment by installing agents and drivers and executing scripts and other Linux-environment code.	VMware ESXi is designed to make the server a computing appliance. Accordingly, VMware ESXi behaves more like firmware than traditional software. VMware has created APIs through which monitoring and management tasks – traditionally done through Service Console agents – can be performed. VMware has provided remote scripting environments such as vCLI and PowerCLI to allow the remote execution of scripts and commands. Tech Support Mode (TSM) provides a command-line interface that can be used by the administrator to troubleshoot and correct abnormal conditions on VMware ESXi hosts.
CLI-Based Configuration	VMware ESX Service Console has a host CLI through which VMware ESX can be configured. VMware ESX can also be configured using vSphere CLI (vCLI) or vSphere PowerCLI.	The vSphere CLI (vCLI) is a remote scripting environment that interacts with VMware ESXi hosts to enable host configuration through scripts or specific commands. It replicates nearly all the equivalent COS commands for configuring ESX. VMware vSphere PowerCLI is a robust command-line tool for automathing all aspect of vSphere management, including host, network, storage, virtual machine, guest operating system, and more. Notes: vCLI, PowerCLI, and vSphere SDk for Perl are limited to read-only access for the free vSphere Hypervisor edition. To enable full functionality of vCLI on a VMware ESXi host, the host must be licensed with vSphere Essentials, vSphere Essential Plus, vSphere Standard, vSphere Advanced, vSphere Enterprise, or vSphere Enterprise Plus. Certain COS commands have not been implemented in the vCLI because they pertain to the management of the COS itself and not ESXi.
Scriptable Installation	VMware ESX supports scriptable installations through utilities like KickStart.	VMware ESXi supports scriptable installations using a mechanism similar to Kickstart, and includes the ability to run pre- and post-installation scripts. VMware ESXi also provides support for post installation configuration using PowerCLI- and vCLI-based configuration scripts.
Boot from SAN	VMware ESX supports boot from SAN. Booting from SAN requires one dedicated LUN per server.	VMware ESXi may be booted from SAN. This is supported for Fibre Channel SAN, as well as iSCSI and FCoE for certain storage adapters that have been qualified for this capability.
Serial Cable Connectivity	VMware ESX supports interaction through direct-attached serial cable to the VMware ESX host.	VMware ESXi does not support interaction through direct-attached serial cable to the VMware ESXi host at this time.
SNMP	VMware ESX supports SNMP.	VMware ESXi supports SNMP when licensed with vSphere Essentials, vSphere Essential Plus, vSphere Standard, vSphere Advanced, vSphere Enterprise, or vSphere Enterprise Plus. The free vSphere Hypervisor edition does not support SNMP. .
Active Directory Integration	VMware ESX provides native support for Active Directory integration.	VMware ESXi provides native support for Active Directory integration.
HW Instrumentation	Service Console agents provide a range of HW instrumentation on VMware ESX.	VMware ESXi provides HW instrumentation through CIM Providers. Standards-based CIM Providers are distributed with all versions of VMware ESXi. VMware partners include their own proprietary CIM Providers in customized versions of VMware ESXi. These customized versions are available either from VMware’s web site or the partner’s web site, depending on the partner. Remote console applications like Dell DRAC, HP iLO, IBM RSA, and FSC iRMC S2 are supported with ESXi.
Software Patches and Updates	VMware ESX software patches and upgrades behave like traditional Linux based patches and upgrades. The installation of asoftware patch or upgrade may require multiple system boots as the patch or upgrade may have dependencies on previous patches or upgrades.	VMware ESXi patches and updates behave like firmware patches and updates. Any given patch or update is all-inclusive of previous patches and updates. That is, installing patch version “n” includes all updates included in patch versions n-1, n-2, and so forth. Furthermore, third party components such as OEM CIM providers can be updated independently of the base ESXi component, and vice versa.
vSphere Web Access	vSphere Web Access is only experimentally supported in VMware ESX.	VMware ESXi does not support web access at this time.
Diagnostics and Troubleshooting	VMware ESX Service Console can be used to issue command that can help diagnose and repair support issues with the server.	VMware ESXi has several ways to enable support of the product: Remote command sets such as the vCLI include diagnostic commands such as vmkfstools, resxtop, and vmware-cmd. The console interface of VMware ESXi (known as the DCUI or Direct Console User Interface) has functionality to help repair the system, including restarting of all management agents.
Jumbo Frames	VMware ESX 4.1 fully supports Jumbo Frames.	VMware ESXi 4.1 fully supports Jumbo Frames.

AIDE with Centos 5

In recent days, the subject of intrusion detection systems for centos has come up. To cover this and hopefully help some folks out, I’ve decided to do a brief writeup of Aide, the IDS which comes with CentOS. Please don’t confuse this with SELinux. SELinux is a Mandatory Access Control style permissioning system. SELinux stops people from getting into your system via protected applications. Aide lets you know if they actually get beyond SELinux and onto your system.

Installing Aide
yum install aide
What? You expected it to be harder? Now that we have aide installed, we need to configure it. The default config file should be okay for most folks who haven’t relocated things on the distro too much. Double check to make sure that all the directories you want to scan are listed. If you want to fine-tune the aide config, then you’ll need to edit /etc/aide.conf.

Initializing Aide’s Records

The next thing we need to do is create the initial aide database. For this, you need to run the following command:
# /usr/sbin/aide --init

This will take a little bit of time to run, and you’ll have some disk churn for minute or two while aide investigates your system and creates a baseline. Once this is done, we’re going to run an initial query of the system, just to make sure that everything’s working properly. To do this, run the command below:
# cp /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz
# /usr/sbin/aide --check

This copies the initial database to the current database, then checks them against each other. In theory you should not have any differences. If you do, investigate them. As we’re still setting this up, they’re likely to be mundane .viminfo files or something similar. Keep in mind that when you update applications via ‘yum update’ that you may see aide go a bit nuts, just as tripwire or others would. You’re replacing files on your system when you update, and this is exactly what aide is designed to warn you about. In a perfect world, you should get some output like the text below:

# aide --check
AIDE, version 0.13.1
### All files match AIDE database.

Once we’re satisfied that aide is working as we expect, it’s time to set up a periodic check of the system. Only you can determine what’s often enough for your servers. I personally run aide as weekly cron, by creating a file in /etc/cron.weekly/ called aide.cron, with the following contents:


#!/bin/bash
/usr/sbin/aide --check | /bin/mail -s "Weekly Aide Data" email@host.com

This runs my check once a week. That’s pretty much it to setting up aide. If you want to see more options for aide, please check out the documentation in /usr/share/doc/aide-*/
Update:

So it seems that by default, aide requires selinux to be enabled, or at least permissive so that it can record the selinux contexts of the files it watches. If for some reason you really, truly want to have selinux disabled, but you still want aide to watch the system, use the config file below. It is identical to the default scan, but with the selinux bits removed.

VMware vSphere 4 - Versions

QLogic HBAs and other PCI devices may stop responding in ESX/ESXi 4.1 when using Interrupt Remapping for Vsphere 4.1

This is very new bug in Vsphere 4.1 ESXi as Vmware release the solution on Apr 21, 2011.

I had facing 2 downtime since we update to Vsphere 4.1 since May 2011.

Please take on this.

When using Interrupt Remapping on some servers, you may experience these symptoms on ESX/ESXi 4.1:

HBAs stop responding
Other PCIs devices may also stop responding
You see an an illegal vector shortly before an HBA stops responding to the driver. For example:

vmkernel: 6:01:34:46.970 cpu0:4120)ALERT: APIC: 1823: APICID 0x00000000 - ESR = 0x40

The HBA stops responding to commands. For example:

vmkernel: 6:01:42:36.189 cpu15:4274)<6>qla2xxx 0000:1a:00.0: qla2x00_abort_isp: **** FAILED ****
vmkernel: 6:01:47:36.383 cpu14:4274)<4>qla2xxx 0000:1a:00.0: Failed mailbox send register test

The HBA card gets marked offline. For example:

vmkernel: 6:01:47:36.383 cpu14:4274)<4>qla2xxx 0000:1a:00.0: ISP error recovery failed - board disabled

Note: This issue only applies if you see the specific alert: ALERT: APIC: 1823: APICID 0x00000000 - ESR = 0x40 in the vmkernel/messages log files. If you do not have this message, you are not experiencing this issue.


Resolution
This issue is currently under investigation by VMware engineering.

ESX 4.1 introduces interrupt remapping code that is enabled by default. This code is incompatible with some servers. You can work around this issue by manually disabling interrupt remapping on the affected servers.

To disable interrupt remapping, perform one of these options:

Run the commands:

# esxcfg-advcfg -k TRUE iovDisableIR
# reboot

To check if interrupt mapping is set after the reboot, run the command:

# esxcfg-advcfg -j iovDisableIR

iovDisableIR=TRUE

In vSphere Client:
Click Configuration > (Software) Advanced Settings > VMkernel.
Select VMkernel.Boot.iovDisableIR and click OK.
Reboot the ESX host.