Tuesday, December 25, 2012

Citrix XenDesktop Health Monitoring - Desktop Availability

Monitoring is a crucial task in a critical environment. Proper monitoring can help mitigate critical situations proactively while minimizing the damage. As a Systems Engineer I always prefer the concept of being notified when something out of the ordinary happens in the systems that I am responsible of keeping alive. In other words I love knowing what is coming towards me than saying "Oh, that's what hit me!" after I get hit. 

A XenDesktop environment isn't always critical but when it grows with more desktops, more infrastructure servers and especially when it has many customers depending on its proper functionality to fulfill their tasks the availability of the desktops is crucial. Unless you are working only with Citrix XenDesktop in a daily manner but work with many other different infrastructures simultaneously you would hardly find time to navigate to the console and check how the desktops are performing every day. "Hey Nimantha, I've been trying to log in to this Citrix Desktop for some time but it takes ages and still does not log me in!", then it comes to my mind "damn, what is going on there... I didn't log into it in.. like.. months...!". Then only we get the idea of opening the console and see what is out of the ordinary but now the problem has already occurred. What if I got a report of the health of the XenDesktop environment early in the morning stating that there are a number of them doing not-so-well. Well, then as soon as the guy says, "Hey Nimantha, I've been trying to log in to this Citrix Desktop for..." I would know "Aha, this must be the issue..." or I may already have solved that issue as soon as I saw the report in the morning and the customer never had to experience it.

Long story short, what I did is I created a powershell script that would examine the status of the desktops and the critical infrastructure (the Delivery Controllers, the Provisioning Servers, the XenServers, etc.) at a given schedule and send a report including the statistics in an email to me and my team. For me this report saves a lot of time that I had to spend logging on to the Delivery Controller, then opening the Desktop Studio, then navigating to each desktop group and check the status of the desktops. To be honest although I had the idea of creating this script I had no idea where to start and neither did I know powershell. But, one day, thanks to Miguel Contreras who posted a skeleton script for this purpose in one of his blog posts I was determined to do this somehow and that led me to creating this script and also to mastering powershell along the way.

This script does 6 things.

1. Look for pooled VMs that have not rebooted since the last user logged off and resets them if there are any.
2. Look for VMs that are powered on and unregistered.
3. Look for VMs that are in maintenance mode.
4. Check the availability of the infrastructure servers using ping.
5. Send the summary of 2, 3, and 4 as a report to specified recipients in an email.
6. Log the summarized report to a specified location.

Here is the script:

##----- Start of the script -----

# Load Citrix PowerShell modules
Asnp Citrix.*

$recipients = "someone@somewhere.com","anyother@somewhere.com"
$fromEmail = "XDHealthCheck@somewhere.com"
$SMTPserver = "smtp.somewhere.com"
$currentTime =
Get-Date -Format "yyyy-MM-dd | hh:mm:ss tt"
$logTime = Get-Date -Format "yyyy-MM-dd_hh-mm"
$LogFile = "C:\Monitoring\Logs\HealthReport-$logTime.html"
$servers = "DeliveryController1","ProvisioningServer1","XenServer1"
$site = "New York"

#Reset VMs that failed to reboot after user log off
$badVMs = Get-BrokerDesktop -PowerActionPending $false -PowerState On -SummaryState Available -WillShutdownAfterUse $true -MaxRecordCount 5000
If ($badVMs)
   foreach($vm in $badVMs)
      New-BrokerHostingPowerAction -MachineName $vm.HostedMachineName -Action 'Reset'

$RptHeader = "<!DOCTYPE html>"
$RptHeader+= "<html>"
$RptHeader+= "<head>"
$RptHeader+= "</head>"
$RptHeader+= "<body>"
$RptHeader+= "<h2><font color='purple' face='cambria'>XenDesktop Health Status Report $currentTime </font></h2>"

#Get unregistered VMs
$RptBody += "<b><font color='purple' face='cambria'><u>UNREGISTERED MACHINES</u></font></b><br><br>"
$unregVM = Get-BrokerDesktop -MaxRecordCount 5000 | ? {($_.RegistrationState -eq 'Unregistered') -and ($_.PowerState -eq 'On')}
if (!$unregVM){
    $RptBody += "<font color='green' face='cambria'> *** None *** </font>"
    $RptBody += "<table border=0>"
    $RptBody += "<tr>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Machine Name</u></font></th><th/>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Desktop Group Name</u></font></th><th/>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Reason for Last Deregistration</u></font></th>"
    $RptBody += "</tr>"

    foreach($unregVM in (Get-BrokerDesktop -MaxRecordCount 5000 | ? {($_.RegistrationState -eq 'Unregistered') -and ($_.PowerState -eq 'On')}))    {
        $RptBody += "<tr><td>" + $unregVM.HostedMachineName + "</td>" + "<td/>" + "<td>" + $unregVM.DesktopGroupName + "</td>" + "<td/>" + "<td>" + $unregVM.LastDeregistrationReason + "</td></tr>"
    $RptBody += "</table>"
$RptBody += "<br><br>"

#Get maintenance mode VMs
$RptBody += "<b><font color='purple' face='cambria'><u>MACHINES IN MAINTENANCE MODE</u></font></b><br><br>"
$maintModeVM = (Get-BrokerDesktop -MaxRecordCount 5000 | ? {$_.InMaintenanceMode -eq 'True'})
if (!$maintModeVM){
    $RptBody += "<font color='green' face='cambria'> *** None *** </font>"
    $RptBody += "<table border=0>"
    $RptBody += "<tr>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Machine Name</u></font></th><th/>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Desktop Group Name</u></font></th><th/>"
    $RptBody += "<th><font color='midnightblue' face='cambria'><u>Reason for Last Deregistration</u></font></th>"
    $RptBody += "</tr>"

    foreach($maintModeVM in (Get-BrokerDesktop -MaxRecordCount 5000 | ? {$_.InMaintenanceMode -eq 'True'}))    {
        $RptBody += "<tr><td>" + $maintModeVM.HostedMachineName + "</td>" + "<td/>" + "<td>" + $maintModeVM.DesktopGroupName + "</td>" + "<td/>" + "<td>" + $maintModeVM.LastDeregistrationReason + "</td></tr>"
    $RptBody += "</table>"
$RptBody += "<br><br>"

#Check the availability of the Infrastructure
$RptBody += "<b><font color='purple' face='cambria'><u>STATUS OF THE INFRASTRUCTURE</u></font></b><br><br>"
$RptBody += "<table border=0>"

foreach ($server in $servers){   
    If (Test-Connection -ComputerName $server -Count 2 -Quiet){
        $RptBody += "<tr><td>$server</td>" + "<td></td>" + "<td><font color='green' face='cambria'>UP!</font></td></tr>"
        $RptBody += "<tr><td>$server</td>" + "<td></td>" + "<td><font color='red' face='cambria'>DOWN!</font></td></tr>"
$RptBody += "</table>"

$RptBody += "<br><br><font color='silver' face='cambria'>Report generated by $DDC</font><br>"

$RptFooter += "</body></html>"

$emailBody = $RptHeader + $RptBody + $RptFooter

$emailBody | out-file $LogFile
send-mailmessage -from $fromEmail -to $recipients -subject "XenDesktop Health Check | $currentTime | $site" -body $emailBody -SmtpServer $SMTPserver -BodyAsHtml -Priority High


##----- End of the script -----

The resulting email will look like this:

You'll find a LOT (I mean it! :-D) of HTML formatting in this code. What can I say, I am an engineer who has an artistic mind. :-)
... and who likes to look at a boring black and white report anyway. ;-)

If you feel like doing some tweaking or adding some modifications you'll find Getting Started with PowerShell in XenDesktop 5 useful.

I would like to thank the customers for their feedback on the current system, the good and the bad. If not for them I might not have created this script in the first place.

P.S. If you need to copy the script please use CTRL+C. Right click is disabled. :-)
You can also download the script here.

Inspired by: http://blogs.citrix.com/2012/10/27/xendesktop-monitoring-desktop-availability/

Sunday, June 3, 2012

The Flame

"Iran's Nightmare" aka The Flame Malware
A massive, highly sophisticated piece of malware has been newly found infecting systems in Iran and elsewhere and is believed to be part of a well-coordinated, ongoing, state-run cyber-espionage operation.

The Flame computer virus not only stole large quantities of information from various Iranian government agencies, but apparently even disrupted its oil exports by shutting down oil terminals, information security firm Symantec Israel said yesterday.

The Flame virus recently found in Iran could be used to infect other countries, according to the International Telecommunications Union. As the United Nations agency charged with helping members protect their data networks, the ITU plans to issue a warning about the danger of Flame.

Iran's National Computer Emergency Response Team (Maher) said in a statement that the detection and clean-up tool was finished in early May and is now ready for distribution to organisations at risk of infection.Flame was discovered after the UN's International Telecommunications Union asked for help from security firms to find out what was wiping data from machines across the Middle East.An investigation uncovered the sophisticated malicious programme which, until then, had largely evaded detection.

The virus is about 20 times the size of Stuxnet, malware that targeted the controls of an Iranian nuclear facility. The largest concentration of infected machines is in Iran, followed by the Palestinian-controlled West Bank, Sudan, Syria, Lebanon, Saudi Arabia, and Egypt.

Flame is a Swiss Army Knife of malware in the sense that it can intercept everything imaginable, but it is not a pile of existing malware code thrown together. It is very cleverly crafted. Like Stuxnet, it has multiple propagation vectors – USB keys, printer sharing, and domain controller rights to name a few.

No security vendor has pointed out a single country or group as being responsible for Flame's creation.The central matter of contention at the moment is whether the malware was made by a private group or a nation state.

Flame Malware Removal Toolkit by BitDefender Labs

64bit version -> http://labs.bitdefender.com/wp-content/uploads/downloads/2012/05/TrojanFlamer_BDRemovalToolDropper_x64.exe

source: http://thehackernews.com/2012/05/flame-malware-21st-century-massive.html

Saturday, February 18, 2012

How to set up remote logins via VNC for Debian, Solaris and Red Hat Clients

Debian Linux (6)

       1. Run the following commands respectively.

root# apt-get update

root# apt-get upgrade

root# apt-get dist-upgrade

       2. Reboot the OS.

root# reboot

       3. Install the GNOME desktop.

root# apt-get install gnome-desktop-environment

**  if "apt-get install gnome-desktop-environment" says the packages are missing ensure "deb http://ftp.us.debian.org/debian squeeze main contrib non-free" is added to the sources.list file.

 **********adding the link to sources.list************************************************************************

                         I.  root# vi /etc/apt/sources.list

                       II.  Paste "deb http://ftp.us.debian.org/debian squeeze main contrib non-free" without quotes.

                      III.  Restart the process from step 1.


            4. Install some (required) fonts for the VNC server GNOME session.

  root# apt-get install xfonts-100dpi
  root# apt-get install xfonts-100dpi-transcoded
  root# apt-get install xfonts-75dpi
  root# apt-get install xfonts-75dpi-transcoded
       root# apt-get install xfonts-base

       5. Install VNC.

                     root# apt-get install vnc4server

       6. Run VNCserver manually and set up a password.

   root# vncserver

       7. Making VNCserver start at system start up. 

*        Run the following command to create the vncserver file.

root# vi /etc/init.d/vncserver

*        Copy the following script, paste and save.

 8. Replace “my-vnc-server” with the hostname and keep with quotes. (Optional)

#-----Beginning of the script----------------------------------------------------------------------------------------

#!/bin/sh -e
# Provides:          vncserver
# Required-Start:    networking
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6


# The Username:Group that will run VNC
export USER="root"

# The display that VNC will use

# Color depth (between 8 and 32)

# The Desktop geometry to use.

# The name that the VNC Desktop will have.

OPTIONS="-name ${NAME} -depth ${DEPTH} -geometry ${GEOMETRY} :${DISPLAY}"

. /lib/lsb/init-functions

case "$1" in
log_action_begin_msg "Starting vncserver for user '${USER}' on localhost:${DISPLAY}"
su ${USER} -c "/usr/bin/vncserver ${OPTIONS}"

log_action_begin_msg "Stopping vncserver for user '${USER}' on localhost:${DISPLAY}"
su ${USER} -c "/usr/bin/vncserver -kill :${DISPLAY}"

$0 stop
$0 start

exit 0

#-----End of the script-----------------------------------------------------------------------------------------------

       9. Grant permissions for everyone to execute the file.

     root# chmod +x /etc/init.d/vncserver

      10. Register the VNCServer service to run at startup.

     root# update-rc.d vncserver defaults

       *  If VNC login shows a gray screen

     root# chmod 755 /etc/X11/xinit/xinitrc

Solaris (10)

1. Download the latest VNC package for Solaris from http://www.realvnc.com/products/download.html

       2. Go to the download location of the file in the terminal and run the following commands.

root# gunzip vnc-4_1_3-sparc_solaris.pkg.gz

root# pkgadd –d vnc-4_1_3-sparc_solaris.pkg

   Solaris 10 is shipped with a basic VNC service mostly configured. This is the procedure to enable it.

       3. Find VNC service.

root# svcs -a | grep -i vnc
disabled 13:47:12 svc:/application/x11/xvnc-inetd:default 

       4. Enable VNC service.

root# svcadm enable svc:/application/x11/xvnc-inetd:default 

       5. Note that VNC is broken by default, some changes will be required.

root# svcs svc:/application/x11/xvnc-inetd:default
maintenance 14:22:41 svc:/application/x11/xvnc-inetd:default

      6. Append VNC to the /etc/services.

root# echo "vnc-root\t5900/tcp\t\t\t# Xvnc" >>/etc/services

       7. Check /etc/services.

root# tail /etc/services
snmpd 161/udp snmp # SMA snmp daemon
vnc-root 5900/tcp # Xvnc

        8. Note, the GNU display manager is not customized yet, and needs correction.

root# ls -al /etc/X11/gdm/custom.conf
/etc/X11/gdm/custom.conf: No such file or directory

        9. Enable and configure GNU display manager for VNC.

root# cat >/etc/X11/gdm/custom.conf <<!

       10. Check the customization configuration file.

root# ls -al /etc/X11/gdm/custom.conf
-rw-r--r-- 1 root root 85 Dec 19 14:43 /etc/X11/gdm/custom.conf

       11. Disable and re-enable, and validate the VNC service.

root# svcadm disable svc:/application/x11/xvnc-inetd:default
disabled 14:46:29 svc:/application/x11/xvnc-inetd:default

root# svcadm enable svc:/application/x11/xvnc-inetd:default

root# svcs svc:/application/x11/xvnc-inetd:default
online 14:46:43 svc:/application/x11/xvnc-inetd:default 

       12. Edit /etc/default/login and comment out the following line. 
             (This permits to log on as Root.)

                     * Force-save using wq! as it is read-only.

                             # CONSOLE=/dev/console

RHEL (Red Hat Enterprise Linux) 

If the RHEL installation is licensed then this is not difficult at all. You just need to run

# yum install tigervnc-server -y

at a terminal. However this gets complicated when the installation of RHEL is not licensed.

After a considerable number of disappointing hours of searching, testing, and some research I finally found a solution to this. Here I am going to use TigerVNC as the VNC server and RealVNC viewer as the client (you can use any VNC viewer you prefer).

1. Navigate to the TigerVNC downloads which at the moment of this blog post is hosted as a SourceForge project here  and download the latest build corresponding to the bit version of your linux installation.

2. Extract the contents into a suitable location.

3. Within the extracted contents a folder named bin exists and it contains all the binaries required for the sound operation of VNC server (except the one which as they say is obsolete now). Locate this folder and copy the contents of it to /bin directory in linux. The following files need to be copied.
  • vncconfig
  • vncpasswd
  • vncserver
  • Xvnc

4. Execute the command vncsever at a terminal.

# vncserver

  • This will require you to set a password for logging in via VNC remotely.
  • This will also create the necessary files required to start a xsession for a particular user when logging in via VNC.
5. Usually after the above step you are good to go but in some cases you might end up with an error Connection timed out(1006) at the VNC viewer when you try to log in. This is because the host in which the VNC server runs is blocking the VNC communication, i.e. it's blocking the VNC port-5900. Here you have to allow the port 5900 at the firewall as a trusted port or if security is not a big issue with the Red Hat host then you can simply disable it.

Friday, February 17, 2012

How to enable DDNS (Dynamic DNS) in DHCP Clients

     Debian Linux (6) 

       1. Configuring the OS to obtain an address from DHCP.

*        Edit the “interfaces” file.

 root# vi /etc/network/interfaces 

*        Copy and paste the following in it.

auto eth0
iface eth0 inet dhcp

        2. Configuring the OS to register with the DNS

*        Edit the “dhclient.conf” file.

root# vi /etc/dhcp/dhclient.conf

*        Uncomment the following line by removing the ‘#’. If the line is not present enter it to the file.

# send host-name "hostname";

*        Replace the hostname with the real hostname and keep with quotes.

 Solaris (10)

 1. Log into the terminal as root and proceed with the following steps.

        2. Change the FQDN of the host.

*        Enter the FQDN of the host and save the file.

 root# vi /etc/nodename 

*   This command will edit the file nodename file if it exists. If not it will create the file.

        3. Configure the OS to update the DNS.

*        Enter the following line in the file hostname.network_interface and save. 

*        Replace the network_interface with the network interface’s name.

         inet hostname

*        Replace the hostname with the actual hostname.

root# vi /etc/hostname.network_interface
*   This command will edit the hostname.network_interface file.

RHEL / Fedora / CentOS Linux

 1. Open the configuration file for the network interface.
# vi /etc/sysconfig/network-scripts/ifcfg-eth0

   * Here it is done for the interface eth0.
2. Append the following line to it.

3. Save and close the file. Restart the network service.

# service network restart

  ** Please refer to dhclient.conf man page for more information.

$ man dhclient.conf

     Windows (if somehow the default configurations aren’t working)


         * Validate whether the ‘Register this connection’s address in DNS is checked.

*        Start -> Run -> ncpa.cpl -> right click on the relevant connection -> properties -> double click on TCP/IP -> Advanced -> DNS