Monthly Archives: June 2009

Standard

Posted by

Posted on

June 5, 2009

Posted under

ISA Server

Comments

Leave a comment

ISA Server 2006 NLB Issue with the L3 Switches

Recently I worked on the similar issue with a customer and thought of writing a definite solution for this issue. We resolved the issue by enabling the multicast mode on the ISA Servers. By default, the NLB on ISA Server works in Unicast mode. For getting it worked with the L3 switch we need to change it to the multicast mode. This issue is related to the Windows NLB but since ISA Server uses the Windows Integrated NLB so it inherits the same limitation.

I came across a nice article which describes the Unicast and the Multimode with NLB http://blogs.technet.com/networking/archive/2008/12/09/balancing-act-what-you-really-ought-to-know-about-windows-server-nlb.aspx

To change the Unicast mode to the Multicast mode follow the below steps

  1. Make sure you have the SP1 installed for the ISA Server 2006
  2. Remove the NLB settings from the ISA nodes and Disable the NLB on is the ISA nodes
  3. Follow the article http://support.microsoft.com/kb/938550
  4. Download the Hotfix from the above article and unzip it on your ISA machine
  5. Run the following command to see what mode are you running in

    Cscript KB938550.wsf /array:OBSERVER1 /show

  6. If it shows that the interface you are tyring to enable the NLB is in Unicast mode then run the following command to change it to the Multicast mode

    Cscript KB938550.wsf /array:<Array Name> /nlb:multicast /net1:<network Name>

    Where: Array Name is the name of your Array and the Network Name is the name of the network on which you are trying to enable the NLB

  7. To change the ISA servers back to the Unicast mode run the following command

    Cscript KB938550.wsf /array:<Array Name> /nlb:unicast /net1:<network Name>

     

At times you may come across an issue where you tried to enable the NLB with Unicast mode and then were not able to make it working because you were using the L3 switch. Since, you enabled the NLB and if you tried to assign any virtual IPs then it won’t allow you to remove those. If you are not able to remove them even after removing the NLB settings manually then follow the below settings

  1. Stop the Firewall service
  2. Manually deleted the subkeys under the following Registry keys from both ISA Servers

    HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesWlbsParametersAdapters

    HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesWlbsParametersInterface

  3. Removed the VIPs from the ISA NIC
  4. Restart the Firewall Service

 

Enjoy