Fabian Williams SharePoint Blog

Solving problems with SharePoint day and night

Setting up NLB for your WFE’s in SharePoint


great article originally found here:  http://www.west-wind.com/presentations/loadbalancing/NetworkLoadBalancingWindows2003.asp

NLB – Network Load Balancing

WFE – Web Front Ends

Setting up NLB

In order to utilize the Windows Server Network Load Balancing features you will need two machines running Windows Server 2003. Each machine needs to have at least one network card and at least one fixed IP address. Although running with one adapter works well, for best performance it’s recommended that you have two adapters in each machine – one mapped to the real IP Address (Microsoft calls this the Dedicated IP) and one mapped to the ‘virtual’ IP Address (Microsoft calls this the Cluster IP). Be aware that NLB uses some advanced networking features of network adapters, so it’s possible that some low end adapters (especially those for non-server machines) may not support the required NDIS protocols.

In addition you will also need one more machine for testing (3 machines total). The test machine should be external as you can’t use a machine from the pool to test – it will only fire request on the local machine since the IP requests are not traveling over the network when you hit the virtual IP address – it goes to the local machine.

I’m going to use two ‘servers’ here to demonstrate how to set up and run NLB. Assume the IP addresses for these machines are 111.111.111.1 and 111.111.111.2. To create a virtual IP address (Cluster IP) you need to pick an available IP Address on the same Class C network segment. In my example here I’ll use 111.111.111.10.

Unlike previous versions of NLB the new version has a central manager application that you can use to create a cluster from a single machine. Gone are the hassles of having to manually configure each machine manually – you can do it all from a single machine over the network which is a welcome change.

To start setting up this cluster bring up the Network Load Balancing Manager from the Administrative Tools menu. Figure 1 shows what the cluster manager looks like.

clip_image001

Figure 1 – To set up a new NLB cluster bring up the Network Load Balancing Manager and right click to createa a new cluster.

Right-click on the root node to add a new cluster. Next configure the basic cluster configuration, which will consist of assigning the Cluster or virtual IP address. Figure 2 shows what this dialog looks like filled out for our test network.

clip_image002

Figure 2 – Configuring the Cluster IP. This is the ‘virtual’ IP address
that will service all servers in the cluster. Note that you should set the
operation mode to Multicast if you are using a single adapter.

The IP Address is the virtual IP address for the cluster that will be used to address this cluster. NLB will actually create a new IP address on each machine in the cluster and bind it to the specified network adapter (in the next step). Choose a subnet mask – make sure you use the same one for all servers in the cluster. The Full Internet name is only for reference and is used here primarily for displaying the name of the server. But if you have a domain configured for the server you should use that domain name.

Cluster operation mode is very important. Unicast mode means that NLB takes over the network card it is bound to and doesn’t allow any additional network traffic through it. This is the reason why two adapters are a good idea – one that NLB can take over and one that can still handle all other network traffic directed at the dedicated IP address of the server. If you’re using a single adapter you should probably select Multicast which allows both the NLB traffic and the native IP traffic to move through the same network adapter. Multicast is slower than Unicast as both kinds of traffic need to be handled by the network adapter but it’s the only way to remotely configure all machines centrally. You can run a single adapter in Unicast mode, but the cluster manager will not be able to communicate with the server after it’s configured. As a general rule use Unicast for two adapters, Multicast for a single adapter. With my network cards I had to use IGMP mode in order to get the cards to converge properly – you may have to experiment with both modes to see what works best for you.

Leave the Allow Remote Control option unchecked. This allows you to reconfigure the nodes and port rules remotely, although I found little need to do so. Any changes made to the cluster are automatically propagated down to the nodes anyway, so there’s little need to do this with the exception of changing the processing priority. If you do want this functionality I suggest you enable it after you have the cluster up and running.

The next dialog called Cluster IP Addresses allows you to add additional virtual IP addresses. This might be useful if you have a Web server that is hosting multiple Web sites each of which is tied to a specific IP address. For our example here, we don’t need any and can just click next as shown in Figure 3.

clip_image003

Figure 3 – If you need to add additional IP addresses to be load balanced
you can add them here. This is needed only if you host multiple sites
on separate IP addresses and you need separate IPs for these.

Next we need to configure port rules. Port rules determine which TCP/IP port is handled and how. Figure 3 shows the Port Rules dialog with two port rules defined for Port 80 (http) and 443 (SSL). The default port configuration set up by NLB handles all ports, but in this case that rule is too broad. Port rules can’t overlap so if you create specific rules you either have to create them for each port specifically or create ranges that fit your specific ports.

clip_image004

Figure 4 – The Port Rules dialog shows all of the port rules defined for
cluster. By default a rule for all ports – 0 – 65365 is defined. Here I’ve

Created to specific port rules for port 80 and 443.

To add a new port rule click on the Add button which brings up the dialog shown in Figure 5. Here you can configure how the specific port is handled. The key property is the Filtering Mode which determines the affinity of requests. Affinity refers to how requests are routed to a specific server. None means any server can service the incoming request. Single means that a specific server has to handle every request from a given IP address. Generally None is the preferred mode as it scales better in stateless applications. There’s less overhead in NLB as it doesn’t have to route requests in many cases. Single mode is useful for server connections that do require state, such as SSL connections for HTTPS. Secure Server Certificates performs much better with a persistant connection rather than having to create new connections on each of the servers in the pool for requests. Figure 1 shows the configuration for the standard Web Server port – port 80.

clip_image005

Figure 5 – Setting port rules lets you configure how the cluster
responds to client requests. Affinity in particular determines
whether the same server must handle all requests from
a specific IP address (single) or Class C IP address range (Class C).

To set up the second rule for the SSL Port I added another rule and then changed the port to 443 and changed the affinity to single.

Although you can’t do it from here, another important setting is the priority for each machine for each port rule. You can set up Machine 1 to take 80% of the traffic and the second 20% for example. Each rule can be individually configured. We’ll see a little later why this is important for our SSL scenario.

The rules set in this dialog are propagated to all the cluster servers, which is significant, because the cluster port rules must be configured identically on each of the cluster node servers. The configuration tool manages this by remotely pushing the settings to each of the cluster nodes Network Connections IP configuration settings. This is a big improvement over previous versions where you manually had to make sure each machine’s port rules matched and stayed matching.

Up to this point we have configured the cluster and the common parameters for each node. Now we need to add individual nodes to the cluster. Figure 6 shows the dialog that handles this step for the first node as part of the configuration process.

clip_image006

Figure 6 – Adding a node by selecting the IP address and picking a specifc
network adapter.

When you click Next you get to another dialog that lets you configure the cluster node. The main feature to configure on this dialog is the Priority which is a unique ID that identifies each node in the cluster. Each node must have a unique ID and the lower the number the higher the priority. Node 1 is the master which means that it typically receives requests and acts as the routing manager although when load is high other machines will take over.

clip_image007

Figure 7 – Setting the node parameters involves setting a priority for
the machine, which is a unique ID you select. The lower the number
the higher the priority – this machine acts as the master host.

Click finish and now we have one node in our cluster.

Actually, not quite so fast. Be patient, this process isn’t instant. When you click finish the NLB manager actually goes out and configures your network adapter for you. It creates a new IP address in your network connections, enables the Network Load Balancing service on your network adapter(s) you chose during setup and configures the setting we assigned on the NLB property sheet.

You’ll see your network connection flash on and off a few times during this configuration process on the machine you are configuring to be a host. This is normal, but be patient until you see your network connection back up and running.

If all goes well you should see your network connection back up and running and see a new node in the NLB Manager sitting below the cluster (see Figure 8 which shows both nodes). If everything is OK the Status should say Converged. If it does node 1 is ready.

But we’re not quite done yet – we still need to add the second node. To do so right-click on the cluster, after which you go through the steps shown in Figure 7 and 8 one more time. Again be patient, this process is not super fast – it takes about 20 seconds or so to get a response back from a remote machine. Once you click finish the process of Converging can take a minute or more.

clip_image008
Figure 8 – The final cluster with both nodes converged and ready to process requests.

Troubleshooting Tips

I’ve had a few problems getting convergence to happen for the first time. It helps to follow the steps here closely from start to finish and if for whatever reason you end up removing nodes make sure you double check your network settings first before re-adding nodes.

You can check what NLB did in the Network Connections for your machine (Figure 9). Click on the Load Balancing section to see the settings made there. Remember that the settings should match between machines with the exception of IP Addresses assigned for each machine. You should also see the new IP address added in the Internet Protocol settings’ Advanced page.

clip_image009

Figure 9 – All of the setting that NLB makes are made
to the network adapter that the virtual IP is bound to.
You can click on the Network Load Balancing item to
configure the node settings as described earlier. The Virtual
IP also has been added in the Internet Protocol | Advanced
dialog.

If things look Ok, make sure that the machines can ping each other with their dedicated IPs. Figure 10 shows what you should see for one of the machines and you should run this test on both of them:

clip_image010
Figure 10 – Checking whether the machines can see each other.

Use IPCONFIG to see adapter information and you should see both your physical adapter and the virtual IP configured. Make sure that you don’t get any errors that say that there’s a network IP address conflict. If you do it means that the virtual IP is not virtual – ie. It’s entered but it’s not bound to the NLB service. In that case remove the IP and then configure the NLB first, then re-add the IP address. Alternately remove everything then try adding it one more time through the NLB manager.

I’ve also found that it helps to configure remote machines first, then configure the machine running the NLB Manager (if you are using it in the cluster) last. This avoids network issues on the manager machine – plain network access gets a little weird once you have NLB configured on a machine. Again this is a great reason to use two adapters rather than one.

Advertisements

April 28, 2009 - Posted by | IIS, Windows Administration

2 Comments »

  1. Hi Fabian,

    great article to nlb in moss environment.

    Do you have experience as well in nlb and kerberos setup??

    Do I have to create and spn to the clusternode to?? At the moment I have only configured the machines itsself.

    Would be great to hear from you.

    Best regards,
    Knut

    Comment by Knut | November 16, 2010 | Reply

    • If you touch kerb you need spns

      Comment by fabiangwilliams | November 25, 2010 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: