Microsoft Exchange and Remote Desktop Services Specialists

SEMblog

Microsoft Exchange Server and
Blackberry Enterprise Server news, views and fixes.

Kemp Release Free Load Balancer Virtual Appliance

Kemp have released a free load balancing virtual appliance. If you have a small environment and don't need the high availability of two load balancers, then this could be an ideal solution. 


There are some limitations, particularly around the throughput (only 20mps) but if you are using a small environment or a lab, then it could be all that you need. Absolutely no reason to use Windows Network Load Balancing any more. 

If you have Kemp load balancers in your production environment, then it is an ideal way to have the same in your test environment. It will also make this a valuable learning tool for Exchange and server administrators. 

No support included, but that is to be expected. 

It looks like it is pretty much the complete feature-set from Kemp, including:

Layer 4/7 load balancing
Content switching
Caching, compression engine
MS Exchange 2010/2013 optimized
Pre-configured virtual service templates

The only thing it is missing features wise is Active/Hot Standby redundant operation.

If you are going to use this in a production site, then I would watch that maximum throughput though. 

Changes to SSL Certificates

There have been a lot of changes to the way that SSL certificates are issued and the impact of those changes are now being particularly felt within the Exchange community. 

What has changed?

The CA/Browser forum (made up of the companies that issue the certificates and the browser developers who use them) decided that that all certificates issued with an expiry date after 1st November 2015 will be restricted to internet resolvable FQDN's only. 
This means that you cannot have an SSL certificate with:
- Single name hosts - such as intranet, server, exch01
- Internal only domains - such as server.example.local
- Internal IP addresses (both Ipv4 and Ipv6). 
This applies to both the common name and any additional names on the certificate. 

Furthermore, if you have a certificate that is still in force with an invalid name from the list above, then it will be revoked on 1st October 2016. 

How does this affect Exchange?

Exchange 2003 isn't really affected by this, because most people simply purchased standard single name SSL certificates. 

Exchange 2007 and later however are being impacted. 
During the early life of Exchange 2007 the advice for SSL certificates was to include both the internal and external host names of the Exchange server. This was because the default configuration of Exchange uses the server's real name and therefore did not require additional modification.

However it quickly became apparent that this wasn't the best way to deploy Exchange web services, as end users were entering the same address internally as they were externally. Split DNS was the answer there http://semb.ee/splitdns

Following the changes to the guidelines for issuing certificates, the changes to Exchange, including setup of a split DNS system is almost mandatory.
I have instructions on how to do that on my main web site at http://semb.ee/hostnames 

Going Forwards

With this change, you can now get away with just two host names on an SSL certificate for full client support:
- host.example.com
- autodiscover.example.com
With our own certificates coming with five "names" available by default, and unlimited server licence, this means you can use the other slots to secure additional services. Once the certificate has been installed on the Exchange server, export it and then import the certificate in to other servers that need it - along side the required intermediate certificate. 
If your DNS provider supports SRV records, then you can even use a standard single name SSL certificate. However mobile devices in particular seem to have some problems with the SRV autodiscover method, so if you are going to deploy mobile devices, stick with a UC (Unified Communications) type certificate. One of the cheapest sources for those is our own site CertificatesforExchange.com http://semb.ee/certs

If you have a certificate with internal names that expires after 1st October 2016, then you should get it rekeyed with the internal names removed, so the certificate is not revoked. 

What else is changing?

From April 2015, the maximum period a certificate can be issued for is being reduced to 39 months. This is to ensure that the names on certificates are checked frequently that they still belong to the original purchaser.

SHA-1 certificates are being phased out very quickly and in 2017 Microsoft will stop trusting them. However a lot of browsers will start showing warning messages on these kinds of certificates in 2016. Therefore to protect yourself, ensure that you are requesting SHA-2 certificates and have replaced any SHA-1 certificates by the end of 2015.

Action Points

What should you do about your own SSL certificates?

  1. Check whether they are SHA-1 or SHA-2. 
    To do that, browse to the SSL site, then open the SSL certificate. Click on the Details tab and then look for Signature Hash Algorithm. It should NOT say SHA1. 
    Do not confuse with Thumbprint Algorithm, which will always say SHA1, no matter the type of the certificate.
    If they are SHA1, then get them rekeyed to SHA-2. If your provider doesn't allow that, then change provider. http://semb.ee/certs

  2. Check your server configuration and start to move everything over to use the same host name internally and externally. This is easily done by setting up a split DNS system, then changing the Exchange configuration. If your certificate still contains the internal names they will continue to work until you change the SSL certificate, providing a time to educate the end users about the names to use. 
Remember if you replace a certificate before it has expired, revoke the old one. This will often happen automatically when you get a certificate rekeyed, but it does no harm to do that yourself anyway. 

Net Framework 3.5 Installation errors Windows 2012/2012 R2

Recently tried to install Net Framework 3.5 on to an existing server which had been in production for a few months. 
Constantly failing with an error about being unable to find the source files, even though it was using an ISO which was used to build this and many other servers in the past. 

Clutching at straws, discovered that the server had a Windows Update installed, released in September 2014 for Net Framework 3.5, even though it wasn't installed. Some research on the internet indicated that it was one of these three:

KB2966826
KB2966827
KB2966828

Removing the update then attempting the installation again was successful. 

Once Net Framework had been installed, I ran Windows Update to reinstall the update I removed, plus numerous others that were required for Net Framework. 

Cross Site DAG Issue When Using A Load Balancer

Just deployed a new Kemp Load Balancer with a client which promptly broke their cross site DAG.

Usual horrible error:

[PS] C:\Windows\system32>Get-DatabaseAvailabilityGroup -Status

WARNING: Unable to get Primary Active Manager information due to an Active Manager call failure. Error: An Active

Manager operation failed. Error The Microsoft Exchange Replication service may not be running on server XXX-3. Specific

 RPC error message: Error 0x6ba (The RPC server is unavailable) from cli_AmGetDeferredRecoveryEntries.

 

(server xxx-3 is the remote server).

Discovered that the problem was due to an option enabled on the Kemp called Enable Server NAT (SNAT). You can find this under System Configuration, Miscellanious Options, Network Options. Disabling that corrected the issue almost immediately. Seems that the NAT broke the DAG. 

Experiences with IPv6

 

IPv6 has and continues to cause a lot of confusion for network administrators. I suspect that a lot of it is down to misunderstanding about the new system and therefore people blame it for problems because it is new. 
In forums, I see a lot of people who simply post that the problem is "IPv6" and it should be disabled because it is "known to cause problems". I have been asked to clean up Exchange deployments where IPV6 has been disabled, simply because the installer believed it would cause issues if it wasn't. 

In many cases this is simply not true - IPv6 is not the cause of many problems and in the default configuration on Windows 2008/2008 R2 will not get in the way of day to day operations. 

However IPv6 is not going to go away and very soon most network administrators are going to need to do something with it, whether they like it or not. Therefore getting experience with it now, before being forced to do so can only benefit the network administrator. 

Back in the summer, just after the World IPv6 day, I decided to look at using IPv6 myself and have been running an IPv6 network at home ever since. The web sites that I operate are also IPv6 enabled and if you are already using IPv6 you may well be reading this blog posting having accessed the site using Ipv6. 

Initial Experiences with IPv6

Having now lived with IPv6 for a few months, I thought I would write up my experiences with using it. In brief, I have found it to be largely trouble-free. I was caught out by a few small issues at the beginning, but after being setup correctly, it has been largely set and forget. 

IPv6 Addresses and Getting Started

The first thing you have to do is get hold of an IPv6 subnet. Most ISPs are currently not issuing IPv6 addresses, so you have to source them from elsewhere. Having researched on the easiest way to do this, I settled on using a tunnel broker, specifically Hurricane Electric (HE). 
It was easy enough to sign up with them, and before long I had the single address required. HE are giving out the addresses free, and allow you to choose where to create tunnel to. I chose London, being in the UK. 

I built a Windows 7 machine in a virtual machine, and followed their instructions to enable it - which was simply a matter of entering some commands in to a Command Prompt to configure the tunnel, which Windows 7 supports natively. 
For the tunnel to create, HE need to be able to ping your EXTERNAL IPv4 address, so a firewall change might be required. 

Once entered, I tried to ping ipv6.google.com, only for it to fail. 

Therefore I hit the first problem, which is probably the most common issue with IPv6 - hardware support. 

I am fortunate that I have dual internet connections, regular ADSL and a cable internet. My ADSL had a Cisco router on it, and I quickly discovered that Cisco only support IPv6, even pass through, on specific OS types and I had the wrong one. I wasn't paying for the upgrade for a test (And was planning to drop the Cisco router a few months later when I got fibre internet), so I decided to use the second connection. 

Switching over to my cable internet connection, which used DDWRT, the tunnel passed through immediately and I was on the internet using IPv6. 

I have to say, it was a rather underwhelming experience - it just worked. 

Putting the Network on to IPv6

With the single machine on IPv6, I thought I would see if I could put my entire home network on as well. This meant the router needed to support it. I played around with DDWRT for a while, but found it wasn't easy to configure with IPv6 information. Therefore I changed my attention to my public web server. 

The web site that you are reading this posting on is actually a virtual server. The firewall that protects it is a VM, and generally I can do what I like with the system. A look around at other software firewalls it quickly became clear that the best one for IPv6 was monowall. It didn't take long to install a fresh VM with monowall and used a spare external IPv4 address. 

After requesting a second subnet from HE I entered them in to the firewall and I was online. Took minutes to configure monowall. I then set the firewall rules so I could ping out etc and all was good. I modified the firewall rules to allow inbound traffic and after a few minutes configuring the DNS records, I was able to browse my web server with IPv6 from my test system at home. 

Using IPv6 also gives the strange sensation of the same IP address internally as well as externally. No NAT involved. I haven't been in that situation since a job back in 1999, where the employer had enough public addresses that we could use them internally as well. 

IPv6 Addresses

One of the things that did start to cause me a headache was dealing with the IP addresses themselves. You have so many and they are so long. However I quickly learned about the "::" shortcut, which allows you to shorten the addresses. What this means is that instead of using:

2001:470:1f09:1ab5:0000:0000:0000:0090

I can use this instead:

2001:470:1f09:1ab5::90

From an understanding point of view, I found that using the same number at the end of the IP address for both IPv4 and IPv6 made managing the addresses much easier. For example this blog is on 85.234.131.90, so I used ::90 at the end of the IPv6 Address. 

With the addresses configured on the web server, it means I can just look at the last number to ensure that I am putting in the correct bindings to the web server. 

Boyed by the success of getting the public web site to work, I looked again at my home network. Switching the DDWRT router for a monowall virtual machine meant that I was able to configure the home network for IPv6 quickly, and also meant that I now had a static IPv6 address running over my dynamic IP address cable internet connection. 

With the addresses the length they are, you have a lot of addresses available to you. 

I subnetted my allocation down further, which has allowed my labs to have their own IPv6 subnet. This means my labs "could" be seen from the internet, if I set the rules to allow that to happen in the firewall. Once IPv6 is widely used, I can see that as a major advantage, particularly if you are testing email servers. 

DHCP or Not

One of the features of IPv6 is that removes the need to have traditional DHCP. Instead you enter information on your router and it is able to "announce" that information which IPv6 clients are able to find. 

Microsoft do provide an IPv6 DHCP server, which I had some success configuring, but as the information from the router was correct, it wasn't something I pursued. 

Most of the systems on my network I entered static IP address information for, but I did find they were getting an automatic address as well, which must be a feature of IPv6. However when reviewing WSUS for example, the static address I assigned is being entered in to WSUS as the server's IP address. 

The impression I am getting though is that the IP address of the system is going to be less important, at least internally. I have set static addresses in the public DNS and on the servers, and those work correctly, but internally the network is also using the automatic addresses. 

With the length of the IP address, remembering them for doing testing isn't going to be easy. Therefore I can see in the future that DNS will become more important, so that you can simply ping or nslookup the host name, to get its IPv6 address, then work from there. 

DNS

I mentioned DNS above. 

Backwards compatibility for DNS on IPv6 works really well - you have two entries for most things - an IPv4 A record and an IPv6 AAAA record. The AAAA record takes priority. 
This can of course give unexpected results, particularly when troubleshooting. Therefore what I have done is create three records for hosts where I am likely to want to do troubleshooting (mail servers mainly).

  1. The regular host name - so host.example.com - both A and AAAA records. 
  2. A IPv6 specific host name - so ipv6.host.example.com  - this is an AAAA record only. 
  3. An IPv4 specific host name - so ipv4.host.example.com - no AAA record. 

A good example of this in action can be seen on a basic IP address display site I built. 

If you browse to the site normally then you will have the IPv6 address displayed if you are using IPv6 and the IPv4 if you are using IPv4. 

http://ip.sembee.info/ 

However further down the page are links to other versions of the page - one for displaying just the IPv4 address and then one that displays both IPv4 and IPv6. Moving through the pages you will notice that the host name in the browser bar changes, so that the correct DNS entry is used. 

Email

As an Exchange MVP, I was of course interested in how this would work for email. 
For email it is just a matter of adding the additional AAAA record for the host name. 
MX records point at host names, and then the client resolves the host name to an IP address. 
Therefore my MX record hosts have both IPv4 and IPv6 addresses. 
Although monitoring the email I have found that I have received just three emails (all marketing) from IPv6 hosts. 

Drawbacks with IPv6

The major issues with IPv6 is support of the address type. 
I quickly discovered that the antispam solution I am using cannot cope with IPv6 addresses, but as spammers aren't using it, it hasn't been a problem so far. 
I also discovered that my web stats application doesn't support IPv6, so I have no real ideal how many people are accessing my web sites with IPv6. Certain applications also have issues with IPv6, but in a lot of cases this is only if it is pure IPv6, not a mixed network. 
The length of the IP address I think will be something that many network admins will find difficult and will miss being able to type in the four sets of numbers from IPv4. I fully expect IPv4 to be used internally for some time to come, perhaps with IPv6 being used just for internet traffic.

Conclusion

As you can test Ipv6 with almost no disruption to the production network, it is something that network admins should take a look at, so that they simply get their heads around it. Then as it becomes more widely used, they  have already been through the learning curve.