Microsoft Exchange and Remote Desktop Services Specialists

SEMblog

Microsoft Exchange Server and
Blackberry Enterprise Server news, views and fixes.

Experiences with Grey Listing

I have heard from many sources that grey listing can be an effective weapon for fighting spam, yet hadn't had an opportunity to try it out.

However one of my clients was being hammered hard with spam, with 700+ messages a day being filtered by Intelligent Message Filter, and lots of messages getting past "I Hate Spam" from Sunbelt Software. Therefore I thought they would be a good site to test the method with.

I installed Vamsoft's ORF (http://www.shareit.com/product.html?productid=169362&affiliateid=200023740) on to the gateway machine and left it to get on with it, enabling just the grey listing and the automatic white list* feature.

* The automatic white list is built by watching outbound email and recording the email address used. When the external recipient replies, the message comes straight back in as the server then knows that the email address is legitimate.

The effect was immediate and noticeable. I watched the logs of the application very carefully to ensure that no legitimate email was being blocked. The amount of spam that was blocked by the application was considerable. After a running a week, the application reported that over 85% of all email that was being received was spam.

That doesn't count messages that were dropped by the filter on unknown users (http://www.amset.info/exchange/filter-unknown.asp).

The process isn't 100% effective, IMF was still catching some messages - but this was down to 20 or 30 a day, a massive reduction in the pre-grey listing number.
Users were also reporting that a few items were reaching their inboxes, but nothing like the level they had been receiving.

I have since deployed the application on four other sites, including my own Exchange server and seen similar significant drops in the number of spam messages being received.

As with user filtering, this technique also saves the bandwidth, as the messages are not even delivered to your server, so don't have to be processed.

The Vamsoft product works with any IIS SMTP mail server, so if you have Exchange 2000 then you can use it as well. It also features Active Directory filtering, which Exchange 2003 has built in, allowing users of the older version of Exchange to benefit.

How Does It Work?

Grey Listing is very simple idea.

A server attempts to deliver the message to the server. If the server hasn't received an email from that sender before, then it rejects the message with a temporary failure.

The systems that spammers use don't care about failure messages. They aren't interested in the failure and will therefore not try again. Spammers want to drop and run, before any system blocks the IP address that they are sending their email messages from.

However a legitimate email server will try again. Most email servers will try again for up to 48 hours, so you will get the email message eventually.

Are there any risks?

Any anti spam technique comes with risks. Unless you have a human looking at every message, you are relying on the computer making the decision whether the message is spam or not.

This technique will introduce a delay for new email messages - I have seen the delay as short as 90 seconds up to 20 minutes or more. If your business cannot tolerate any delay in email message delivery then this technique is not for you.

I have also seen a few email messages fail to be delivered from some sites that generate large amounts of email - such as eBay and a few ISPs. This is because each message appears to come from a different IP address in their server cluster.
With eBay, white listing their domain isn't advised as that will also allow in phishing emails.

Conclusion

While spammers don't comply with the RFC on SMTP email delivery and try just the once to deliver their email messages, this technique will be an effective first strike weapon in the war on spam. It shouldn't be considered the only weapon, but combined with other techniques can make spam more manageable.

Disappearing Blog

If you have visited this site in the last couple of days, you may have received odd HTTP error messages. This blog "disappeared".
On Wednesday (23rd) afternoon, the server that hosted this blog, along with the amset web sites suffered a hardware failure. It was the worst kind of failure for a web server - the hard disk.
The hosting company replaced the machine quickly and I was able to get the amset web sites running around six hours later.
Unfortunately the blog had to wait as it required more work, including configuring Community Server to operate in the way that I liked.
I have now completed the configuration and am in the process of populating the blog with my old articles.
I kept a copy of the articles offline on my home system, along with original publishing dates so I can easily restore the content.
I also took the opportunity to update the server to the latest release of Community Server.
For those of you reading this through RSS, I apologise for the number of "new" articles that you have seen popup. That was me repopulating the blog and couldn't be helped.

Internet Service Separation

One of the tactics I have been using with my clients for many years is something I call internet service separation.
This is where I use different providers for different aspects of the internet service that the client needs. 
This doesn't go down well with many internet companies (whether this is Internet service providers, web hosts etc). They like to have control over everything, get you to use their service for everything etc.
This isn't for your benefit despite what they may say in their sales brochures. It is for their benefit as it makes it much more difficult to leave them. You have to juggle all of the services being disconnected at the same time. For many people, especially those who don't understand how the internet works, they will not want the hassle. It is that reluctance to move that allows companies to get away with poor service.
You should have different companies for the following tasks:

  • Domain Registration.
    Use a specialist such as 123-reg.co.uk here in the UK, or Go Daddy or register.com in the US. Don't use them for anything else (despite what they might tempt you with).
    Use a big provider, which limits the chances of them going down. Although most of the domain name registrars are actually using the services of one of the others, so in the event of a failure you may be able to rescue the domain name. 
  • Internet connection.
    This should come from a service provider who gives you the best deal. Unless you are on a managed service, use your own kit. Routers etc, so that you have control.
    The only thing they should be giving you is IP addresses. Everything else should come from other suppliers
  • Web Hosting.
    This should be with a dedicated host. The web hosting market is so competitive that the choice is endless.
    Try to steer clear from free web hosts - the old adage of "get what you pay for".
    However you don't have to pay over the odds for hosting - especially if the site is a simple static brochure type site. 
  • Email.
    Ideally you should be using your own email server. I am an Exchange specialist and this posting is from an Exchange server point of view. 
    Although, if you have more than five or six staff, you are getting to the point where you can justify your own server. This doesn't have to be Exchange - there are many low end options that will provide you with in-house email services without the complexity of Exchange.  

Your Domain Name
The thing that internet service companies all want is to get control of domain name. Preferably transferred to their own domain name registrar, or in to the master account at their pet domain name registrar if they aren't one already.
As that is your company identity, you don't want to loose it. Once they have control over the domain name, they can effectively hold you to ransom.
Resisting attempts to gain control over your domain name is very difficult, and trying to get hosting companies to comply with something else can be a challenge. They can do it - they just don't want to - as there is nothing in it for them.
I have even had companies say that they cannot do what I need them to do - which is a outright lie. Very shortly afterwards they will usually lose the business. For one UK ISP this meant a loss of over £20k in annual revenue as I took a large number of home user accounts, a leased line and other services away as well - I actually had an account manager on the phone begging to be given another chance and crying when they found out.

Despite what any web hosting company, ISP or whoever states - you do NOT have to transfer your domain name to them to use their service.

A domain name transfer is just a way of getting control and also earning themselves some more money from the transfer fee. 
All you need to do is ask them for their name servers, ask them to put your domain name in to their name servers, then enter the name servers in to the relevant option at the domain name registrar.
You have maintain complete control. In the event that you want to move your web site to another host, then you just need to change the name servers. The hosting company doesn't need to know anything about it. I have changed hosts many times, and the first the old company knows about it is when I ask to terminate their service. At that point I am not using them for anything, so if they cut me off immediately, it doesn't impact my web sites in any way.
If you do change the name servers, then you need to use the web hosting company to manage your DNS. Make sure that you have the correct entries in place first.

A better option is not even use their name servers.

Ask for the IP address of the web site and enter that in to your DNS at your domain name registrar. This is often a good idea when you are hosting your own email, as it is not uncommon for web hosts or ISPs to "reset" their DNS records which set the MX records back to their email servers rather than yours.
Protect the domain name like you would any other asset of the company. Make sure that you do whatever it takes to ensure that it remains under your control at all times.

Three Rules of Microsoft Licensing

I have been posting these three rules of software licensing in various forums for a couple of years now, so it made sense to include them here.
Purchasing software licenses for Microsoft products is daunting, with multiple choices and schemes available to you. However as long as you consider these three rules, you shouldn't go too far wrong.

  1. Get at least three opinions, including one from Microsoft.
    Even some people at Microsoft don't understand all the options, so if you aren't sure on something then make sure that you get three opinions. 
  2. Get in writing.
    Without it in writing, it is worth nothing if you are audited. 
  3. The most expensive option will be the correct one.
    That is pretty obvious I think.

It will not make licensing any easier, but it will help you sleep at night in the knowledge that you have at least tried to do the right thing.

Out of the Office Messages to the Internet

When setting up the Exchange server, you need to consider whether to allow Out of the Office Messages (OOTO) to the internet or not.
These are not sent to the internet by default on Exchange, you have to actually go in to the system and set the option.
However should you enable the option?
Some people consider them to be important, others a hindrance.
If you are a member of any email distribution list, then you will almost always get at least one out of the office response if you post to the list.
The decision on enabling OOTO messages to the Internet is probably not something for the Exchange administrator to decide. As they can play a part in the internal business processes, it should be considered by the management of the business to ensure that they fit in with those processes.
Remember that internal OOTO messages are not affected and will always be sent.

What are the issues with OOTO?

There are a number of key issues that need to be considered when the OOTO status is being reviewed.
There are four major issues with OOTO messages.

  1. Security.
    The OOTO message could contain information that the person receiving it shouldn't have. Mobile phone numbers, names and numbers of other contacts in the company etc.
    The message could also indicate that the person is out of the country, whether on holiday or on a business trip. It is clearly identifying that the home is empty. If the staff member is a director, then their home details could be easily discovered, and the home broken in to shortly afterwards.
  2. Technical issues.
    Not so much an issue with OOTO on Exchange, but other systems will use automatic replies instead of an OOTO system. These can cause email loops. The message bounces back to someone with an automatic reply and then bounces back in, and back out and so on. Eventually one server will crash. 
  3. Guaranteed Response
    Any spam gets a response. That confirms the address is live and means more spam.
  4. Can leave a bad impression on the recipients.
    If any staff are members of distribution lists then these lists may get the OOTO messages. These are just annoying for list members.
    Some people consider OOTO to be poor business behaviour as the are effectively saying that no one else is monitoring your email. You should get someone to monitor your email while you are away from the office, in case something important does occur

What can you do about OOTO?

While it is considered good practise to have OOTO and other automatic replies and forwards disabled to the Internet, this is not always practical to fit in with the business practises.

  • Review whether you need to have OOTO going out to the internet. If better practises can be adopted, such as team members monitoring the email, then those should be used instead. 
  • Standardise on the message that is used in an OOTO. Make sure that it states that you are unable to read email and who to contact instead. Give a general phone number - switchboard etc as the contact instead of a direct number or mobile.
  • As an Exchange server administrator, make sure that you have made the registry change to suppress OOTO messages. http://support.microsoft.com/default.aspx?kbid=825370
  • If you have specific external clients who you would like to receive OOTO messages, then you can enable them on a per domain basis.
    Open ESM and choose Global Settings, Internet Message Formats. Right click in the right pane and choose New, Domain. Then enter the information as required. The SMTP domain is the name after the @ sign.

Whichever decision is made, ensure that the staff know which method is being used. If the OOTO is being kept for internal use only, then the messages used can be tailored for that audience.

Future - Exchange 2007

The OOTO behaviour in Exchange 2007 is much improved, with more control over the message, including different messages depending on whether people are in your contact list. The OOTO can be programmed ahead of time to be turned off when you are due to return, instead of having to remember to disable it.