Sembee

Anyone who has done a cross forest public folder migration will almost certainly be reliving their nightmares about it simply from reading the title.
I was just the same.

Extract the content to a PST file, either manually (selecting about 1000 items at a go) or by using a rule, move the PST file to a machine in the new forest, then import.

Slow, mind-numbing dull and therefore not the most fun part of a migration and always the bit that I don't look forward to. 

However a recent migration was done almost completely hands free. I moved almost 9gb of data in an afternoon, while I went to the cinema.

To do this, I took advantage of the new feature of Outlook 2010 that allows it to connect to two different Exchange organisations at the same time.

This allowed me to create a rule to move the content between the two public folders. Once the rule was set, I left it to get on with it. The speed wasn't great, but compared to moving it manually, it was a considerable time saver. After returning from cinema I was able to do more of the migration work, while I waited for the rule to finish.
Furthermore, by using multiple machines, I could move lots of large public folders at once. Once the process was completed, the rule was discarded.
Even before moving the data, when creating the new folders it was easy to setup the permissions as I could compare them side by side.

Where an item was corrupt and couldn't be moved, or the few items that didn't match the rule, I simply moved those items manually or deleted them. In most folders this was only a few hundred items at most.

You still can't copy and paste large numbers of items, as the problem with trying to copy/cut and paste more than about 1500 items is still in Outlook, but a rule effectively moves each item individually, so that isn't a problem.
For the folders with small number of items, a straight copy and paste works well.

I used the same procedure to move a stubborn mailbox which wouldn't move on the regular cross forest move mailbox procedure. Much faster than exporting the mailboxes out to PST file and then importing them. It also allowed me to identify the corrupt items and deal with them.

Even if you aren't deploying Outlook 2010 for your migration, it is worth downloading the trial for this feature alone. Then once the migration is complete, discard the trial software.

Exchange 2003, Exchange 2007, Exchange 2010, MS Exchange Server

A common complaint in forums for some time has been SSL certificate prompts from Outlook 2007, when running Exchange 2003.
The error is usually along the lines of

"The name on the security certificate is invalid or doesn't match the name of the site."

Often the first response will be connected to RPC over HTTPS, as this is the only part of Exchange 2003 that can use SSL certificates for Outlook connectivity.

However the real cause of this is because of the changes made to Outlook 2007 to accommodate the changes to Exchange 2007 and its move to web services. Web services are used to reduce the dependency on Public Folders.

The specific cause of this is a process known as autodiscover. Anyone who has managed Exchange 2007 will be very familiar with Autodiscover, as it can be a key pain point.

Outlook 2007 will attempt to connect to autodiscover.example.com - where example.com is the part of your email address after the @ sign. It will also attempt to connect to a number  of other URLs if that one fails.
 
If your domain does not have an entry for autodiscover, but does have a wildcard entry in its DNS (which is common) then you may get this issue.

Therefore from a client where you have the problem, attempt to ping 

autodiscover.example.com

Where example.com is your email domain, then repeat with your internal Windows domain.

If it resolves, pinging either autodiscover.example.com, example.com or similar, even if it fails, then you may well be on to the cause. The final test is to bring up a web browser and type in autodiscover.example.com and see what happens.
It is likely that you will get the same SSL certificate prompt that Outlook receives and then it will load another web site completely.

The reason for this is quite simple.
Web hosts will often share the IP address of their server with a number of web sites, could be 100s. However to use SSL, a web site must have a dedicated IP address. Therefore a single web site with that IP address will have SSL support.
By using a wildcard in your DNS (so anythingyoulike.example.com resolves) means that all hosts will resolve to the same IP address.
As SSL cannot share an IP address, and does not see the host name being used, it will connect, and generate the SSL certificate mismatch.

How to resolve? Either remove the wildcard entry on the external DNS or have an entry for autodiscover.example.com put in to your domain with a dummy IP address - 127.0.0.2 for example. This will cause the host name to resolve, but fail to connect. See the single host replacement method on this page for instructions on how to do it: http://www.amset.info/netadmin/split-dns.asp

However if you ever deploy Exchange 2007 or higher then remember to remove it!

Exchange 2003, SSL Certificates, Outlook Exchange 2003, SSL Certificates, Outlook

One of the most useful online tools to come out of Microsoft for the Exchange product is their testexchangeconnectivity site - or to give it's correct name - the Microsoft Exchange Remote Connectivity Analyzer (ExRCA). However the URL is a mouthful, and if you are typing it as often as I do, it is easy to make mistakes.

Therefore I have setup a short URL for it using our Exchange community site exbpa.com  - you can get to it via http://et.exbpa.com/

I was going to use te.exbpa.com (which also works) but I thought et would be easier to remember.

Exchange 2003, Exchange 2007, Exchange 2010, MS Exchange Server Exchange 2003, Exchange 2007, Exchange 2010, MS Exchange Server

I have recently seen an issue with the POP3 connector which I haven't seen before, but will be very widespread. In this particular circumstance it caused the client's server to get blacklisted and have a server processing many thousands of messages which it shouldn't need to.

It is yet another reason why using the POP3 Connector is a bad idea. I have blogged on the POP3 connector being a bad option in the past: http://blog.sembee.co.uk/archive/2006/09/25/25.aspx .

This client was not only using a POP3 connector, but they were also using a catch all mailbox at the ISP - I have posted today why using a catch all is a bad idea here:  http://blog.sembee.co.uk/archive/2010/02/15/117.aspx (posting that item was inspired by this one).

The Problem

The actual problem was quite simple, and something that Exchange could have dealt with on its own if the server was setup for SMTP delivery. However it became a noticeable issue because of the way this particular server was configured.

The domain was subject to an NDR or directory harvest attack (I cannot tell which due to the nature of the SBS Connector) and ended up with large numbers of email messages in their queues.

What puzzled the client was that port 25 wasn't open to the internet, and they had followed my guides on recipient filtering and authenticated user relay so that the server was secure ( http://www.amset.info/exchange/spam-cleanup.asp ).
As I wrote in that article, messages can continue to appear in the queues for some hours after the initial clean-up due to the way Exchange displays the queues when there are a very large number of messages in the queues. However for this client, the messages continued to appear for weeks. Eventually, fed up with cleaning the queues daily, I was asked to look at the server.

What I found was that the messages in the queues were all from postmaster@ so had the classic hallmarks of an NDR or direct harvest attack, but the client was using the POP3 Connector.

Due to the way the POP3 connector works, messages that come in to the server through it are not subject to the recipient filter. The recipient filter works at the connection point, but the POP3 connector simply drops the in to the queue for delivery. This is the key point and the result was the same as a standard NDR attack through SMTP without recipient validation  - the messages that could be delivered were, and the messages with invalid external recipients, or where there was a delivery problem, hung around in the queues. As time went on, the server became blacklisted by most major ISPs for being a source of spam and back scatter.

Furthermore, the client also had the POP3 connector setup to send a copy of messages that could not be delivered to a valid user  in to a mailbox, so not only were the messages being delivered there (and the client had what they considered to be a major spam problem) but the NDRs were going out as well. The user concerned thought they were receiving large amounts of spam - when in actual fact they were receiving email that wasn't even addressed to them.

In short, it was a complete mess.

This will be a widespread problem

In many respects, the client was not to blame for this problem. This configuration is quite common, and would therefore affect everyone using the POP3 connector with a catch all mailbox. However you may not see the messages in the queues and therefore be unaware that your server is a source of spam or backscatter.

The most common configuration when SBS is used with a POP3 connector is to route email OUT through a smart host - usually the ISPs SMTP Server. If you are doing that in combination with a catch all mailbox then you wouldn't see the symptoms of this problem. When a smart host is used, Exchange is sending the email straight back out again and the smart host is responsible for the delivery of the email.

It was only because this client was using direct delivery rather than a smart host that the email messages were shown in the queue causing further investigation. The client had accepted large amounts of spam in the mailbox as something that happens - and asked me to look at that as another issue - not realising that it was all caused by the same thing.

If the server had been configured in the usual way for POP3 use, that is to use a smart host, then the first the client would have known there is a problem is when their ISP called to tell them - although many do not.

Furthermore the email messages also do not appear in message tracking logs as they do not pass through Exchange, but simply bounce off SMTP. The only messages that do appear in message tracking are those delivered to the user set to receive the messages that could not be delivered.
Therefore a server could be the source of back scatter and the administrators (whether in house or an external support company) would be completely oblivious to the issue.

I haven't been able to verify if the email messages showed in the volume reported by the SBS Reporting tool, because as with most SBS Servers I see, it wasn't turned on.

The Solution

Changing the client to SMTP delivery of email resulted in the spam level dropping immediately. In the 24 hours after the change, the number of messages the server dropped for non-valid recipients was measured in 1000s. The account which received a copy of the unmatched addresses from the POP3 connector saw the level of spam almost completely drop away - as most of the spam wasn't addressed to the user.

Conclusion

There is a very simple conclusion to this blog posting.
Don't use a catch all mailbox with the POP3 Connector. Ideally you shouldn't use the POP3 connector at all.

If you are using the POP3 connector and do not wish to move to SMTP delivery, then you should look at switching to user specific POP3 mailboxes instead of a catch all. While that is more tedious to setup, it does mean you are only downloading email that you may want, rather than lots of spam that you almost certainly do not, only for it to be rejected.

Exchange 2003, MS Exchange Server, Small Business Server, Anti Spam, Vamsoft ORF Exchange 2003, MS Exchange Server, Small Business Server, Anti Spam, Vamsoft ORF

This is another post in my serious of articles on why you shouldn't use certain features in Exchange, even though they are there. As with the other articles, the article does NOT tell you how to enable the feature in question.
In this post I am going to outline why a "catch all" mailbox is a bad idea.
Many of the points in this article also apply to enabling the option to have a copy of any Non Delivery Reports delivered to someone else in the Exchange org.
This post applies to all email servers, not just Exchange though.

I actually completed this post some time ago, I just never got round to putting it on the blog. However I have recently seen a problem with a SBS Server where a catch all mailbox was used, which I am going to blog on separately, so thought this article should go up first.

The other articles in this series to date are:

Why you shouldn't use logos in signatures ( http://blog.sembee.co.uk/archive/2008/04/14/76.aspx )
Why you shouldn't enable the POP3 Server ( http://blog.sembee.co.uk/archive/2008/03/03/71.aspx )
Why you shouldn't use the POP3 connector ( http://blog.sembee.co.uk/archive/2006/09/25/28.aspx )
Why you shouldn't use a self generated SSL certificate ( http://blog.sembee.co.uk/archive/2006/03/05/9.aspx )
Why you shouldn't put Exchange 2003 in to a DMZ ( http://blog.sembee.co.uk/archive/2006/02/23/7.aspx )

Where does the request come from?

Newcomers to Exchange will often ask where is the "catch all" option, particularly if they are used to that option provided by their ISP with POP3 mailboxes, or are coming from the POP3 connector on Small Business Server.
It may also be asked by a manager, in the mistaken belief that they are missing out on important emails when someone mistypes the email address.

Of course Exchange doesn't support catch all mailboxes, which is why the question is asked.

Similarly, I have seen servers with the "Send copy of Non Delivery Report to..." option set on the SMTP virtual server so that someone gets a copy of the message which can be forwarded on to the relevant person. Of course that is just a COPY, the sender will already have received the NDR, and then may get confused when their message is replied to, despite getting an NDR saying it wasn't delivered.

Why are they a bad idea?

Catch all mailboxes have been a bad idea since the late 1990s, with the growth of worms on the internet that make up email addresses.
In short, a catch all mailbox means that every email address on your server is valid. Therefore the bots that create email addresses based on common name combinations will be able to successfully deliver their messages to your server.

As such, if you enable a catch all, then someone needs to monitor the mailbox constantly for the odd valid email message. Depending on the number of users, the number of messages that could by saved by the catch all may be one or two a day at most.

Meanwhile the person monitoring the mailbox will be deleting the vast majority of the messages, as they will be spam and virus infected messages.
The fact that messages have been delivered at all is also a security risk. If the message is opened or the attachment looked at because it seems legitimate, then the payload could be executed. However if the message had been dropped then it would not even get the chance.

By dropping messages to invalid recipients you will save on bandwidth as the messages do not have to be delivered and on processing power, as the messages do not have to be processed by your AV, Antispam and then Exchange.

Furthermore, if a spammer decides to launch an NDR attack, or simply sends a large amount of spam to your domain, then the messages will be delivered. You may find that you have a mailbox that Outlook cannot open because it has 150,000 messages in it.

I have posted on this blog in the past about VAMSOFT ORF which uses emails to non-valid addresses as a feature to block spam, to great effect. If you are receiving 10,000 messages a day to non-valid address then that would be a tremendous waste of bandwidth - I have a client who drops this kind of level a day.

What is the problem with "Send copy of Non Delivery Report to..." option?

With the "Send copy of Non Delivery Report" option, if you have that set, you are actually being a poor internet citizen. To receive a copy of the Non Delivery Report (NDR) you need to allow the message to be delivered. The server then attempts to send the original NDR back to the sender. However if the message is a virus or spam message (which is most likely) then the sender will be spoofed. Your server is then a source of "back scatter" which could lead to a poor spam score or even blacklisting.
During the last major email-borne virus attack, there were more back scatter NDRs going back and forth than infected messages.
 
Your server is also exposed to an NDR attack or NDR spam attempt, as the server will accept the message and then try and send it back to the "sender" who is the real target of the message.
I have more on NDR attacks on my spam clean up page: http://www.amset.info/exchange/spam-cleanup.asp

Then there is the internal security aspect. If someone senior makes a typo in a confidential email address, this could be seen by someone else, who possibly should not. The original sender will be unaware of this, because they will still get a copy of the NDR.

What are the alternatives?

If you have Exchange 2003 or higher on Windows 2003 SP1 or higher, then enable the recipient filter and tar pit option (instructions: http://www.amset.info/exchange/filter-unknown.asp ). Anyone who sends an email to the wrong address will get a failure immediately. If they are a legitimate sender then they will call or email someone else to get the correct email address.
On older versions of Exchange or if you can't set the tar pit, for example when Exchange 2003 is installed on Windows 2000 where the option isn't available, then setting recipient filtering can actually expose your server to attack, as it cannot defend itself from a directory harvest - therefore a third party tool is required such as Vamsoft's ORF to do recipient filtering and the tar pit.
For other email server products, you should check for recipient validation functionality. If it doesn't exist, but an LDAP lookup option is available, then something like VAMSOFT ORF can query an LDAP database for valid addresses, so could be used as an SMTP gateway. ( http://www.amset.info/exchange/gateway.asp )

If you are aware of a common misspelling, for example Steven and Stephen, then add the misspelling to the user's account as an additional email address. That will ensure that the common misspellings are delivered, without exposing your server.

Exchange 2003, MS Exchange Server, Why you shouldn't..., Exchange 2007, Vamsoft ORF, Anti Spam Exchange 2003, Exchange 2007, MS Exchange Server, Why you shouldn't..., Anti Spam, Vamsoft ORF

I am pleased to announce that the domain exbpa.com has been saved for the Exchange community.
This was a domain that Microsoft first used a few years ago to point to their (at the time) recently released Exchange Best Practises Analyser. There are thousands of links to this domain across the internet as well as in books and magazines.

However Microsoft recently decided to allow the domain to lapse and early this morning it was finally deleted.

Fortunately I was able to register it myself through my consultancy company Sembee Ltd and therefore kept it out of the hands of a domain squatter. 

I have uploaded a slightly modified version of the list of Exchange resources that I maintain at Daniel Petri's forum, which as well as the links to the Exchange Best Practises Analyzer, also contains links to other Microsoft tools, blogs etc.

http://exbpa.com/

While it is not the best designed web site in the world, it does the job. Hopefully the Exchange community will find it of some use.

Exchange 2003, Exchange 2007, MS Exchange Server, Web Sites, Amset IT Solutions Ltd. / Sembee Ltd., Exchange 2010 Exchange 2003, Exchange 2007, MS Exchange Server, Web Sites, Amset IT Solutions Ltd. / Sembee Ltd., Exchange 2010

Blacklisting.
For some email administrators Blacklists are the greatest weapon against spam. It cannot be denied that they can have a significant effect on the amount of email that your server has to process, and they do meet the primary objective of spam detection - dealing with the email at the point of delivery, therefore  reducing back scatter. They are also free, and once setup require little to no maintenance by the administrator.

However personally I dislike blacklists. I don't like the idea of someone else (either human or computer) deciding on what email I should receive, based on lists and reports that I have no control over.
Furthermore, from a business perspective, using a blacklist may cause potential clients to be rejected, as one of my specialism's is the cleanup of servers that have been abused and are likely to be blacklisted.

However, if I could blacklist IP addresses that I know are trying to send spam to me, in real time, where I have complete control over all aspects of the filter, then that could be something of use. A new feature in Vamsoft ORF has introduced exactly that, and has actually got to the point where I have turned off the antispam features in Exchange.

I have written about Vamsoft ORF before, using it for Greylisting (http://blog.sembee.co.uk/archive/2006/09/18/24.aspx) and as part of an SMTP gateway configuration (http://www.amset.info/exchange/gateway.asp).

With the latest version at the time of writing, 4.3, they have introduced a feature called Honey Pot. The simple way that this works is to block IP addresses that attempt to send email to addresses in the Honey Pot list.
In the Vamsoft setup guide it gives you some ideas on how to publish the honey pot addresses, however I found that I didn't need to publish anything.
Going through logs on my backup SMTP gateway, which does recipient validation through Vamsoft rather than Exchange, I noticed that the same non-valid addresses were being used time and time again. These were addresses that I had NEVER used, would never be likely to.

IMPORTANT: The use of addresses that have never been used is the key here. Adding addresses that were in use will provide you with false positives, because that could be legitimate email. If you decide to follow this practise then ensure that you only use addresses that have NEVER been used.

Therefore what I did was turn off recipient validation on my primary SMTP point of entry and configured  that function to be done by Vamsoft ORF. This allowed me to see the addresses that were being sent to on that server as well. I was then able to compile a list to use as my honey pot.
I review the logs frequently to see if new email addresses are being tried, which can be added to the list of honey pot addresses.

This means I am using three tests for spam - recipient validation (which should be something that every site does) greylisting, and honey pot.

The effect was significant. I have been using this setup for a number of weeks and the amount of spam I am seeing in my mailbox or caught by IMF (so got through the initial greylisting and honey pot) is almost zero. One or two messages a week. I have actually now turned off IMF on my Exchange servers.

Why is this being so effective?
The simple reason this is being so effective is that the spammer's list of email addresses will contain a mixture of valid and invalid addresses. As soon as the spammer's server attempts to send an email to a non-valid address that is on my honey pot list, it is blacklisted. Even if that IP address subsequently tries to send to a valid address it will be blocked.
Combined with greylisting, which sends away the initial connection, the even if a legitimate address is used first, the spam doesn't get though. The first attempt is greylisted, then if the list of email addresses contains one of the bogus ones, then it gets blocked. The server attempts to deliver again after greylisting and its connection is blocked.

I also think this is more effective than regular blacklisting because it is in real time and is based on email received by my servers.

I have combined this with an SQL backed database for Vamsoft ORF so that both of my SMTP gateways share the same information, meaning that a blacklisting that is set by one server, is also used by the other.

Finally, I have also combined this with custom NDR text, that points people to a special page on my web site. This page explains what is happening, and other ways to contact me. If required, I can then white list to allow the legitimate messages through and take the spam hit for a short time.

Vamsoft ORF: http://www.shareit.com/product.html?productid=169362&affiliateid=200023740

Exchange 2003, Exchange 2007, MS Exchange Server, Anti Spam, Vamsoft ORF Exchange 2003, Exchange 2007, MS Exchange Server, Anti Spam, Vamsoft ORF

The database of an Exchange server is something that seems to raise a lot of questions with Exchange administrators. Many of the questions appear to be around the size of the database and its limits.
This article should help to increase the understanding of the database size and limits. I have also touched on the thorny topic of offline defrags.

First some terminology.
Where I mention VERSION, this is Exchange 2000, 2003, and 2007.
Where I mention EDITION, this is Standard or Enterprise. Where I mention Standard edition that also applies to the SBS variant.

Unless stated otherwise, references to Exchange 2003 also apply to Exchange 2000.
To the best of my knowledge at the time of writing, Exchange 2007 references also apply to Exchange 2010. However if I find that is not the case, I will update this article.

This is a background article, it does not tell you how to do anything (just in case you came here via Google expecting to be told how to do X with your database).

Myths of the Exchange Database

There are a lot of myths around the Exchange database size and limits which I hope this article will help to dispel

• The store will dismount when you hit a physical size of 75gb
• Adding up the mailboxes listed in ESM should equal the size of the database
• Regular offline defrags are required.

Then there is the confusion with many administrators that the database doesn't shrink in size, even after the users have deleted lots of data. I will cover that as well.

Exchange Database Basics

Lets start with some basics of the database.

With Exchange 2003, the database is made up of two separate files. An EDB and an STM file. These combined are referred to as a store and come in two flavours - Mailbox and Public Folder.
Mailbox and Public Folder stores can be grouped together in to Storage Groups.

The EDB file should be thought of as the MAPI database and will consist mainly of internal email.
The STM file should be thought of as the SMTP database and will consist mainly of external email.
Email sent by Outlook Express users or other internal non Exchange servers would be considered external email.
However some information from the mail in the STM file is held in the EDB file.

The two files should be treated as one.

Mailbox Store and Storage Group Capacities

With Exchange 2000 and 2003 Standard edition you can have one storage group consisting of one database of each type.
With Exchange 2000 and 2003 Enterprise edition, you can have four storage groups consisting of a maximum of four mailbox stores in each group.
With Exchange 2007 Standard edition you can have up to four storage groups with a single mailbox store  or public folder store in each, or a single Storage Group with four mailbox stores.
With Exchange 2007 Enterprise the  number of Storage Groups goes up to 50.

Database Size

The size of the database is a source of much confusion with newcomers to Exchange.
The simple fact with the PHYSICAL size of the database is that it will never shrink without intervention from the administrator. When content is removed from the database then the Exchange server marks that space as white space, and should use that space first for new content before increasing the physical size of the database.

However in practise, that often does not happen. What you will usually find is that if users are asked to clean out their email, more external email will be removed (spam etc) but more internal email is generated.

Database Limits

The database limits are probably the are that causes the most concern for the Exchange administrators, so lets clear that up to begin with.

Exchange 2000 Standard has a database limit if 16gb, which can be increased to 17gb via a registry hack.
Exchange 2003 Standard RTM and Service Pack 1 is also subject to the same limit.
Exchange 2003 Standard with Service Pack 2 has a soft limit of 18gb, which can be increased to 75gb via a registry change.
Exchange 2007 Standard has a soft limit of 50gb in RTM and 250gb in Service Pack 1 which can be removed/changed with a registry change.

Enterprise edition of all versions have a technically unlimited database size, although if you are picky it is 8TB with Exchange 2000/2003.

If you update Exchange 2003 from Standard edition to Enterprise edition, then the registry setting for the soft limit is not removed, so the database may still dismount when it hit the size stated. You need to remove the key completely for that to stop happening

Soft Limit

Soft limits are basically a way for an administrator to ensure that the database doesn’t get out of control. The Exchange server will react when a soft limit is reached by dismounting the store.

Database Limit Enforcement

The way that the database limit is enforced changed with Exchange 2003 Service Pack 2 and subsequent versions.
With Exchange 2000 and Exchange 2003 RTM and Service Pack 1, the limit was simply the physical size of the two database files combined.
With Exchange 2003 Service Pack 2 and later, the limit is now a logical limit. The limit is the physical size of the two files, minus the white space.

The white space is reported by event ID 1221 during the night.
The logical limit of the database is not reported by Exchange until you change the default limit of 18gb.

The registry keys for increasing the 18gb limit in Exchange 2003 are in Microsoft KB article 912375 (link at the end) however I suggest that you read the Technet Article on how to work with the limit and setting the registry key for the warnings.

When setting the check time, ensure that it is AFTER the maintenance window configured on your Exchange server (ie after event ID 1221 has reported) so that content removed that night is taken in to account.

If you hit the limit -whether it is a limit below 75gb or the maximum 75gb limit and the database dismounts, you can mount it again. However it will dismount again the next day.

Offline and Online Defragmentation of the Database

When it comes to the database size and reducing it, most Exchange administrators will be referred to an offline defrag. However Exchange also does an online defrag. While they are related there are some key differences to what they do. 

The online defrag is part of the nightly maintenance that Exchange does on its databases and is what finds and marks the white space for use. Its results are reported by event ID 1221. If that process does not run, the space gained by deleting content will not be used.

Am offline defrag will take the database and create a new one, consisting of the same data, minus the white space. Therefore the physical size of the database will be reduced. An offline defrag is the only way to reduce the physical size of the database.

The offline defrag is not risk free, and can take a considerable amount of time. The process speed is hardware dependant and can vary between 1 and 4gb per hour. Therefore if you have a 50gb store you could be looking at anything between 12 and 50 hours for the process to complete. Once started, it cannot be stopped. If it is, then both the source and the destination files are useless and a copy will need to be put in place.
The Exchange services have to be stopped while the process runs - so requires total downtime of the server. If you have multiple databases on the server then you can dismount the store you are working on and allow the others to run, however if you are in a position to run multiple databases, then you do not need to do an offline defrag, as I will explain below.

Some Exchange administrators  claim that a regular offline defrag is required to keep the server running at the peak of performance. This is not the case and Microsoft specifically state that an offline defrag should not be considered something that needs to be done regularly.

The reason why there can appear to be a performance gain is because an offline defrag creates a new database. As with many things, if you replace with new then you will see some performance gains. Minor imperfections in the database structure can be removed and things generally cleaned up. However because it will skip data that it cannot read, that can mean there will be data loss.

With Exchange 2007, and Exchange 2003 Service Pack 2, or Exchange Enterprise edition (any version) an offline defrag is not necessary and is a waste of time.

Why?
With Exchange 2003 SP2 standard, due to the way that the database is reported, you gain nothing by doing an offline defrag. All you could do is lose data during the process. If you hit the limit, you can remount the database and then remove content.

With Exchange 2007 (all editions) And Exchange Enterprise Edition  (all versions) the process is unnecessary. Simply create another mailbox store, move all of the mailboxes to that store and then drop the original one and delete the database file. You can then create a replacement and move the content back. Zero risk, zero downtime.

If the store you are replacing is the original first store, then it will also hold some system mailboxes. Those will be recreated in another database when the system attendant service is recreated, so you should do that as soon as possible after dropping the original store.

The only reason why you want to do an offline defrag is because you are tight on physical storage, however you will need considerable space to do the offline defrag (At least 110% the size of the store) which will mean additional storage somewhere, so you may as well add it to the original server.

Mailbox Size - Exchange 2000/2003 only.

Many Exchange administrators will be unaware that the list of mailboxes in ESM is not showing the true size of the mailbox. This is clearly shown by the number of questions on forums from administrators who add up the size of their mailboxes and then ask why there is a X gb difference between that total and the sum of their physical database sizes.

In Microsoft KB article number 828070 (link at the end), Microsoft state:

 "When you view the space that a mailbox uses in Exchange System Manager, the amount only includes the space that is used by the Priv.edb file. The amount does not include the space that the Priv.stm file uses."

Therefore a significant difference between the size of the mailboxes and the total of the physical database size should be expected.
This difference is further increased when you take in to account single instance storage and deleted item retention.

Single Instance Storage is a mechanism used within the Exchange database to keep the size of the database down. If you send an email with a 5mb attachment to 10 users, rather than using 50mb of space, it only uses 5mb. The attachment is only removed from the store when the last of those ten recipients removes it from their mailbox.

Deleted Item Retention (aka dumpster) is a feature of the Exchange database, where an item that is deleted from the mailbox or public folder (including removal from the Deleted Items folder) is stored in the database where it can be recovered.

Conclusion

Day to day administration of the Exchange database is not something that most administrators should fear or have any concerns about. As long as you monitor the size of the database regularly, then issues around the size should not come as a surprise.

References

Exchange Server 2003 mailbox store does not mount when the mailbox store database reaches the 16-GB limit
http://support.microsoft.com/kb/828070/

Database Size Limit Configuration and Management (Exchange 2003 SP2)
http://technet.microsoft.com/en-us/library/aa998066.aspx

How to increase the Exchange Server 2003 Service Pack 2 18-gigabyte database size limit
http://support.microsoft.com/kb/912375

How to Modify a Database Size Limit (Exchange 2007)
http://technet.microsoft.com/en-gb/library/bb232092.aspx

Related Articles

Recover Deleted Items: http://www.amset.info/outlook/recoverdeleteditems.asp

Exchange 2003, Exchange 2007, MS Exchange Server, Small Business Server Exchange 2003, Exchange 2007, MS Exchange Server, Small Business Server

Over the weekend one of my clients suffered an authenticated user attack on the SMTP interface of the Exchange 2003 server. This was detected by the monitoring tool I use, HoundDog (http://www.hounddogiseasy.com/referrer.html?code=YNPX) .

The attack was unsuccessful, as I have all of the authentication options disabled.

However what was interesting was the list of usernames that were tried. Some of them are to be expected, but others maybe not so. I have included the list at the end of this posting.

What this list tells you is the usernames that should be avoided, as some of them may well be used as test accounts, with basic or no passwords and therefore may well be easily compromised.
As authenticated user relaying is enabled by default on Exchange 2000 and 2003, if an account can be compromised, even with limited privileges, it can be used to relay spam through your server.

If you do not have anyone using POP3/IMAP accounts on your Exchange server, then authenticated relaying should be disabled completely. It is not required for the correct operation of Exchange with MAPI, Outlook RPC over HTTPS, Outlook Web Access and Windows Mobile or Blackberry use.
If you do have POP3/IMAP users then lock down the authenticated relay to those specific users only. I have added a link to my article on amset.info with instructions on how to do that below.

If you are a victim of an authenticated user attack then remember that most of them are not against you or your company directly, but a spammer wanting to use your bandwidth to send their messages, whether this is to sell something or a phishing attack.

Related Articles
Securing the authenticated relaying: http://www.amset.info/exchange/smtp-relaysecure.asp
Spam Cleanup: http://www.amset.info/exchange/spam-cleanup.asp

List of Usernames Targeted During Authenticated User Attack

webmaster
service
web
info
root
backup
tech
test
Administrateur
administrator
admin
tunnel
nagios
visitor
access
account
data
server
user

Exchange 2003, MS Exchange Server, Small Business Server Exchange 2003, MS Exchange Server, Small Business Server

Last year I wrote an article about how spam could not originate from inside your network and go through your Exchange server, because that was simply too much hard work.
You can read the original article here: http://blog.sembee.co.uk/archive/2008/03/13/73.aspx
As a brief summary, I basically said that a spammer needs to get their BOT inside the network, find the Outlook installation and then abuse it, or they could take the simple route of having their own SMTP engine and send email directly.
Remember a spammer doesn't want to be found, and sending a large amount of messages through Exchange will alert network administrators that something is wrong.

Alas despite that posting some people continue to believe that someone who is being flooded with spam in their queues will have a compromised client, despite the changes being very slim.

The purpose of this posting is to help you diagnose whether the message has come from a client inside your network, or not.
Do that, you need the SMTP headers of one of the messages.

Background

First a little bit of background about Exchange and SMTP headers.
When you send a message from a MAPI client (so that is Outlook, Outlook Web Access, a Windows Mobile device using Exchange ActiveSync or a Blackberry using a BES) then the SMTP headers are not actually put on to the message by the client, but by Exchange when the message hits the SMTP server. That is why you do not see any headers on internal email messages or show the internal IP address of the machine that originated the message on external email messages.  

If you send the message via an SMTP client (Outlook Express etc, a PDA with IMAP/POP3 access etc), the SMTP header is generated by the application originating the message.

If you bounce the message off Exchange's SMTP virtual server (relaying) then the header stays intact and  Exchange simply treats it like any other SMTP message that is passing through.

It is the originating application line that will show you whether the message has come from inside.

Getting the Headers

If the server has already been blacklisted, then on many of the blacklists you can see the message that was received, which will show the complete headers.
If you have a large number of messages in the queues, then you simply need to get hold of one of those messages. You will find them in \exchsrvr\mail root\vs 1\queue.
Drop one on to a notepad document, which should open for you.

What you are looking for.

When you can see the raw header, you are looking for something like the below.

If the message originated outside of Exchange, then look for something along the lines of these examples:

• X-Mailer: Microsoft Outlook Express 6.00.2600.0000
• X-Mailer: The Bat! (3.0.2.8) Professional
• User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209)

If the message originated on Exchange, then you will see something similar to these:

• Exchange 2000:
  X-MimeOLE: Produced By Microsoft Exchange V6.0.6619.12
• Exchange 2003:
  X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.3790.4325
  X-MimeOLE: Produced By Microsoft Exchange V6.5
• Exchange 2007:
  Received: from host.domain.local ([fe80:xxx:xxxx:xxxx:xxxx]) by  host.domain.local ([fe80:xxxx:xxxx:xxxx:xxxx]) with mapi;
  Received: from host.domain.local ([123.123.123.123]) by  host.domain.local ([123.123.123.123]) with mapi;

Of course the headers are not completely fail safe. Antivirus and anti spam applications can strip content off the headers, as can other third party tools that will scan SMTP traffic. However the presence of non-Microsoft products in the headers is a clear sign that the message originated outside of your network.

Exchange 2003, Exchange 2007, MS Exchange Server Exchange 2003, Exchange 2007, MS Exchange Server