Chimera Computing Ltd

EFS - Duplicate Messages


  • Why do I get Duplicate Messages?!

BACK

Some of you may experience a problem where users receive multiple copies of the same message. This is not a bug in EFS - it is a feature of some mail servers. Usually, when your ISP receives a message, they will just place the message in your Pop account, even if the message has been addressed to more than one recipient at your domain. Other ISPs actually scan the message and create as many messages as are recipients for it. For example, if you sent a message...

To: bob@mydomain.co.nz, fred@mydomain.co.nz

...then your ISP mail server will hold 2 separate messages in your single Pop account (if the message was also addressed to henry@mydomain.co.nz then 3 messages would be created, and so on) In this case, EFS will pickup and deliver the message, as usual, addressing it to the users for your domain - however, because there are 2 messages and both still contain the same 2 recipients, EFS will deliver the message to each user twice.

 
  • How does EFS get around this?
If you understand how EFS gets around this, you will have a better insight on how to set up the required option for your network. There are basically 2 methods you can use to get around this, however the second option again depends on the function of your ISP's mail server.

Method 1: (EFS 5.0 and higher) "Fix Duplicate Messages" (removal of duplicate messages)
This can be enabled by simply clicking the "Fix Duplicate Messages" option in the Advanced section. It works like this:

- Each message on the Internet has (or should have) it's own unique Message ID (often made up of a combination of the current date/time, host name etc, for example: "4C2568BF.00711D59.00@mail.mydomain.com")
- EFS initially downloads all message headers first
- It then compares the Message-ID of that message to ID's of all other messages
- If the Message-ID does NOT exist, EFS downloads that entire message then writes its Message-ID to memory
- If the Message-ID DOES exist, it assumes that the message has already been downloaded and so removes it
- When EFS sends, the message still has the entire recipient field in the To: or Cc: field and therefore forwards the message to all local domain recipients.


Method 2: "Custom Header Checking" (specifically include/exclude fields to check - depends on ISP)
This option depends on whether your ISP creates a unique header field containing a single recipient address for each duplicate message it creates (in most cases, this is highly likely). For example, your ISP creates a duplicate of each message (as above) but also adds a header field "X-Rcpt-To:" that contains each INDIVIDUAL users email address in each message (as shown in bold below). If a message was addressed...

To: bob@mydomain.co.nz, fred@mydomain.co.nz

...then the (basic) message header would look like:

Duplicate Message 1.
To: bob@mydomain.co.nz, fred@mydomain.co.nz
From: another@person.com
Subject: Budgets for March
Message-ID: 123456789@mydomain.co.nz
X-Rcpt-To: bob@mydomain.co.nz

Duplicate Message 2.
To: bob@mydomain.co.nz, fred@mydomain.co.nz
From: another@person.com
Subject: Budgets for March
Message-ID: 123456789@mydomain.co.nz
X-Rcpt-To: fred@mydomain.co.nz

As you can see, the ISP has created an additional field "X-Rcpt-To:" in the header that splits up the users in the To field and put them in each message. You can then use Custom header checking in EFS by clicking the "Headers" (or "Custom" tab or "Custom Header Checking" under SMTP in older versions of EFS). Select "Custom Fields, To:" and enter X-Rcpt-To You also need to click to ignore the "To" (and possibly "Cc") fields otherwise EFS will search these fields and the users will receive duplicate messages.

If you are able to use Method 2, then you also have the option of using Method 1. If this is the case, you should decide on which method to use dependant on the average number of and size of your emails. If most emails are large (have attachments) then Method 1 would be beneficial as EFS only downloads the single duplicate. If most emails are small (text messages) then Method 2 is preferred as EFS does not need to perform the process of retrieving headers and comparing them which obviously takes up more processing time and resources. Method 2 is preferred.


  • How do I know if I am getting duplicate messages?
It is HIGHLY RECOMMENDED you perform the following steps to verify how your ISP interprets each message. The following will show you how to tell and should be done before considering this option:

1. Get someone external to your company to send a SINGLE email to 2 users at your domain, eg: User1@yourdomain.com ; User2@yourdomain.com
2. Set "Don't forward" under the SMTP section of EFS and Apply.
3. DISABLE "Fix Duplicate Messages" under the Advanced section and Apply.
4. Click the CHECK button from the Log screen.

If EFS downloads 2 messages, then it's likely that your ISP is creating duplicates for any message that is addressed to two or more recipients. You MUST double check this (incase someone else sent a message to you at the same time) To do this, open Windows Explorer and browse to C:\Program Files\E.F.S\Inbox and open the 2 message files in notepad. If they are the same message (other than the subject & body, you can check that both messages have the same "Message-ID:"), then you NEED one of the above Method's enabled - either "Fix Duplicate Messages" or "Custom Header Checking" set on. Once you've checked this, remove the 2 message files from the "Inbox". Restore settings from Step 2 and 3 above then perform Step 1 again. If you choose "Fix Duplicate Messages", verify that EFS retrieves only ONE of the duplicated messages, but both recipients receive it. If you choose to use "Custom Header Checking" then verify that EFS retrieves the same message TWICE, but each recipient only receives the message once.

TOP OF PAGE


(Chimera Computing Ltd)