November 10, 2009

Troubleshooting Email Notifications

Surround SCM
Our support group is commonly contacted with email notification problems. This can be tricky to troubleshoot because there are many moving pieces and it is difficult to isolate the cause. This post includes some troubleshooting tips I learned as a member of the support group. Before we delve into details, let's take a high-level look at how notifications work. This post applies to both Surround SCM and TestTrack. For this post, when something applies to both the TestTrack and Surround SCM Servers, I will refer to them as the Seapine Server. The usual flow of a notification starts when a user performs an action, like a defect assignment (TestTrack Pro) or checking in a file (Surround SCM). The action fires the notification rule that generates the email. Once the message is generated, the Seapine Server places the email in its mail queue while it attempts to establish a handshake with the mail server. The Seapine Server behaves like a mail client. It does not handle the actual delivery of the email. Instead, it hands the email to a mail server for delivery. If a handshake is established, the email is handed off to the mail server for delivery. If used properly, the mail queue can be your best friend when troubleshooting mail issues. If you can see the message in the mail queue you have eliminated the following:
  • The user is not invoking the correct action
  • The rule is set up incorrectly
The big benefit is knowing that the rule is set up correctly. You don't have to spend time playing around with the rules, hoping that something automagically fixes the problem. At this point, you are left with some possible causes:
  • The SMPT Host is incorrect
  • The SMTP port is incorrect
  • The SMTP host is accepting the handshake and message but refuses delivery
  • The recipient is getting the email, but it was caught by their SPAM filter
Troubleshooting Steps Both TestTrack and Surround SCM have a way to pause the notification process at the point of establishing the handshake with the SMTP server. Messages are still generated and placed in the mail queue, but the corresponding server does not attempt to contact the SMTP host. The ability to do this is done in the Server Options dialog.  In Surround SCM, you access this using the Surround SCM client through Tools > Administration > Server Options.  In TestTrack, you access this using the TestTrack Server Admin utility and clicking on the Server Options button. Both dialogs have an email configuration tab or section that similar to the following image: [caption id="attachment_1190" align="aligncenter" width="523" caption="Send Mail Settings"]mailsettings[/caption] Whether you are using MAPI or SMTP, both have the option to pause sending mail. Follow these steps when you need to troubleshoot mail issues:
  1. Select the "Pause sending via SMTP/MAPI" option.
  2. Perform the action that should trigger the notification.
  3. Check the mail queue. To access the mail queue in Surround SCM, select Tools > Administration > Mail Queue in the Surround SCM client. In TestTrack, click on the Mail Queue button in the TestTrack Server Admin utility.If the mail queue is empty, then either the rule is set up incorrectly or the action invoked is not the correct one. Examine these to find the problem. Repeat this process until you see a message in the Mail Queue as shown in the following image.If the rule is set up correctly, you should see something similar to the following image: [caption id="attachment_1194" align="aligncenter" width="601" caption="Mail Queue"]Mail Queue[/caption]
  4. Once you see a message in the Mail Queue, go back to the Send Mail options dialog and clear the "Pause sending via SMTP/MAPI" option.
  5. Go back to the Mail Queue. The message might still be there, as it may take a few seconds/minutes for the server to establish a handshake with the mail server.
If the message disappears from the Mail Queue, then you can be pretty sure that the server established a handshake with the mail server. At this point you are left with some possible causes:
  • The reply to address is something that your mail server does not allow. This is configured per project in TestTrack under Tools > Administration > Project Options > Email and in Surround SCM under Tools > Administration > Server Options > Email servers > Email Notifications. Verify that the notification account name and email address are valid.
  • There is something in the body of the email that your spam filter is catching.
  • The send to address is something that your mail server does not recognize or allow.
Check your mail server's Mail Queue and log for possible clues. If the message never leaves the Mail Queue, there should be a meaningful error. Keep in mind that any error returned by the Mail Server can show up there, so you may need to consult your Mail Server documentation to figure out what the error means.