6 – Monitoring and notifications 150 Figure 6.2: Database Mail account settings associated with the Notifications profile. Having configured Database Mail with a default profile and account, both tasks thankfully having guided wizards, you can send a test mail using the stored procedure, sp_send_dbmail. The options for this stored procedure are many but a simple test can be performed with the code shown in Listing 6.1. msdb sp_send_dbmail @recipients = N'rlandrum13@cox.net', @subject = N'Mail must work or else ', @body = N'This is a level 1 alert please wake up to failure.' Listing 6.1: Sending a test mail using sp_send_dbmail. NOTE You may notice that sp_send_dbmail is now located in the MSDB database, whereas xp_sendmail, in versions prior to SQL Server 2005, was located in the Master database. 6 – Monitoring and notifications 151 If all went well with the test, you should receive a test message similar to that shown in Figure 6.3 (and yes, that is Outlook Express. I will not apologize). Figure 6.3: Mail received from sp_send_dbmail. While Database Mail is certainly important for sending mail from code, such as maintenance stored procedures, it is only one part of the notification system that the DBA will use, if he or she is diligent. You will also want to configure SQL Agent to use Database Mail, for scheduled jobs that execute the code. In this way, you will have built in redundancy of notifications, one message coming from the code and one message coming from the SQL Agent service that executed the job code. To configure SQL Agent notifications, right-click SQL Server Agent in SSMS, select "Properties" and then choose the Alert System page, as shown in Figure 6.4. 6 – Monitoring and notifications 152 Figure 6.4: Configuration for SQL Server Agent mail. Here, I have chosen to enable the mail profile and selected "Database Mail" as the mail system, along with the "Notifications" profile created previously. Setting up an operator Having configured Database Mail and SQL Server Agent, the final step is to setup an operator i.e. the person (or people) who will receive the messages from any failures, either internally in the code or from the SQL Agent job. This can be a single email address but it is far better to use a distribution list, such as DBATeam@companyname.com, so that every member of the team receives the notification messages. NOTE Of course, you should use a single account to validate everything works, prior to setting this server to production, so that other team members do not get inundated with false alarms during testing. 6 – Monitoring and notifications 153 It's important that the whole team is aware of any errors or failures that occur, even if they are not on-call. Generally, the on call DBA will have his or her mobile device configured in such a way that a "fail" or "error" will cause a raucous and marriage-damaging alert to scream forth when the fated message hits it, while other DBAs can continue to sleep soundly, having set their devices to a phone- only profile. However, it does also mean that if the on-call DBA does not respond, for whatever reason, someone else can. Setting up an operator is as easy as right-clicking Operators in SSMS and selecting "New Operator". This opens the dialogue shown in Figure 6.5. Here is where you will set the e-mail address for the operator. Figure 6.5: Setting up an operator group mail account. With SQL Agent configured for mail, and Database Mail successfully tested with the operator, all I need to do is configure each job about which the team needs to be notified. This can be done from the "Notifications" section of the Properties page of each job. Having done this, I can rest assured that any jobs that fails will notify me at 2:34 AM so that I can be jolted from dreaming about work into actually doing some work. 6 – Monitoring and notifications 154 NOTE It is important to remember to restart the SQL Agent Service after you enable mail. Otherwise you may receive an error that states that attempt to send mail failed because no email session has been established. Backup failure notification As much as we might not like it, or are astonished to see it, failures do happen and happen regularly. The goal of attaining the 4 9s availability looks good on paper but to pull it off costs exorbitantly. Database backups are certainly not immune from failure. There are many problems that cause backup failures, such as network outages (don't tell the network admin this as he or she will say everyone always tries to blame the network), SAN issues or other general failures, such as backup shares disappearing or filling up. Database backups are probably, no, certainly the most important aspect of the DBA's job. Without good backups, you cannot recover business-critical data in the event of a disaster. Whether you're performing a full, differential or log backup, success has to be guaranteed. If a failure occurs during a backup, the DBA has to respond quickly, especially on production servers. If an error occurs at 1 AM, waiting until the next day is not generally deemed acceptable, as it would probably violate the Service Level Agreement (SLA) that IT has agreed with the organization in regard to the maximum acceptable number of hours of data loss. If it is agreed upon that 6 hours is the maximum acceptable amount of data loss, then the backup schedule can be setup to accommodate this. The shorter the period the more diligence and expense it entails. If, for example, the SLA dictates a maximum of 15 minutes data loss then this will require log backups to be taken every 15 minutes, increasing the risk of failure due to outages or some other unforeseen circumstance. As noted earlier, I perform the bulk of my backups using Red Gate's SQL Backup tool. Besides the benefits of compression, speed and security via encryption, it has the added bonus of a built-in SMTP mail client. Back in the days of SQL 2000, where no such SMTP client existed natively, Red Gate's SMTP mail client proved to be invaluable for notifications. I have a backup script that rolls through a set-based list of databases to be backed up, taking account of the fact that databases like TempDB should be omitted. The script runs as a scheduled SQL Agent job, and sequentially backs up every database on the list, one at a time, using SQL Backup. With this set up, I am interested in two different types of notification, in the event of a backup job failing. I want to see "redundant" notifications of the failure both from the SQL Agent backup job itself and the code that performs the backup. . from the code and one message coming from the SQL Agent service that executed the job code. To configure SQL Agent notifications, right-click SQL Server Agent in SSMS, select "Properties". Mail and SQL Server Agent, the final step is to setup an operator i.e. the person (or people) who will receive the messages from any failures, either internally in the code or from the SQL Agent. shown in Figure 6.4. 6 – Monitoring and notifications 152 Figure 6.4: Configuration for SQL Server Agent mail. Here, I have chosen to enable the mail profile and selected "Database