during a pilot period The alert should be tested by others not involved in its development Many institutions have committees with multidisciplinary expertise available to approve alerts before roll-out After implementation, data should be collected on alert frequency and responses by clinician, which can be done with EHR log data and/or inperson qualitative assessments Many EHR systems allow alerts to also send messages in the background that can notify the alert design team who can then follow up and review the clinical processes An ongoing monitoring system is important, as alerts may display differently or stop firing altogether with later modifications of the system