Introduction This purpose of this document is to define a set of test cases that systems implementing the OpenADR 2.0a profile must successfully pass in order to obtain certification. Scope OpenADR 2.0a is an application layer message exchange specification used for twoway communication of Demand Response (DR), price, and Distributed Energy Resource (DER) signals between the electricity service provider and their customers. OpenADR 2.0a is defined in the following documents: • The OpenADR 2.0a Profile Specification, Revision 1.1 • The OpenADR 2.0a schema The testable requirements are defined as a set of conformance rules in the OpenADR 2.0a Profile Specification. These rules are referenced in both OpenADR 2.0 PICS and this document. Appendix B provides a mapping between each of these conformance rules and the test cases defined in the test specification. The scope of the test cases will focus on validating the successful exchange of messages between VTNs and VENs, or VTNVEN pairs, validating that any exceptions are handled gracefully and without service interruptions, and that security mechanism are properly implemented. Validating the functional behavior of the DUT in response to payload content is out of scope for the certification tests. For instance, the test cases will test that a VEN acknowledges the receipt of a DR Event but it will not validate that the VEN actually sheds load as a result of this message. For product certification, the certification test suite will be will be run over top of the specific transport protocols supported by the DUT. If multiple transports are supported by the DUT, the full conformance test will be run over one transport and a representative set of test cases will be run over the remaining transports (See Appendix C). Conformance testing of the supported transports is out of scope for the certification tests. However, limited testing will be done to validate OpenADR specific requirements for implementation of the transports, such as the inclusion of specific http headers. ...
OpenADR 2.0a Certification Test Specification Version 1.1.0 Revisions: Version 0.50 0.51 0.96 0.97 • • • • • • • • • • • • • • • • • • • • • • 0.98 • 0.52 0.6 0.61 0.70 0.90 0.91 0.92 0.93 0.94 0.95 0.98.1 0.98.2 0.98.3 0.98.4 0.98.5 0.98.6 0.98.7 0.99.1 1.0.0 • • • • • • • • • • • • • • • • Changes Initial Draf Added visual checks by test engineer for event execution Edits based on feedback from Rolf Minor edits Add negative test cases Added perquisite notes Misc updates, input from Rish Added balance of negative test cases Added transport test cases Updated to reflect conformance rule changes at test event Renumbered test cases Added security tests General Clean Up Minor Edits Update several test cases, fix minor errors Add prompts to test case definitions Minor Edits Changes to implementation limits Updated conformance rule 26, added rules 50 an 51 Added test cases 280, 380, and 670 Added test case 285 Removed conformance rule test and just provided reference to the OpenADR Profile Spec Numerous changes related to the 03/23/12 changes to conformance rules and schema Completed Appendix B, test case to conformane rule mapping Added Appendix D with a test case to event state mapping Added missing test E2/E3_0655 Minor edits Minor edits Test case additional and deletions as a result of 4/2/12 test event Minor edits Deleted test E2_0660 Changes related to new conformance rules 65 and 66 Modified test cases E2/E3 – 0685 Added test cases E2/E3- 0432 and E0/E1 – 0267/268 Minor Edits Minor Edits Minor Edits Tweaked Security Section a bit Addition of Security Tests Date/Editor 01/18/12 – JZ 01/19/12 – JZ 01/23/12 – JZ 01/24/12 – JZ 01/25/12 – JZ 01/31/12 – JZ 02/06/12 - JZ 02/10/12 – JZ 02/18/12 – JZ 02/20/12 – JZ 02/23/12 – JZ 03/02/12 – JZ 03/06/12 – JZ 03/09/12 – JZ 03/25/12 – JZ 03/26/12 – JZ 04/05/12 – JZ 04/08/12 – JZ 04/16/12 – JZ 04/17/12 – JZ 04/25/12 – JZ 05/01/12 – JZ 05/23/12 – JZ 08/09/12 – JZ 1.0.1 1.0.2 1.0.4 1.0.5 • • • • • • • • 1.0.6 • • • • • • • 1.1.0 • Added prompts to verify *** CertificateRequest Minor Edits Edits from Test Event E3_0655, T9_1200 - Prompt has been modified E2_0530/E3_0530 - Prompt has been modified Scenario for test case E2_0527 has been modified Removed the following test cases E0_0030, E1_0030, E0_0050, E1_0050, E0_0084, E1_0084, E0_0088, E1_0088, E0_0295, E0_0350, E1_0350, E0_0382, E1_0382, E0_0386, E1_0386, E2_0464, E3_0464, E2_0466, E3_0466, E2_0476, E3_0476, E2_0478, E3_0478 Removed the following security tests S0_1310, S0_1320, S0_1330, S0_1340, S0_1350, S0_1360, S0_1370, S1_1410, S1_1420, S1_1430, S1_1440, S2_1510, S2_1520, S2_1530, S2_1540, S2_1550, S2_1560, S2_1570, S3_1610, S3_1620, S3_1630, S3_1640, S3_1650, S3_1660 08/09/12 – JZ 08/09/12 - JZ 08/27/12 - JZ 2/12/14 – EP, BD Removed all references to startBefore Mantis 139: Added Adjacent Event Execution - E0_0292/E1_0292 Mantis 164: Delete Test Case E0_0325/E1_0325 - MarketContext can have a wild card, so can't negative test Mantis 166: Modified conformance rule check to verify that "Z" is at the end of every dateTime string Mantis 173: Conformance rule check modified to validate ordering of intervals in reports Mantis 186: Added test cases to verify expired certificates: cases S1_1405 and S3_1605 Revised tests E1_0285, E0_0285, E2_0510, E3_0510 and deleted tests E2_0520, E3_0520, E2_530, and E3_0530 as the result of changes to rule regarding overlapping events 2/22/16 - JZ Version Change Only Table of Contents Introduction This purpose of this document is to define a set of test cases that systems implementing the OpenADR 2.0a profile must successfully pass in order to obtain certification Terms and Acronyms The following terms and abbreviations are used in this document: Term DR DUT DUT_VEN DUT_VTN TH TH_VEN TH_VTN VEN VTN Description Demand Response Device Under Test – This is used to refer to the sofware implementation of OpenADR and any supporting hardware necessary to execute the sofware Device Under Test VEN Device Under Test VTN Test Harness - A sofware implementation of OpenADR playing the role of a VEN or VTN for the purpose of running test cases Test harness playing the role of the VEN Test harness playing the role of the VTN Virtual End Node Virtual Top Node Scope OpenADR 2.0a is an application layer message exchange specification used for two-way communication of Demand Response (DR), price, and Distributed Energy Resource (DER) signals between the electricity service provider and their customers OpenADR 2.0a is defined in the following documents: • • The OpenADR 2.0a Profile Specification, Revision 1.1 The OpenADR 2.0a schema The testable requirements are defined as a set of conformance rules in the OpenADR 2.0a Profile Specification These rules are referenced in both OpenADR 2.0 PICS and this document Appendix B provides a mapping between each of these conformance rules and the test cases defined in the test specification The scope of the test cases will focus on validating the successful exchange of messages between VTNs and VENs, or VTN/VEN pairs, validating that any exceptions are handled gracefully and without service interruptions, and that security mechanism are properly implemented Validating the functional behavior of the DUT in response to payload content is out of scope for the certification tests For instance, the test cases will test that a VEN acknowledges the receipt of a DR Event but it will not validate that the VEN actually sheds load as a result of this message For product certification, the certification test suite will be will be run over top of the specific transport protocols supported by the DUT If multiple transports are supported by the DUT, the full conformance test will be run over one transport and a representative set of test cases will be run over the remaining transports (See Appendix C) Conformance testing of the supported transports is out of scope for the certification tests However, limited testing will be done to validate OpenADR specific requirements for implementation of the transports, such as the inclusion of specific http headers The certification test suite will be run utilizing each of the security mechanisms supported by the DUT Conformance testing of the security mechanisms is out of scope for the certification tests However, limited testing will be done to validate OpenADR specific configuration requirements for implementation of the security mechanisms, such as use of x509 certificates, TLS versions, and cipher suites Exception handling tests will be limited to a small set of representative real world scenarios The intent of these test cases will not be to exhaustively test all possible exception scenarios, but to validate that the DUT has sound exception handing capabilities For instance, a test might validate that the DUT trigger an error if it is sent a payload with an unknown VEN or VTN ID Boundary and limit testing will utilize values that are deemed to be practical from a test execution perspective and typical from an implementation perspective, as specific implementation limits have not been defined by the OpenADR 2.0a Profile Specification For instance, testing that includes a responseDescription string would use as most a few hundred characters, not 10’s of thousands Optional elements will be testing will be tested as follows: Most payloads will include only mandatory elements only Where practical all optional elements will be used in a single payload Otherwise, the entire set of optional payload elements will be exercised cumulatively across the set of test cases for a given payload Optional elements not need to be included in the payload, but if they are, the VEN or VTN receiving the payload must understand and act upon those optional elements Test Methodology The test harness will play the role of the opposite party (VEN or VTN) in interactions with the DUT Each test case will define a set of perquisites, a test scenario consisting of a sequence of VEN/VTN message exchanges, and an expected result Execution of that test scenario will result in payload exchanges between the DUT and the Test Harness to be analyzed The analysis will consist of the following: • • • • The message interaction patterns are as expected This includes… o Correct response payloads For instance, receiving a oadrCreatedEvent payload in response to a oadrRequestEvent would not be correct o Appropriate request payloads For instance a request containing oadrResponse payload would not be correct That payloads contain well formed XML That payloads conform to the alliance OpenADR 2.0a schema That the specific conformance rules defined in the OpenADR Profile Specification are followed For instance, an OpenADR 2.0a conformance rule states that the payload element signalType must contain the string “simple” This is not validated by the schema, so it is done as a separate conformance rule analysis step • That the intent of the test case is met The test case may expect the VEN to send an optType of “optOut” and if this is not received, the test case will fail Note that some of the OpenADR conformance rules can be validated by simply analyzing the payloads of all message exchanges that occur during execution of the certification test suites These rules are summarized in Appendix A Other conformance rules will require specific test scenarios to validate A few test cases will require the test engineer executing the test to observe the behavior of the device under test in order to determine the pass fail status Test Organization Each test case will be classified into one or more groups as follows: • VEN_Push –Tests where the DUT target is a VEN in a push message exchange pattern • VTN_Push - Tests where the DUT target is a VTN in a push message exchange pattern • VEN_Pull – Tests where the DUT target is a VTN in a pull message exchange pattern • VTN_Pull – Tests where the DUT target is a VTN in a pull message exchange pattern • Positive - This will be the bulk of the tests on successful message exchange A small subset of the positive tests will be tests focused on determining if the DUT can handle values at the boundary/limits of what is typical from an implementation perspective • Negative - Tests focused on the devices ability to gracefully handle unexpected message exchange scenarios and payload contents • Transport - Tests for OpenADR 2.0a specific transport requirements • Security - Tests for OpenADR 2.0a specific security requirements Test case numbering uses the following pattern: XY_ZZZZ X = A letter designating the service or special test type • E= EiEvent Tests • T = Transport Tests • S = Security Tests Y = A digit indicating the type of implementation: = VEN Push = VEN Pull = VTN Push = VEN Pull 9=N/A ZZZZ = A numeric sequence number for the test case with initial increments of 10, with test cases that use the same definition for both push and pull sharing the same number Test cases definitions within this document will be organized by operational sequences and further subdivided by role and exchange pattern (VEN/VTN, Push/Pull) Each test case will be defined in a table as shown below The narrative text on the right describes each row in the table Optional test cases will be indicated by the string “(Opt)” in the title Test ID - name, Sequence (Opt) Prerequisites Scenario Expected Results {Notes} Test Engineer observations Any prerequisite actions required prior to the execution of the test scenario This may include configuring the DUT VTN with specific events, or instructions on the specific payload values to be used in payloads sent by the test harness Prerequisite pending or active events in the VTN or VEN will be programmatically generated by the test case unless indicated otherwise in the test case The specific message exchange scenario required Note that conditional steps based on whether the DUT supports a push or pull model are enclosed in brackets {} In addition to the common pass verdicts, the specific expected behavior as the result of the test scenarios execution Optional support required to run test case, objectives of test case, and other notes to clarify the implementation or execution of the test case Instruction to the test engineer to observe the DUT to confirm the status of the event execution DUT Configuration Requirements In order for many test cases to run successfully, the DUT must be capable of being configured to initiate action or respond to requests For instance, there should be a way to cause a DUT_VEN to respond to an event with an optOut The following is a list of required configurable device behavior for DUTs being submitted for certification • • • • • • • • • • • VEN implementations submitted for certification testing must provide some visual means for a test engineer to determine that an event is currently active Some backchannel way of setting the marketContext URI so that the DUT and test harness can interact with respect to the same marketContext A way to seed a VTN with events and to modify and/or cancel the events A way to opt out of an event with a VEN A way to trigger requests including oadrRequestEvent, and send oadrDistributeEvent A way to initialize the DUT such that all events are cleared from the data store A way to configure or determine the assigned VEN or VTN ID If supported, to configure additional identification values used by the VEN and VTN o groupID o resourceID o partyID o VenID A way to set the polling frequency on the VEN A way to set responseRequired for event configuration on VTN VTN Test Interface Testing of a VTN requires various event scenarios to be configured on the VTN In order to provide a simple and easy to use interface for the test house that will be doing the certification, the vendor submitting the device for certification must provide the following interface: • • • • A drop down menu listing an option for each point in the test case execution where a VTN control panel configuration or action is requested via a prompt The naming convention for the drop down menu should be the test case name followed by an index indicating the prompt sequence within the test case, such as E3_0420_FirstPrompt for the first prompt in the test case requiring operator intervention Submitting or clicking on the menu option should one of the following: o The first request for operator intervention should clear the data store of any pending or active events and then add the requested event(s) o Subsequent prompts to modify, cancel, or add additional events should be executed as instructed when the menu option is selected o Prompts in push scenarios to resend the events in the existing data store should just that Events should be grouped by common push or pull functionality and sorted alphabetically in the menu DUT Implementation Limits The implementation being submitted for certification must support the following minimum capabilities in order to successfully execute the certification test suite Note that these limits not imply minimum market needs for an “a” profile implementations • • • • • • responseDescription, vtnComment – 250 characters venID, vtnID, requestID, uid, groupID, partyID, resourceID, eventID – 50 characters SignalName, MarketContext – 100 characters Maximum size of oadrDistributeEvent: A payload with events each with intervals per signal and a payload with event containing 24 intervals Number of instances of resourceID in eiTarget – Number of eventResponses in oadrResponse - Common Pass and Fail Verdicts In order to avoid repeating common expected results throughout test case definitions, unless stated otherwise in the test case definition, the following criteria will be used to determine if a test case passes or fails: • Common Pass Verdict o Test case scenario runs to completion AND o All exchanged payloads validate against the OpenADR 2.0a schema AND o All OpenADR 2.0a conformance rule validation checks in Appendix A pass 10 VEN Push Security Test Scenarios S0_1300 – VEN Push Security, RSA/ECC Prerequisites Enable all security configuration properties at a test case level Scenario Prompt: “You are currently configured for {HTTP_Security} If this is a formal certification test, please make sure that this configuration matches the security level indicated on the PICS document.” Current state for SHA256 RSA/ECC cipher TH_VTN: oadrDistributeEvent – CPP Event DUT_VEN: Transport Acknowledgement DUT_VEN: oadrCreatedEvent TH_VTN: oadrResponse , success Expected Results 1) Common Pass/Fail verdicts 2) VTN DUT sends event as expected Notes Can VTN DUT send a basic event payload over secure transport using the RSA cipher? 79 VEN Pull Security Test Scenarios S1_1400 – VEN Pull Security, RSA/ECC Prerequisites Enable all security configuration properties at a test case level Scenario Prompt: “You are currently configured for {HTTP_Security} If this is a formal certification test, please make sure that this configuration matches the security level indicated on the PICS document.” \onfigure current state for SHA256 RSA cipher DUT_VEN: oadrRequestEvent TH_VTN: oadrDistributeEvent – CPP Event DUT_VEN: oadrCreatedEvent TH_VTN: oadrResponse , success Expected Results 1) Common Pass/Fail verdicts 2) VTN DUT sends event as expected Notes Can VTN DUT send a basic event payload over secure transport using the RSA cipher? 80 S1_1405 – VEN Pull Security, Expired Certificate Prerequisites None Scenario Port the B profile test case G1_4007_TH_VTN Expected Results 1) Common Pass/Fail verdicts 2) VTN DUT refuses to interoperate Notes Determine whether VEN DUT rejects an attempt to communicate with a VEN that has an expired certificate This test case does not have a corresponding self test routine Use test case S3_1605 for self test 81 VTN Push Security Test Scenarios S2_1500 – VTN Push Security, RSA/ECC Prerequisites Enable all security configuration properties at a test case level Scenario Prompt: “You are currently configured for {HTTP_Security} If this is a formal certification test, please make sure that this configuration matches the security level indicated on the PICS document.” onfigure current state for SHA256 RSA and ECC cipher DUT_VTN: oadrDistributeEvent – CPP Event TH_VEN: Transport Acknowledgement TH_VEN: oadrCreatedEvent, OptIn DUT_VTN: oadrResponse Expected Results Notes 1) Common Pass/Fail verdicts 2) VEN DUT opts into the event as expected Can the VEN DUT receive and opt into an event over a secure transport? 82 VTN Pull Security Test Scenarios S3_1600 – VTN Pull Security, RSA/ECC Prerequisites Enable all security configuration properties at a test case level Scenario Prompt: “You are currently configured for {HTTP_Security} If this is a formal certification test, please make sure that this configuration matches the security level indicated on the PICS document.” Configure current state for SHA256 RSA and ECC cipher TH_VEN: oadrRequestEvent DUT_VTN: oadrDistributeEvent – CPP Event TH_VEN: oadrCreatedEvent, OptIn DUT_VTN: oadrResponse Expected Results 1) Common Pass/Fail verdicts 2) VEN DUT opts into the event as expected Notes Can the VEN DUT receive and opt into an event over a secure transport? 83 S3_1605 – VTN Pull Security, Expired Certificate Prerequisites None Scenario Port the B profile test case G1_4007_TH_VEN Expected Results 1) Common Pass/Fail verdicts 2) VTN DUT refuses to interoperate Notes Determine whether VEN DUT rejects an attempt to communicate with a VEN that has an expired certificate This test case does not have a corresponding self test routine Use test case S1_1405 for self test 84 Appendix A – Conformance Rules Payload Validation The following conformance rules are validated by checking the received payloads independent of the specific test scenario Other conformance rules will be checked via test scenarios Refer to the OpenADR 2.0a profile specification for detailed descriptions of the conformance rules General • Rule 21 • Rule 45 oadrRequestEvent • Rule 23 • Rule 64 oadrDistributeEvent • Rule • Rule • Rule • Rule • Rule 13 • Rule 14 • Rule 15 • • • • • • Rule 29 Rule 40 Rule 41 Rule 50 Rule 63 Rule 70 oadrCreatedEvent • Rule 22 • Rule 35 • Rule 42 • Rule 62 Additional conformance checks include: • Make sure that payloads that contain eiResponse:responseCode has a success code 2xx, otherwise flag them as errors • For oadrCeratedEvent, make sure that eventResponses:eventResponse:response code has a success indication • Check that if an oadrDistibuteEvent has mulitple events in the same payload, the eventID's are unique • Verify that createdDateTime is within plus or minus minute of the UTC time of the PC This will identify problems with implementations not doing UTL time correctly 85 • Verify that the time in createdDateTime and date-time uses zulu notation and ends in Z Expected format is: "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" or "yyyy-MM-dd'T'HH:mm:ss'Z'" Appendix B – Conformance Rules to Test Case Mapping The following table is a cross reference between alliance specific conformance rules and test cases that validate their conformance Refer to section 11.2 of the OpenADR Alliance 2.0a Profile Specification for detailed descriptions of each conformance rule Conformance Rule Test Cases -Proper format validated on each payload by schema validation -Validated by Conformance Rule Payload Validation (see Appendix A) -E0 & E1 = 0250, 285 -E2 & E3 = 510, -Validated by Conformance Rule 15 Payload Validation (see Appendix A) -E2 & E3 = 0452 -E2 & E3 = 0470 0474, 0480, 0490, 0492, 0494, 0496, 0498 -E0 & E1 = 0130 -Validated by Conformance Rule Payload Validation (see Appendix A) -Validated by Conformance Rule Payload Validation (see Appendix A) -Validated by Conformance Rule Payload Validation (see Appendix A) 10 -E2 & E3 = 0492, 0494, 0496, 0498 12 -E0 & E1 = 0082, 0086, 0090, 0092, 0094, 0096, 0098, 0100, 0102, 0104 13 -E2 & E3 = 0527 -Validated by Conformance Rule 13 Payload Validation (see Appendix A) 14 -Validated by Conformance Rule 14 Payload Validation (see Appendix A) 15 -E2 & E3 = 0510 0525 -Validated by Conformance Rule 15 Payload Validation (see Appendix A) 16 -This is not testable as the VTN may continue to send the event to the VEN even afer it has opted out 17 -E0 & E1 = 0280 -E2 & E3 = 0500 18 -E0 & E1 = 0285 86 19 20 -E0 & E1 = 0360 -E0 & E1 = 0094, 0096, 0098 -E2 & E3 = 0484, 0490 21 -E0 & E1 = 0345 -Validated by Conformance Rule 21Payload Validation (see Appendix A) 22 -E0 & E1 = 0070, 0300, most test cases -Validated by Conformance Rule 22 Payload Validation (see Appendix A) 23 -Validated by Conformance Rule 23 Payload Validation (see Appendix A) 25 -E0 & E1 = 0360 27 -E2 & E3 = 0655 29 -Validated by Conformance Rule 29 Payload Validation (see Appendix A) 30 -E0 & E1 = 0260 31 -E0 & E1 = 0210, 0220, 0230, 0240, 0250, 0260 -x-eiRampUp and x-eiRecovery are not testable requirements on the VEN 32 - This is not testable in the profile “a” as we have no method to test the individual interval duration signalPayload executed by the VEN 33 -E0 & E1 = 0390 -E2 & E3 = 0780 -Mismatched venID and vtnID only 35 -Validated by Conformance Rule35 Payload Validation (see Appendix A) 36 -E0 & E1 = 0100, 0102 37 -Test Framework validates 38 -E2 & E3 = 0670 40 -Validated by Conformance Rule 40 Payload Validation (see Appendix A) 41 -E2_0660 and E2/E3_0740-Validated by Conformance Rule 41 Payload Validation (see Appendix A) 42 -Test harness expected events handler verifies each oadrCreatedEvent response against the requestID -Validated by Conformance Rule 42 Payload Validation (see Appendix A) 43 -E0 & E1 = 0190 44 -Default test templates send empty requestID’s 87 45 -Validated by Conformance Rule 45 Payload Validation (see Appendix A) 46 -E0 & E1 = 0300,, 0340, 0360 -E2 & E3 = 0710, 0720, 0730 -Validated by Conformance Rule 15 and 22 Payload Validation (see Appendix A) 47 -E0 & E1 = 0250 48 -E0 & E1 = 0300,, 0330, 0340, 0360 -E2 & E3 = 0710, 0720, 0730, 0740 49 -E0 & E1 = 0300,, 0330, 0340, 0360 -E2 & E3 = 0710, 0720, 0730, 0740 -Tests validate a 4xx response 50 -E2 & E3 = 0680 -Validated by Conformance Rule 50 Payload Validation (see Appendix A) 51 -E2 & E3 = 0670 52 -E2 & E3 = 0657 53 T9_1200 54 -E0 & E1 = 0297 55 -E0 & E1 = 0297 56 -E0 & E1 = 0082, 0086 57 -E0 & E1 = 0090, 0092, 0094, 0096, 0098 58 -E0 & E1 = 0392 59 -E0 & E1 = 0100, 0102, 0104 60 -E0 & E1 = 0345 61 -E0 & E1 = 0370 62 -Validated by Conformance Rule 62 Payload Validation (see Appendix A) 63 -Validated by Conformance Rule 63 Payload Validation (see Appendix A) 64 -Validated by Conformance Rule 64 Payload Validation (see Appendix A) 65 E0 & E1 = 0267, 0268 66 E2 & E3 = 0685 88 67 S0 = 1300 S1 = 1400 S2 = 1500 S3 = 1600 68 S0 = 1300 S1 = 1400 S2 = 1500 S3 = 1600 69 70 Informational -Validated by Conformance Rule 70 Payload Validation (see Appendix A) 89 Appendix C – Additional Transport Test Case List When more than one transport is supported, the full certification test will be run on one transport and the following subset of test cases shall be run on the additional transports: Push VEN Test Case E0_0060 – 1, Send Event with Optional Elements, VEN Push oadrDistributeEvent E0_0082 – Send CPP Event, VEN Push oadrDistributeEvent E0_0090 – 1, Send modified pending event , VEN Push oadrDistributeEvent E0_0100 – 1, Cancelled event , VEN Push oadrDistributeEvent E0_0200 – Limit Values, VEN Push oadrDistributeEvent Pull VEN Test Case E1_0060 – 1, Send Event with Optional Elements, VEN Pull oadrDistributeEvent E1_0082– Send CPP Event, VEN Pull oadrDistributeEvent E1_0090 – 1, Send modified pending event , VEN Pull oadrDistributeEvent E1_0100 – 1, Cancelled event , VEN Pull oadrDistributeEvent E1_0200 – Limit Values, VEN Pull oadrDistributeEvent 90 Push VTN Test Case E2_0452 – OptIn to one event, VTN Push oadrDistributeEvent E2_0454 – OptOut of one event, VTN Push oadrDistributeEvent E2_0430 – OptIn optional elements, VTN Push oadrDistributeEvent E2_0460 – OptOut afer optIn, VTN Push oadrDistributeEvent E2_0474 – OptOut of modified event, VTN Push oadrDistributeEvent Pull VTN Test Case E3_0452 – OptIn to one event, VTN Pull oadrDistributeEvent E3_0454 – OptOut of one event, VTN Pull oadrDistributeEvent E3_0440 – OptOut/In one oadrCreatedEvent, VTN Pull oadrDistributeEvent E3_0450 – OptOut/In two oadrCreatedEvent, VTN Pull oadrDistributeEvent E3_0500 – OptOut afer optIn, VTN Pull oadrDistributeEvent 91 Appendix D– Event State Sequences The following table shows the possible permutations of event status, modification number, response required, previous acknowledgements, and the VEN response Not all combinations are legal, logical, or even useful to test The columns to the far right (DUT VTN and DUT VEN) indicate whether a tests were included for the particular permutation Not all sequences tested in a pending state were repeated for the active state and a limited number of tests were run where there was no response from the VEN (Response Required = never) Some but not all invalid or illogical permutations were tested Pending Pending Mod Numbe Mod Numbe r r Pending Pending Always Never No Yes optOut None Pending Active 0 Never Always No No None optIn Pending Active Always Always No No optIn optOut 10 Pending Active Always Always No Yes optOut optIn 11 Active Always Yes optOut 12 Active Never No None 13 Active Never Yes None 14 Active Always No optIn 15 Active Always No optOut 16 Active Always Yes optIn 92 17 Active Always Yes optOut Event Status Response Response Required Required Previously Previously Ack'd Ack'd VEN VEN Response Response Always Never No No optIn None Comment DUT VTN E2_0452 E3_0452 E2_0480 E3_0480 E2_0454 E3_0454 E2_0468 E3_0468 E2_0484 E3_0484 E2_0470 E3_0470 E2_0490 E3_0490 E2_0474 E3_0474 DUT VEN E0_0092 E0_0082 E1_0092 E1_0082 E0_0086 E1_0086 E0_0094 E1_0094 E0_0090 E1_0090 E0_0096 E1_0096 E0_0098 E1_0098 Event Status Mod Response Numbe Event Status Required r Never Cancelled - Act Previously Mod Ack'd Numbe r No VEN Response Response Required None Always Yes optOut Never Cancelled - Pend Yes None Never No None Always Cancelled - Act No optIn Never No None Always Cancelled - Act No optOut Never Yes None 18 Active 19 Active 20 Cancelled- 21 Cancelled 22 Cancelled Always Yes optIn 23 Cancelled Always Yes optOut 24 Cancelled Never No None 25 Cancelled Never Yes None 26 Cancelled - Pend Always No optIn 27 Cancelled - Act Always No optIn 28 Cancelled - Pend Always No optOut 29 Cancelled - Act Always No optOut 30 Cancelled - Pend Always Yes optIn 31 Cancelled - Act Always Yes optIn 32 Cancelled - Pend Always Yes optOut 33 34 35 36 Previously Comment VEN DUT VTNComment DUT VEN Ack'd Response Illogical 93 E2_0492 E3_0492 E0_0100 E1_0100 E0_0102 E1_0102 E0_0384 E1_0384 E2_0494 E3_0494 E2_0496 E3_0496 DUT VTN DUT VEN E2_0498 E3_0498 E0_0104 E1_0104 ... provider and their customers OpenADR 2.0a is defined in the following documents: • • The OpenADR 2.0a Profile Specification, Revision 1.1 The OpenADR 2.0a schema The testable requirements are defined... and payload contents • Transport - Tests for OpenADR 2.0a specific transport requirements • Security - Tests for OpenADR 2.0a specific security requirements Test case numbering uses the following... payloads conform to the alliance OpenADR 2.0a schema That the specific conformance rules defined in the OpenADR Profile Specification are followed For instance, an OpenADR 2.0a conformance rule states