OpenADR 2.0a Certification Test Harness User Manual

Overview OpenADR 2.0a is an application layer message exchange protocol used for twoway communication of Demand Response (DR), price, and Distributed Energy Resource (DER) signals between the electricity service provider and their customers. The technical requirements for the protocol are defined in the OpenADR 2.0 Profile Specification. Devices that implement OpenADR 2.0a must pass a certification test. The requirements for the certification test are spelled out in the OpenADR 2.0a Certification Test Specification. Each test case defined in the test specification is implemented in the OpenADR 2.0a Certification Test Harness. This document describes how to use the OpenADR 2.0a Certification Test Harness to execute a suite of tests used to certify OpenADR implementations. There are four test suites supported by the Test Harness: • VEN Push Test Suite • VEN Pull Test Suite • VTN Push Test Suite • VTN Pull Test Suite All OpenADR 2.0a interactions are between a Virtual End Node (VEN) and a Virtual Top Node (VTN). When a test is run, the test harness plays the role opposite that of the device under test. For instance, when testing a VEN Push implementation, the test harness will play the role of a VTN Push implementation. In order to successfully pass each of the certification tests, the device under test must have certain capabilities as outlined in the following sections of the OpenADR 2.0a Test Specification: • DUT Configuration Requirements • VTN Test Interface • DUT Implementation Limits The Test Harness is built on top of the open source Eclipse integrated development environment. The test harness consists of a large body of code referred to as the Test Framework that implements the OpenADR 2.0 protocols, and a series of small test case programs that implement the test casespecific logic. Running a test case is as simple as selecting the test case from the Test Harness “Package Explorer” and clicking the green Run toolbar button, watching the text case execute in the Console window, and reviewing the test results in a browser. The following figure shows the major functional areas of the Test Harness GUI. ...

OpenADR 2.0a Certification Test Harness User Manual Version 1.1.1 Revisions: Version 0.9.0 0.9.1 0.9.2 1.0.0 1.0.1 1.0.4 1.0.5 1.0.6 1.1.0 1.1.1 • • • • • • • • • • Changes Initial Draf Additional Edits Clean up by Laura Add security Testing details Edits to security section Update to reflect changes from Test Event Update to reflect the NetworkFX security changes Removal of SHA1 Version Change Html log file added Date/Editor 05/18/12 – JZ 05/23/12 – JZ 05/29/12 - LP 08/12/12 – JZ 08/16/12 – JZ 08/28/12 - JZ 02/26/14 - BD 11/2/15 - JZ 2/22/16 – JZ 6/26/16 - JZ Table of Contents Overview OpenADR 2.0a is an application layer message exchange protocol used for two-way communication of Demand Response (DR), price, and Distributed Energy Resource (DER) signals between the electricity service provider and their customers The technical requirements for the protocol are defined in the OpenADR 2.0 Profile Specification Devices that implement OpenADR 2.0a must pass a certification test The requirements for the certification test are spelled out in the OpenADR 2.0a Certification Test Specification Each test case defined in the test specification is implemented in the OpenADR 2.0a Certification Test Harness This document describes how to use the OpenADR 2.0a Certification Test Harness to execute a suite of tests used to certify OpenADR implementations There are four test suites supported by the Test Harness: • • • • VEN Push Test Suite VEN Pull Test Suite VTN Push Test Suite VTN Pull Test Suite All OpenADR 2.0a interactions are between a Virtual End Node (VEN) and a Virtual Top Node (VTN) When a test is run, the test harness plays the role opposite that of the device under test For instance, when testing a VEN Push implementation, the test harness will play the role of a VTN Push implementation In order to successfully pass each of the certification tests, the device under test must have certain capabilities as outlined in the following sections of the OpenADR 2.0a Test Specification: • • • DUT Configuration Requirements VTN Test Interface DUT Implementation Limits The Test Harness is built on top of the open source Eclipse integrated development environment The test harness consists of a large body of code referred to as the Test Framework that implements the OpenADR 2.0 protocols, and a series of small test case programs that implement the test case-specific logic Running a test case is as simple as selecting the test case from the Test Harness “Package Explorer” and clicking the green Run toolbar button, watching the text case execute in the Console window, and reviewing the test results in a browser The following figure shows the major functional areas of the Test Harness GUI The run button is used to execute a test case This is the test code which the user should not need to alter Test Cases appear in the Package Explorer Figure – Eclipse IDE During test execution progress indications will appear in the console window Once a test case has completed execution, a pass/fail indication is shown in the Console window Detailed transaction logs are recorded for each test execution, and these transaction logs can be used to isolate the cause of a test case failure Figure shows the Test Result Summary screen When a tracelog link is clicked, a detailed transaction log will be displayed in a text editor Figure – Test Result Summary Figure – Transaction Log Installation Java The security tests cases require the test harness be run on top Java 1.7 or later This can be obtained at the following site: http://www.oracle.com/technetwork/java/javase/downloads/java-se-jdk-7-download-432154.html Eclipse IDE For Java Developers The test harness uses the Eclipse Indigo IDE, which can be downloaded from the following site: http://www.eclipse.org/downloads/packages/release/indigo/sr1 Note that the test code is not sensitive to the version of Eclipse that is used as long as it is Juno or later The narrative in this user’s guide presumes the Juno version of Eclipse is being used Eclipse Configuration Two changes needs to be made to the Eclipse environment (Fig 4a and Fig 4b) From the Main Menu select Window, Preferences, Java Expand Compiler, and then select the Errors/Warning option On the right panel under Deprecated API, change the Forbidden reference to Warning Also from Window, Preferences, select General, Workspace, and check Refresh using native hooks or polling Figure 4a – Eclipse Configuration Setting Figure 4b – Eclipse Configuration Setting Test Harness The OpenADR 2.0a test Harness is distributed as an Eclipse project zip file The process for installing the Test Harness in the Eclipse workspace is as follows: • • • • Extract the project into a temporary directory Rename the project folder to something unique, and then copy the project directory into the Eclipse workspace directory Note that the project folder name will become the project name inside Eclipse Use the Eclipse File, Import, General, "Existing Project into Workspace" option You can then move over any property settings from the previous release and then delete the previous release if desired The test cases are located under “src” in the following areas: • testcases.ven.pull • testcases.ven.push • testcases.vtn.pull • testcases.vtn.push It is easier to traverse the test cases when running tests if you use the drop down menu in the Package Explorer to change the package presentation to hierarchical (Fig 5) Test cases listed under "DR_PRogram_Guide" are not part of the message exchange certification and they may be ignored Figure – Package Presentation Test Cases Test Case Naming Test cases names are structured as xx_xxxx_yy_zzz, such as E1_0010_TH_DUT • xx_xxxx is the test name used in the test specification • yy can be “TH” for Test Harness or “DUT” for a device under test (used for self test) • zzz is the role that the is being played by the test code (VEN or VTN) When testing a real device, only the TH test cases will be used For instance, to test a real pull VEN you would run the test case E1_0010_TH_VTN In this case the test harness would be playing the role of the VTN Test Logs Test execution can be observed in the Eclipse Console window, and you will see a pass or fail indication at the end of test case execution However, detailed transactions records and failure information can be found in a log file generated for each test case in the log directory (configured in Properties) Two test cases under “util” will allow you to clear the log directory and to launch a browser to see the log file summary, which will show all log files with a pass/fail status Click on the link to see the detailed log Afer running a test case, you will need to refresh the browser to see 10 • • VEN Push VTN Pull VTN Push • • • • • • • • • • • If running self test, start self-test routine If testing real device, make sure its polling feature is enabled If running self test, start self-test routine If testing real device, make sure its push functionality is enabled Start test case Follow directions on prompt and click resume If running self test, start self-test routine If testing real device, make sure device is enabled Start test case Follow directions on prompt and click resume Start test case Follow directions on prompt and click resume If running self test, start self-test routine Note that when running self-test routines it is not necessary to configure events or to wait for specific time periods as requested by the prompts 12 Test Case Setup In order for the test suite to work with a device under test, a number of properties need to be set in the test suite The Configuration properties are located under “src.properties” The configuration file openadrconfig.properties acts as a pointer to vendor-specific configuration files The contents of this file look as follows: #################################################### # OpenADR Properties # Test harness configuration properties are read from # the file point to by the Properties_File key # listed below ##################################################### Properties_File=vendor.isy.openadrconfig.properties ###################################################### To create your own configuration file, copy and paste selfest configuration file (vendor.selfest.openadrconfig.properties) and make your own edits The most important item to set up in your vendor-specific properties is the test report directory Modify the following property to point to the directory as shown in the example below: logPath=C:\\OpenADR_Test_Logs\\ Each time a test case is run, a text file with a time stamp is written to this directory and the file Logfile.xml and logfile.html is updated Appendix A provides detailed information on how to set up both the DUT and the test harness for testing Appendix B provides a summary of all configuration settings Appendix C shows a sample of the HTTP_SHA256_Security.properties Optional Test Cases Some test cases are dependent on the ability to configure specific values on the Device Under Test If these values are not configurable, the OpenADR 2.0a PICS document provides a list of questions that determine which optional test cases can be skipped This can be found under the heading Optional Test Case Guidelines 13 Security Configuration Security is controlled using the following default properties: • HTTP_Security=HTTP_Basic_Or_No_Security Valid values for this property are listed in comments in the property file Note that this security property controls which of the security profile files are used at run time The test harness is preconfigured with test certificates provide by the Alliance Certificate Authority, NetworkFX Vendors should configure their devices with test certificates from the OpenADR/NetworkFX portal (http://www.networkfx.net/testcerts) The test harness certificate stores are configured as follows: • Truststore contains both root and intermediate certs • Keystores used by the test harness when playing the the role of a VEN or VTN contain the device cert and private key VENs being submitted for certification must have host authentication of the x.509 certificate CN field disabled to avoid needing to reconfigure the test harness server certificates The VTN test certificates used as part of the test harness have been generated with an CN name of "vtn" As long as the device under test has host authentication disabled, these certificates will work correctly in the test harness It is beyond the scope of this manual to provide detailed guidance for rebuilding trust and keystore files with new certificates on the test harness Please review the following helpful hints in the following section prior to doing security testing: Disabling Host Verification 14 Helpful Hints Please note the following: Active Event Indicator – A number of test cases will ask the test operator to observe the transition of an event to Active using the Active Event indicator However, if you have access to a DUT Control Panel that shows the start and end of an event, as well as the eventStatus, it may not be necessary to actually wait for an event to end if you can see from the DUT console that the event has been scheduled to end at a specific time Judgment must be exercised! Set polling to 10 seconds – For Pull VENs, set the polling interval to around 10 seconds The test cases deal in events that are just minutes long, so a longer poll time may result in false failures Time Synchronization – While all the payload validation is based on the createdDateTime in the payload, some false failures may occur if there is a delta between the time on the Device Under Test and the time on the PC executing the test suite Note that there is a built-in conformance rule check that validates that the DUT is reasonably synced with the test harness, which should avoid most issues Errors in the Log Window – We shut down the test suite http server afer each test case execution During the shutdown process, an async payload from the DUT may generate an error in the Console window However, always check the test report in the log directory, as typically you will see that the test case did run successfully even though an error appears at the end of the Console window trace Failure information in test reports – The Console window just shows status and an overall pass/fail status For detailed conformance errors you will need to scan through the test report log file for specific errors that caused the test to fail Processes Left Running – If you start and then abort a test case, you may leave processes running If you don’t kill these processes in the Eclipse Debug perspective or allow them to time out, subsequent test cases may not run correctly Clearing the event data store – Generally speaking, implicit cancellation when testing VENs takes care of flushing the DUT at the start of each test case However, it is critical on VTNs that the event data store be cleared before the next test VTN Pull test cases will prompt you to clear the data store at the start of each test case The VTN Push test cases will prompt at the end of each test case and also wait 10 seconds for any retries to occur before ending the test case Conformance Rule Disable – Conformance rules can be disabled selectively by inserting code at the top of the test() method in test cases using the following format: ConformanceRuleValidator.setDisableCreatedEvent_ResponseCodeSuccess_Check(true); Self Tests – Make sure DUT processes have stopped before running the next test You can toggle the Console window to show the DUT window and use the red square to kill the process 15 Clean Project – Sometimes Eclipse gets confused and starts claiming that it cannot access the “main” routine in the test case The solution is to use the Project, Clean menu option and/or exit Eclipse and restart it Refresh Project – If you attempt to edit the properties while a test case process is running, you may get a project error stating that the a file in the bin directory cannot be deleted Simply refresh the project afer making sure all the processes are terminated Firewalls – In most test scenarios, the test harness is playing the role of an http server It is necessary to open up any Windows or network firewalls to allow traffic to and from the IP address of the DUT We have discovered in some circumstances just opening up a port is not adequate and disabling the firewall is necessary Wireshark - The test framework writes the payloads to the log file before it hands them it to the transport layer Because we are using Restlet on top of the jetty http server, our application code is a couple of layers removed from transport failures, and the failures don't percolate back up to the test code If it appears that you have a communication problem, it is best to monitor traffic with Wireshark, as the test logs can sometimes provide a false positive that traffic is being exchanged Disabling TH Host Verification – When security is enabled in properties, the URL IP address is verified against the certificates received by the test harness client The verification can be disabled in the properties file using the vtnClientVerifyHostName and venClientVerifyHostName properties This will allow the Test Harness self test routines to run without needed to rebuild the certificate files with the localhost IP address and will allow the test harness to accept certificates from DUT devices with nonmatching host or IP addresses Disabling DUT Host Verification – The test harness, when playing the role of the TLS server, uses a certificate with a CN and SubAltName of "vtn" You will need to disable host name verification on the DUT or this server certificate will be rejected 16 Appendix A – Configuration Tables Pull VEN Configuration Characteristic VEN ID of the DUT VTN ID of the test harness Market Context Associated with VEN If the DUT supports partyD, resourceID, or groupID, make sure that the test harness and DUT have matching values If the DUT requires a special string to invoke a test event, add it to properties Create a directory to contain test results and add it to properties Use a double backslash for the path delimiter Set the port number that the test harness VTN will monitor for inbound request s from the VEN Configure the DUT VEN with the URL of the test harness If the DUT VEN supports communication with multiple VTNs configure matching VTN/VEN values If the DUT VEN supports SHA-256 hash Test Harness (VTN) Property DUT VEN Configuration VEN_ID=DUT_VEN DUT_VEN VTN_ID=TH_VTN TH_VTN DR_MarketContext_1_Name= http://MarketContext1 DR_MarketContext_2_Name= http://MarketContext2 Party_ID=Party_123 ResourceID=Resource_123 GroupID=Group_123 http://MarketContext1 http://MarketContext2 Party_123 Resource_123 Group_123 TestEventString=Test Event logPath=C:\\Users\\TestResults\\ VtnURL_Port=8080 http://IP_Address :Port/OpenADR2/Simple/EiEven t VEN_ID_2=DUT_VEN2 VTN_ID_2=TH_VTN2 DUT_VEN2 TH_VTN2 HTTP_Security=HTTP_SHA256_Securi ty SHA-256 17 Push VEN Configuration Characteristic VEN ID of the DUT Test Harness (VTN) Property DUT VEN Configuration VEN_ID=DUT_VEN DUT_VEN VTN ID of the test harness Market Context Associated with VEN VTN_ID=TH_VTN TH_VTN DR_MarketContext_1_Name= http://MarketContext1 DR_MarketContext_2_Name= http://MarketContext2 Party_ID=Party_123 ResourceID=Resource_123 GroupID=Group_123 http://MarketContext1 http://MarketContext2 If the DUT supports partyD, resourceID, or groupID, make sure that the test harness and DUT have matching values If the DUT requires a special string to invoke a test event, add it to properties Create a directory to contain test results and add it to properties Use a double backslash for the path delimiter Set the port number that the test harness VTN will monitor for inbound requests from the VEN (note in a push model, the VEN can still pull) Configure the DUT VEN with the URL of the test harness if the DUT VEN supports pull while in a push mode Configure the test harness with the address of the DUT VEN If the DUT VEN supports communication with multiple VTNs, configure matching VTN/VEN values Party_123 Resource_123 Group_123 TestEventString=This is a test Event logPath=C:\\Users\\TestResults\\ VtnURL_Port=8080 http://IP_Address : 8080/OpenADR2/Simple/EiEven t VenURL=http://IP_Address :Port/OpenADR2/Simple/EiEvent VEN_ID_2=DUT_VEN2 VTN_ID_2=TH_VTN2 18 DUT_VEN2 TH_VTN2 If the DUT VEN supports SHA-256 hash HTTP_Security=HTTP_SHA256_Securi ty SHA-256 Pull VTN Configuration Characteristic VEN ID of the test harness VTN ID of the device under test Market Context Associated with VTN If the DUT supports partyD, resourceID, or groupID, make sure that the test harness and DUT have matching values Create a directory to contain test results and add it to properties Use a double backslash for the path delimiter Configure the Test harness VEN with the address of the DUT VTN If the DUT VTN supports SHA-256 hash Test Harness (VEN) Property DUT VTN Configuration VEN_ID=DUT_VEN DUT_VEN VTN_ID=TH_VTN TH_VTN DR_MarketContext_1_Name= http://MarketContext1 DR_MarketContext_2_Name= http://MarketContext2 http://MarketContext1 http://MarketContext2 Party_ID=Party_123 ResourceID=Resource_123 GroupID=Group_123 Party_123 Resource_123 Group_123 logPath=C:\\Users\\TestResults\\ VtnURL=http://IP_Address :Port/OpenADR2/Simple/EiEvent HTTP_Security=HTTP_SHA256_Securi ty 19 SHA-256 Push VTN Configuration Characteristic VEN ID of the test harness VTN ID of the device under test Market Context Associated with VTN If the DUT supports partyD, resourceID, or groupID, make sure that the test harness and DUT have matching values Create a directory to contain test results and add it to properties Use a double backslash for the path delimiter Configure the test harness VEN with the port number to monitor Configure the DUT with the URL of the test harness VEN Configure the Test harness VEN with the address of the DUT VTN If the DUT VTN supports SHA-256 hash Test Harness (VEN) Property DUT VTN Configuration VEN_ID=DUT_VEN DUT_VEN VTN_ID=TH_VTN TH_VTN DR_MarketContext_1_Name= http://MarketContext1 DR_MarketContext_2_Name= http://MarketContext2 http://MarketContext1 http://MarketContext2 Party_ID=Party_123 ResourceID=Resource_123 GroupID=Group_123 Party_123 Resource_123 Group_123 logPath=C:\\Users\\TestResults\\ VenURL_Port= 8080 http://IP_Address : 8080/OpenADR2/Simple/EiEven t VtnURL=http://IP_Address :Port/OpenADR2/Simple/EiEvent HTTP_Security=HTTP_SHA256_Securi ty 20 SHA-256 Appendix B – Configuration Properties Configuration properties are located under “src” at openadrconfig.properties and included at the end of this section The following is a brief description of each configuration property: asyncResponseTimeout – An overall timeout for the test case in milliseconds During prompts, this timeout is suspended createdEventAsynchTimeout – The time to wait in milliseconds for all the expected oadrCreatedEvent responses to be received DR_MarketContext_1_Name, DR_MarketContext_2_Name –The market context used for message exchange If the test harness is playing the role of the VTN, this value is placed in the outbound oadrDistibuteEvent payloads If the test harness is the VEN, it is the expected marketContext of the events being received from the DUT VTN it is communicating with Global Conformance Rule Control – Each payload exchanged is checked for a variety of conformance rules These rules can be disabled on a global basis if a particular bug in an implementation is preventing test cases from running GroupID - This is the Group ID used by the test harness to populate the oadrDistibuteEvent eiTarget object This value should match the configuration of the VEN being tested logFileName = The name of the summary log files containing pointers to individual test file transaction logs If deleted, this file is recreated next time a test is run Login_Name -A login name used for http basic authentication by VTN or VEN client operations Login_Password - A password used for http basic authentication by VTN or VEN client operations logPath - The directory path to the logFileName defined elsewhere in properties Party_ID – The Party ID used by the test harness to populate the oadrDistibuteEvent eiTarget object This value should match the configuration of the VEN being tested Report_Header_Info_1, 2, and - Text placed in the header of the test execution log report ResourceID - The Resource ID used by the test harness to populate the oadrDistibuteEvent eiTarget object This value should match the configuration of the VEN being tested SchemaFile – The name of the schema file Users should not alter this value Security_Debug – This controls whether a detailed TLS handshake transaction log is output to the console during message exchange Some security test cases force this setting to true 21 HTTP_Security – Controls which security configuration file is read when a test case is run The security configuration files are located in src.properties.security Can be set to HTTP_Basic_Or_No_Security, or HTTP_SHA256_Security Test Case Specific Configuration Keys - These configuration values are used for specific test cases and, as a general rule, should not be changed • MaxLen_VTN_ID=01234567890123456789012345678901234567890123456789 • MaxLen_MarketContext=http://789012345678901234567890123456789012345678901234567 890123456789012345678901234567890123456789 • VEN_ID_2=DUT_VEN2 • VTN_ID_2=TH_VTN2 testCaseBufferTimeout – The time to wait afer the test case is completed to capture any post execution traffic TestEventString - This string is placed in the oadrDistributeEvent testEvent element when testing whether a device supports the test event VEN_ID – The VEN ID used for message exchange If the test harness is playing the role of the VEN, this value is placed in the outbound payloads If the test harness is the VTN, it is the expected ID of the VEN it is communicating with VTN_ID - The VTN ID used for message exchange If the test harness is playing the role of the VTN, this value is placed in the outbound payloads If the test harness is the VEN, it is the expected ID of the VTN it is communicating with VEN_ID_2 – If the DUT VEN supports communication with multiple VTNs, this would be its secondary VEN ID This value is used to validate that the VEN responding to the VTN VTN_ID_2 – If the DUT VEN supports communication with multiple VTNs this is the secondary VTN ID that it will communicate with This value is included in payloads sent to the DUT VEN VenURL - The full path to the EiEvent service endpoint to talk to a VEN Must include “OpenADR2/Simple/EiEvent” afer the base path VenURL_Port - The port number that the test harness listens on when playing the role of a VEN Suggest setting to 8080 VenURL_Resource - This property must be “/OpenADR2/Simple” Users should not change VtnURL - The full path to the EiEvent service endpoint to talk to a VTN Must include “OpenADR2/Simple/EiEvent” afer the base path VtnURL_Port – The port number that the test harnesses listens on with playing the role of VTN Suggest setting to 8080 22 VtnURL_Resource - This property must be “/OpenADR2/Simple” Users should not change Configuration Listing: ################################################################### # Vendor Specific Properties # To use this file set Properties_File key in # in openadconfig.properties to point to this file name #################################################################### asyncResponseTimeout=40000 createdEventAsynchTimeout=15000 testCaseBufferTimeout=10000 #Info for Report Header Report_Header_Info_1=Name of Implementation Report_Header_Info_2=Version Number Report_Header_Info_3=Additional Info VEN_ID=TH_VEN VTN_ID=TH_VTN Party_ID=Party_123 ResourceID=Resource_123 GroupID=Group_123 Login_Name= Login_Password= TestEventString=This is a test Event DR_MarketContext_1_Name=http://MarketContext1 DR_MarketContext_2_Name=http://MarketContext2 #** Users should not alter the schema file name ** SchemaFile=oadr_20a.xsd logPath=C\:\\OpenADR_Test_Logs\\ logFileName=logfile.xml # VTN URL # Do not use https as this is automatically set based on the value in the property HTTP_Security VtnURL=http://127.0.0.1:8080/OpenADR2/Simple/EiEvent # VEN URL # Do not use https as this is automatically set based on the value in the property HTTP_Security VenURL=http://127.0.0.1:8081/OpenADR2/Simple/EiEvent # Below is required to start local VTN resource VtnURL_Port=8080 VtnURL_Resource=/OpenADR2/Simple # Below is required to start local VEN resource VenURL_Port= 8081 23 VenURL_Resource=/OpenADR2/Simple #TestCase specific properties #For Test Case E0_0200 and E1_0200 MaxLen_VTN_ID=01234567890123456789012345678901234567890123456789 MaxLen_MarketContext=http://78901234567890123456789012345678901234567890123456 7890123456789012345678901234567890123456789 #For test case E0_0295 and E1_0295 (two VTNs) VEN_ID_2=DUT_VEN2 VTN_ID_2=TH_VTN2 #**** Security ****** #Global settings for security behavior for all test cases #Overridden locally in security test cases # HTTP_Basic_Or_No_Security, HTTP_SHA256_Security, HTTP_SHA256_Expired HTTP_Security=HTTP_SHA256_Security # This controls whether a detailed TLS handshake transaction log is output to the console during message exchange Security_Debug=false #Global Conformance Rule Control - true or false disableDistributeEvent_RequestIDPresent_Check=false disableDistributeEvent_MaketContext_Check=false disableDistributeEvent_SignalNameSimple_Check=false disableDistributeEvent_UIDValid_Check=false disableDistributeEvent_MultipleVENIDFound_Check=false disableDistributeEvent_VtnIDValid_Check=false disableDistributeEvent_ResponseCodeSuccess_Check=false disableDistributeEvent_EventIDsUnique_ValidCheck=false disableDistributeEvent_EventStatusValid_ValidCheck=false disableDistributeEvent_CreatedDateTimeWithinOneMin=false disableDistributeEvent_AtLeastOneEiTargetMatch_ValidCheck=false disableDistributeEvent_EventOrderValid_Check=false disableDistributeEvent_ActiveCurrentValueMatchCurrentPayload_Check=false disableDistributeEvent_NonActiveCurrentValueValid_Check=false disableDistributeEvent_PayloadFloatLimitValid_Check=false disableDistributeEvent_ActivePeriodDurationValid_Check=false disableDistributeEvent_EIResponsePresent_Check=false disableDistributeEvent_MultipleEventSignalpPresent_Check=false disableDistributeEvent_AllPreviousEvntPresent_Check=false disable_isDateTimeinZulu_Check=false disableCreatedEvent_VenIDValid_Check=false disableCreatedEvent_ResponseCodeSuccess_Check=false disableCreatedEvent_RequestIDValid_Check=false disableCreatedEvent_EventIDValid_Check=false disableCreatedEvent_EIResponsesValid_Check=false disableRequestEvent_VenIDValid_Check=false disableRequestEvent_isAllPrevCreatedEvntsRecvd_Check=false disableResponseEvent_ResponseCodeSuccess_Check=false 24 Appendix C – Security Configuration Properties ########################################## ######## HTTP_SHA256_Security ############ ########################################## # Global settings for security behavior for all test cases # Controlled locally in security test cases # Use client certificates to login to the XMPP Server # If set to true SSL is used regardless of the security_Enabled property setting # If set to false, plain authentication is used XMPP_SASL_External= #If set to true, all test cases will run with SSL Security_Enabled=true # Selection of TLS version TLSv1.2 (currently this only accepted cipher) TLS_Version=TLSv1.2 # OpenADR Alliance supported ciphers - Do not change Client_TLS12_Ciphers=TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC _SHA256 Server_TLS12_Ciphers=TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CB C_SHA256 # Controls the test harness verification of the servers X.509 CN field Recommend leave set to false vtnClientVerifyHostName=false venClientVerifyHostName=false # X.509 certificates # Paths are relative to the TH directory # IMPORTANT NOTE: Openfire has its own set of trust and keystores, as well as cipher settings VTN_trustStorePath=certs/NetworkFX/SHA256/truststore.jks VTN_truststorePassword=password VTN_needClientAuth=true VTN_keystorePath_HTTP=certs/NetworkFX/SHA256/keystore_vtn_vtn.jks VTN_keystorePath_XMPP= VTN_keystorePassword=password VTN_keystoreType=JKS VEN_trustStorePath=certs/NetworkFX/SHA256/truststore.jks VEN_truststorePassword=password VEN_needClientAuth=true VEN_keystorePath_HTTP=certs/NetworkFX/SHA256/keystore_ven_111111111111.jks VEN_keystorePath_XMPP= VEN_keystorePassword=password VEN_keystoreType=JKS 25 Appendix D – Updated Tests V1.10 - The following tests were updated to either correct or enhance the behavior: • Version change only V1.06 - The following tests were updated to either correct or enhance the behavior: • Removed all references to startBefore • Added Adjacent Event Execution - E0_0292/E1_0292 • Delete Test Case E0_0325/E1_0325 - MarketContext can have a wild card, so can't negative test • Modified conformance rule check to verify that "Z" is at the end of every dateTime string • Conformance rule check modified to validate ordering of intervals in reports • Added test cases to verify expired certificates: cases S1_1405 and S3_1605 • Revised tests E1_0285, E0_0285, E2_0510, E3_0510 and deleted tests E2_0520, E3_0520, E2_530, and A_E3_0530 as the result of changes to rule regarding overlapping events 26 ... Certification Test Specification Each test case defined in the test specification is implemented in the OpenADR 2.0a Certification Test Harness This document describes how to use the OpenADR 2.0a Certification. .. in the OpenADR 2.0 Profile Specification Devices that implement OpenADR 2.0a must pass a certification test The requirements for the certification test are spelled out in the OpenADR 2.0a Certification. .. Certification Test Harness to execute a suite of tests used to certify OpenADR implementations There are four test suites supported by the Test Harness: • • • • VEN Push Test Suite VEN Pull Test Suite