Available online at www.sciencedirect.com Available online at www.sciencedirect.com Procedia Engineering Procedia Engineering 00 (2011) 000–000 Procedia Engineering 15 (2011) 3505 – 3510 www.elsevier.com/locate/procedia Advanced in Control Engineeringand Information Science Confidence Measures Analysis of Software Security Evaluation Zhengping Rena,b a*, Song Huangb, Yi Yaob, Yu Hongb a Institute of Communication Engineering, PLAUST, BiaoYing 2#, Yu Dao Street, Nanjing, 210007, China b Institute of Command Automation, PLAUST, Hai Fu Xiang 1#, Nanjing, 210007, China Abstract Security evaluation technologies are important to software developers and users, especially in security-critical systems However, there is very lack of effective method to acquire confidence measures of software security evaluation supported by mathematical theories In this paper, the dependability in security evaluation conclusion was studied, and confidence measures of software security evaluation were modeled from parametric estimation angle based on practical experiences and statistics theory Furthermore, a confidence measures analysis method towards evaluation conclusion of specific values was presented based on Bayes analysis to solve confidence measure acquiring problem © 2011 Published by Elsevier Ltd Selection and/or peer-review under responsibility of [CEIS 2011] Keywords: Software security evaluation; Evaluation system; Confidence measures; Parametric estimation; Bayes analysis Introduction With the rapid development of computer technologies, people depend on software increasingly Meanwhile, all kinds of attacks bring serious security problem to software users In security-critical systems such as military applications, security is one of the most important quality characteristics of software As the result, security evaluation technologies became research hotspots of software * Corresponding author Tel.: +86-025-80824569 E-mail address: zhengpingren@gmail.com 1877-7058 © 2011 Published by Elsevier Ltd doi:10.1016/j.proeng.2011.08.656 3506 Zhengping Ren / Procedia Engineering 15 (2011) 3505 – 3510 Zhengping Ren et et al/al Procedia Engineering 00 (2011) 000–000 engineering Software security evaluation can give the conclusion if the software product meets security requirements, using security measurement information from development and testing process It shows the security quality and builds the confidence of development, and gives important reference to user if the software product is acceptable In this paper, the confidence measures of software security evaluation were studied by means of mathematical statistics The status quo of studies in software security evaluation was summarized first After giving the definition of confidence measures of software security evaluation from parametric estimation angle, a confidence measures analysis method towards evaluation conclusion of explicit values was presented based on Bayes analysis Software Security Evaluation Methods According to literature [1], software security is an ability of software product to protect information and data, which makes the information and data can not be read or modified by unauthorized people, and can not reject the access of authorized ones In fact, security is historically been defined more often in terms of its most popular subfactors: availability, integrity, and privacy However, security is a relatively complex concept and cannot be adequately addressed merely in terms of three parts above Typical decomposition of security can be find in [2]。Unfortunately，there is no widely accepted, industrystandard decomposition of security into a taxonomy of its component quality subfactors, and these quality subfactors not have industry definitions As the result, there is no uniform software security evaluation method in industry People have to establish different evaluation systems according to software types (such as embed system and network application) and select different evaluation methods Existing software evaluation methods mainly include three classes: Security Evaluation based on Security Level (SESL), Security Evaluation based on Formalism (SEF), and Quantitative Security Evaluation (QSE) SESL uses corresponding standards as evaluating criteria, such as [3][4] Literature [5] proposed SSE-CMM (System Security Engineering Capability Maturity Model) for security evaluation based on existing standards Literature [6] combines influence of availability, integrity, and privacy, using fuzzy synthetical decision-making method to evaluate system SEF proves specific security characteristic under given security hypothesis by mathematic analysis and model checking, and it mainly used in protocol analyzing and proving [7] Literature [8] established a formal evaluation model based on combined independent security factors, combined complementary security factors, and combined correlative security factors Literature [9] proposed a security evaluation method based on Bayesian function networks QSE analyzed and evaluated software system using quantitative indices, and used mathematical statistic to process the measurement data The final evaluation conclusion generally is explicit value No matter what method, the security process is similar First, software engineering expert established an evaluation system based on measurement theory and software type Second, trained evaluators evaluate the software security using the evaluation system on specifically steps, and obtained the evaluation conclusion Data needed in establishing evaluation system can obtain from software testing, system applying process, similar systems and simulation Since the establishing process of evaluation system must depend on the knowledge and experience of experts, they bring subjectivity to the evaluation system inevitably This is the main reason why some users of evaluation system are suspicious of evaluation conclusion and even the evaluation system itself in some situations, especially the evaluation conclusion is not according with intuitions From this angle, confidence measures are as important as the evaluation conclusion to software security evaluation Hence, it is necessary to introduce confidence measures acquiring method in software security evaluation to help the decision-maker judge the real security level of the software product Zhengping RenRen et al / Procedia Engineering 15 00 (2011) 3505 – 3510 Zhengping et al / Procedia Engineering (2011) 000–000 3507 3 Confidence Measures Defining of Software Security Evaluation Software security evaluation can be described with a function E=Me(S), where Me is an evaluating method for some kind of specified system and S is referred to as the object software entity to be evaluated Here we only consider the evaluation methods whose results are explicit values, and three hypotheses are made for security evaluation process: (1) Expert knowledge introduced during the establishing and applying of the evaluation system is independent; (2) Knowledge from every single expert would introduce errors to the evaluation conclusion; and (3) Software security evaluation is small samples situation It is obvious that the expert knowledge of the establishing phase and applying phase are independent, and even in the same phase, knowledge from different experts is also independent from each other Therefore the impacts on the final evaluation conclusion from different expert knowledge are uncorrelated As for the subjectivity of expert knowledge, different experts hold different opinions for the same thing, so the evaluation deviation induced by different expert knowledge would be a stochastic variable Hence, we can find that E is a stochastic variable form hypothesis (1) and (2) According to the Linderberg central-limit theorem, if a stochastic variable is the sum of a set of stochastic numbers which are independent and uniform, its probability distribution approximates to the normal distribution [10], and so E conforms to the normal distribution, that is E~N (μ, σ2) where μ is the objective security attribute value of the software to be evaluated For software security evaluation, it is impossible to collect great amount of evaluation data of similar software systems, thus the evaluation can be made only upon small amount of data Under this situation, the unbiasedness and effectiveness can not be assured by the traditional methods Instead, according to the Bayesian theory, we make full use of the prior information to realize the calculation of the confidence measures with particular small samples Confidence measures describe the dependability level of software security evaluation conclusion according with the objective security attribute value, and they measure the match degree between the evaluation conclusion and the actual security quality In this paper, we can define the confidence measures of software security evaluation as the follows, using parametric estimation methods of mathematical statistic theory: Suppose S is the object to be evaluated, (e1, e2, …, en) is the final result obtained after the nth evaluation of S using Me, and μ is the objective efficiency value which is an unknown parameter of the population distribution For the given γ(0