Data Analysis Machine Learning and Applications Episode 2 Part 4 doc

25 247 0
Data Analysis Machine Learning and Applications Episode 2 Part 4 doc

Đang tải... (xem toàn văn)

Tài liệu hạn chế xem trước, để xem đầy đủ mời bạn chọn Tải xuống

Thông tin tài liệu

A Pattern Based Data Mining Approach 333 2. Science converges. Concepts in one area of science is applicable in another area. Patterns support these processes. This potential is comparable to the promises of Systems Theory. 3. Decision for a specific algorithm can be postponed to later stages. A solution path as a whole will be sketched through patterns and algorithms need only be filled in immediately prior to processing. Using differnet algorithms in places will not invalidate the solution path, creating “late binding” at the algorithm level. Current Data Mining applications occasionally provide the user with first traces of pattern based DM. Figure 5 shows the example of Bagging of Classifiers within the TANAGRA project and its graphical user interface (Rakotomalala (2004)). Bag- ging cannot be described with a pure data flow paradigm, rather a nesting of a clas- sifier pattern within the bagging pattern is needed. This nested structure will then be pipelined with pre- and postprocessing patterns. Fig. 5. Screenshot of Tanagra Software Further steps in our project are to • collect a list of patterns which are useful in the whole knowledge dis- covery process and data mining (list will be open-ended). • integrate these patterns into data mining software to help design ad-hoc algorithms, choose an existing one or have guidance in the data mining process. • develop a software prototype with our pattern and make experiments with users: how it works and what are the benefits. 334 Boris Delibaši ´ c, Kathrin Kirchner and Johannes Ruhland References ALEXANDER, C. (1979): The Timeless Way of Building, Oxford University Press. ALEXANDER, C. (2002a): The Nature of Order Book 1: The Phenomenon of Life, The Center for Environmental Structure, Berkeley, California. ALEXANDER, C. (2002b): The Nature of Order Book 2: The Process of Creating Life, The Center for Environmental Structure, Berkeley, California. CHAPMAN, P., CLINTON, J., KERBER, R., KHABAZA, T., REINARTZ, T., SHEARER, C. and WIRTH, R. (2000): CRISP-DM 1.0. Step-by-step data mining guide, www.crisp- dm.org. COPLIEN, J.O.(1996): Software Patterns, SIGS Books & Multimedia. COPLIEN, J.O. and ZHAO, L. (2005): Toward a General Formal Foundation of Design - Symmetry and Broken Symmetry, Brussels: VUB Press. ECKERT, C. and CLARKSON, J. (2005): Design Process Improvement: a review of current practice, Springer Verlag London. FAYYAD, U.M., PIATETSKY-SHAPIRO, G. and UTHURUSAMY, R. (Ed.) (1996): Ad- vances in Knowledge Discovery and Data Mining, MIT Press. GAMMA, E., HELM, R., JOHNSON, R. and VLISSIDES, J. (1995): Design Patterns. Ele- ments of Reusable Object-Oriented Software, Addison-Wesley. HIPPNER, H., MERZENICH, M. and STOLZ, C. (2002): Data Mining: Einsatzpotentiale und Anwendungspraxis in deutschen Unternehmen, In: WILDE, K.D.: Data Mining Studie, absatzwirtschaft. RAKOTOMALALA, R. (2004): Tanagra – A free data mining software for research and edu- cation, www.eric.univ-lyon2.fr/∼rico/tanagra/. WITTEN, I.H. and FRANK, E. (2005): Data Mining: Practical machine learning tools and techniques, Morgan Kaufmann, San Francisco. A Procedure to Estimate Relations in a Balanced Scorecard Veit Köppen 1 , Henner Graubitz 2 , Hans-K. Arndt 2 and Hans-J. Lenz 1 1 Institut für Produktion, Wirtschaftsinformatik und Operations Research Freie Universität Berlin, Germany {koeppen, hjlenz}@wiwiss.fu-berlin.de 2 Arbeitsgruppe Wirtschaftsinformatik - Managementinformationssysteme Otto-von-Guericke-Universität Magdeburg, Germany {graubitz, arndt}@iti.cs.uni-magdeburg.de Abstract. A Balanced Scorecard is more than a business model because it moves perfor- mance measurement to performance management. It consists of performance indicators which are inter-related. Some relations are hard to find, like soft skills. We propose a procedure to fully specify these relations. Three types of relationships are considered. For the function types inverse functions exist. Each equation can be solved uniquely for variables at the right hand side. By generating noisy data in a Monte Carlo simulation, we can specify function type and estimate the related parameters. An example illustrates our procedure and the corresponding results. 1 Related work Indicator systems are appropriate instruments to define business targets and to mea- sure management indicators together. Such a system should not be just a system of hard indicators; it should be used as a system with control in which one can bring hard indicators and management visions together. In the beginning of the 90’s Johnson and Kaplan (1987) published the idea how to bring a company’s strategy and used indicators together. This system, also known as Balanced Scorecards (BSC), is developed until now. The relationships between those indicators are hard to find. According to Marr (2004), companies understand better their business if they visualise relations between available indicators. However, some indicators influence each other in cause and effect relations which increases the validity of these indicators. Unusually, compared to a study of Ittner et al (2003) and Marr (2004) 46% of questioned companies do not or are not able to visualise cause-and-effect relations of indicators. Several approaches try to solve the existing shortcomings. A possible way to model fuzzy relations in a BSC is described in Nissen (2006). Nevertheless, this leads to restrictions in the variable domains. 364 Veit Köppen et al. Blumenberg et al (2006) concentrate on Bayesian Belief Networks (BBN) and try to predict value chain figures and enhanced corporate learning. The weakness of this prediction method is that it does not contain any loops which BSCs may contain. Loops within BSCs must be removed if BBN are used to predict causes and effects in BSCs. Banker et al (2004) suggest calculating trade-offs between indicators. The weak- ness of this solution is that they concentrate on one financial and three nonfinancial performance indicators and try to derive management decisions. A totally different way of predicting relations in BSCs is the usage of system dynamics. System Dynamics is usually used to simulate complex dynamic systems (Forrester (1961)). Various publications exist of how to combine these indicators with dynamics systems to predict economic scenarios in a company, e.g. Akkermans et al (2002). In contrast to these approaches we concentrate on existing performance indicators and try to predict relationships between these indicators instead of pre- dicting economic scenarios. It is similar to the methods of system identification. In contrast, our approach calculates in a more flexible way all models within the de- scribed model classes (see section 3). 2 Balanced scorecards ”If you can’t measure it, you can’t manage it” (Kaplan and Norton (1996), p. 21). With this sentence the BSC inventors Kaplan and Norton made a statement which describes a common problem in the industry: you can not manage a company if you don’t have performance indicators to manage and control your company.Kaplan and Norton presented the BSC – a management tool for bringing the current state of the business and the strategy of the company together. It is a result of previous indicator systems. Nevertheless, a BSC is more than a business system (Friedag & Schmidt 2004). Kaplan & Norton (2004) emphasise this in their further development of Strategy Maps. However, what are these performance indicators and how can you measure it. PreiSSner (2002) divides the functionality of indicators into four topics: operational- isation (”indicators should be able to reach your goal”), animation (”a frequent mea- surement gives you the possibility to recognise important changes”), demand (”it can be used as control input”) and control (”it can be used to control the actual value”). Nonetheless, we understand an indicator as defined in (Lachnit 1979). But before a decision is made which indicator is added to the BSC and the corre- sponding perspective the importance of the indicator has to be evaluated. Kaplan & Norton divide indicators additionally into hard and soft, short and long-term objec- tives. They also consider cause and effect relations. The three main aspects are: 1. All indicators that do not make sense are not worthwhile being included into a BSC; 2. While building a BSC, a company should differentiate between performance and re- sult indicators; 3. All non-monetary values should influence monetary values. Based on these indicators we are now able to build up a complete system of indicators which A Procedure to Estimate Relations in a Balanced Scorecard 365 turns into or influences each other and seeks a measurement for one of the follow- ing four perspectives: (1) Financial Perspective to reflect the financial performance like the return on investment; (2) Customer Perspective to summarize all indicators of the customer/company relationships; (3) Business Process Perspective to give an overview about key business processes; (4) Learning and Growth Perspective which measures the company’s learning curve. Financial Profitability Customer Lower Costs Increase Revenue More customers Lowest Prices Internal Improve Turnaround Time OnŦtime flights Align Ground Crews Learning Fig. 1. BSC Example of a domestic airline By splitting a company into four different views the management of a company gets the chance of a quick overview. The management can focus on its strategic goal and is able to react in time. They are able to connect qualitative performance indi- cators with one or all business indicators. Moreover the construction of an adequate equation system might be impossible. Nevertheless the relations between indicators should be elaborated and an approx- imation of the relations of these indicators should be considered. In this case mul- tivariate density estimation is an appropriate tool for modeling the relations of the business. Figure 1 shows a simple BSC of an airline company. Profitability is the main figure of interest but additionally seven more variables are useful for manag- ing the company. Each arc visualizes the cause and effect relations. This example is taken from "The Balanced Scorecard Institute" 1 . 1 www.balancedscorecard.org 366 Veit Köppen et al. 3 Model To quantify the relationships in a given data set different methods for parameter esti- mation are used. Measurement errors within the data set are allowed, but these errors are assumed to have a mean value of zero. For each indicator within the data set no missing data is assumed. To quantify the relationships correctly it is further assumed that intermediate results are included in the data set. Otherwise the relationships will not be covered. Heteroscedasticity as well as autocorrelations of the data is not con- sidered. 3.1 Relationships, estimations and algorithm In our procedure three different types of relationships are investigated. The first two function types are unknown because the operators linking the variables are unknown: z = f (x,y)=x⊗y (1) where ⊗ represent an addition or a multiplication operator. The third type includes a parametric type of real valued function: y = f T (x)= ⎧ ⎪ ⎨ ⎪ ⎩ px≤ a c 1+e −d·(x−g) + ha< x ≤ b qx> b (2) with T =(abcdgh) and p = c 1+e −d·(a−g) +h and q = c 1+e −d·(b−g) +h. Note, that all three function types are assumed to be separable, i.e. uniquely solvable for x or y in 1 and x in 2. Thus forward and backward calculations in the system of indicators are possible. As a data set is tested independently with respect to the described function types a ˆ Sidàk correction has to be applied (cf. Abdi (2007)). Additive relationships between three indicators (Y = X 1 + X 2 ) are detected via multiple regression. The model is: Y = E 0 + E 1 ·X 1 + E 2 ·X 2 + u (3) where u ∼ N(0, V 2 ). The relationship is accepted if level of significance of all ex- planatory variables is high and E 0 = 0, E 1 = 1 and E 2 = 1. The multiplicative rela- tionship Y = X 1 ·X 2 is detected by the regression model: Y = E 0 + E 1 ·Z + u with Z = X 1 ·X 2 ,u ∼N(0,V 2 ). (4) The relationship is accepted if the level of significance of the explanatory variable is high and E 0 = 0andE 1 = 1. The nonlinear relationship between two indicators according to equation 2 is detected by parameter estimation based on nonlinear re- gression: Y = c 1+ e −d·(X−g) + h + u ∀a < x ≤ b;u ∼N(0,V 2 ). (5) A Procedure to Estimate Relations in a Balanced Scorecard 367 In a first step the indicators are extracted from a business database, files or tools like excel spreadsheets. The number of extracted indicators is denoted by n. In the second step all possible relationships have to be evaluated. For the multiple regression scenario n! 3!·(n−3)! cases are relevant. Testing multiplicative relationships demands n! 2·(n−3)! test cases. The nonlinear regression needs to be performed n! (n−2)! times. All regressions are performed in R. The univariate and the multivariate linear regression are performed with the lm function from the R-base stats package. The nonlinear regression is fitted by the nls function in the stats package and the level of significance is evaluated. If additionally the estimated parameter values are in given boundaries the relationship is accepted. The pseudo code of the the complete environment is given in algorithm 3.1. Algorithm 1 Estimation Procedure Require: data matrix data[M t×n ]witht observations for n indicators significance level, boundaries for parameter Ensure: detected relationships between indicators 1: for i =1ton −2 AND j = i +1 ton − 1 AND k = j +1 ton do 2: estimation by lm(data[,i] data[,j] + data[,k]) 3: if significant AND parameter estimates within boundaries then 4: Relationship ”Addition” found 5: end if 6: end for 7: for i =1ton AND j =1ton − 1 AND k = j +1ton do 8: if i!=jANDi!=k then 9: set Z := data[,j] · data[,k] 10: estimation by lm(data[,i] Z) 11: if significant AND parameter estimates within boundaries then 12: Relationship ”Multiplication” found 13: end if 14: end if 15: end for 16: for i =1ton AND j =1ton do 17: if i!=jthen 18: estimation by nls(data[,j] c/(1+exp(-d+g*data[,i])) + h) 19: if significant then 20: ”Nonlinear Relationship” found 21: end if 22: end if 23: end for 4 Case study For our case study we create an artificial model with 16 indicators and 12 relation- ships, see Fig. 2. It includes typical cases of the real world. 368 Veit Köppen et al. IndicatorPlus 1 IndicatorPlus 2 Indicator 1 Indicator 3 Indicator 4Indicator 2 IndicatorExp 2 exp IndicatorPlus 3 x IndicatorMultiply 3 IndicatorPlus 4 + x IndicatorMultiply 4 exp IndicatorExp1 IndicatorExp 4 exp x IndicatorMultiply 1 exp IndicatorExp 3 x IndicatorMultiply 2 + + + Fig. 2. Artificial Example Indicators 1-4 are independently and randomly distributed. In Fig. 2 they are dis- played in grey and represent the basic input for the simulated BSC system. All other indicators are either functional dependent on two indicators related by an addition or multiplication or functional dependent on an indicator according to equation 2. Some of these indicators effect other quantities or represent leaf nodes in the BSC model graph, cf. Fig. 2. Based on the fact that indicators may not be precisely measured we add noise to some indicators, see Tab. 1. Note, that IndicatorPlus4 has a skewed added noise whereas the remaining added noise is symmetrical. In our case study we hide all given relationships and try to identify them, cf. section 3. Table 1. Indicator Distributions and Noise Indicator Distribution Indicator added Noise Indicator Noise Indicator1 N(100, 10 2 ) IndicatorPlus1 N(0,1) IndicatorExp1 N(0, 1) Indicator2 N(40, 2 2 ) IndicatorPlus4 E(1) −1 IndicatorExp4 U(−1,1) Indicator3 U(−10,10) IndicatorMultiply1 N(0, 1) Indicator4 E(2) IndicatorMultiply4 U(−1,1) 5 Results The case study runs in three different stages: with 1k, 10k, and 100k randomly dis- tributed data. The results are similar and can be classified into four cases: (1) if a A Procedure to Estimate Relations in a Balanced Scorecard 369 relation exists and it was found (displayed black in Fig. 3), (2) if a relation was found but does not exist (displayed with a pattern in Fig. 3) (error of the second kind), (3) if no relation was found but one exists in the model (displayed white in Fig. 3) (error of the first kind), and (4) if no relation exists and no one was found. Additionally the results have been split according to the operator class (see Tab. 2). Table 2. Identification Results Observations 1k 10k 100k +*Exp+*Exp+*Exp (2) 032705480249 (3) 103103103 560 1680 240 560 1680 240 560 1680 240 Hence, Tab. 2 shows that the results for all experiments are similar for the oper- ators addition and multiplication. For non-linear regression, relationships could not be discovered properly. The additive relation of IndicatorPlus4 was the only non-detective relation, see observation (3) in Tab. 2. This is caused by the fact that the indicator has an added noise which is skewed. In such a case the identification is not possible. IndicatorPlus 1 IndicatorPlus 2 Indicator 1 Indicator 3 Indicator 4Indicator 2 IndicatorExp 2 IndicatorPlus 3 IndicatorMultiply 3 IndicatorPlus 4 + IndicatorMultiply 4 exp IndicatorExp1 IndicatorExp 4 exp IndicatorMultiply 1 exp IndicatorExp 3 IndicatorMultiply 2 x x x + x x + + x exp Fig. 3. Results of the Artificial Example for 100k observations 370 Veit Köppen et al. 6 Conclusion and outlook Traditional regression analysis allows estimating the cause and effect dependencies within a profit seeking organization. Univariate and multivariate linear regression exhibit the best results whereas skewed noise in the variables destroys the possibility to detect these relationships. Non-linear regression has a high error output due to the fact that optimization has to be applied and starting values are not always at hand. The results from the non-linear regression should only be carefully taken into account. In future work we try to improve our results while removing indicators for which we calculate a nearly 100% secure relationship. Additionally we plan to work on real data which also includes the possibility of missing data for indicators. Research aims at creating a company’s BSC with relevant business figures while looking only at a company’s indicator system. References ABDI, H. (2007): Bonferroni and Sidak corrections for multiple comparisons. In: N.J. Salkind (Ed.): Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage: 103– 107. AKKERMANS, H. and VAN OORSCHOT, KIM (2002): Developing a balanced scorecard with system dynamics in Proceeding of 2002 International System Dynamics Conference. BANKER, R. D. and Chang, H. and JANAKIRAMAN, S. N. and KONSTANS, C. (2004): A balanced scorecard analysis of performance metrics. in European Journal of Operational Research 154(2): 423–436. BLUMENBERG, STEFAN A. and HINZ, DANIEL J. (2006): Enhancing the Prognostic Power of IT Balanced Scorecards with Bayesian Belief Networks. In HICSS ’06: Pro- ceedings of the 39th Annual Hawaii International Conference on System Sciences IEEE Computer Society, Washington, DC, USA FORRESTER, J. W. (1961). Industrial Dynamics Waltham, MA: Pegasus Communications. FRIEDAG, H.R. and SCHMIDT, W. (2004): Balanced Scorecard. 2nd edition. Haufe, Planegg. ITTNER, C.D. and LARCKER, D.F. and RANDALL, T. (2003): Performance implications of strategic performance measurement in financial service firms". Accounting Organization and Society, 2nd edition. Haufe, Planegg. JOHNSON, T.H. and KAPLAN, R.S. (1987): Relevance lost: the rise and fall of management accounting . Harvard Business Press, Boston. KAPLAN, R.S. and NORTON, D.P. (1996): The Balanced Scorecard. Translating Strategy Into Action. Harvard Business School Press, Harvard. KÖPPEN, V. and LENZ, H J. (2006): A comparison between probabilistic and possibilistic models for data validation. In: Rizzi, A. & Vichi, M. (Eds.) Compstat 2006 ˝ U Proceedings in Computational Statistics , Springer, Rome. LACHNIT, L. (1979): Systemorientierte Jahresabschlussanalyse. Betriebswirtschaftlicher Verlag Dr. Th. Gabler KG, Wiesbaden. MARR, B. (2004): Business Performance Measurement: Current State of the Art. Cranfield University, School of Management, Centre for Business Performance. [...]... 39 41 35 25 20 37 Complexity J4.8 J4.8 0 17 J4.8(cv) 18 RPart0 18 RPart1 15 QUEST 18 CTree 86 J4.8(cv) RPart0 RPart1 QUEST CTree 1 0 0 2 0 0 0 0 5 3 18 0 0 13 15 18 16 0 14 15 13 5 4 0 10 14 3 2 8 0 64 24 6 42 43 3 25 64 81 47 45 Table 3 Median linear order consensus rankings for algorithm performance 1 2 3 4 5 6 Misclassification Complexity J4.8(cv) RPart1 J4.8 RPart0 RPart0 QUEST CTree CTree RPart1... CTree J4.8 RPart0 J4.8(cv) RPart1 J4.8(cv) QUEST J4.8(cv) CTree J4.8(cv) RPart1 RPart0 QUEST RPart0 CTree RPart0 QUEST RPart1 CTree RPart1 CTree QUEST 0.5 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) () ( ) () ( ) () ( ) 80 60 40 20 0 20 Misclassification difference (in percent) Breast Cancer: Misclassification J4.8(cv) J4.8 RPart0 J4.8 RPart1 J4.8 QUEST J4.8 CTree J4.8 RPart0 J4.8(cv) RPart1 J4.8(cv) QUEST J4.8(cv)... 3 .2 Pima Indians Diabetes: Misclassification J4.8(cv) J4.8 RPart0 J4.8 RPart1 J4.8 QUEST J4.8 CTree J4.8 RPart0 J4.8(cv) RPart1 J4.8(cv) QUEST J4.8(cv) CTree J4.8(cv) RPart1 RPart0 QUEST RPart0 CTree RPart0 QUEST RPart1 CTree RPart1 CTree QUEST ( ) ( ) ( ) ( ) ) ( ( ) ( ) ( ) ) ( ( ) ( ) ) ( ( ) ) ( ( 2. 5 1.5 ) 0.5 0.0 Pima Indians Diabetes: Complexity J4.8(cv) J4.8 RPart0 J4.8 RPart1 J4.8 QUEST J4.8... J4.8(cv) RPart1 RPart0 QUEST RPart0 CTree RPart0 QUEST RPart1 CTree RPart1 CTree QUEST Complexity difference Breast Cancer: Complexity ( ) ( ) ( ( ) ( ) ( ) ( ( ( ( ) ) ) ) ( ( ) ) ( ( ) ) ) ( 1.0 0.5 ) 0.0 0.5 1.0 Misclassification difference (in percent) J4.8(cv) J4.8 RPart0 J4.8 RPart1 J4.8 QUEST J4.8 CTree J4.8 RPart0 J4.8(cv) RPart1 J4.8(cv) QUEST J4.8(cv) CTree J4.8(cv) RPart1 RPart0 QUEST RPart0... similar 3 94 Michael Schauerhuber et al Table 2 Summary of predictive performance dominance relations across all 18 datasets based on misclassification rates and model complexity (columns refer to losers, rows are winners) Misclassification J4.8 J4.8 0 4 J4.8(cv) 5 RPart0 6 RPart1 4 QUEST 7 CTree 26 J4.8(cv) RPart0 RPart1 QUEST CTree 2 9 9 11 8 0 8 9 11 9 6 0 7 10 7 4 1 0 8 6 2 2 5 0 7 6 7 8 9 0 20 27 38 49 ... other use is made of such data than statistical analysis 12 5 x 10 P2P 4. 5 upstream traffic 4 downstream traffic 3 5 6 7 8 Applications Chat Mail 4 Streaming News 3 DB FTP 0 Control 2 0.5 Games 1 1 Others 2 1.5 Web 2. 5 Unknown Volume (in byte) 3.5 9 10 11 12 Fig 1 Volume of the traffic on the applications 6 14 x 10 12 Volume (in bytes) 10 8 6 4 2 0 0 5 10 15 20 25 hours Fig 2 Average hourly volume Figure... between C4.5 and CART, and heuristic choice of hyper-parameters – we can conclude: (1) The fully cross-validated J4.8(cv) and RPart0 perform better than their heuristic counterparts J4.8 (with fixed hyper-parameters) and RPart1 (employing a 1-standard-error rule) (2) In terms of predictive performance, no support for the claims of (clear) superiority of either algorithm can be found: J4.8(cv) and RPart0... 57 ringnorm ∗ 1000 20 sonar 20 8 60 spirals ∗ 1000 2 threenorm ∗ 1000 20 tictactoe 958 9 titanic 22 01 3 twonorm ∗ 1000 20 influence of this parameter, we compare the default J4.8 algorithm with a tuned version where C and the minimal leaf size M (default: M = 2) are chosen by cross-validation (J4.8(cv)) A full grid search for C = 0.01, 0.05, 0.1, , 0.5 and M = 2, 3, , 10, 15, 20 is used in the cross-validation... the remaining 12 data settings yield equivalent performances Therefore superiority of J4.8(cv) above J4.8 is questionable In contrast the superiority of RPart0 vs RPart1 seems to be more reliable but still the number of data settings producing tied results is high A comparison of the figures of CTree and the RPart versions confirms previous findings (Hothorn et al., 20 06) that CTree and RPart often perform... environment RPart, an open-source implementation of CART, has been available for some time in the S/R package rpart (Therneau and Atkinson, 1997) while the open-source implementation J4.8 for C4.5 became available more recently in the Weka machine learning package (Witten and Frank, 20 05) and is now accessible from within R by means of the RWeka package (Hornik, Zeileis, Hothorn and Buchta, 20 07) With . 029 911839 J4.8(cv) 40 8911 941 RPart0 560710735 RPart1 641 08 625 QUEST 42 2 50 720 CTree 76789037  26 20 27 38 49 37 Complexity J4.8 J4.8(cv) RPart0 RPart1 QUEST CTree  J4.8 010 020 3 J4.8(cv) 17 0 0 0 5 3 25 RPart0 18. Ŧ RPart1 QUEST Ŧ RPart1 CTree Ŧ RPart0 QUEST Ŧ RPart0 RPart1 Ŧ RPart0 CTree Ŧ J4.8(cv) QUEST Ŧ J4.8(cv) RPart1 Ŧ J4.8(cv) RPart0 Ŧ J4.8(cv) CTree Ŧ J4.8 QUEST Ŧ J4.8 RPart1 Ŧ J4.8 RPart0 Ŧ J4.8 J4.8(cv). Ŧ RPart1 QUEST Ŧ RPart1 CTree Ŧ RPart0 QUEST Ŧ RPart0 RPart1 Ŧ RPart0 CTree Ŧ J4.8(cv) QUEST Ŧ J4.8(cv) RPart1 Ŧ J4.8(cv) RPart0 Ŧ J4.8(cv) CTree Ŧ J4.8 QUEST Ŧ J4.8 RPart1 Ŧ J4.8 RPart0 Ŧ J4.8 J4.8(cv)

Ngày đăng: 05/08/2014, 21:21

Từ khóa liên quan

Tài liệu cùng người dùng

Tài liệu liên quan