20 zan creditrating

Credit Rating Analysis with Support Vector Machines and Neural Networks: A Market Comparative Zan Study Huang, Hsinchun Chen, Chia-jung Hsu, Andy Chen, Soushan Wu AI Seminar Artificial Intelligence Lab The University of Arizona 08/16/2002 Agenda Agenda • • • • • • • • Introduction Credit Risk Analysis Literature Review Research Questions Analytical Methods Data Sets Experiments Results and Analysis Discussion and Future Directions Introduction Credit Rating Rating Credit • Credit Rating is valuable information – Widely used measure for the riskiness of the companies and bonds • Credit Rating is expensive information – Costly to obtain • Credit Rating prediction is important – For investors: estimate riskiness of unrated companies – For companies: monitor the companies’ credit rating, predict the future rating Credit Rating Rating Prediction Prediction Credit • Rating agencies: subjective judgment is important, not predictable • Researchers: satisfactory results have been obtained using statistical and AI methods • Prediction Assumption – Risk evaluation expertise embedded in historical rating data • Beyond Prediction – Interpretation of models  Market characteristics Our Study Study Our • Apply a relatively new machine learning technique, Support Vector Machines, with a classic technique, Neural Networks • Interpretation of the model – Variable contribution analysis • Cross market analysis – United States and Taiwan market Credit Risk Analysis Credit Rating Rating Credit • Two types of ratings – Debt issue rating – bond rating, issue credit rating – Debt issuer rating – conterparty credit rating, default rating, issuer credit rating • Significant implication for investment community – Interest yield of the debt issue – Investment regulation (“investment” level ratings) – Conveys information about the value of the firm Credit Rating Rating Process Process Credit • Typical process – Issuing company contacts rating agency requesting rating – Issuing company submits evaluation package – Rating agency form evaluation team – Evaluation team submits rating report – Rating committee makes final decision • Time and labor intensive • Emphasizes on subjective judgment of financial analyst and rating committee members Literature Review: Bond Rating Prediction Variable Selection Selection Variable • ANOVA test – Whether the differences of each financial variable among different rating classes were significant – uninformative variables removed from the data set • Final data sets – Taiwan: 14 financial ratios and balance measures – United States: 12 financial ratios and balance measures Financial Variables Variables Financial X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 X17 X18 X19 X20 X21 Financial Ratio Name/ Description Total assets Total liabilities Long-term debts/ total invested capital Debt ratio Current ratio Times interest earned (EBIT/interest) Operating profit margin (Shareholders’ equity + long-term debt)/ fixed assets Quick ratio Return on total assets Return on equity Operating income/ received capitals Net income before tax/ received capitals Net profit margin Earnings per share Gross profit margin Non-operating income/ sales Net income before tax/ sales Cash flow from operating activities/ current liabilities (Cash flow from operating activities / (capital expenditures + increased in inventory + cash dividends)) in last years (Cash flow from operating activities – cash dividends)/ (fixed assets + other assets + working capitals) ANOVA BetweenGroup P-Value 0 0.12 0.36 0 0.37 0.01 0.04 0 0 0.02 0.81 0.84 0.64 0.08 Experiment Results and Analysis Experiment Results Results Experiment • Models (Frequently used variables, full set of variables) – TW I: Rating = f(X1,X2,X3,X4,X6,X7) – TW II: Rating = f(X1, X2, X3, X4, X6, X7, X8, X10, X11, X12, X13, X14, X15, X16, X18, X21) – US I: Rating = f(X1,X2,X3,X6,X7) – US II: Rating = f(X1, X2, X3, X6, X7, X8, X10, X11, X12, X13, X14, X15, X16, X21) Experiment Results Results (cont.) (cont.) Experiment • Results – SVM did not outperform neural networks – The small set of frequently used financial variables contained most relevant information TW I TW II US I US II SVM Results 79.73% 77.03% 78.87% 80.00% NN Results 75.68% 75.68% 80.00% 79.25% Difference 4.05% 1.35% -1.13% 0.75% Experiment Results 81.00% 80.00% 79.00% 78.00% SVM Results 77.00% 76.00% 75.00% NN Results 74.00% 73.00% TW I TW II US I US II Within-1-class accuracy accuracy Within-1-class Predicted Rating Predicted Rating Acutal Rating twBB Acutal Rating twAAA twAA twA twBBB twAAA twAA twA twBBB twBB twAAA 0 twAAA twAA 10 0 twAA 0 twA 23 twA 22 twBBB 16 twBBB 0 17 twBB 0 twBB 0 TW I: within-1-class accuracy: 91.89% TW II: within-1-class accuracy: 93.24% Predicted Rating Acutal Rating Predicted Rating AA A BBB BB B Acutal Rating AA 20 0 A 178 BBB 23 33 BB B 0 AA A BBB BB B AA 13 0 A 165 12 0 BBB 16 37 0 BB 0 0 B 0 US I: within-1-class accuracy: 97.74% US II: within-1-class accuracy: 98.44% Variable Contribution Contribution Analysis Analysis Variable • Research of credit rating prediction using Artificial Intelligence methods has been solely focused on prediction accuracy • Low level understanding of the market – Credit rating analyst rate companies (consciously or unconsciously) based on a specific set of financial variables • Higher level understanding – What are the relative importance of individual financial variables in the process of credit rating? - Variable Contribution Analysis Variable Contribution Contribution Analysis Analysis (cont.) (cont.) Variable • Difficult for both Neural Networks and Support Vector Machines • Substantial literature in interpreting neural network models – Mainly extracts information from the connection strengths (inter-layer weights) of neural network model – Measures of relative importance – Garson 1991, Yoon 1994 – Symbolic rules derived from connection weights – Taha 1999 – Optimal neural network structure construction and better understanding of the models - Engelbrecht 1998 Measure of of Relative Relative Importance Importance Measure • First order derivatives of the network parameters – Neural network model  =f() – Contribution measure: • Garson 1991 ∂yi / ∂xj ∑ – Without direction • Yoon 1994 Conik = – With direction • Conik relative contribution of input i on out k Connection strengths between input, hidden and output layers are denoted as w ji and v jk | w ji || v jk | J j =1 ∑ I i =1 ∑ ∑ | w ji | | w ji || v jk | I J i =1 j =1 ∑ I i =1 w v ∑ = ∑ ∑ w | w ji | J Conik ji j =1 I J i =1 j =1 jk ji v jk Variable Contribution Contribution Analysis Analysis Variable • Garson’s measure • Optimal set of variables for the two markets – TW III: Rating = f(X1, X2, X3, X4, X6, X7, X8) – US III: Rating = f(X1, X2, X3, X4, X7, X11) X1 X2 X3 X4 X6 X7 X8 X11 Financial Variable Name/ Description Total assets Total liabilities Long-term debts/ total invested capital Debt ratio Times interest earned (EBIT/interest) Operating profit margin (Shareholders’ equity + long-term debt)/ fixed assets Return on equity Contribution Analysis Analysis Results Results Contribution Variable Contribution (United States) Variable Contribution (Taiw an) 0.3 0.3 AA 0.25 A 0.2 BBB 0.15 BB 0.1 B 0.05 Co nt ribut io n Measure Co nt ribut io n Measure 0.35 0.25 tw AAA 0.2 tw AA 0.15 tw A tw BBB 0.1 tw BB 0.05 X1 X2 X3 X4 X7 X11 X1 Financial Variable X2 X3 X4 X6 X7 X8 Financial Varilables X1 X2 X3 X4 X6 X7 X8 X11 Financial Variable Name/ Description Total assets Total liabilities Long-term debts/ total invested capital Debt ratio Times interest earned (EBIT/interest) Operating profit margin (Shareholders’ equity + long-term debt)/ fixed assets Return on equity Cross Market Market Analysis Analysis Cross • US Model – X1, X2, X3, X7 | X4, X11 – Most important: total assets, total liabilities, long-term debts/total invested capital • TW Model – X4, X7, X8 | X1, X2, X3, X6 – Most important: operating profit margin, debt ratio Discussion and Future Directions Discussion Discussion • We need expertise from credit rating industry to evaluate and interpret the results – Some positive response: “Size is not (that) important in Taiwan.” – Dr Soushan Wu • The reason for the prediction accuracy improvement over previous studies • The reason for SVM’s failure to improve Future Directions Directions Future • Data mining + text mining – Add important financial variables from the text format annual report • Larger scale cross market analysis – Mainland China, Taiwan, Hong Kong and United States markets • Multidimensional financial data visualization and exploration