Java 2 Network Security pdf

SG24-2109-01 International Technical Support Organization http://www.redbooks.ibm.com Java 2 Network Security Marco Pistoia, Duane F. Reller Deepak Gupta, Milind Nagnur, Ashok K. Ramani Java 2 Network Security Marco Pistoia, Duane F. Reller Deepak Gupta, Milind Nagnur, Ashok K. Ramani Foreward by Li Gong Distinguished Engineer and Chief Java Security Architect Sun Microsystems, Inc. June 1999 SG24-2109-01 International Technical Support Organization © Copyright International Business Machines Corporation 1997 1999. All rights reserved. Note to U.S Government Users – Documentation related to restricted rights – Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp. Second Edition (June 1999) This edition applies to Java 2 SDK, Standard Edition, V 1.2. Comments may be addressed to: IBM Corporation, International Technical Support Organization Dept. HZ8 Building 678 P.O. Box 12195 Research Triangle Park, NC 27709-2195 When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you. Before using this information and the product it supports, be sure to read the general information in Appendix F, “Special Notices” on page 659. Take Note! © Copyright IBM Corp. 1999 3 Foreword As the person who led the JavaSoft team that developed the Java security technology discussed in this book, it is extremely gratifying to see people spend their precious time writing about our technology and products. Every engineer’s dream is to have his or her technology deployed and used by thousands of others, and this book is a great help to Java developers who write security-aware applications. Security is a difficult subject to write about. On the one hand, security is in people’s daily consciousness so that it appears easy to get across (to the reader) some of the basic concepts. On the other hand, security applied to computer and networking is often subtle and unexpected. Security also is pervasive in that it touches all aspects of the computing technology, including hardware, software, operating system, software libraries, communication software, networking infrastructure, application software, user interface, and management software. In order to understand security in any situation, one has to understand the entire system under consideration as well as each individual component so that one can identity their strengths and weaknesses and design the appropriate solutions. Java security is one of the more recent additions to the family of security technologies. Ever since Sun Microsystems announced Java technology in the spring of 1995, there has been strong and growing interest (in industry, research laboratories, and academia) around the security of the Java platform as well as new security issues raised by the deployment of Java technology. Such close attention being paid to security is almost unprecedented in that new computing technologies normally ignore security considerations when they emerge initially. Most of them remain unsecured forever. In the few cases where efforts are made to secure them later, the efforts are typically not very successful because retrofitting security is usually very difficult, if possible at all, and often causes backward compatibility problems. Therefore, it is extremely fortunate that the Java technology had security as a primary design goal from the very beginning. (Hats off to the original Java development team. I joined JavaSoft only in 1996.) Although the initial security model was very simplistic, it enabled later improvements in the security architecture. The Java language is a general-purpose object-oriented programming language and is specifically designed to be platform independent so that application developers can write a program once and then run it securely 4 Java 2 Network Security everywhere on the Internet. To achieve this platform independence, a Java program is compiled to a bytecode instruction set and binary format defined in the Java Virtual Machine Specification. The Java platform consists of the Java language and its associated tools (such as compilers), together with the Java Virtual Machine (JVM) and its associated libraries that define a rich set of application programming interfaces (APIs). Security for the Java platform has multiple layers. First of all, the Java language is strongly typed and does not include any unsafe constructs, such as array accesses without index checking, because such unsafe constructs may result in unspecified and unpredictable program behavior that can lead to security compromises. Type safety is checked both at the time a piece of bytecode is loaded into the JVM and throughout the lifetime of the bytecode (that is, during run time) until it is no longer used and garbage collected. Second, mechanisms (for example, class loaders) are in place to ensure a sufficient degree of separation between multiple Java programs so that they do not interfere with each other in undesirable ways. Third, access to crucial system resources is mediated by the JVM. A security manager is installed to deny all requests for unauthorized access. The access control model, in the initial release of the Java Development Kit (JDK 1.0), was to grant full access to local code (that is, trust such code and let it do anything it wants) and to grant very restricted access to code loaded over the network because such code (often referred to as applets) may not be trusted. JDK 1.1 introduced a notion of trusted applets and granted full access to these applets. The latest release, JDK 1.2 (also called Java 2), incorporates a new security architecture that supports policy-driven, fine-grained, flexible, and extensible access control. (For design rationales of this architecture, as well as difficulties and subtleties we encountered during JDK 1.2 development, please refer to my book Inside Java 2 Platform Security.) On top of type safety and access control, there are the Java Cryptography Architecture (implemented in JDK 1.2 and in the Java Cryptography Extension 1.2), support for secure communication (the Java Secure Socket Extension), and a framework for user-based authentication and access control (the Java Authentication and Authorization Service). These technologies are at various stages in the development and release cycle. Finally, applications can provide their own specific security features and can customize security features that are built into the Java platform. Our colleagues at IBM, among other industrial partners, have been closely involved with the recent development of Java security technology. They have supported our efforts in many ways, and have provided excellent technical suggestions. This latest book from IBM is a comprehensive guidebook that 5 provides the programmer/reader with well-organized details of the Java security APIs and their usage. The book is also broad in its coverage of the wider security context and related issues. I am very excited to see such a good book being published on Java security. It will contribute greatly toward making the Java platform the most popular deployment environment for secure computing. Li Gong Distinguished Engineer and Chief Java Security Architect Sun Microsystems Cupertino, California May 1999 6 Java 2 Network Security © Copyright IBM Corp. 1997 1999 vii Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii The Team That Wrote This Redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Comments Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Part 1. Introduction to Java and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. An Overview of Java and Security . . . . . . . . . . . . . . . . . . . . 3 1.1 Java Is Not Just a Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 What Java Does. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Java Is Not an Island: Java as a Part of Security . . . . . . . . . . . . . . . . . 5 1.3.1 Safety and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Java as an Aid to Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.3 Java as a Threat to Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.3.4 Writing Secure Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3.5 Staying One Jump Ahead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3.6 The Vigilant Web Site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4 Understanding Java 2 Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4.1 An Example of Applet Security in Java 2 . . . . . . . . . . . . . . . . . . 14 1.4.2 An Example of Application Security in Java 2 . . . . . . . . . . . . . . . 26 1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Chapter 2. Attack and Defense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.1 Components of Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.1.1 The Development Environment. . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.1.2 The Execution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.1.3 Interfaces and Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.2 Java 2 and Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.2.1 Cryptographic Tools in Brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 2.2.2 Java Cryptography Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.2.3 United States Export Rules for Encryption . . . . . . . . . . . . . . . . . 57 2.2.4 Signed Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.2.5 The Other Side of the Coin – Access Control . . . . . . . . . . . . . . . 59 2.3 Attacking the World of Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 2.3.1 Perils in the Life of Remote Code . . . . . . . . . . . . . . . . . . . . . . . . 59 2.3.2 Vulnerabilities in Java Applications. . . . . . . . . . . . . . . . . . . . . . . 66 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Chapter 3. The New Java Security Model . . . . . . . . . . . . . . . . . . . . . . . 69 3.1 The Need for Java Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 viii Java 2 Network Security 3.2 Evolution of the Java Security Model . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.2.1 The JDK 1.0 Sandbox Security Model . . . . . . . . . . . . . . . . . . . . 70 3.2.2 The Concept of Trusted Code in JDK 1.1 . . . . . . . . . . . . . . . . . . 72 3.2.3 The Fine-Grained Access Control of Java 2 . . . . . . . . . . . . . . . . 74 3.2.4 A Comparison of the Three Java Security Models . . . . . . . . . . . 78 3.3 Java 2 Protection Domain and Permissions Model . . . . . . . . . . . . . . . 80 3.4 New Class Search Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.4.1 Boot Class Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.4.2 Extensions Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.4.3 Application Class Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.4.4 Class Search Paths in Summary . . . . . . . . . . . . . . . . . . . . . . . . 89 3.5 Java 2 Class Loading Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.5.1 Run-Time Access Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.6 The Policy File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 3.6.1 The Default System-Wide Policy File . . . . . . . . . . . . . . . . . . . . . 96 3.7 Security Manager vs Access Controller . . . . . . . . . . . . . . . . . . . . . . . 98 3.8 Security Management with Java 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.8.1 Applying a Security Manager to Applets and Applications. . . . . . 99 3.8.2 Applying a User-Defined Security Policy. . . . . . . . . . . . . . . . . . . 99 3.8.3 Java Security Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Part 2. Under the Hood. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Chapter 4. The Java Virtual Machine. . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.1 The Java Virtual Machine, Close Up. . . . . . . . . . . . . . . . . . . . . . . . . 109 4.1.1 The Class Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 4.1.2 The Class File Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.1.3 The Heap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.1.4 The Class Area. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.1.5 The Native Method Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.1.6 The Security Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.1.7 The Execution Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.1.8 Just-in-Time Compilers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Chapter 5. Class Files in Java 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 5.1 The Traditional Development Life Cycle . . . . . . . . . . . . . . . . . . . . . . 117 5.2 The Java Development Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . 119 5.3 The Java 2 Class File Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 5.3.1 Decompilation Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 5.4 The Constant Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 5.4.1 Beating the Decompilation Threat. . . . . . . . . . . . . . . . . . . . . . . 134 [...]... 187 187 190 1 92 1 92 193 195 20 4 20 5 20 6 20 6 21 1 21 8 22 4 Chapter 8 Security Configuration Files in the Java 2 SDK 8.1 A Note on java. home and the JRE Installation Directory 8 .2 Keystores 8 .2. 1 The Certificates KeyStore File cacerts 8.3 The Security Properties File, java. security 8.4 Security Policy Files... 8.4 .2 grant Entries 8.5 An Example of Security Settings in the Java 2 Platform 8.5.1 The Count Application Source Code 22 5 22 5 23 0 23 3 23 4 24 2 24 2 24 3 24 8 24 8 ix 8.5 .2 A Sample Text File 8.5.3 Compiling the Application 8.5.4 Running the Application without a Security. .. 9.3 .2 Observations on the jarsigner Verification Process 9.3.3 Tampering with a Signed JAR File 9.4 Policy File Creation and Management Tool 9.4.1 Observations on the Use of the Policy Tool 25 9 25 9 25 9 26 1 26 2 26 9 27 0 27 1 27 4 27 5 28 0 28 4 28 6 28 8 29 5 Chapter 10 Security APIs in Java 2 ... 317 318 321 321 322 322 322 Java 2 Network Security 10.3 The Package java. security. cert 10.4 Package java. security. interfaces 10.5 The Package java. security. acl 10.6 Examples Using the Java 2 Security APIs 10.6.1 Signature and Signature Verification 10.6 .2 Using Keystores ... Java Cryptography Architecture The set of the Java core classes (which are the Java classes shipped with the Java platform 3) can be divided into two subsets: • Security related core classes 3 • Other core classes In this book, the Java 2 Platform, Standard Edition, V1 .2 (J2SE) is often referred to as Java platform or Java 2 platform 10 Java 2 Network Security The Security related core classes in Java. .. 12. 2 Signed Code Scenario in JDK 1.1 and Sun HotJava 12. 2.1 Creating the CA Key Database 12. 2 .2 Creating the Server Key Database 12. 2.3 Creating and Signing a JAR File 385 385 387 3 92 3 92 393 393 395 397 xi 12. 2.4 Running the Applet 12. 2.5 Creating the Client... Running the Application with the Default Security Manager 8.5.6 Policy File Modification 8.6 File Read Access to Files in the Code Base URL Directory 8.7 Security Properties and Policy File Protection 8.8 How to Implement a Policy Server 24 9 24 9 25 0 25 0 25 0 25 2 25 2 25 2 Chapter 9 Java 2 SDK Security Tools 9.1 Key... Enterprise Java 501 14.1 Browser Add-On Applets 501 14 .2 Networked Architectures 501 14 .2. 1 Applying the Java 2 Access Control Mechanisms 5 02 14 .2. 2 Two-Tier Architecture 503 14 .2. 3 Three-Tier Architecture 503 14 .2. 4 Network Security ... evaluation form found at http://www.redbooks.ibm.com/ • Send your comments in an internet note to redbook@us.ibm.com xix xx Java 2 Network Security Part 1 Introduction to Java and Security © Copyright IBM Corp 1997 1999 1 2 Java 2 Network Security Chapter 1 An Overview of Java and Security The purpose of this chapter is not only to introduce the themes of the book to those who will later read the more... the high-level Java language should be seen as safety features, not as security features An Overview of Java and Security 7 1.3 .2 Java as an Aid to Security Sometimes, discussions of Java and security focus only on the perils of Java, as though there was only a downside to using it, from a security point of view anyway But this is not the whole story Java can be a great help to the security of a system, . . 24 3 8.5 An Example of Security Settings in the Java 2 Platform . . . . . . . . . . 24 8 8.5.1 The Count Application Source Code. . . . . . . . . . . . . . . . . . . . . 24 8 x Java 2 Network Security 8.5 .2. . . . . 69 3.1 The Need for Java Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 viii Java 2 Network Security 3 .2 Evolution of the Java Security Model . . . . . . . . . 22 4 Chapter 8. Security Configuration Files in the Java 2 SDK . . . . . . . . 22 5 8.1 A Note on java. home and the JRE Installation Directory. . . . . . . . . . 22 5 8 .2 Keystores. . . . .