java power toolsa

373 9.1 Instant Messaging in a Development Project 373 9.4 Authenticating Users in an External Database 3749.5 Authenticating Users Against a POP3 Server 377 9.7 Extended Functionality w

Java ™ Power Tools

John Ferguson Smart

Beijing • Cambridge • Farnham • Köln • Paris • Sebastopol • Taipei • Tokyo

Java Power Tools

by John Ferguson Smart

Copyright © 2008 John Ferguson Smart All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472

O’Reilly books may be purchased for educational, business, or sales promotional use Online editions

are also available for most titles (http://safari.oreilly.com) For more information, contact our corporate/ institutional sales department: (800) 998-9938 or corporate@oreilly.com.

Editor: Mike Loukides

Production Editor: Loranah Dimant

Production Services: GEX, Inc.

Cover Designer: Karen Montgomery

Interior Designer: David Futato

Illustrator: Robert Romano

Printing History:

Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of

O’Reilly Media, Inc Java Power Tools, the image of a drill press, and related trade dress are trademarks

of O’Reilly Media, Inc.

Many of the designations uses by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc was aware of a trademark claim, the designations have been printed in caps or initial caps.

While every precaution has been taken in the preparation of this book, the publisher and author assume

no responsibility for errors or omissions, or for damages resulting from the use of the information tained herein.

con-ISBN: 978-0-596-52793-8

[C]

1209067353

This book is dedicated to my wonderful wife Chantal, and my two lovely boys, James and William, who are my constant source of inspiration, wonder, and joy.

Table of Contents

Foreword xvii Preface xix Introduction xxxiii

Part I Build Tools

1 Setting Up a Project Using Ant 5

1.5 Customizing Your Build Script Using Properties 17

1.11 Using Maven Dependencies in Ant with the Maven Tasks 49

2 Setting Up a Project Using Maven 2 61

2.4 Declarative Builds and the Maven Project Object Model 64

v

2.6 The Maven Directory Structure 79

2.9 Looking for Dependencies with MvnRepository 91

2.11 Creating a Project Template with Archetypes 96

2.18 Using Plug-Ins to Customize the Build Process 1132.19 Setting Up an Enterprise Repository with Archiva 1222.20 Setting Up an Enterprise Repository Using Artifactory 135

Part II Version Control Tools

3 Setting Up Version Control Using CVS 165

3.5 Working with Your Files—Updating and Committing 170

4.5 Setting Up a New Subversion Project 195

4.10 Seeing Where You’re At: The Status Command 205

4.16 Making Locked Files Read-Only with the svn:needs-lock Property

219

4.18 Change History in Subversion: Logging and Blaming 2234.19 Setting Up a Subversion Server with svnserve 224

4.21 Setting Up a WebDAV/DeltaV Enabled Subversion Server 229

4.24 Installing Subversion As a Windows Service 2364.25 Backing Up and Restoring a Subversion Repository 238

Part III Continuous Integration

5 Setting Up a Continuous Integration Server with Continuum 271

5.5 Running the Continuum Server in Verbose Mode 276

Table of Contents | vii

5.11 Managing Users 282

5.17 Automatically Generating a Maven Site with Continuum 290

6 Setting Up a Continuous Integration Server with CruiseControl 295

6.5 Setting Up a Maven 2 Project in CruiseControl 309

7.5 Using Project Variables for Version Numbering 326

7.8 Reporting on Test Coverage in Luntbuild Using Cobertura 333

8.11 Authentication and Security 364

9 Setting Up an Instant Messaging Platform with Openfire 373

9.1 Instant Messaging in a Development Project 373

9.4 Authenticating Users in an External Database 3749.5 Authenticating Users Against a POP3 Server 377

9.7 Extended Functionality with Openfire Plug-Ins 379

9.11 Sending Jabber Messages from a Java Application Using the

Part IV Unit Testing

10 Testing Your Code with JUnit 389

10.3 Setting Up and Optimizing Your Unit Test Cases 392

11 Next-Generation Testing with TestNG 413

Table of Contents | ix

11.2 Creating Simple Unit Tests with TestNG 413

12 Maximizing Test Coverage with Cobertura 441

12.3 Checking the Code Coverage of TestNG Tests 445

12.7 Integrating Coverage Tests into the Maven Build Process 453

Part V Integration, Functional, Load, and Performance Testing

13 Testing a Struts Application with StrutsTestCase 463

14.6 Replacing Values 493

15 Performance Testing with JUnitPerf 517

15.4 Load-Testing Tests That Are Not Thread-Safe 52315.5 Separating Performance Tests from Unit Tests in Ant 52315.6 Separating Performance Tests from Unit Tests in Maven 524

16 Load and Performance Testing with JMeter 527

16.5 Using the JMeter Proxy to Record a Test Case 541

17 Testing Web Services with SoapUI 547

18 Profiling and Monitoring Java Applications Using the Sun JDK Tools 569

18.1 The Sun JDK Profiling and Monitoring Tools 56918.2 Connecting To and Monitoring a Java Application with jConsole 56918.3 Monitoring a Remote Tomcat Application with jConsole 57218.4 Detecting and Identifying Memory Leaks with the JDK Tools 57418.5 Diagnosing Memory Leaks Using Heap Dumps, jmap, and jhat 579

Table of Contents | xi

19 Profiling Java Applications in Eclipse 585

19.2 The Eclipse Test & Performance Tools Platform 585

19.6 Studying Memory Use with the Basic Memory Analysis Results 593

20 Testing Your User Interfaces 603

20.1 Testing Your Web Application with Selenium 603

Part VI Quality Metrics Tools

21 Detecting and Enforcing Coding Standards with Checkstyle 647

21.1 Using Checkstyle to Enforce Coding Standards 647

21.4 Customizing Checkstyle Rules Using the XML Configuration

21.5 Customizing Checkstyle: Common Rules That You Can Do

21.6 Defining Rules for Source Code Headers with Checkstyle 660

22 Preemptive Error Detection with PMD 667

22.9 Using PMD in Ant 679

23 Preemptive Error Detection with FindBugs 685

23.3 Selectively Suppressing Rules with FindBug Filters 688

24 Inspecting the Results—Semiautomated Code Review with Jupiter 697

24.1 Introducing Jupiter—A Code Review Tool for Eclipse 697

24.3 Understanding the Jupiter Code Review Process 699

25.5 Focusing on a Task with Context Management 726

26 Monitoring Build Statistics 733

26.2 Source Code Management Metrics with StatSCM 741

Table of Contents | xiii

Part VII Issue Management Tools

27 Bugzilla 747

27.8 Managing Groups of Products with Classifications 758

28 Trac—Lightweight Project Management 769

28.9 Tailoring the Trac Web Site: Using the Wiki Function 783

28.15 Managing Progress with Trac Roadmaps and Timelines 797

Part VIII Technical Documentation Tools

29 Team Communication with the Maven 2 Project Web Site 807

29.1 The Maven 2 Project Web Site As a Communication Tool 807

29.8 Customizing the Look and Feel of Your Site 826

30 Automatically Generating Technical Documentation 831

30.1 Visualizing a Database Structure with SchemaSpy 83130.2 Generating Source Code Documentation with Doxygen 84030.3 Embedding UML Diagrams in Your Javadoc with UmlGraph 849

Bibliography 855 Index 857

Table of Contents | xv

Designing, coding, and deploying working Java applications isn’t easy Doing it in apredictable manner rapidly with an acceptable level of quality is even harder In addi-tion to having to understand what stakeholders want or the varying skills of teammembers or even the myriad web, data access, and utility frameworks one can choosefrom, you’ve got to actually manage the development process itself!

The coding of requirements is challenging enough, but as anyone who’s ever delivered

a working application knows, in the grand scheme of things, that’s one sliver of thedevelopment process—in fact, some may say that’s the easiest part Think about allthe techniques and processes that aggregate up to produce a software application.First, you’ve got to figure out how to deliver the working application in a predictablemanner At a high level, this means three things: tracking changes to source code assets,keeping up with any uncovered issues, defects, or feature requests, and assembling theapplication in a reliable and repeatable manner

Next, you’re going to want to actually ensure the application under development

ac-tually works—ideally during development This means writing tests early Of course,

this process is easier said than done Although arguably there are few standard testingframeworks from which to chose, there is a cornucopia of associated tools that accel-erate writing developer tests by addressing specific challenges

What’s more, as the code base grows, you’ll probably want to understand what’s beingcoded and how well it’s being developed Although tests cancertainly verify code func-tionality, you may also want lighter-weight tools that can report on various metrics,such as complexity, coding standards, or even the coverage of tests

Of course, if you’ve got a mechanism for assembling your application in a repeatableand reliable manner, it makes sense to augment this process by running tests and evenanalysis tools What’s more, given that you want to produce working code quickly, it

makes sense to assemble the application often—in fact, assembling it continuously

facilitates discovering issues as they arise

xvii

Finally, you’re going to want to enable easy maintenance of the code base so that tures can be added often—in the same rapid and repeatable manner that the applicationwas built.

fea-John has assembled what I think is the “A” list of tools and techniques that will helpyou meet each and everyone of the challenges above In fact, John presents multiplechoices in some cases, giving you the opportunity to decide which tool works best foryou Whether you decide to use Ant or Maven for delivering a working application in

a predictable manner, TestNG or JUnit for early developer testing, PMD or FindBugsfor code analysis, or CruiseControl or Luntbuild for Continuous Integration, this bookaddresses the fundamental techniques for effectively employing these tools (and a mul-titude of others as well)

Rapid development of Java applications (which have an acceptable level of associatedquality) is still hard as ever; however, after reading John’s magnum opus on the toolsand techniques that ultimately enable predictability, confident, and accelerated deliv-ery in a software development process, you’ll find that designing, coding, and deployinghigh-quality Java applications rapidly just got a whole lot easier

—Andrew Glover, President, Stelligent Incorporated

Here is Edward Bear * coming downstairs now, bump, bump, bump, on the back of his head, behind Christopher Robin It is, as far as he knows, the only way of coming down- stairs, but sometimes he feels that there really is another way, if only he could stop bumping for a moment and think of it.

—“We are introduced to Winnie-the-Pooh and some bees, and the stories begin,”

Winnie the Pooh, A A Milne

Thus does A A Milne introduce that classic character of children’s literature, Winniethe Pooh As you can see, Winnie the Pooh seems to have some issues with the way hegoes downstairs (we probably wouldn’t be too far off if we were to speak of “painpoints”)

Software development sometimes feels like this It is easy to get so bogged down in thedetails, under the pressure of tight deadlines and changing requirements, that you forgetthat there might just be a better way A high number of bugs, a difficult integrationprocess, long and painful builds, and poor project visibility become an accepted part

of the developer’s life

The good news is that there are in fact many easy ways to improve your software velopment lifecycle

de-A judicious use of tools can go a long way in boosting developer productivity Forexample, many distributed development teams use nothing more sophisticated than aCVS or Subversion repository for storing application code and a mailing list for dis-cussion Imagine a system in which, whenever someone commits a change, issues areautomatically closed or updated based on the commit comment The issue manage-ment system then automatically notifies the issue owner of the change of status.Meanwhile, a dedicated build server detects the commit, builds and tests the system,and notifies the developer of any failures In a few relatively simple steps, you’ve takenyour development platform to a new level of reactivity and responsiveness

* For nonnative English readers: Edward Bear is a round-about way of saying “Teddy Bear.” In the original book, this text is accompanied by a drawing of Christopher Robin, a boy of about four years old, going down a flight of stairs dragging his teddy bear behind him.

xix

This is just a first step Soon you will be integrating automatically running unit andpossibly integration tests, code coverage tools, style checking tools, and more to create

a highly reactive, informative, finely tuned project infrastructure

Tools are the key to making such an infrastructure work efficiently Once you start toadopt a set of SDLC tools that suit your project and your organization, you will see arevolutionary shift in a groups development practices Agile development authors such

as Alistair Cockburn rightly point to optimal communication as being the cornerstone

of productive development Developing in a team without continuous integration islike rock climbing without a rope In addition, developing in a team without collabo-ration and connective development infrastructure is like developing in a boring beigeoffice space where everyone comes to work on time, enters a closed office, closes thedoor, and starts programming without ever stopping to have meetings In other words,programming without a good set of SDLC tools is very much the equivalent of fighting

a modern war with swords and armor—the adoption of SDLC tools is a generationalshift, the programmers just coming into the workforce will never know an alternative,but the programmers who haven’t experienced this shift are in danger of missing thetrend altogether

Now you shouldn’t go thinking that SDLC tools are only for large teams or big izations They aren’t In fact, most SDLC tools are easy to setup, and almost allorganizations can benefit, even a small outfit with only two or three developers.Most of these tools and techniques are not particularly difficult to put into place, butthey do require a minimum of effort, to step back and take a long, hard look at yourcurrent practices Chances are, there are things that can be improved

organ-This book is part of the O’Reilly Power Tools series, which began with the illustriousUnix Power Tools back in 1993 It has no relation to the “Java Power Tools” library

(http://www.ccs.neu.edu/jpt/), which is a software research project in the Java GUI field,

conducted by College of Computer & Information Science at the Northeastern versity, Boston, under the direction of Richard Rasala

Uni-How This Book Is Organized

One of the most characteristic traits of the open source (http://opensource.org) Java

world is choice It seems that, for any given task, there are always at least two opensource tools that can do the job To reflect this, I have tried to give a representativesurvey of the available open source tools for each area of software development that

we cover The book does not try to “sell” any one tool, or even any one tool in eachdomain On the contrary, we try to give readers enough information so that they candecide for themselves which tool is the most appropriate for their particular organiza-tion, and give enough practical information to get them up and running

This book is organized into sections, with each section covering a particular aspect ofthe software development lifecycle (or SDLC) With each section, we look at a number

of available tools that can be used to improve this aspect of the SDLC

Build Tools

In the first section, we cover possibly the most fundamental tool of all (after the compilerand the IDE, that is): the build tool Indeed, when used well, the build tool becomesthe cornerstone of your SDLC process It is this tool that coordinates, federates, andbinds the other SDLC tools together into a single, coherent process And the build toolhelps to ensure that your project can be built on any machine, in any environment

In the Java world, two tools dominate this area The first is Ant, the traditional Javabuild tool, which uses a straightforward procedural approach and benefits from a verylarge user base and a rich set of extensions The second is Maven 2, which uses apowerful, declarative approach to project build management and, as we will see, goesmuch further than being a simple build tool We will look at each of them in somedetail in this section

Version Control Tools

The second section covers that other fundamental component of any software opment infrastructure: a version control system not only provides critical backups ofyour source code, it also lets developers work together on the same project withoutgetting in each other’s way Version control systems also allow you to identify versionsand coordinate releases and (if necessary) rollbacks Finally, as we will see in Part VIII,

devel-it is a cornerstone of any Continuous Integration environment

In this section, we will look at the two most prominent open source version controltools on the market: CVS and Subversion

Unit Testing

Unit testing is another important best practice of modern software development though testing is certainly not new in itself, unit testing, and practices such as Test-Driven Development, have been gaining popularity over recent years Not only doesproper unit testing help ensure that your code works; it also fosters cleaner, moremodular, and better designed code Automated unit testing takes this a step further

Al-By simply integrating your unit tests into your standard build process, and runningthem automatically with every build, you can go a long way toward increasing thequality and reliability of your code

Writing tests is good, but it is even better to be sure of what code you are actuallytesting Test coverage tools help you check how much of your application is actuallybeing executed during your unit tests This in turn helps you identify untested codeand improve the overall quality of your tests

Preface | xxi

In this section, we will cover the latest tools in the unit testing domain, including JUnit

4 and TestNG, and see how these tests can be integrated smoothly into the build ess We also will look at how to verify test coverage with Cobertura, a powerful opensource coverage tool

proc-Integration, Load, and Performance Testing

Unit testing is not the end of the story as far as testing goes This section looks at othertesting techniques such as integration, load and performance, and user interface testing.All of these are important, and all can benefit from being integrated into the buildprocess

In this section, we will see how to integrate performance tests into your unit tests, how

to load-test your application, and how to automatically test web services, Swing faces, and web interfaces

inter-Quality Metrics Tools

It is important to be able to measure the quality of your code in objective terms Codequality has a direct bearing on the number of bugs and the ease of maintenance later

on Code quality metrics can also be a good way to bring inexperienced developers up

to speed with coding conventions and best practices This section looks at a range ofautomated tools that measure different aspects of code quality, including CheckStyle,PMD, FindBugs, and Jupiter

Technical Documentation Tools

A professional project needs professional documentation A significant part of thisdocumentation can (and should) be generated automatically, based on the source codeand comments This is the most reliable way to get consistently up-to-date technicaldocumentation This section describes tools that can help you generate good technicaldocumentation without having to spend too much effort writing and maintaining it

Issue Management Tools

We will look at that vital communication tool in the SDLC, the issue tracking system

Of course, issue tracking systems can be used by testers to raise bugs and by developers

to document bug fixes But they can also be used to help organize and document leases, to plan iterations, and to assign work tasks to team members

re-There are literally hundreds of issue tracking systems out there, both open source andcommercial In this section, we will look at two of the more interesting open sourcesolutions seen in the Java world The first is Bugzilla, the original open source issuetracking system The second is Trac, which excels by its excellent Subversion integra-tion and its innovative project management and wiki features

Continuous Integration Tools

Finally, we look at a tool to wrap it all up together under a single process This is theproverbial “one tool to rule them all.” This process is called continuous integration

In software development, it is a common observation that the longer you wait to grate your team’s code, the harder it gets Continuous Integration is based on the ideathat you can greatly facilitate this process by committing small changes regularly, andthen running automatic builds whenever code changes are committed Whenever adevelopper commits new or modified code to the source code repository, the buildserver checks it out and runs a build In the very least, this makes sure that the modi-fications compile correctly However, why stop at simply checking that everythingcompiles? While you’re at it, you might as well check that all the unit tests still run notforgetting, of course, the integration, performance, and user interface tests

inte-Indeed, virtually all of the tools and techniques that we have discussed above can benefitfrom being run automatically on a regular basis

Although this sort of integration is certainly possible with a well-tailored shell scriptand a cron job, nowadays there are a lot of tools that can save you a great deal of timeand effort in this area In this section, we will be looking at some of the more interestingopen source CI tools: Continuum, CruiseControl, LuntBuild, and Hudson

Who Should Read This Book

This is, fundamentally, a techie book It is a hands on tour of a wide range of tools, forpeople who like to get their hands dirty

If you are a Java developer, these tools can help to improve your development practices,and make your life easier in the process Lead developers, architects, and people inter-ested in the wider picture will be able to glean from these pages some useful ideas aboutimproving your project infrastructure and best practices You will learn about the dif-ferent build tools of the Java world You will learn how to set up a version control server

or a Continuous Integration server using open source tools You will find tools to port your coding standards and design recommendations, and automatically generatehigh-quality technical documentation You will find out how to get the most out ofyour unit and integration tests

sup-Readers are expected to have a basic knowledge of Java and XML Many build serversrun on Linux boxes, so there will be a mixture of Windows and Linux examples, whenoperating systems issues are discussed No prior experience of any of the tools isrequired

Preface | xxiii

What This Book Doesn’t Cover

This book cannot come close to covering all the good software tools on the market.Some aren’t here for lack of space, or simply because I wasn’t familiar enough withthem to do them justice

This book is limited to open source tools This is not because there are not commercialtools in the software development lifecycle field: there are Nor is it because these toolsaren’t worth considering for your project or organization; again, they may be No,commercial tools are off limits for a simple question of scope (I did want to finish thisbook one day), and, to be fair to everyone, it would be hard to include one tool withoutincluding all the others

Having said this, there are a few excellent commercial tools on the market which itwould be a shame not to mention These commercial tools are often of very high quality,with many innovative features And, as in any field, competition is always an excellentdriver for innovation

Two organizations that deserve special mention in this area are Atlassian and JetBrains.Atlassian is behind the very popular JIRA issue tracking system They also market Bamboo, an innovative Continuous Integration server, as well as a set of integratedtools such as Clover, a test coverage tool; FishEye, a tool that helps you to visualize thecontents of your source code repository; and Crucible, a code review tool

JetBrains is the author of the well-known and highly innovative Java IDE IntelliJ cently, JetBrains have also developed TeamCity, a next-generation Continuous Inte-

Re-gration server that builds and tests code before it is committed to version control.

At the time of this writing, both of these companies offered free licencing arrangementsfor open source products

Contributing Authors

This book was not written alone Indeed, it has been a collaborative effort, with thedirect and indirect participation of many people The following are those who gener-ously contributed their time and energy into providing valuable material for this book:

Brian Agnew

Brian Agnew is the founder and principal consultant with OOPS Consultancy Ltd,located in London, U.K He holds a B.Eng in Electrical and Electronic Engineeringfrom Sheffield University He advises and works with major financial houses andleading consultancies on a wide range of projects, including trading systems, net-work management infrastructures and grid-based architectures, working in Javamainly, C++ when he has to, and Perl when nothing else will do

Brian contributed material on XMLTask, an Ant task that provides sophisticatedXML manipulation

Jettro Coenradie

Jettro Coenradie is a Java (enterprise) specialist living in the Netherlands Jettrohas been working in the ICT for about 10 years now He likes to try out newframeworks that are focused on quality and productivity, so he is interested in toolslike Luntbuild, Continuum and Maven In addition, Jettro is a software architectfocusing on the Spring framework You can find more information about Jettro on

his blog, http://www.gridshore.nl/blog.

Jettro contributed an article on integrating Maven with LuntBuild

Keith Coughtrey

Keith gained a wealth of development experience at some of the U.K.’s largestcompanies before moving to New Zealand in 2004 He is currently leading devel-opment at First NZ Captial, New Zealand’s largest stockbroker, where he isapplying many of the tools and techniques espoused in this book

Keith contributed material on Mylyn

With more than eight years of experience, Masoud Kalali is a senior staff engineer

at E-peyk He is mostly responsible for the design and technology architecture ofE-peyk enterprise framework, which is a service-oriented framework based on Java

EE 5 and other open standards Masoud’s area of expertise are SOA and web ices in addition to performance monitoring and management Masoud is a con-tributor of several open source projects that are utilized in the E-peyk framework.Masoud contributed material on SchemaSpy and on SoapUI

serv-Avneet Mangat

Avneet Mangat has six years’ experience with Java/JEE He is currently working

as the Lead Developer at Active Health Partners, U.K He is a Sun-certified grammer and web developer and is also Prince2-certified He is also the lead

pro-developer of DBBrowser open source database browsing tool (http://database

browser.sourceforge.net/) Outside interests are photography and traveling.

Avneet contributed material on setting up a Maven repository with Artifactory

Eric Redmond

Eric Redmond has been involved in the Maven community for over two years as auser, contributor, speaker, author, and patch provider, as well as a founder of twoprofessional education and consulting companies surrounding Maven and otheraspects of large-scale application lifecycle management He is currently enjoyingsome downtime, meaning a simple life as a Ruby (and Rails) developer, yet keeping

a keen eye on JRuby He is the proprietor of Propellors Consulting and an active

blogger at http://blog.propellors.net.

Eric contributed material on Maven 2 and Achiva

Alex Ruiz

Alex Ruiz is a Software Engineer in the development tools organization at Oracle

(http://www.oracle.com) Alex enjoys reading anything related to Java, testing,

OOP, AOP, and concurrency, and has programming as his first love Alex hasspoken at JavaOne, JavaPolis, Desktop Matters, and SD West He also has publi-cations with IEEE Software, Dev2Dev, JavaWorld and Objective View Beforejoining Oracle, Alex was a consultant for ThoughtWorks Alex maintains a blog

at http://www.jroller.com/page/alexRuiz.

Alex has contributed material on FEST

Tim O’Brien also helped out with ideas for the introduction

Technical Reviewers

The following people were brave enough to accept the task of formally reviewing thefinished manuscript:

Nigel Charman

Nigel is a Java consultant living in Wellington, New Zealand, with a special interest

in developer testing and code quality In his spare time, he’s currently working onthe JiBX/WS web services framework and helping to organize the Wellington JavaUser Group

Paul Duvall

Paul M Duvall is the CTO of Stelligent Incorporated in Reston, VA—an Agile

infrastructure consulting firm He coauthored Continuous Integration: Improving

Software Quality and Reducing Risk (Addison-Wesley, 2007), authors a series for

IBM developerWorks called “Automation for the People,” and is a contributing author to the No Fluff Just Stuff Anthology (Pragmatic Programmers, 2007) and the

“UML 2 Toolkit” (Wiley, 2003) He actively blogs on TestEarly.com andIntegrateButton.com

Greg Ostravich

Greg Ostravich works for the Colorado Department of Transportation in Denver,Colorado, where he leverages the knowledge from the local Denver Java User

Group and books such as Java Power Tools to implement best practices in software

development for the State of Colorado In addition to being a technical editor for

Java Power Tools, he is currently the President of the Denver Java User Group and

did the technical editing for JBoss at Work, No Fluff Just Anthologies (2006 and

2007), and GIS for Web Developers (2007) He also wrote reviews for Pragmatic

Unit Testing in Java with JUnit and Pragmatic Project Automation, which can be

found on the Denver Java User Group and The Server Side web sites

Many other people also reviewed individual chapters or sections, including CédricBeust, Keith Coughtrey, Richard Hancock, Jimmy Kemp, Gregor Lawson, AndrewMcDowell, Brett Porter, Bill Ross, and Martin White.

Conventions

Most of the conventions used in this book should be fairly self-explanatory Literal textsuch as file names, class names, and so on, is represented using a fixed width font.Commands intended to be executed on the command line are written in constant width italic Code listings are written like this:

Windows equivalent would be something like “C:>.” Commands that are typed by are

written like this System output is written in normal type:

$ mvn compile

[INFO] Scanning for projects

Downloading: ssh-external/1.0-alpha-5/wagon-ssh-external-1.0-alpha-5.pom

http://buildserver:8080/artifactory/repo/org/apache/maven/wagon/wagon-5K downloaded

The commands themselves are generally the same under Unix and Windows, with theexception of the usual things such as class pathes You might also see the occasional

ls rather than dir in some of the examples.

When a command is split over several lines for readability, I use the Unix convention

of putting a “\” at the end of each line except the last one In Windows, you will have

to put everything on one line (excluding the trailing “\” characters):

Source Code

There are a lot of practical examples throughout this book You can download thesource code for these examples from the publisher’s web site, or from the Java Power

Tools web site (http://www.javapowertools.com) In general, the code is designed to

illustrate how to use the various tools, and to make it possible for you to experimentwith the tools As a rule, it is not of production code quality, as production quality codetends to be more complex and project-specific

About the Title

This book is part of the O’Reilly Power Tools series, which began with the illustriousUnix Power Tools back in 1993 It has no relation with the “Java Power Tools” library

(http://www.ccs.neu.edu/jpt/), which is a software research project in the Java GUI field,

conducted by College of Computer & Information Science at the Northeastern versity, Boston, under the direction of Richard Rasala

First and foremost, I would like to deeply thank my wife, Chantal, and my two boys,James and William, whose great love, patience, and support over the last two yearshave made this book possible Writing this book involved a seemingly unending stream

of late night writing and grumpy mornings, as well as frequently delegating importanttasks such as playing with the kids, putting them to bed and story-telling Chantalnevertheless provided unrelenting support and love throughout the whole endeavor

I would also like to thank the team at Equinox, New Zealand Equinox provided valuable time and resources for this book, as well as many challenging opportunities

in-to research and work with the in-tools and techniques described in the book First andforemost, a warm thanks to Roger Dalgleish and Paul Ramsay, without whose help thisbook would simply not have been possible Also, thanks to all the Equinox staff mem-bers who took the time to read the (sometimes very) rough draft chapters and providevalued feedback, and to those who helped in all sorts of other ways: Willy Bartholo-meusz, Keith Coughtrey, Candi Cunningham, Nev Flaws, Richard Hancock, JimmyKemp, Gregor Lawson, Brian Levy, Brendon Livingstone, Craig McLean, AndrewMcDowell, Robin Newton, Bill Ross, Pete Tanesey, Kenneth Wells, Martin White,Yolanda van Dorrestein, and everyone else at Equinox

A big thanks to all the contributors—Brian Agnew, Jettro Coenradie, Keith Coughtrey,John Hurst, Masoud Kalali, Avneet Mangat, Eric Redmond, and Alex Ruiz—withoutwhose help this book would have been considerably poorer

Thanks also to the reviewers who spent uncounted time and energy reading the draftmanuscript and providing many valuable comments and suggestions: Nigel Charman,Paul Duvall, Greg Ostravich, and also Cédric Beust, Keith Coughtrey, Richard Han-cock, Jimmy Kemp, Gregor Lawson, Andrew McDowell, Tim O’Brien, Bill Ross, andMartin White

Warm thanks to Andy Glover, for his valued support and friendship I am honored tohave him as the author of the Foreword Groovy, man!

I also want to thank the many organizations where I have been able to put these toolsinto practice during the writing of the book: notably, the staff of the National Library

of New Zealand—in particular Jenny McDonald and Paul Curtis—and the staff at theInland Revenue department of New Zealand, especially the members of the O2C2team: Susanna McSweeny, Yujia Huang, Eduard Letifov, Leah Xu, Bryden Davenport,and Aijun Kang

Thanks to my parents for their love and support, and for reading me Winnie the Pooh

as a child

And to Maryse and Emmanuel Consul, for their kindness and hospitality during allthose years, and for letting their daughter depart for a far-off foreign land

Preface | xxix

Thanks also to my “kiwi” family, Diana and Owen Lowe, and Yvonne and CorneliusVan Veen, for their warm welcome, permanent support, and for those many eveningsduring which wine, beer, and friendship rejuvenated my mind between chapters.

A book like this would not be possile without the vibrant Java open source community

in existance today Thanks to tool authors and contributors the Wellington Java UsersGroup and to other members of the Java community with whom I have exchangedwords, emails, and ideas over the last few years: Cédric Beust, Mandy Chung, ScottDavis, Mohamad Easawy, Jeremy Flowers, Rob Harwood, John Hurst, Mik Kersten,Surjendu Kuila, Steve Loughran, Brett Porter, Cameron Purdy, Matt Raible, and RobWilliams

Using Code Examples

This book is here to help you get your job done In general, you may use the code inthis book in your programs and documentation You do not need to contact us forpermission unless you’re reproducing a significant portion of the code For example,writing a program that uses several chunks of code from this book does not requirepermission Selling or distributing a CD-ROM of examples from O’Reilly books doesrequire permission Answering a question by citing this book and quoting examplecode does not require permission Incorporating a significant amount of example codefrom this book into your product’s documentation does require permission

We appreciate, but do not require, attribution An attribution usually includes the title,

author, publisher, and ISBN For example: “Java Power Tools by John Ferguson Smart.

Copyright 2008 John Ferguson Smart, 978-0-596-52793-8.”

If you feel your use of code examples falls outside fair use or the permission given above,

feel free to contact us at permissions@oreilly.com.

Safari® Enabled

When you see a Safari® Enabled icon on the cover of your favorite nology book, that means the book is available online through the O’ReillyNetwork Safari Bookshelf

tech-Safari offers a solution that’s better than e-books It’s a virtual library that lets you easilysearch thousands of top tech books, cut and paste code samples, download chapters,and find quick answers when you need the most accurate, current information Try it

for free at http://safari.oreilly.com.” (http://safari.oreilly.com)

Since the dawn of time, people have been using tools to make their life easier Tools letyou solve a problem at hand more quickly and more efficiently, so that you can spendyour time doing more interesting things The stone axe, for example, enabled yourprehistoric hunter to cut up meat more efficiently, thus leaving the tribe with time to

do much more satisfying activities such as grilling mammoth steaks and painting oncave walls

Of course, not all tools are equal, and tools have a tendency to evolve If you go down

to your local hardware store nowadays, you can probably find something even better

to chop up your firewood or to chop down a tree In all things, it is important to findthe tool that is most appropriate for the job at hand

The software industry is no exception in this regard There are thousands of tools outthere, and there is a good chance that some of these can help you work more efficiently

If you use them well, they will enable you to work better and smarter, producing higherquality software, and avoiding too much overtime on late software projects The hardestthing is knowing what tools exist, and how you can put them to good use

That’s where this book can help In a nutshell, this book is about software developmenttools that you can use to make your life easier And, in particular, tools that can helpyou optimize your software development life cycle

The Software Development Life Cycle (or SDLC) is basically the process you follow toproduce working software This naturally involves coding, but there is more to building

an application than just cutting code

You also need a build environment

When I talk of a build environment, I am referring to everything that contributes toletting the developers get on with their job: coding This can include things like a versioncontrol system (to store your source code) and an issue management system (to keeptrack of your bugs) It can also include integration, testing, and staging platforms, inwhich your application will be deployed at different stages But the build environmentalso includes tools that make life easier for the developer For example, build scriptsthat help you compile, package and deploy your application in a consistent and

xxxiii

reproducible manner Testing tools that make testing a less painful task, and thereforeencourage developers to test their code And much more.

Let’s look at a concrete example My picture of an efficient, productive build ment goes something along the following lines

environ-You build your application using a finely tuned build script environ-Your build script is clean,portable, and maintainable It runs on any machine, and a new developer can simplycheck the project out of the version control system and be up and running immediately

It just works

You store your code in a central source code repository Whenever you commit yourcode, a build server detects the change to the code base, and automatically runs a fullsuite of unit, integration, and functional tests If any problems crop up, you (and therest of the team) are immediately notified You have learned from experience that com-mitting small, regular changes makes the integration process go a whole lot smoother,and you organize your work accordingly

You use an issue tracking system to keep tabs on features to implement, bugs to fix, orany other jobs that need doing When you commit changes to your source code repo-sitory, you mention the issues that this change addresses in the commit message Thismessage is automatically recorded against these issue in the issue management system.Furthermore, if you say “Fixes #101,” the issue automatically will be closed

Conversely, when you view an issue in the issue management system, you can also seenot only the commit messages but also the exact modifications that were made in thesource code When you prepare a release, it is easy to compile a set of reliable releasenotes based on the issues that were reported as fixed in the issue tracking system sincethe last release

Your team now writes unit tests as a matter of habit It wasn’t easy to start with, butover time, coaching and peer programming have convinced everyone of the merits oftest-driven development Automatic test coverage tools help them ensure that their unittests aren’t missing out on any important code The extensive test suite, combined withthe test coverage reports, gives them enough confidence to refactor their code as nec-essary This, in turn, helps keep the code at a high level of reliability and flexibility.Coding standards and best practices are actively encouraged A battery of automaticcode auditing tools checks for any violations of the agreed set of rules These tools raiseissues relating to coding standards as well as potential defects They also note any codethat is not sufficiently documented

These statistics can be viewed at any time, but, each week, the code audit statistics arereviewed in a special team meeting The developers can see how they are doing as ateam, and how the statistics have changed since last week This encourages the devel-opers to incorporate coding standards and best practices into their daily work Becausecollective code ownership is actively encouraged, and peer-programming is quite fre-quent, these statistics also helps to foster pride in the code they are writing

Occasionally, some of the violations are reviewed in more detail during this meeting.This gives people the opportunity to discuss the relevence of such and such a rule incertain circumstances, or to learn about why, and how, this rule should be respected.You can view up-to-date technical documentation about your project and your appli-cation at any time This documentation is a combination of human-written high-levelarchitecture and design guidelines, and low-level API documentation for your appli-cation, including graphical UML class diagrams and database schemas The automaticcode audits help to ensure that the code itself is adequately documented.

The cornerstone of this process is your Continuous Integration, or CI, server Thispowerful tool binds the other tools into one coherent, efficient, process It is this serverthat monitors your source code repository, and automatically builds and tests yourapplication whenever a new set of changes are committed The CI server also takes care

of automatically running regular code audits and generating the project tion It automatically deploys your application to an integration server, for all to seeand play around with at any time It maintains a graphical dashboard, where teammembers and project sponsors can get a good idea of the general state of health of yourappplication at a glance And it keeps track of builds, making it easier to deploy aspecific version into the test, staging, or production environments when the time comes.None of the tools discussed in this book are the silver bullet that will miraculously solveall your team’s productivity issues To yield its full benefits, any tool needs to be usedproperly And the proper use of many of these tools can entail significant changes inthe way you work For example, to benefit from automated testing, developers mustget into the habit of writing unit tests for their code For a continous integration system

documenta-to provide maximum benefits, they will need documenta-to learn documenta-to commit frequent, small changes

to the version control system, and to organize their work accordingly And, if you want

to generate half-decent technical documentation automatically, you will need to makesure that your code is well-commented in the first place

You may need to change the way people work, which is never easy This can involvetraining, coaching, peer-programming, mentoring, championing, bribing, menacing,

or some combination of the above But, in the long run, it’s worth it

Introduction | xxxv

PART I Build Tools

“It just shows what can be done by taking a little trouble,” said Eeyore “Do you see,

Pooh? Do you see, Piglet? Brains first and then Hard Work Look at it! That’s the way to

build a house.”

—“A House is Built at Pooh Corner for Eeyore,” The House at Pooh Corner, A A Milne

Putting some thought and effort into planning your build process from the outset canpay off abundantly further on down the line, when the going gets tough and the pressure

is on This is where a well-designed build process and fine-tuned build tools show theirworth

Like many things, in IT and elsewhere, build tools are primarily the fruit of humanlaziness Compiling C or C++ (or Java, for that matter) from the command line is aterribly tedious affair And, in the Unix world, where scripts abound, the next step wasnatural: why not write a script to do it for you? A basic shell script written to compile

a few C source code files was probably the oldest ancestor of our modern Java buildtools such as Ant and Maven

Shell scripts work fine for a small number of source code files, but this approach isdifficult to scale to larger applications This is where Make enters the scene Make isthe principal Unix build tool, and anyone familiar with linux or Unix will have comeacross it at some stage A makefile (the name of the scripts run by Make) is basically alist of instructions used to compile your application The idea is to automate the buildprocess, by working out exactly what files need to be compiled, and in what order You

do this by defining dependency rules, which tell Make when it should compile a ticular file, and how it should go about compiling it A very simple makefile is shownhere:

par-# top-level rule to create the program.

# linking the compiled files.

main: main.o

gcc -g main.o -o main

# Remove generated files

clean:

/bin/rm -f main main.o

This makefile will compile and link the C program contained in the main.c source codefile Real-world makefiles can get much bigger and more complicated than this, andMake does a lot more than what can be gleaned here Indeed, Make is a powerful tool:

it is used regularly to build very large and complex C and C++ applications, includingthe Linux kernal itself It marks an important step in the history of automating the buildprocess Make, along with Unix/Linux, also helped to promote the idea of a portablebuild: you should be able to build an application from the source code on any machine

Of course, the use of libraries in Linux and Unix makes this a bit more complicatedthen that, but the idea is there

However, as we will see, nowadays there are build tools that are much better adapted

to Java development: leave Make to the C and C++ programmers

The history of builds in Windows environments is slightly different In the days of yore,when Turbo Pascal was king, you usually would write, compile, and build your appli-cation directly from within the IDE This remained the tendency for a very long time

—builds would be performed on an individual developer’s machine from within hisIDE

This approach is still used in many organizations However, it is not a good idea forseveral reasons A build process that relies on an IDE is likely to depend on how theIDE is installed This in turn makes it dependent on the configuration of particularmachines If your build process depends on how a particular developer has configuredhis or her machine, you’re in trouble

A good build process has a certain number of characteristics, for example:

• Builds should be portable A new developer should be able to check out the sourcecode of a project, and run a build, independent of the IDE Builds should also beportable between operating systems Nowadays, it is common to develop on one

OS and to build on another

• You should be able to run a build without human intervention This is one of theunderlying principles of Continuous Integration, which we look at in Part VIII ofthis book A build that needs human intervention reveals a very fragile and vuner-able build process

• A build tool should federate a set of tools and processes into a single, coherentbuild process Building a software application can involve many steps—compilingthe code, of course—but also other steps such as downloading dependencies, run-ning unit, integration and functional tests, performing automatic code audits,