Theoretical Introduction to Programming Bruce Mills Theoretical Introduction to Programming With 29 Figures Bruce Mills, BEng, BSc, PhD British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Control Number: 2005926335 ISBN-10: 1-84628-021-4 Printed on acid-free paper ISBN-13: 978-1-84628-021-4 © Springer-Verlag London Limited 2006 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regula- tions and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Printed in the United States of America (SPI/SBA) 987654321 Springer Science+Business Media springeronline.com v Preface This book is organised into a large number of brief, self-contained entries. Admittedly, there is no such thing as a self-contained entry. For exam- ple, you need some knowledge of English to understand this paragraph. But, the principle is that each entry, of one or two pages, is a conceptual whole as well as a part of a greater whole (see note 20) in the same way that a car has four whole wheels, and not eight half wheels. Some entries are intended to demonstrate a technique, or introduce an historically contingent fact such as the actual syntax of a contemporary language, or in this case, a specific issue regarding this book. Others are intended to illustrate a more eternal truth. They may be about a contemporary language, but stress a philosophical position or broadly based attitude. Both of these I have called notions. Finally, there are entries that are intended to cause the reader to do something other than just nodding their head as a sign of either agreement or an incipient dormant state. These are the exercises. The distinction can only be arbitrary; the classification is merely a guide to suggest the sense in which the pages are intended. In many cases, entries that are not specifically labelled as exercises in- volve generic opportunities for self-study. As this is a book on computer programming, it is natural and strongly advised that the reader try im- plementing each concept of interest as it arises. With this in mind, I have tried hard not to leave out pragmatic details whose omission would leave the reader with nothing but the illusion of understanding. Nev- ertheless, actually cutting practice code makes a big difference in the ability of the programmer to use the concepts when the need arises. At the end of the book are the notes explaining short and simple issues or (paradoxically) issues that are too complex to explain in this book. If a note became too lengthy while being written it was converted into a notion or an exercise. vii Contents Preface v Chapter 1. The Abstract Rational Outlook 1 Abstract Computation 2 Rational Thought 4 Human Psychology 6 Mythological Language 8 Literate Programming 10 Hand-Crafted Software 12 Technical Programming 14 Chapter 2. A Grab Bag of Computational Models 17 Abstract and Virtual Machines 18 State Machines 20 State Machines in Action 22 Turing Machine 26 Non-Deterministic Machines 30 Von Neumann Machine 34 Stack Machine 36 Register Machine 38 Analogue Machine 39 Cellular Automata 40 Unorthodox Models 41 The Game of Life 42 The Modern Desktop Computer 44 Aspects of Virtual Machines 46 Aspects of Programming 48 Register Indirection 50 Pure Expression Substitution 52 Lists Pure and Linked 54 Pure String Substitution 56 The Face Value of Numerals 58 Solving Equations 62 Pure Unification 64 Equality of Expressions 66 Equational Reasoning 68 viii Theoretical Introduction to Programming Unification Reduction 70 Code Reduction 74 Programming With Logic 76 Negation in Logic Programming 78 Impure Lambda Calculus 80 Pure Lambda Calculus 82 Pure Lambda Arithmetic 84 Pure Lambda Flow Control 86 S-K Combinators 90 Chapter 3. Some Formal Technology 92 The Ellipsis Is Not a Definition . 93 The Summation Operator 95 Propositional Calculus 97 Boolean Algebra 99 Predicate Calculus 101 Formal Mathematical Models 102 The Formal State Machine 103 Several Types of Networks 105 Informal Petri Nets 107 Formal Turing Machine 109 The Table-Driven State Machine 110 Factors of Graphs 111 Products of Graphs 113 Constructive Numerics 115 Prime Programs 117 Showing that Factorial Works 119 Reasoning About Code 123 Logical Conditions 127 Chapter 4. Limitations on Exact Knowledge 131 Finite-State Limitations 132 N log N sorting 133 Russell’s Paradox. 134 Pure Lambda Paradoxes 136 Godel’s Theorem 138 Non-Computability . 140 Solving Polynomials 142 ix Churche’s Thesis 143 Algorithmic Complexity 144 P and NP 146 NP completeness 148 Turing Test 149 Natural Language Processing 150 The Computable Reals 151 The Diagonal Argument 152 Chapter 5. Some Orthodox Languages 154 C Pointers to Functions 159 Taking C on Face Value 161 Functions and Other Data in C 163 The CPreprocessor 166 C Functions are Data Again 167 Java Code 169 Pointer Casting 171 The Object Data Type 177 Manual Objects 179 Inheritance and Dynamic Type 181 CODASYL and Objects 183 Typecasting 185 The Concept of Type 187 Type-Checking 188 Subtypes and Programming 189 New Datatypes 190 Scheme Code 193 Declarative and Imperative 195 Sorting with Pure Substitution . 197 Fast Sorting in Haskell 199 Logic in Prolog 201 Functions in Prolog 204 Arithmetic in Prolog 205 Meta-Logic in Prolog 207 What Is HTML Code? 209 Illogical markup language 211 HTML Forgive and Forget 212 Expanding Beyond Recognition 213 x Theoretical Introduction to Programming Chapter 6. Arithmetic Computation 214 Natural Arithmetic 215 Modulo Arithmetic 217 Integer Arithmetic 219 Rational Arithmetic 221 Complex Arithmetic 223 Exact Arithmetic 225 Showing That a Power Loop Works 227 When Is a Proof Not a Proof? 229 Real-Valued Memory 231 Cellular Matrix Multiplication 232 Chapter 7. Repetitive Computation 235 The Use of Recursion 236 Doing Without the While Loop 238 Defining the Generic While-Loop 240 Design of the Power Function 244 Powers by Multiplication 246 Computing Powers by Squaring 248 Language or Algorithm? 250 Repetitive Program Design 253 Recursive Code Compilation 254 Functions as Data 256 Lambda Expressions in Java 258 The Y -combinator definition 260 Y -combinator factorial 263 Y -combinator Fibonacci 264 Chapter 8. Temporal Interaction 265 Virtual Interaction 266 Incorruptible Operations 268 Temporal Computing 270 Multi-Threaded Code 272 Graphs of State Machines 273 Direct Thread Composition 274 Concurrent Thread Interference 276 Control Structures 278 xi Thread Point of Execution 280 The Transition Network 281 High-Level Interference 285 Incorruptible Commands Again 286 Thread Interaction 288 Pure String Interaction 292 Showing That a Parser Works 295 Mutual Exclusion 296 Good Mutual Exclusion 298 A Partial Mutex Protocol 299 Guarded Commands 300 Blocking Commands 306 Hardware Assistance 307 Proving That a Protocol Works 308 Two Partial Exclusion Protocols 309 The Peterson Protocol 310 The Decker Protocol 312 Proving That a Protocol Works 314 Chapter 9. Container Datatypes 315 Abstract Arrays 316 Pure Containers 318 Generic Maps 322 Showing That Infinite Lists Work. 325 Generic Lists 326 Computing with Infinite Lists 328 Sequence Builder 330 Infinite Lists in Haskell . 333 Infinite Lists in Scheme 334 Primitive List Recursion 336 Appendices 339 End notes 340 Bibliography 351 Glossary 353 Index 355 Chapter 1 The Abstract Rational Outlook In which we discover that programming is about being human. That to truly master a technology we must first master ourselves. That philo- sophical esoterica will bite us on the backside if we do not pay them enough attention. We discuss the effect that eternal truth, pure sci- ence, rational thought, group behaviour, and contemporary fashion have on our daily programming activities. We discover that identification of computation is a matter of opinion, that programming is an outlook on life in general, that the task of a programmer is to add a little wisdom to the inanimate. In short, this theme contains the bulk of the material that most readers will pay scant attention to, until it is too late. You may now skip to the next theme. [...]... believe another system to be intrinsically more difficult and awkward, regardless of whether it really is or not The familiar is erroneously believed to be intrinsically easier and more natural Further, old ideas are often repackaged with a new name and new jargon, alienating the older system and gaining promotion for the organisation that invented the new jargon The roots of many concepts go significantly... of the material at hand, small enough and simple enough to submit to analysis, realistic enough to be relevant, we admit that a programmer likewise needs a collection of software models: pure archetypical computational mechanisms that assist analysis and design of practical software in the real and very impure world We recognise that every piece of software is a virtual machine And so, study a collection... Machine You do not understand a virtual machine until you have written several programs for it In this exercise, we try to understand Turing machines One direct way to specify a Turing machine is a transition table The columns are old-state, old-symbol, new- symbol, new- state, and movement, where L means go left, R means go right, and H means halt The starting state is s, and the halting state is h... we must be able to push techniques to their limit and frequently learn new techniques to help us to solve new problems, or to solve old problems more efficiently This book expounds fundamental and generic abstractions of computation that have been developed, tested, and debugged by many people over the course of the twentieth century At one time complex and esoteric, these ideas can now be well learned... exercise in classification, and in detail we have many different combinations, and permutations and subsystems In my use of the terms, a paradigm is an outlook that contains unjustified existential ideas, while a mythology is an outlook that contains unjustified empirical ideas We should use as minimal a mythos as possible, and we should be aware, and gain experience in, several distinct and conflicting mythologies... tool It thus helps to understand that tool In particular, this book is about rational programming, about making the thought processes involved in programming available to the conscious mind, and thus to introspection and adjustment This requires effort, practice and discipline The human mind is made from conscious and subconscious parts The subconscious has the greater capacity and speed It provides the... responds by showing information on its display and updating the information that it stores cancel cancel P Start today date start time new enter enter new date R ok 3 stop time 2 enter 1 Recall Channel As above, we can draw a diagram, a network of nodes and links that represents the states and transitions of the state machine The input is written on the links, and the output (not indicated in the above... merge and split, and it is unclear where one stops and another begins Abstractions never apply precisely in practice, but they may apply sufficiently well while we have the power to maintain their pre-conditions If the pre-conditions are violated, then the conclusions from the abstraction may be invalid For example, if we count rabbits and combine them in a box, some may be born, and some die and we... readily understandable by those who try (as distinct from those who do not) First and foremost, a program is a literate work, from one human being to another, even if only from you to yourself Technology should be made human, and yes, it is possible, but we have stopped trying, and stopped promoting this attitude This book emphasises the idea that software is primarily a work of literature and science,... mechanical, and was largely the same as the modern electronic digital computer Pneumatics has many advantages over mechanics, e.g., an air hose can be bent around easily, while rods and wheels need careful alignment Pneumatic computers are less affected by the environment and were used for industrial control into the 1980s The basic element in many computers is the nand-gate It has two signal inputs and one . easier and more natural. Further, old ideas are often repackaged with a new name and new jargon, alienating the older system and gaining promotion for the organisation that invented the new jargon Dynamic Type 181 CODASYL and Objects 183 Typecasting 185 The Concept of Type 187 Type-Checking 188 Subtypes and Programming 189 New Datatypes 190 Scheme Code 193 Declarative and Imperative 195 Sorting. 161 Functions and Other Data in C 163 The CPreprocessor 166 C Functions are Data Again 167 Java Code 169 Pointer Casting 171 The Object Data Type 177 Manual Objects 179 Inheritance and Dynamic