Your unixlinux the ultimate guide (third edition) – part 1

Third Edition A separate chapter on the essentials of C programming Coverage of the GNU debugger (gdb), which is a more effective, powerful debugger than dbx, the original UNIX debugger The popular Concurrent Version System (CVS), which is found on all Linux systems The bc calculator utility, an efficient computing tool for shell programmers Coverage of the pico editor and relocates the chapter on emacs to an appendix The popular Red Hat Package manager (RPM) A wealth of real-life examples Numerous learning aids such as Notes, Tips, Caution boxes, and a separate icon that highlights Linux references Your UNIX/Linux The Ultimate Guide Das MD DALIM #1173349 12/18/11 CYAN MAG YELO BLK Visit the Your UNIX/Linux website at www.mhhe.com/das for solutions, slides, source code, links, and more Your UNIX/Linux The new edition of Your UNIX/Linux includes: The Ultimate Guide Your UNIX/Linux: The Ultimate Guide, Third Edition, is written with both users and programmers in mind and is the ultimate UNIX/Linux text Both pedagogical tool and exhaustive reference, it is well suited to any course that includes UNIX or Linux A strong pedagogical framework sets it apart from similar texts and allows beginning students to gain a firm grasp of fundamental concepts, while chapters on advanced topics inspire the more experienced reader to move beyond the basics Nearly a thousand exercises and self-test questions provide a way for students to test and reinforce their understanding of the material Third Edition Sumitabha Das Your UNIX/ Linux The Ultimate Guide Third Edition Sumitabha Das das76205_fm_i-xI.indd i 12/14/11 4:36 PM YOUR UNIX/LINUX: THE ULTIMATE GUIDE, THIRD EDITION Published by McGraw-Hill, a business unit of The McGraw-Hill Companies, Inc., 1221 Avenue of the Americas, New York, NY 10020 Copyright © 2013 by the McGraw-Hill Companies, Inc All rights reserved Previous editions © 2006 and 2001 Printed in the United States of America No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written consent of The McGraw-Hill Companies, Inc., including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning Some ancillaries, including electronic and print components, may not be available to customers outside the United States This book is printed on acid-free paper 1234567890 DOC/DOC 1098765432 ISBN 978–0–07–337620–2 MHID 0–07–337620–5 Vice President & Editor-in-Chief: Marty Lange Vice President of Specialized Production: Janice M Roerig-Blong Editorial Director: Michael Lange Publisher: Raghothaman Srinivasan Marketing Manager: Curt Reynolds Project Manager: Melissa M Leick Design Coordinator: Brenda A Rolwes Cover Design: Studio Montage, St Louis, Missouri Cover Image: © Creatas/PunchStock RF Buyer: Sherry L Kane Media Project Manager: Balaji Sundararaman Compositor: Cenveo Publisher Services Typeface: 10/12 Times Roman Printer: R R Donnelley All credits appearing on page or at the end of the book are considered to be an extension of the copyright page Library of Congress Cataloging-in-Publication Data Das, Sumitabha Your UNIX/LINUX : the ultimate guide / Sumitabha Das — 3rd ed p cm Rev ed of: Your UNIX : the ultimate guide / Sumitabha Das 2006 ISBN-13: 978-0-07-337620-2 (alk paper) ISBN-10: 0-07-337620-5 (alk paper) UNIX (Computer file) Linux Operating systems (Computers) I Das, Sumitabha Your UNIX II Title QA76.76.O63D3495 2013 005.4’32—dc23 2011042979 www.mhhe.com das76205_fm_i-xI.indd ii 12/14/11 4:36 PM To my wife Julie and daughter Sohini, whose contribution can no longer be ignored das76205_fm_i-xI.indd iii 12/14/11 4:36 PM iv das76205_fm_i-xI.indd iv 12/14/11 4:36 PM Contents in Brief PART I 10 11 UNIX for the User Introducing UNIX Becoming Familiar with UNIX Commands 25 The File System 56 File Attributes .92 The vi/vim Editor .122 The Shell 156 The Process 188 The Shell—Customizing the Environment 213 Simple Filters 241 Filters Using Regular Expressions—grep and sed 265 Networking Tools .296 PART II 12 13 14 15 16 17 18 UNIX for the Programmer 329 Filtering and Programming with awk 331 Shell Programming .359 perl—The Master Manipulator 404 Introducing C .440 Program Development Tools .485 Systems Programming I—Files 535 Systems Programming II—Process Control 570 PART III 19 System Administration 607 System Administration 609 v das76205_fm_i-xI.indd v 12/14/11 4:36 PM Your UNIX/Linux: The Ultimate Guide vi A B C D E F G H I das76205_fm_i-xI.indd vi Appendixes The C Shell—Programming Constructs .643 The Korn and Bash Shells—Exclusive Programming Constructs 650 The GNU emacs Editor 657 vi/vim Command Reference .686 The Regular Expression Superset 693 The HOWTO .696 The ASCII Character Set 701 Glossary 705 Solutions to Self-Test Questions 728 12/14/11 4:36 PM Contents List of Tables xxix Preface xxxiii PART I UNIX for the User .1 Chapter Introducing UNIX .3 1.1 The Operating System 1.2 The UNIX Operating System .5 1.3 Knowing Your Machine 1.4 The System Administrator 1.5 Logging In and Out 1.5.1 Logging In 1.5.2 Logging Out 1.6 A Hands-On Session 1.6.1 System Information with date and who 10 1.6.2 Viewing Processes with ps .10 1.6.3 Handling Files .11 1.6.4 Handling Directories .12 1.7 How It All Clicked 14 1.7.1 Berkeley: The Second School 15 1.7.2 UNIX Gets Fragmented 15 1.7.3 The Internet 16 1.7.4 The Windows Threat .16 1.8 POSIX and the Single UNIX Specification 16 1.9 Linux and GNU .17 1.10 The UNIX Architecture 17 1.10.1 Division of Labor: Kernel and Shell .18 vii das76205_fm_i-xI.indd vii 12/14/11 4:36 PM viii Your UNIX/Linux: The Ultimate Guide 1.10.2 The File and Process 19 1.10.3 The System Calls .19 1.11 Features of UNIX 20 1.11.1 A Multiuser System .20 1.11.2 A Multitasking System Too 20 1.11.3 A Repository of Applications 21 1.11.4 The Building-Block Approach .21 1.11.5 Pattern Matching .21 1.11.6 Programming Facility 21 1.11.7 Documentation 22 Summary 22 Self-Test 23 Exercises .24 Chapter Becoming Familiar with UNIX Commands 25 2.1 Command Basics 26 2.1.1 The PATH: Locating Commands 26 2.1.2 Where Is the Command? .27 2.2 Command Structure .28 2.3 Flexibility of Command Usage 30 2.4 man: On-Line Help 31 2.4.1 Navigation and Search 33 2.4.2 Further Help with man -k and man -f 33 2.5 The man Documentation 34 2.5.1 Understanding a man Page 35 2.5.2 Using man to Understand man 35 2.6 echo: Displaying Messages 38 2.7 printf: Alternative to echo .39 2.8 script: Recording Your Session 40 2.9 Using Email with mailx 40 2.9.1 Sending Mail 41 2.9.2 Receiving Mail 41 2.9.3 mailx Internal Commands 42 2.10 passwd: Changing Your Password 44 2.11 uname: Your Machine’s Name and Operating System 44 2.12 who: Know the Users .45 das76205_fm_i-xI.indd viii 12/14/11 4:36 PM Contents ix 2.13 date: Displaying the System Date 46 2.14 stty: When Things Go Wrong 47 2.14.1 Changing the Settings .48 2.15 The X Window System 49 2.15.1 The Terminal Emulator 50 2.15.2 The File Manager 51 Summary 52 Self-Test 53 Exercises .54 Chapter The File System 56 3.1 The File 57 3.1.1 Ordinary (Regular) File 57 3.1.2 Directory File 58 3.1.3 Device File 58 3.2 What’s in a (File)name? 58 3.3 The File System Hierarchy 59 3.4 The UNIX File System .60 3.5 Using Absolute Pathnames with Commands 61 3.6 The HOME Variable and ~: The Home Directory 62 3.7 pwd and cd: Navigating the File System 62 3.8 Relative Pathnames ( and ) 64 3.9 mkdir: Making Directories .66 3.10 rmdir: Removing Directories 67 3.11 ls: Listing Files 68 3.11.1 ls Options 69 3.12 cp: Copying Files 71 3.12.1 cp Options 72 3.13 mv: Renaming Files 73 3.14 rm: Deleting Files 73 3.14.1 rm Options 74 3.15 cat: Displaying and Concatenating Files .76 3.16 more: The UNIX Pager 76 3.17 pico: Rudimentary Text Editing 78 3.17.1 Navigation 79 3.17.2 Text Editing 80 das76205_fm_i-xI.indd ix 12/14/11 4:36 PM Your UNIX/Linux: The Ultimate Guide 314 11.9.1 Remote Login and Command Execution (ssh and slogin) SSH features two commands, ssh and slogin, both derived from the Berkeley tools (rsh and rlogin) for remote login If you have previously run ssh-agent and ssh-add to save your private key in memory, then you can log in without supplying a passphrase: $ ssh mercury Last login: Fri Nov 28 10:45:23 2003 from saturn.heavens.com $ _ However, if you have not used ssh-agent and ssh-add, then ssh will prompt for the passphrase: $ ssh mercury Enter passphrase for key ‘/home/sumit/.ssh/id_rsa’: Using the -l (el) option, you can also log in using a different user-id, provided the other user has permitted you to so by saving your public key in ~/.ssh/authorized_keys on the server: ssh -l charlie mercury The slogin command behaves similarly and also supports the -l option Sometimes, you may want to run a remote command without logging in ssh can that too Simply use the hostname and command name as arguments: ssh saturn ls -l ls -l executed on host saturn Though ls is run remotely, the output is seen on your terminal To save this output in the remote machine as well, escape the >: ssh saturn ls -l \> dir.lst > interpreted remotely If you use wild cards that have to be interpreted remotely, you need to quote or escape them too so that your local shell doesn’t interfere 11.9.2 File Transfer with sftp and scp The sftp command is similar to ftp except that it has a restricted set of internal commands Log in with or without a passphrase: $ sftp mercury Connecting to mercury sftp> _ The sftp prompt Invoke help, and you’ll find that it uses an “l” (el) prefix to run a command on the local machine This prefix was seen only in the lcd command in ftp, but here you have to use lpwd, lls, and lmkdir to run pwd, ls, and mkdir on the local machine Here’s how you use sftp to upload a file: sftp> cd workc sftp> pwd das76205_Ch11_296-328.indd 314 12/13/11 10:47 AM Chapter 11: Networking Tools 315 Remote working directory: /home/sumit/workc sftp> lpwd Local working directory: /home/sumit sftp> lcd personal/workc sftp> lpwd Local working directory: /home/sumit/personal/workc sftp> put shell.c Uploading shell.c to /home/sumit/workc/shell.c sftp> quit Even though the feature is not yet documented, you can use the mput and mget commands in the same way they are used in ftp scp has one advantage over both ftp and sftp: It can copy subdirectory structures scp accesses a remote file as hostname: filename This is how a file is copied in both directions: scp saturn:/home/henry/shell.c shell.c scp shell.c saturn:/home/henry/ From remote to local From local to remote If the file has to be copied from the user’s home directory, then you can shorten the command line further You can also use wild cards to retrieve multiple files: scp henry@saturn:shell.c scp henry@saturn:”*”.c For copying a directory structure, use the -r option Both of the following commands copy henry’s home directory tree from saturn to the local machine: scp -r saturn:/home/henry scp -r henry@saturn: Two dots Shell programmers can now use the scp command in shell scripts 11.10 Internet Mail Electronic mail was first discussed in Section 2.9, and the mailx command was used to move mail between users on the same host Mail programs generally need no special configuration to deliver mail in these single-host situations A user’s email address is simply her username On the Internet, email addresses take one of these two forms, both of which include the domain name: romeo@heavens.com juliet floyd Generally, we don’t see FQDNs (that include the hostname) in email addresses Here, romeo could have an account on saturn and juliet could have hers on mercury Yet, the email address of both hide the hostname Note that juliet’s email address is embedded within < and > On the Internet, we use email addresses of this form, though the minimal form also works The full name is not used for routing the mail, only the actual email address das76205_Ch11_296-328.indd 315 12/13/11 10:47 AM 316 Your UNIX/Linux: The Ultimate Guide The mechanism used to move mail on the Internet is a little complex Unlike TELNET and SSH, which work within a simple client-server framework, Internet mail handling requires the work of at least three agencies: • Mail user agent (MUA)—For reading the mailbox and sending mail • Mail transport agent (MTA)—For transporting mail between machines • Mail delivery agent (MDA)—For delivering mail to the recipients’ mailboxes In this three-tier arrangement, the mail user agent (MUA) like mailx or pine acts as the user’s frontend The MUA reads incoming mail from the mailbox and hands over outgoing mail to the mail transport agent (MTA) The MTA also has two functions: it both sends and receives mail At the sending end, the MTA identifies the recipient’s address and delivers the message directly to the MTA at the other end At the receiving end, the MTA passes on mail to the mail delivery agent (MDA) Both of these functions are handled universally by the Simple Mail Transfer Protocol (SMTP) The MTA doesn’t deliver mail It’s the MDA that accepts mail from the receiving MTA and delivers it to the actual user’s mailbox This is handled by separate programs like /usr/lib/mail.local on Solaris and procmail on Linux A fourth tier comes in when the user’s host connects to the mail server intermittently This is the case with dialup lines In this arrangement, users typically use their ISP’s facilities to handle their mail The ISP stores the user’s mail on their server, and the user fetches the mail using a separate program There are two protocols in use today for fetching mail—Post Office Protocol (POP3) and Internet Message Access Protocol (IMAP) Most character-based clients like mailx and pine can only view mail that has been delivered to the host on which they are running; they can’t retrieve mail from a POP/IMAP server However, if you compose and receive mail on your own workstation using a GUI client like Mozilla Thunderbird, then you need to specify the following parameters as part of your client’s setup: • The outgoing SMTP server • The incoming server that may use the POP3 or IMAP protocol • Your user-id on these servers If your own workstation is set up to act as a mail server, then you can use the generic name localhost to signify your own hostname or 127.0.0.1 if you prefer to specify the IP address DNS Identifies the Mail Server Apart from performing FQDN-address resolution, DNS also specifies the mail servers (called mail exchangers) meant to handle mail for that domain When a message is addressed to juliet floyd , the resolver of the sending host contacts its own DNS server to obtain the IP address of the mail server for heavens.com (which could be, say, mail.heavens.com) The MTA of the sending host then transfers the mail to the MTA of mail.heavens.com das76205_Ch11_296-328.indd 316 12/13/11 10:47 AM Chapter 11: Networking Tools 11.10.1 317 ~/.signature and ~/.forward: Two Important Files Most mailers make use of the signature facility to append some static text to every outgoing message This text is often the sender’s personal details that are saved in $HOME/.signature The contents of this file are attached to the user’s outgoing messages, provided the MUA is configured properly Some mailers like Mozilla also use the signature file, but it is located differently When you travel, you can consider the automatic forwarding facility to redirect all of your incoming messages to another address Simply place the address where you want all mail to be forwarded to in $HOME/.forward If romeo’s forward contains an entry like this: romeo@oldstamps.com the local MTA forwards the mail to the mail server of the oldstamps.com domain without delivering it to romeo’s local mailbox No mailer needs to be configured to use this facility because forwarding through forward is a feature of sendmail—the program that uses SMTP and delivers most of our mail on the Internet Caution 11.11 A problem arises when you forward mail with forward to another host and then set up a reverse forwarding facility there to redirect it back This situation can occur for mobile users Forwarding at both ends creates a loop, and your message never gets delivered It shuttles to and fro before sendmail intervenes and breaks the loop MIME: Handling Binary Attachments in Mail Every mail message consists of several lines of header information Some of them are inserted by the MUA and the others by the MTA A typical message shows at least the first four of the following fields: Subject: creating animations in macromedia director from GIF89a images Date: Fri, 08 Nov 2002 15:42:38 +0530 From: joe winter To: heinz@xs4all.nl Cc: psaha@earthlink.net The message body follows next, preceded by a blank line The body contains mostly text, but it can also contain attachments, which are held as a single multipart message The original SMTP protocol had two limitations that disallowed the mailing of binary attachments: • Only 7-bit ASCII characters could form a mail message • The line length could not exceed 1000 characters The Multipurpose Internet Mail Extensions (MIME) standard addresses these issues MIME extends the definition of mail to include binary files and multiple data formats in a single message MIME also imposes no restriction on line length Binary attachments das76205_Ch11_296-328.indd 317 12/13/11 10:47 AM Your UNIX/Linux: The Ultimate Guide 318 are encoded (converted to printable characters) so they can be saved in text format in the user’s mailbox But the MIME standard requires two additional headers to be sent with the message: Content-Type: application/pdf; name=”interior.pdf” Content-Transfer-Encoding: base64 The Content-Type: header defines the type/subtype of the data following the header Here, pdf is a subtype of application The Content-Transfer-Encoding: header defines the encoding techniques used on the data We have here a PDF document as an attachment having application/pdf as its content type and encoded using the base64 technique These two headers provide the necessary information that would enable a MIMEcompliant mail application at the other end to decode the content Two configuration files play a vital role here: mime.types on the sender’s side and mailcap on the receiver’s side When you attach a PDF document to an outgoing message, your MUA looks up the file’s extension in mime.types to determine the Content-Type: header Here’s a sample entry for a PDF file (extension: pdf): application/pdf pdf The MUA sets the content type for the PDF portion of the message to application/pdf At the receiving end, the MUA may not have the capability to handle this content type It then looks up the file mailcap for the helper application (an external program) that is specified for this content type Here’s an entry from this file: application/pdf; acroread %s Note the delimiter is ; This entry directs the MUA to call up the Acrobat Reader (the executable acroread) to view the PDF document Many UNIX systems maintain a systemwide mailcap database in /etc, but many MUAs (like Netscape Messenger) maintain their own If mailcap doesn’t specify a helper application for a content type, then the MUA will seek your approval for saving the file to disk Even though MIME was designed to deliver multimedia attachments with mail messages, the standard applies equally well to newsgroup messages and Web resources We’ll revisit MIME when we discuss HTTP Note 11.12 The sender looks up mime.types to identify the content type to be inserted in the mail header The receiver looks at mailcap to identify the helper application that can handle the attachment Using X Window on a TCP/IP Network The X Window system that was first discussed in Section 2.15 provides UNIX with a graphical user interface (GUI) X was also built to run in a network, but using a reversed client-server paradigm Its architecture places the responsibility of handling the display on the server, while the application itself runs as a client The server in X is called das76205_Ch11_296-328.indd 318 12/13/11 10:47 AM Chapter 11: Networking Tools 319 the display, which comprises the screen, terminal, and mouse X enables you to run a graphical program (client) on a remote machine and have its display on the local one (and vice versa) 11.12.1 The Display You may decide to run a client on a remote machine, possibly because of its superior computing power or because it’s not available on your machine Imagine that you want to run the xcalc program that is available on a remote machine (uranus) However, the display of the program must appear on your local machine (saturn) so you can input data from your keyboard There are two things that you have to ensure before you can make that happen: • The server (on your machine, saturn) must enable others to write to its display This is done by using the xhost command on the local machine • The client program (on the remote machine, uranus) must be directed to write its output to another display This is achieved by setting either the DISPLAY variable or by using the -display option with the client We’ll first use xhost on our local machine to enable any user on uranus to write to our display: $ xhost +uranus uranus being added to access control list You can turn off the xhost setting with the - symbol or enable your display for all machines with +: $ xhost Disables display for others access control enabled, only authorized clients can connect $ xhost + Enables display for others access control disabled, clients can connect from any host You can now run xcalc on uranus by logging on to the host using telnet or ssh You then have two ways of running an X client on that host, and they are considered in Sections 11.12.2 and 11.12.3 11.12.2 Using the DISPLAY variable X uses the DISPLAY shell variable to determine where the output of an X client should be displayed After you have logged in to uranus, define and export the DISPLAY variable at the shell prompt of a terminal emulator (like xterm or dtterm): DISPLAY=saturn:0.0 export DISPLAY uranus must be able to access saturn by name Else use the IP address Here, saturn is the hostname, :0 is the instance of the X server program, and is the screen number of the display The hostname should either be present in /etc/hosts or be accessible using DNS If it is not, then you have to use the IP address in place of the hostname das76205_Ch11_296-328.indd 319 12/13/11 10:47 AM Your UNIX/Linux: The Ultimate Guide 320 The preceding setting signifies that any X client that will subsequently be run on uranus will use the display of saturn, rather than its own Now you can run the program xcalc on uranus: xcalc & You executed xcalc on a remote machine, and the calculator pops up on your local machine! 11.12.3 Using the -display Option The other technique is to use the -display option offered by every X client The parameter to this option is the complete display name—the same value assigned to DISPLAY: xcalc -display saturn:0.0 & Depending on the system you are using, the -display option may override any previous DISPLAY setting If that happens on your system (as in Linux), you don’t need to use the -display option any more to run client programs from this emulator Tip If you have a number of clients to run on a remote machine with the display on your local one, then it is preferable to use DISPLAY, which needs to be set only once Many Linux systems set DISPLAY automatically when you log in, so you may not need to set it at all! 11.13 HTTP and the World Wide Web The World Wide Web was originally conceived by Tim Berners-Lee at CERN in Switzerland as a simple mechanism for interconnecting documents It quickly went beyond the original vision of its creator, and today it functions as a “one-stop shop” for practically everything that’s discussed in this chapter The Web kept the traditional Internet services (email, FTP, and Net News) alive, but made obsolete its immediate ancestors, Archie and Gopher Even though the Web appears to be a conglomeration of multiple services, it works within the framework of the simple client-server model Web service uses the Hyper Text Transfer Protocol (HTTP), and Web servers, also known as HTTP servers, listen for requests at port 80 If ps -e shows httpd running, then your host is a Web server The Web’s access (client) tool is called the browser A Web browser fetches a document (or any resource) residing on Web servers and formats it using the formatting instructions provided in the document itself It also displays pictures if they are in GIF, JPEG, or PNG formats If there’s a format it can’t understand, it will call up a plugin or a helper application (11.11) The World Wide Web is indeed a “web”—a vast collection of hypertext (or hyperlinked) documents that are linked to one another This linkage is based on the principle that if a resource is available on one server, then it makes no sense to have it on another These links are specified by Uniform Resource Locators (URLs) In this das76205_Ch11_296-328.indd 320 12/13/11 10:47 AM Chapter 11: Networking Tools 321 way, the user “wanders and roams” without needing to know where she is, and initiates a new connection with a simple keystroke or a mouse click Web documents are written in the Hyper Text Markup Language (HTML), a textbased portable language HTML can specify the text attributes that should appear on the display (like bold, red in color, etc.), but its real power lies in its hypertext capability: HTML text contains hypertext links to other Web pages Activating a link can take you to another place in the same document, another document on the same server, or any page anywhere on the Internet Text and pictures can also point to each other 11.13.1 The Uniform Resource Locator (URL) A resource is described by a Uniform Resource Locator (URL) or Uniform Resource Identifier (URI)—a form of addressing that combines the FQDN of the site and the pathname of the resource The simplest URL is one that specifies only the FQDN of the server, and is entered through the URL window of the browser: http://www.oracle.com This displays the home page of Oracle’s site Web servers are often configured to send the file index.html when you specify the FQDN as above The home page shows a number of hyperlinks in the form of underlined text Clicking on a link fetches a different page, and your URL could change to something like this: http://www.oracle.com/technetwork/articles/java/index.html The URL syntax ranges from the simple to the complex, but in general, a URL is a combination of three or four things: • The protocol (usually http:// ) used in transferring the resource A Web browser supports other protocols, so you can use ftp:// to transfer a file HTTP is the default protocol, so you may drop the protocol prefix from the URL • The port number, which is not usually specified if the server uses the default port, 80 The preceding URL is equivalent to http://www.oracle.com:80/technetwork/articles/java/index.html Note the colon before the port number • The FQDN of the host (here, www.oracle.com) • The pathname of the resource (here, /technetwork/articles/java/index.html ) This need not always be a resource to fetch, but could be a program to run on the server The Web has a strong UNIX tradition, so frontslashes are the rule Like FTP servers, Web servers also have their own root directory, which is distinctly different from the file system’s root In other words /technetwork is not under the system’s root Note das76205_Ch11_296-328.indd 321 The URL or URI string is not fully case-insensitive The FQDN is case-insensitive all right, but whether the pathname is case-insensitive or not depends on the server’s operating system 12/13/11 10:47 AM Your UNIX/Linux: The Ultimate Guide 322 UNIX is case-sensitive, so if you have seen the pathname /Docs/index.html, then enter it just that way 11.13.2 HTTP: The Protocol of the Web Like the other Internet services, HTTP has separate client and server components A Web page typically contains links to many resources that may be distributed across multiple servers The client requests the server for a document, and the server responds by sending it The client then extracts the URLs of the other resources from the document and then makes separate requests for each resource that has to be fetched to complete the page display The life cycle of a connection using HTTP/1.1 is as follows: The client contacts the server and opens a connection at port number 80 The client requests the Web server for some service This service may be to ask for a document or post some form data back to the server The request consists of a request header followed by the data sent by the client The server now sends a response, which consists of a response header followed by data The server waits for more requests and finally closes the connection On older HTTP/1.0 servers, the server would close the connection after each request Most Web servers today use HTTP/1.1 and its Keep-Alive feature, which makes connections persistent This implies that if a Web page contains five graphic files, a single connection can fetch them using Keep-Alive Without Keep-Alive, six connections would be needed The protocol is also stateless in that each connection is unaware of the other even though they took place sequentially The server’s response header describes the type of data sent from the server HTTP uses the MIME feature that was first used in email for specifying the content type of the data that is to follow For HTML documents, this is text/html 11.13.3 Running External Programs The previous URL specifications referred to static resources—files that reside on a Web server However, a lot of content on the Web is generated dynamically; i.e., it doesn’t exist as files on the server Take for instance this URL, which specifies a search for the string unix system calls on the Teoma site: http://s.teoma.com/search?q=unix+system+calls&qcat=1&qsrc=1 Here, the string search following the FQDN is not the filename of a Web page, but of a program to be run on the server The remaining string following the ? symbol comprises the data to be used by this program, suitably encoded to avoid conflict with the characters used in the rest of the URL string A Web server has no capacity to run these programs The server passes on the request to a Common Gateway Interface (CGI), which makes arrangements to execute the program and returns its standard output to the Web server for onward transmission back to the client We call them CGI programs, and such programs could be written in any language perl is the language of choice for CGI das76205_Ch11_296-328.indd 322 12/13/11 10:47 AM Chapter 11: Networking Tools 323 because of its parsing capabilities using regular expressions However, Java servlets are increasingly being used in this domain 11.13.4 HTML: The Language of Web Pages Web pages are written in HTML Even though the HTML acronym expands to Hyper Text Markup Language, it’s not really a programming language like C, which produces binary executables from text sources Rather, HTML uses tags to “mark up” text It owes its origin to the nroff/troff suite of UNIX systems that used some of these tags several years before the Web was born A few sample lines from a Web page show the use of these tags: Perl: Larry Wall’s Brainchild perl is an interpretive language and is probably the best language yet available for text manipulation It was created by Larry Wall, and made freely available to the world You don’t have to pay for using perl, It’s distributed under the GNU General Public License, which means that no one can impose any restrictions on its distribution You can know more about perl by visiting the Perl site Each tag begins with a < and ends with a >, and most tags have some formatting capability For instance, and its closing counterpart, , serve to boldface text In this HTML source, the word perl appears twice in boldface as shown in Fig 11.2 The browser ignores extra spaces and blank lines, and combines multiple adjacent spaces in a single space Two tags provide the actual hyptertext capability— and Both of them take on attributes in the form attribute=value The tag and its SRC attribute are used to specify the URL of a graphic The browser fetches the graphic file from the server (which could be a different one) and displays it inline within the Web page Here, the tag places the picture of a pearl on the page The anchor tag, , and the HREF attribute behave in a similar manner except that they allow you to click on a section of text or an image to fetch another resource Here, the words the Perl site appear underlined, indicating a hyperlink Clicking on it replaces the current page with the home page of www.perl.org Because HTML documents are text files, they are portable, and you can view them on any browser on any machine running any operating system They are also small in size and thus are ideally suited for use on the Web, where network bandwidth is often a constraint 11.13.5 das76205_Ch11_296-328.indd 323 The Web Browser The Web browser is the HTTP client It accepts a URL either from the URL window or from a bookmark, and fetches the resource from the server If the document contains tags, the browser fetches the images the tags link to—using 12/13/11 10:47 AM Your UNIX/Linux: The Ultimate Guide 324 FIGURE 11.2 An HTML Web Page a single Keep-Alive connection, wherever possible Every browser is also expected to offer these features: • Step back and forth through documents viewed in a session • Save HTML files (and graphics) to the local machine • Bookmark important URLs so they can be fetched later without actually entering the URL • Support other application protocols like FTP and TELNET • Automatically invoke helper applications and special software (plugins) when encountering a file format it can’t handle Like email clients, the earliest Web browsers were character-based, and the lynx browser remained popular until the advent of graphics and X Window Mozilla Firefox, Konqueror, and Opera are the most popular browsers for UNIX and Linux systems today 11.14 Multimedia on the Web: MIME Revisited Web documents today feature a variety of multimedia objects like Java applets, RealAudio, RealVideo, and Shockwave technology MIME technology (11.11) also applies to multimedia files on the Web However, these files are sent by Web servers not as multipart messages but as independent files The server sends the content type to the das76205_Ch11_296-328.indd 324 12/13/11 10:47 AM Chapter 11: Networking Tools 325 client before it sends the file It does this by looking up mime.types, which associates the content type with the file’s extension, as shown here for a PDF document: type=application/acrobat exts=pdf application/pdf pdf Solaris Linux When a browser encounters an unfamiliar data format, it first sees whether there is a plugin in its arsenal A plugin is a piece of software installed (“plugged”) in the browser It is normally small and has the minimal features required for simple viewing (or, in case of audio and video, playing) You can’t invoke a plugin separately as you can call up a helper application (explained next) like Acrobat Reader When a file is viewed with a plugin, it appears inline with the HTML text, and not in a separate window If the browser is not able to locate a plugin for a specific content type, it looks up mailcap to determine the helper application This is a separate standalone application that can also be invoked separately from the UNIX command line We saw one entry in this file in Section 11.11 that specified acroread for application/pdf Unlike in Windows, Firefox doesn’t have this file configured well, so you’ll have to fill it up yourself SUMMARY TCP/IP is a suite of protocols that connects heterogeneous machines in a network It splits data into packets and ensures reliable transmission with full error control Packets pass through routers to reach their destination A host is represented by a unique hostname and a unique IP address comprising four dot-separated octets A host can be accessed both by its IP address and hostname, but TCP/IP packets contain only IP addresses The hostname-IP address translation is performed by /etc/hosts or the Domain Name System (DNS) The hosts file is maintained on all machines of a network DNS understands a host by its fully qualified domain name (FQDN) and distributes the mappings across a number of name servers The resolver queries the hosts file or DNS to perform the translation TCP/IP works in the client-server model Server programs are known as daemons, which run in the background and listen for requests at certain ports telnet is used to run commands on a remote machine and display the output on the local machine ftp transfers files between two hosts You can upload one or more files (put and mput) or download them (get and mget) Anonymous FTP lets you download files from the Internet The secure shell is more secure than telnet and ftp as it encrypts the entire session, including the password It uses a symmetric key for encryption of bulk data, but uses asymmetric keys (public and private) for host and user authentication and key distribution You can log in in a secure manner (ssh and slogin), transfer files (scp and sftp), and run a command remotely (ssh) Internet mail is handled by three agencies You read and compose mail using a Mail User Agent (MUA) The Mail Transport Agent (MTA) transports mail to the MTA at the receiving end using the Simple Mail Transfer Protocol (SMTP) The Mail Delivery Agent (MDA) delivers the mail to the user’s mailbox das76205_Ch11_296-328.indd 325 12/13/11 10:47 AM Your UNIX/Linux: The Ultimate Guide 326 The Web works on the Hyper Text Transfer Protocol (HTTP) at port 80 Web documents written in the Hyper Text Markup Language use hypertext to link one document with another resource An HTML document is cross-platform and can be viewed in any environment The Uniform Resource Locator (URL) combines the FQDN of the site with a pathname It can point to a static resource like a file or a program to be run, using the Common Gateway Interface (CGI) perl is the language of choice for CGI programming The Multipurpose Internet Mail Extensions (MIME) standard enables transmission of binary data in both email and HTTP The Content-Type: and Content-Transfer-Encoding: headers together define the type of data and encoding techniques used The file mime.types associates the content type with a file’s extension, and mailcap specifies the helper application that will handle a specific content type SELF-TEST 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10 11.11 11.12 11.13 11.14 11.15 11.16 das76205_Ch11_296-328.indd 326 Why is TCP termed a reliable protocol? What is the significance of the port number? How will you find out the port number finger uses? Why are the TELNET and FTP services increasingly being disabled on most networks? What are they being replaced with? How can you be sure whether you are working on the local machine or have used telnet or ssh to log on to a remote machine? You copied a graphics file with ftp and the file appears to be corrupted What could be the reason? With which command you upload files to an anonymous FTP site? What is a brute force attack? Why does the security of data mainly depend on the size of the key? To send a large volume of data securely over a network connection, what form of encryption would you adopt? What is the difference between a password and a passphrase? Why is it necessary to have a passphrase? Using scp, how will you noninteractively copy all files from juliet’s home directory on host saturn without knowing the absolute pathname of her home directory? What does this command do? ssh jupiter date \> date How does X solve the problem of running the same program on different displays with different characteristics? Can an X client like xterm running on a Solaris machine display its output on a HP-UX machine? What is the problem with /etc/hosts? Name three top-level domains that were added to the Internet namespace in the year 2000 Is the domain name WWW.suse.COm valid? Explain the significance of the MUA and MTA Whom does the MTA hand over mail to? 12/13/11 10:47 AM Chapter 11: Networking Tools 327 11.17 How are binary files included in mail messages even though SMTP handles only 7-bit data? Name the two mail headers that play an important role here 11.18 The browser can display three types of images without needing external help What are they? 11.19 What is hypertext? Is it confined to text only? 11.20 What is HTTP? Which port number does it use? 11.21 What are CGI programs? How are they invoked? 11.22 How you access the home page of the Web server running on your own machine? EXERCISES 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10 11.11 11.12 11.13 11.14 11.15 11.16 11.17 11.18 11.19 das76205_Ch11_296-328.indd 327 How is a TCP/IP network different from a telephone network? What is an FQDN? Why are hostnames not used on the Internet, but only FQDNs? Describe the role of the resolver when handling (i) simple hostnames (ii) FQDNs Name three important features of DNS What advantages does DNS have over the hosts file? Explain the role of a name server What does a name server if it can’t handle an FQDN? When you change your local directory from inside ftp, will the changed directory still be in place after you quit ftp, and why or why not? When A sends data to B over a network connection using public key cryptography, how does A achieve the goals of (i) authentication, (ii) confidentiality? Public key cryptography is more suitable for key distribution than bulk data encryption Explain how you can use this mechanism to distribute a symmetric key To use SSH, why does a host also need to have a public and private key? Explain how you can generate a public/private key pair for yourself Explain how the ssh-agent and ssh-add programs enable noninteractive logins Cite two reasons why scp is preferable to ftp How is the client-server mechanism in X different from others? How can romeo running Netscape on his machine saturn write its output to juliet’s display on a remote machine uranus? Do both users need to run X? Why is the DISPLAY variable more convenient to use than the -display option? Explain how the general mail handling scheme changes when a user connects to the mail server over a dialup line Explain the significance of each word in the acronym URL What happens if you leave out the port number in the URL? Why is HTTP called a stateless protocol? What is meant by the Keep-Alive feature? Why is the HTML formate especially suitable for Web documents? 12/13/11 10:47 AM 328 Your UNIX/Linux: The Ultimate Guide 11.20 Can you use WWW.PLANETS.COM/CATALOG.HTML instead of www.planets.com/catalog.html as the URL? 11.21 To download a Web page with 10 graphics, how many connections are required in (i) HTTP 1.0, (ii) HTTP 1.1? 11.22 If a browser passes data from an HTML form to the server, how does the server handle the data? 11.23 What is a helper application, and how does it differ from a plugin? Explain the role of the files mime.types and mailcap when using a helper application 11.24 What is MIME? How are the limitations of SMTP in handling mail attachments overcome by MIME? das76205_Ch11_296-328.indd 328 12/13/11 10:47 AM ... . 317 11 .12 Using X Window on a TCP/IP Network . 318 11 .12 .1 The Display 319 11 .12 .2 Using the DISPLAY variable . 319 11 .12 .3 Using the -display Option .320 11 .13 HTTP... 514 16 .11 The Concurrent Version System (CVS) 517 16 .11 .1 Setting Up the Repository 518 16 .11 .2 Importing Files 518 16 .11 .3 Checking Out 519 16 .11 .4 Editing... 17 1. 10 .1 Division of Labor: Kernel and Shell .18 vii das76205_fm_i-xI.indd vii 12 /14 /11 4:36 PM viii Your UNIX/Linux: The Ultimate Guide 1. 10.2 The File and Process 19 1. 10.3 The