Using UNIX

Systems Administration Chapter 4: Using UNIX Page 66 Chapter Using UNIX Introduction A Systems Administrator not only has to look after the computers and the operating system, they also have to be the expert user (or at least a very knowledgeable user) of their systems. When other users have problems where do they go? The documentation? Online help facilities? No, they usually go to the Systems Administrator. Adding to the importance of the material covered in the next few chapters is that a number of the topics introduced here are the foundations on which some of the more complex Systems Administration topics are built. If you don't understand these concepts now you will have problems later on. The following reading aims to start you on the road to becoming an expert UNIX user. Becoming a UNIX guru can only be achieved through a great deal of experience so it is important that you spend time using the commands introduced in this chapter. Other resources Resources explaining the basics about using Linux and UNIX are quite numerous. Some of the other resources that mention similar concepts to this chapter include: · The RedHat Manuals RedHat Linux comes with several manuals. This chapter refers to some of these manuals as a source for more information. · Your Linux Distribution’s Manuals All distributions of Linux come with a considerable volume of documentation and manuals. This chapter refers to some of these manuals as a source for more information. · Online lecture 4 Included on the course website/CD-ROM, online lecture 4 with slides and audio covers complementary material to this chapter. · Linux Installation and Getting Started Guide One of the guides included with the Linux Documentation Project includes some basic information. A copy of the LDP is available on the course website/CD- ROM. What you need to learn This book does not contain all of the concepts and material you will use to learn Linux. Throughout this chapter you will be referred a collection of documents and web pages. It is important that you actually read this material and more importantly you should attempt to practice the commands and practices you learn about. If you don't use it you lose it. Systems Administration Chapter 4: Using UNIX Page 67 In order to be able to really understand the material introduced later in this chapter and to be able to perform the required tasks with a minimum of effort you MUST become familiar with the following: · How to get around the Linux file hierarchy Using the GUI "explorer-like" tools provided by Gnome and KDE are not sufficient. You must be familiar with using commands like cd ls mkdir rm ls cp · How to get the most out of the user interface you are provided with Making use of the GUI, while not a replacement for the command line, will make some tasks easier. · Be able to use simple command line tools I'm repeating it, just in case you ignored it the first time. You MUST BE ABLE to use the simple UNIX commands such as ls, cd, mkdir, rm, cp etc · Be able to use the vi editor As with the UNIX command line, many of you will question why you need to use vi . The simple reason is that it will make your life much easier later in the semester if you learn how to use vi now. · Understand the Linux file permissions system This is especially essential. An understanding of the Linux file permissions system is an absolute necessity for when you move onto the more complex Systems Administration concepts introduced in later chapters. If you don't understand this now you will have major problems later on. · Be able to manipulate and view the processes currently running As with file permissions, the ability to manipulate and view the processes on a Linux box is an essential skill for a Systems Administrator. Introductory UNIX Once upon a time this section used to be quite easy for the majority of people reading this text. Since then the explosion in the exclusive use of Windows and other GUIs means that most people have little or no experience with using the command line. Some of you may not even know what the command line is!!! The command line is the name given to the text-based interface which was common several years ago and is still present in the form of the MS-DOS/command prompt in the Windows world and command-line shells in the UNIX world. This interface uses a process something like this: · computer displays a prompt · user types a command, usually in the format command_name [a list of parameters] (the braces [] indicate parameters are optional) · computer tries to carry out that command and displays any output · computer displays a prompt again (and we loop back to the top) I'm sorry to say but as a Systems Administrator you have to know how to use the command line, even if you’re a Windows Systems Administrator. This means you have to forget about that nice GUI provided by MS Explorer and start to understand the structure of files and directories used by Linux. Systems Administration Chapter 4: Using UNIX Page 68 Why do I need to know the command line? Some possible answers to this question include: · The GUI isn't always available Unlike Windows, a GUI is not a compulsory part of UNIX. This is one of the reasons why a small 386 running Linux can act as the server for a small organisation. It also means that there will be times as a Systems Administrator that you will have to perform tasks without a GUI. Those times are generally when something has broken. You won't have the time to learn how to use the command-line then. Take the time to do it now. · The command line is often more efficient and powerful There are a wide number of tasks which you will have to perform in your computing career that are not suited to using a GUI. These tasks can be done quicker and easier using the command line. · The last reason for those of you studying this course is that your ability to use the command line is assessable. If you don't know how to use it you will lose marks. How do I learn all this stuff? The simple answer is practice. You can't learn this material without experience. First you need to read and understand the material outlined below. Then you must take the time to perform the tasks set and also possible a number of others until you are comfortable with using the command line. Taking the time to do this now will save you time later. Basic UNIX There is a wide range of material on the Internet which will introduce you to the basics of using UNIX. The following is a list of some of those available from the course website/CD-ROM. Please use the resources which best suit you: · Online Lecture 2 Produced for the 1999 offering of the course, this lecture covers using the command line, vi and the file hierarchy. · The Linux Installation and Getting Started Guide This guide was produced as part of the Linux Documentation Project (a mirror of this project is included on the course website/CD-ROM). It has a Linux tutorial which covers much of the basic material. · The Red Hat Linux Getting Started Guide Produced by Red Hat and included on the course website/CD-ROM in PDF and HTML formats, this guide also covers much of the introductory material you will need. The latest version is always available from http://www.redhat.com/ Systems Administration Chapter 4: Using UNIX Page 69 Exercises 4.1. What UNIX commands would you use to: - change to your home directory - display the list of files in the current directory - display my name is fred onto the screen - copy the file tmp.dat from the current directory to the directory data underneath your home directory and after the file has been copied delete it 4.2. What will the following UNIX commands do? Don't execute a UNIX command if you aren't sure what it is going to do. In particular do not try to execute the first command below. rmdir ~ cat /etc/passwd ls / /fred/doc/tmp 4.3. Indicate which of the following paths are full or relative: a. /root/ b. /root/ c. /usr/ /root d. /home/david/ 4.4. Assuming you are currently in the /home/david/tmp/ directory, write the full path of your final location if you perform the following commands a. cd /85321/ b. cd /usr/lib c. cd ~/85321 4.5. Answer the following questions: a. Where would you find the home directory for the root user? b. Where would you store some temporary files? c. Where do you normally find the home directories of "normal" users? d. Assuming you were currently in the directory containing the boot configuration files, how would you change into the directory containing the system configuration files and scripts? UNIX Commands are programs The UNIX commands that have been introduced so far are stored on a UNIX computer as executable files. All the commands on a UNIX system are either stored on the hard-drive as executable files or are understood by a shell (more on these in a later chapter). Most of the standard commands will be stored in standard binary directories such as /bin /usr/bin /usr/local/bin . On my system running RedHat Linux there are over 2200 different files in the directories /bin , /usr/bin and /usr/sbin , which means over 2200 different commands. vi A major task of any user of a computer is editing text files. For a Systems Administrator of a UNIX system, manipulation of text files is a common task because many of the system configuration files are text files. The most common, screen-based UNIX editor is vi . The mention of vi sends shudders through the spines of some people, while other people love it with a passion. vi is difficult to learn, however it is also an extremely powerful editor which can save a Systems Administrator a great deal of time. Systems Administration Chapter 4: Using UNIX Page 70 As you progress through this subject you will need an editor. vi is an anachronistic antique of an editor hated by most people. So why should you use it? Reasons include: · It is very powerful How many editors do you know that can take the first 20 characters of every line in a file and swap them with the second set of 20 characters (something I've had to do)? · It is the only screen editor available on every UNIX system · There will be times when a Systems Administrator cannot use a full screen editor. At times like this you must resort to single line editors like ed and ex . vi grew out of the ex editor and so uses many of the same commands. Learning and using these commands in vi can save you time later on. As a result of all this it is strongly suggested that you use vi wherever possible in studying for this course. Early on you will find using vi a hassle but sticking with it will be worthwhile in the end. An introduction to vi Most people when confronted with vi for the first time are put off by its completely foreign nature and lack of prompts about what to do. vi actually uses a very simple "model of operation". Central to this is the fact that vi is a modal editor. This means vi has a number of different modes and the same action can have completely different meaning in different modes. vi modes vi can be said to have three modes: · command mode This is the default mode vi is in when you start it up. In this mode, most of the keys on the keyboard perform vi commands. For example, hitting the e key during vi command mode moves the cursor onto the next word. Use the list of vi commands in any of the vi command references discussed below to find out more. · insert mode This is the mode in which vi behaves most like other editors. If you hit the k key it will insert k into the current location of the cursor and move the cursor on. · ex mode In ex (sometimes called colon mode) you get to access a range of commands from the ex editor (and you thought vi was hard to use). A common one you will use is :wq which writes/saves the current file and then quits vi ( wq ). vi Transitions Knowing about the vi modes is no good unless you know how to go from one mode to another. For example, you can't actually type anything into a text file you are creating without knowing how to go from command mode to insert mode. Common transitions include: · command to insert A number of vi commands take you from command to insert modes (for example i o O ). Systems Administration Chapter 4: Using UNIX Page 71 · insert to command You'll do this transition when you want to save a file (usually). Hitting the ESC key is usually enough to achieve this. · command to ex Simply hitting the : (colon) key will put you into ex mode. You will know this because the colon will appear at the bottom of the screen. · ex to command Simply hitting enter at the end of an ex command takes you back into command mode. Using vi The section for week 2 in the Link database on the course web site contains a number of resources that introduce you to vi . This includes the 4th online lecture that has a number of slides and examples of using vi . Many “Idiot’s Guides” books about Linux provide useful information on using vi vi provides its own interactive tutorial in the form of the vimtutor command. This is a highly recommended tutorial which will help immensely in understanding what vi can do and how you can control it. An excellent resource to get your hands on is a vi cheat sheet. These sheets list most of the commands and what they do when used within vi . To find one of these simple go to http://www.google.com/ and type “ vi cheat sheet ” into the search field. vi, vim and ^Ms A common problem students have had in the last couple of years is shell scripts that can't run (you'll be getting to shell scripts in a couple of weeks). The problem usually occurs when the student copies a text file created under Windows to Linux. The cause of the problem is that UNIX and Microsoft indicate the end of the line in different ways: · carriage return, line-feed Used by Microsoft operating systems. · line feed Use by Linux. The extra character, the carriage return, causes problems in some situations, for example when you want to run the text file as a shell script. The solution is to remove the extra characters. One method is to use vi . The trouble is that by default vim , the version of vi on Linux, is smart enough to hide the carriage returns. If you have a text file which has carriage returns in it (highly likely if you copied it from a Windows machine), and you want to see the carriage returns, you have to start up vi with the following command: vi -b filename The -b causes vi to work in binary mode and you will now be able to see the carriage returns which look like ^M and appear at the end of each line. Carriage return is Systems Administration Chapter 4: Using UNIX Page 72 actually one character. ^M is the standard UNIX way of representing a single control character. If you try to delete the ^M with the x command you will find that there is only one character. To delete all the carriage returns in a file you can use the following command: 1,$s/^M//g Where you enter this, don't type the ^ character and then the M character. Instead you hold the CONTROL key down and hit the c key and then hit the m key. What this command does should become clear when we talk about regular expressions in a later chapter. Exercises 4.6. Run vimtutor and do the vi tutorial. UNIX commands A UNIX system comes with hundreds of executable commands and programs. Typically each of these programs carries out a particular job and will usually have some obscure and obtuse name that means nothing to the uninitiated. That said, the names of most of these commands actually do make some sort of sense once you have a bit of knowledge. In the following you are introduced to the philosophy and format of UNIX commands. It is also emphasised that there is almost always going to be a UNIX command (or a combination of them) to perform the task you wish to accomplish. You need to become familiar with how to find out about the available commands. Philosophy of UNIX commands There are no set rules about UNIX commands however there is a UNIX philosophy that is used by many of the commands: · small is beautiful UNIX provides the mechanisms to join commands together so commands should do one thing well. · 10 percent of the work solves 90 percent of the problems UNIX was never designed to solve all problems, it was designed to solve most requirements without too much hassle on the programmer's part. · solve the problem, not the machine Commands should ignore any machine specific information and be portable. · solve at the right level, and you will only have to do it once The key to UNIX problem solving is only to do it once, for example pattern matching is only implemented once, in the shell, not in every command. One of the central tenants of the UNIX command philosophy is to provide a flexible, adaptable toolbox approach to solving problems. The idea is not to provide a single large program which does everything. Instead you have small, purpose built commands which can be easily combined to perform much larger tasks… An evolution rather than creation approach to solving problems. Systems Administration Chapter 4: Using UNIX Page 73 UNIX command format UNIX commands all use the following format: command_name [-switches parameter_list] Component Explanation command_name The name of the actual command. Generally this is the name of the executable program that is the command. -switches The - symbol is used to indicate a switch. A switch modifies the operation of a command. parameter_list The list of parameters (or arguments) that the command will operate on. Could be 0, 1 or more parameters. Parameters are separated by white space characters (space, TAB). Table 4.1 UNIX command format Aside: Using square brackets [] around parameters and switches for a command means that they are optional. You will see this standard scheme used throughout all command documentation. Please note: There must be spaces between each component of a UNIX command line. Under MS-DOS it was possible to perform commands like cd/dos to change the current directory into the /dos directory. Under UNIX this command will be interpreted as run the command cd/dos . This means UNIX will normally try to find an executable file called cd/dos . The UNIX shell (the command which interprets the command line and tries to execute commands) uses the space character to tell the difference between the different components of the command line. Under UNIX the command would have to be cd /dos Example commands · ls -l The switch -l is used to modify the action of the ls command so that it displays a long listing of each file. · ls -l /etc/passwd / /var Commands can take multiple parameters. · ls -ld /etc/passwd / /var Multiple switches can also be used. Linux commands take multiple arguments However, space characters must separate the multiple parameters. Exercises 4.7. One of your users has created a file called –tmp (the command cat /etc/passwd > -tmp will do it). They want to delete it but can't. Why might the user have difficulty removing this file? How would you remove it? You may have to refer to the online help section below to find the answer. Systems Administration Chapter 4: Using UNIX Page 74 A command for everything A fairly intelligent and experienced would-be computer professional has just started using UNIX seriously (he was a student in the very first offering of this course). He gets to a stage where he wants to change the name of some files. Being an MS-DOS junkie from way back, what command does he look for? The rename command of course. It doesn't work! "That's a bit silly!” he thinks, "You would think that UNIX would have a rename command." It just so happens that this person has just completed a C programming subject in which one of the assignments was to write a rename command. So he spends the next day trying to write and compile this program. After much toil and trouble he succeeds and follows good administration policy and informs all the other students of this brand new wonderful program he has written. He goes into great detail on how to use the command and all the nice features it includes. They all write back to tell him about the UNIX command mv (the move command) that is the UNIX command equivalent to rename . The moral of the story The moral of this story is that if you want to do something under UNIX, then chances are that there is already a command to do it. All you have to do is work out what it is. Online help UNIX comes with online help called man pages, short references for commands and files on a UNIX system. They are not designed as a means by which newcomers to UNIX can learn the commands. Instead they are a reference to the command to be used by someone who understands the basics of UNIX and UNIX commands. The man pages are divided into different sections. Table 4.2 shows the sections that Linux uses. Different versions of Linux use slightly different sections. Systems Administration Chapter 4: Using UNIX Page 75 Section number Contents 1 User Commands 2 System Calls 3 Library Functions 3c Standard C Library 3s Standard I/O Library 3m Arithmetic Library 3f Fortran Library 3x Special Libraries 4 Special Files 5 File Formats 6 Games 7 Miscellaneous 8 Administration And Privileged Commands Table 4.2 Manual Page Sections Using the manual pages To examine the manual page for a particular command or file, you use the man command. For example if you wanted to examine the man page for the man command you would execute the command man man . Is there a man page for . The command man -k keyword will search for all the manual pages that contain keyword in its synopsis. The commands whatis and apropos perform similar tasks. Rather than search through all the manual pages, Linux maintains a keyword database in the file /var/cache/man/whatis . If at any stage you add new manual pages you should rebuild this database using the makewhatis command. The -K switch (distinct from the –k switch) for the man command forces it to search through all of the manual pages for the word. You should realise that with the size and number of manual pages this operation can take quite a while. If there is a file you wish to find out the purpose of, you might want to try the –f option of the man command or the whatis command. [...]... Describe the contents of section 8 of the manual pages HTML versions of Manual Pages Many websites, including the course website, contain collections of manual pages in HTML format Some UNIX commands There are simply too many UNIX commands for this chapter to introduce all, or even most of them The aim of the following is to show you some of the basic commands that are available To find the remainder you... example of good UNIX commands They do one job well and are designed to be chained together To get the most out of filters you combine them together in long chains of commands How this is achieved will be examined in a later chapter when the concept of I/O redirection is introduced I/O redirection allows you to count the number of people on your computer who have usernames starting with d by using the grep... familiar with the available commands is to: · · look at the filenames in the /bin /usr/bin /usr/local/bin directories These are the “binary” directories which contain the executable programs which are the UNIX commands take the filename and look at the manual page Each of the commands will have a manual page which will explain what the Page 76 command does and how you can use it The commands introduced... and less tail uniq paste grep Basic cut wc Display who is currently on the computer Display a calendar Translate specific characters Count the number of characters, words and lines in a file Table 4.3 UNIX commands Identification Commands who Displays a list of people currently logged onto the computer dinbig:/$ who albert tty1 Feb 5 14:27 whoami Displays who the computer thinks you are currently logged... specific month (the Linux version might not work) bash$ cal 1 2005 January 2005 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Filters Filters are UNIX commands that take input or the contents of a file, modify that content and then display the result on output Later on in this chapter you will be shown how you can combine these filters together... display the last 10 lines of chap1.html display the first 100 bytes of chap1.html display the first 50 lines of chap1.html display the last 100 bytes of chap1.html sort The sort command is used to sort data using a number of different criteria outlined in the following table Switch Result Sort in descending order (default is ascending) -r Sort as numbers (default is as ASCII characters) When sorting numbers... the field delimiter Table 4.4 Switches for the sort command Examples The following examples all work with the /etc/passwd file /etc/passwd is the file that stores information about all the users of a UNIX machine It is a text file with each line divided into seven fields Each field is separated by a : character Use the cat command to view the contents of the file · sort /etc/passwd · sort -r /etc/passwd... is where consecutive lines match exactly sort is often used to get the duplicate lines in a file into consecutive order before passing it to uniq Passing a file from one command to another is achieved using I/O redirection which is explained in a later chapter Examples uniq · uniq names · uniq names uniq.names · uniq -d names remove duplicate lines from the file names and display them on the screen... need to be familiar with regular expressions which are discussed in more detail in a later chapter wc Used to count the number of characters, words and lines in a file By default it displays all three Using the switches -c -w -l will display the number of characters, words and lines respectively bash$ wc /etc/passwd 19 20 697 /etc/passwd bash$ wc -c /etc/passwd 697 /etc/passwd bash$ wc -w /etc/passwd... format: student number, surname, firstname, grade(F,P,C,D,HD), mark, degree code and is available from the course website/CD-ROM at the URL http://infocom.cqu.edu.au/85321/Resources/Lectures/6/results.csv Using previous descriptions in the lecture, this text book and the Linux manual pages, come up with commands to perform the following tasks on this example file: a count the number of students in the class . Systems Administration Chapter 4: Using UNIX Page 66 Chapter Using UNIX Introduction A Systems Administrator not only has to look. to solving problems. Systems Administration Chapter 4: Using UNIX Page 73 UNIX command format UNIX commands all use the following format: command_name