Minimal Perl For UNIX and Linux People 7 potx

252 CHAPTER 8 SCRIPTING TECHNIQUES $answer as undefined This signifies that the variable has been brought into exist- ence, but not yet given a usable value. The solution is to add an additional check using the defined function, like so: (! defined $answer or $answer ne "YES\n" ) and die "\n$0: Hasty resignation averted\n"; This ensures that the program will die if $answer is undefined, and also that $answer won’t be compared to “YES\n” unless it has a defined value. That last prop- erty circumvents the use of a fabricated value in the inequality comparison, and the “uninitialized value” warning that goes with it. With this adjustment, if $answer is undefined, the program can terminate with- out a scary-looking warning disturbing the user. 3 The rule for avoiding the accidental use of undefined values, and triggering the warnings they generate, is this: Always test a value that might be undefined, for being defined, before attempting to use that value. But there is an exception—copying a value, as in $got_switch, never triggers a warn- ing—even when $answer is undefined. That’s because moving undefined values around, as opposed to using them in significant ways, is considered a harmless activity. Tips on using defined The following statement attempts to set $got_switch to a True/False value, accord- ing to whether any (or all) of the script’s switches was provided on the command line: $got_switch=defined $debug or defined $verbose; # WRONG! Here’s the warning it generates: Useless use of defined operator in void context That message arises because the assignment operator (=) has higher precedence than the logical or, causing the statement to be interpreted as if it had been typed like this: 4 ( $got_switch=defined $debug ) or defined $verbose; Perl’s warning tells the programmer that it was useless to include the or defined part, because there’s no way for its result to be used anywhere (i.e., it’s in a void context). As with other problems based on operator precedence, the fix is to add explicit parenthe- ses to indicate which expressions need to be evaluated before others: $got_switch=( defined $debug or defined $verbose ); # Right. 3 Which might result in you being paged at 3 a.m.—prompting you to consider your own resignation! 4 The Minimal Perl approach minimizes precedence problems, but they’ll still crop up with logical op- erators now and then (see “Tips” at the end of section 2.4.5, appendix B, and man perlop). EXPLOITING SCRIPT-ORIENTED FUNCTIONS 253 In many cases, a Perl program ends up terminating by running out of statements to process. But in other cases, the programmer needs to force an earlier exit, which you’ll learn how to do next. 8.1.2 Exiting with exit As in the Shell, the exit command is used to terminate a script—but before doing so, it executes the END block, if there is one (like AWK). Table 8.1 compares the way the Shell and Perl versions of exit behave when they’re invoked without an argument or with a numeric argument from 0 to 255. As indicated in the table, Perl’s exit generally works like that of the Shell, except it uses 0 as the default exit value, rather than the exit value of the last command. Although the languages agree that 0 signifies success, neither has established con- ventions concerning the meanings of other exit values—apart from them all indicating error conditions. This leaves you free to associate 1, for example, with a “required arguments missing” error, and 2 with an “invalid input format” error, if desired. As discussed in section 2.4.4, Perl’s die command provides an alternative to exit for terminating a program. It differs by printing an error message before exiting with the value of 255 (by default), as if you had executed warn "message" and exit 255 . (But remember, in Minimal Perl we use the warn and exit combination rather than die in BEGIN blocks, to avoid the unsightly warning messages about aborted compilations that a die in BEGIN elicits.) The following illustrates proper uses of the exit and die functions in a script that has a BEGIN block, as well as how to specify die’s exit value by setting the “$!” variable, 5 to load the desired value into the parent shell’s "$?" variable: Table 8.1 The exit function Shell Perl Explanation exit exit; With no argument, the Shell’s exit returns the latest value of its "$?" variable to its parent process, to indicate the program’s success or failure. Perl returns 0 by default, to indicate success. a exit 0 exit 0; The argument 0 signifies a successful run of the script to the parent. exit 1-255 exit 1-255; A number in the range 1–255 signifies a failed run of the script to the parent. a. Because it’s justifiably more optimistic than the Shell. 5 Later in this chapter, you’ll learn how to use Perl’s if construct, which is better than the logical and for making the setting of “$!”, and the execution of die, jointly dependent on the success of the matching operator. 254 CHAPTER 8 SCRIPTING TECHNIQUES $ cat massage_data #! /usr/bin/perl –wnl BEGIN { @ARGV == 1 or warn "Usage: $0 filename\n" and exit 1 ; } /^#/ and $ !=2 and die "$0: Comments not allowed in data file\n"; $ massage_data Usage: massage_data filename $ echo $? 1 $ massage_data file # correct invocation; 0 is default exit value $ echo $? 0 $ echo '# comment' | massage_data - # "-" means read from STDIN massage_data: Comments not allowed in data file $ echo $? 2 We’ll look next at another important function shared by the Shell and Perl. 8.1.3 Shifting with shift Both the Shell and Perl have a function called shift, which is used to manage command-line arguments. Its job is to shift argument values leftward relative to the storage locations that hold them, which has the side effect of discarding the original first argument. 6 Figure 8.1 shows how shift affects the allocation of arguments to a Shell script’s positional parameter variables, or to the indices of Perl’s @ARGV array. 6 A common programming technique used with early UNIX shells was to process $1 and then execute shift, and repeat that cycle until every argument had taken a turn as $1. It’s discussed in section10.2.1. Figure 8.1 Effect of shift in the Shell and Perl EXPLOITING SCRIPT-ORIENTED FUNCTIONS 255 As the figure illustrates, after shift is executed in the Shell, the value initially stored in $1 (A) gets discarded, the one in $2 (B) gets relocated to $1, and the one in $3 gets relocated to $2. The same migration of values across storage locations occurs in Perl, except the movement is from $ARGV[1] to $ARGV[0], and so forth. Naturally, the affected Perl variables ( @ARGV and $#ARGV) are updated automatically after shift, just as “ $*”, “$@”, and “$#” are updated in the Shell. Although Perl’s shift provides the same basic functionality as the Shell’s, it also provides two new features, at the expense of losing one standard Shell feature (see table 8.2). The new feature—shown in the table’s second row—is that Perl’s shift returns the value that’s removed from the array, so it can be saved for later access. That allows Perl programmers to write this simple statement: $arg1=shift; # save first arg's value, then remove it from @ARGV where Shell programmers would have to write arg1="$1" # save first arg's value before it's lost forever! shift # now remove it from argument list Another improvement is that Perl’s shift takes an optional argument that specifies the array to be shifted, which the Shell doesn’t support. However, by attaching this new interpretation to shift’s argument, Perl sacrificed the ability to recognize it as a numeric “amount of shifting” specification, which is the meaning shift’s argument has in the Shell. Table 8.2 Using shift and unshift in the Shell and Perl Shell Perl Explanation shift shift; shift removes the leftmost argument and moves any others one position leftward to fill the void. N/A $variable=shift; In Perl, the removed parameter is returned by shift, allowing it to be stored in a variable. shift 2 shift; shift; OR $arg1=shift; $arg2=shift; The Shell’s shift takes an optional numeric argument, indicating the number of values to be shifted away. That effect is achieved in Perl by invoking shift multiple times. N/A shift @any_array; Perl’s shift takes an optional argument of an array name, which specifies the one it should modify instead of the default (normally @ARGV, but @_ if within a subroutine). N/A unshift @array1, @array2; Perl’s unshift reinitializes @array1 to contain the contents of @array2 before the initial contents of @array1. For example, if @array1 in the example contained (a,b) and @array2 contained (1,2), @array1 would end up with(1,2,a,b). 256 CHAPTER 8 SCRIPTING TECHNIQUES Now that you’ve learned how to use defined, shift, and exit in Perl, we’ll use these tools to improve on certain techniques you saw in part 1 and to demonstrate some of their other useful applications. We’ll begin by discussing how they can be used in the pre-processing of script arguments. 8.2 PRE-PROCESSING ARGUMENTS Many kinds of scripts need to pre-process their arguments before they can get on with their work. We’ll cover some typical cases, such as extracting non-filename arguments, filtering out undesirable arguments, and generating arguments automatically. 8.2.1 Accommodating non-filename arguments with implicit loops The greperl script of section 3.13.2 obtains its pattern argument from a command- line switch : greperl -pattern='RE' filename When this invocation format is used with a script having the s option on the she- bang line, Perl automatically assigns RE to the script’s $pattern variable and then discards the switch argument. This approach certainly makes switch-handling scripts easy to write! But what if you want to provide a user interface that feels more natural to the users, based on the interface of the traditional grep? grep 'RE' filename The complication is that filter programs are most conveniently written using the n invocation option, which causes all command-line arguments (except switches) to be treated as filenames—including a grep-like script’s pattern argument: $ perlgrep.bad 'root' /etc/passwd # Hey! "root" is my RE! Can't open root: No such file or directory Don’t despair, because there’s a simple way of fixing this program, based on an under- standing of how the implicit loop works. Specifically, the n option doesn’t start treating arguments as filenames until the implicit input-reading loop starts running, and that doesn’t occur until after the BEGIN block (if present) has finished executing. This means initial non-filename arguments can happily coexist with filenames in the argument list—on one condition: You must remove non-filename arguments from @ARGV in a BEGIN block, so they’ll be gone by the time the input-reading loop starts executing. The following example illustrates the coding for this technique, which isn’t diffi- cult. In fact, all it takes to harvest the pattern argument is a single line; the rest is all error checking: PRE-PROCESSING ARGUMENTS 257 $ cat perlgrep #! /usr/bin/perl -wnl BEGIN { $Usage="Usage: $0 'RE' [file ]"; @ARGV > 0 or warn "$Usage\n" and exit 31; # 31 means no arg $pattern=shift; # Remove arg1 and load into $pattern defined $pattern and $pattern ne "" or warn "$Usage\n" and exit 27; # arg1 undefined, or empty } # Now -n loop takes input from files named in @ARGV, or from STDIN /$pattern/ and print; # if match, print record Here’s a sample run, which shows that this script succeeds where its predecessor perlgrep.bad failed: $ perlgrep 'root' /etc/passwd root:x:0:0:root:/root:/bin/bash The programmer even defined some custom exit codes (see section 8.1.2), which may come in handy sometime: $ perlgrep "$EMPTY" /etc/passwd Usage: perlgrep 'RE' [file ] $ echo $? # Show exit code 27 Once you understand how to code the requisite shift statement(s) in the BEGIN block, it’s easy to write programs that allow initial non-filename arguments to precede filename arguments, which is necessary to emulate the user interface of many tradi- tional Unix commands. But don’t get the idea that perlgrep is the final installment in our series of grep-like programs that are both educational and practical. Not by a long shot! There’s an option-rich preg script lurking at the end of this chapter, waiting to impress you with its versatility. We’ll talk next about some other kinds of pre-processing, such as reordering and removing arguments. 8.2.2 Filtering arguments The filter programs featured in part 1 employ Perl’s AWKish n or p option, to handle filename arguments automatically. That’s nice, but what if you want to exert some influence over that handling—such as processing files in alphanumeric order? As indicated previously, you can do anything you want with a filter-script’s argu- ments, so long as you do it in a BEGIN block. For example, this code is all that’s needed to sort a script’s arguments: 258 CHAPTER 8 SCRIPTING TECHNIQUES BEGIN { @ARGV=sort @ARGV; # rearrange into sorted order } # Normal argument processing starts here It’s no secret that users can’t always be trusted to provide the correct arguments to commands, so a script may want to remove inappropriate arguments. Consider the following invocation of change_file, which was presented in chapter 4: change_file -old='problems' -new='issues' * The purpose of this script is to change occurrences of “problems” to “issues” in the text files whose names are presented as arguments. But of course, the “ *” metacharac- ter doesn’t know that, so if any non-text files reside in the current directory, the script will process them as well. This could lead to trouble, because a binary file might hap- pen to contain the bit sequence that corresponds to the word “problems”—or any other word, for that matter! Imagine the havoc that could ensue if the superuser were to accidentally modify the ls command’s file—or, even worse, the Unix kernel’s file—through such an error! To help us sleep better, the following code silently removes non-text-file argu- ments, on the assumption that the user probably didn’t realize they were included in the first place: BEGIN { @ARGV=grep { -T } @ARGV; # retain only text-file arguments } # Normal argument processing starts here grep selects the text-file (-T; see table 6.1) arguments from @ARGV, and then they’re assigned as the new contents of that array. The resulting effect is as if the unacceptable arguments had never been there. A more informational approach would be to report the filenames that were deleted. This can be accomplished by selecting them with ! -T (which means “non-text files”), storing them in an array for later access, and then printing their names (if any): BEGIN { @non_text=grep { ! –T } @ARGV; # select NON-text-file arguments @non_text and warn "$0: Omitting these non-text-files: @non_text\n"; @ARGV=grep { -T } @ARGV; # retain text-file arguments } # Normal argument processing starts here But an ounce of prevention is still worth at least a pound of cure, so it’s best to free the user from typing arguments wherever possible, as we’ll discuss next. EXECUTING CODE CONDITIONALLY WITH if/else 259 8.2.3 Generating arguments It’s senseless to require a user to painstakingly type in lots of filename arguments— which in turn burdens the programmer with screening out the invalid ones—in cases where the program could generate the appropriate arguments on its own. For example, Uma, a professional icon-designer, needs to copy every regular file in her working directory to a CD before she leaves work. However, the subdirectories of that directory should not be archived. Accordingly, she uses the following code to gen- erate the names of all the (non-hidden) regular files in the current directory that are readable by the current user (that permission is required for her to copy them): BEGIN { # Simulate user supplying all suitable regular # filenames from current directory as arguments @ARGV=grep { -f and -r } <*>; } # Real work of script begins below The <*> expression is a use of the globbing operator (see table 7.14) to generate an initial set of filenames, which are then filtered by grep for the desired attributes. Other expressions commonly used to generate argument lists in Perl (and the Shell) are shown in section 9.3, which will give you additional ideas of what you could plug into a script’s BEGIN block. You can’t always automatically generate the desired arguments for every script, but for those cases where you can, you should keep these techniques in mind. Next, you’ll learn about an important control structure that’s provided in every programming language. We’ve managed without it thus far, due to the ease of using Perl’s logical operators in its place, but now you’ll see how to arrange for conditional execution in a more general way. 8.3 Executing code conditionally with if/else The logical or and logical and operators were adequate to our needs for controlling execution in part 1, where you saw many statements like this one: $pattern or warn "Usage: $0 -pattern='RE' filename\n" and exit 255; However, this technique of using the True/False value of a variable ($pattern) to conditionally execute two functions ( warn and exit) has limitations. Most impor- tant, it doesn’t deal well with cases where a True result should execute one set of state- ments and a False result a different set. So now it’s time to learn about more widely applicable techniques for controlling two-way and multi-way branching. Table 8.3 shows the Shell and Perl syntaxes for two-way branching using if/else, with layouts that are representative of current programming practices. The top panel shows the complete syntax, which includes branches for both the True (“then”) and False ( else) cases of the condition. In 260 CHAPTER 8 SCRIPTING TECHNIQUES both languages, the else branch is optional, allowing that keyword and its associated components to be omitted. The table’s bottom panel shows condensed forms of these control structures, which save space in cases where they’ll fit on one line. We’ll examine a realistic programming example that uses if/else next, and com- pare it to its and/or alternative. 8.3.1 Employing if/else vs. and/or Here’s a code snippet that provides a default argument for a script when it’s invoked without the required one, and terminates with an error message if too many argu- ments are supplied: if (@ARGV == 0) { warn "$0: Using default argument\n"; @ARGV=('King Tut'); } else { if (@ARGV > 1) { # nested if warn "Usage: $0 song_name\n"; exit 255; } } For comparison, here’s an equivalent chunk of code written using the logical and/or approach. It employs a style of indentation that emphasizes the dependency of each subsequent expression on the prior one: @ARGV == 0 and warn "$0: Using default arguments\n" and @ARGV=('King Tut') or @ARGV > 1 and warn "Usage: $0 song_name\n" and exit 255; This example illustrates the folly of stretching the utility of and/or beyond reason- able limits, which makes the code unnecessarily hard to read and maintain. Moreover, Table 8.3 The if/else construct Shell a Perl if condition then commands else commands fi if (condition) { code; } else { code; } if cond; then cmds; else cmds; fi if (cond) { code; } else { code; } a. In the bottom panel, cond stands for condition and cmds stands for commands. EXECUTING CODE CONDITIONALLY WITH if/else 261 matters would get even worse if you needed to parenthesize some groups of expres- sions in order to obtain the desired result. The moral of this comparison is that branching specifications that go beyond the trivial cases are better handled with if/else than with and/or—which of course is why the language provides if/else as an alternative. Perl permits additional if/elses to be included within if and else branches, which is called nesting (as depicted in the left side of table 8.4). However, in cases where tests are performed one after another to select one branch out of several for exe- cution, readability can be enhanced and typing can be minimized by using the elsif contraction for “else { if” (see the table’s right column). Just remember that Perl’s keyword is elsif, not elif, as it is in the Shell. Next, we’ll look at an example of a script that does lots of conditional branching, using both techniques. 8.3.2 Mixing branching techniques: The cd_report script The purpose of cd_report is to let the user select and display input records that rep- resent CDs by matching against the various fields within those records. Through use of the following command-line switches, the user can limit his regexes to match within various portions of a record, and request a report of the average rating for the group of selected CDs: Table 8.4 Nested if/else vs. elsif if/else within else elsif alternative if ( A ) { print 'A case'; } else { # this brace disappears > if ( B ) { print 'B case'; } else { print 'other case'; } } # this brace disappears > if ( A ) { print 'A case'; } elsif ( B ) { print 'B case'; } else { print 'other case'; } • -search='RE' Search for RE anywhere in record • -a='RE' Search for RE in the Artist field • -t='RE' Search for RE in the Title field •-r Report average rating for selected CDs • (default) Print all records, under column headings [...]... interpreter (/bin/sh on Unix) for execution 18 See http://TeachMePerl.com/DQs_in_shell_vs _perl. html for further details EXECUTING OS COMMANDS USING system 275 Table 8.8 The system function Example Explanation system 'command(s)'; command(s) in single quotes are submitted without modification for execution system "command(s)"; command(s) in double quotes are subjected to variable interpolation before being executed... processing In Perl, that protection is always provided, and double quotes aren’t allowed around command interpolations The Shell examples yield all of cmd’s output as one line, whereas the Perl example yields a list of $/ separated records a cmd and cmd2 represent OS commands, var/$var and array/@array Shell /Perl variable names, and function a Perl function name When a Unix shell processes a command substitution,... command interpolation Table 8.6 shows the syntax for typical uses of this facility in the Shell and Perl. 12 12 As indicated in the left column of the table, the Bash and Korn shells simultaneously support an alternative to the back-quote syntax for command substitution, of the form $(command) INTERPOLATING COMMAND OUTPUT INTO SOURCE CODE 269 Table 8.6 Command substitution/interpolation in the Shell and. .. $NO_UL; print $line; # Assemble command in string $command="fmt -$width '$file'"; # e.g., # dashed line # the heading # dashed line "fmt -62 Reuters.txt" $debug and warn "Command is:\n\t$command\n\n" and $command="set -x; $command"; # enable Shell execution trace system $command; # format to fit on screen # show error if necessary ! $? or warn "$0: This command failed: $command\n"; The next step (Line 39)... command’s $? False Perl $o=`command` or warn "message\n"; # warns if output in $o False You can arrange for Perl to do what the Shell does, but because the languages have opposite definitions of True and False, this involves complementing command ’s exit value With this in mind, here’s the Perl counterpart for the previous Shell example: $o=`command` ; ! $? or warn 'message'; # warns if $? False And. .. program handle code that wasn’t present during that program’s initial compilation Examples of tokens that require eval for recognition are keywords, operators, function names, matching and substitution operators, backslashes, quotes, commas, and semicolons Table 8.9 shows the syntax for eval in both the Shell and Perl Table 8.9 The eval function in the Shell and Perl Shell Perl eval 'command' error=$?... that bypasses the tput command to access the Unix terminal information database directly, but it’s much easier to run the Unix command via command interpolation than to use the module INTERPOLATING COMMAND OUTPUT INTO SOURCE CODE 271 Highlighting trailing whitespaces with tput People who do a lot of grepping in their jobs have two things in common: They’re fastidious about properly quoting grep’s pattern... mutually-exclusive switches defined $p and defined $c and warn "$Usage\n\tCan't have -p and -c\n" $X='g'; $ON=$OFF=""; if ($d) { $ON=(`tput smso` and exit 1; # set modifier to perform all substitutions # by default, don't highlight matches # for match-displaying with -d or ""); $OFF=(`tput rmso` or ""); See sections 3.2.3 and 3.3.1 for information on what various grep commands do when given directory arguments... command for news_flash2 was coded (Lines 39, 44) as $command="fmt -$width '$file'"; system $command; 280 CHAPTER 8 SCRIPTING TECHNIQUES rather than more directly as system "fmt -$width '$file'"; Using a variable to hold system’s command has much to recommend it, because it facilitates • printing the text of the command for inspection before running it (Line 41); • showing the text of the command in... running perldoc -f system One of the most useful and powerful services that a script can provide is to compile and execute programs that it constructs on the fly—or that the user provides— while it is already running We’ll discuss Perl s support for this service next 8 .7 EVALUATING CODE USING eval Like the Shell, Perl has a code-evaluation facility It’s called eval, and its job is to compile and execute . records. a. cmd and cmd2 represent OS commands, var/$var and array/@array Shell /Perl variable names, and function a Perl function name. INTERPOLATING COMMAND OUTPUT INTO SOURCE CODE 271 to give you. eliminates the most common need for it 7 To learn about the first Perl beautifier, see http://TeachMePerl.com /perl_ beautifier.html. Table 8.5 String operators for concatenation and repetition Name Symbol. alter- native to the back-quote syntax for command substitution, of the form $(command). 270 CHAPTER 8 SCRIPTING TECHNIQUES When a Unix shell processes a command substitution, a shell of the same