Useful unix tricks - part 3

Update (8/16/11): Check out Useful unix tricks – part 4

Here is part 3 of Useful unix tricks and Useful unix tricks – part 2.

!! is the previous command in the shell history

It is pretty common to want to rerun the previous command, possibly with something new on the beginning or end. !! is that command in the history. For example:

% tail foo
tail: cannot open `foo' for reading: Permission denied

% sudo !!
sudo tail foo
hello world

As you can see, I forgot to sudo the first command. Now, I want to rerun it with a sudo at the front, so I can just do “sudo !!” and press enter. The shell will print out the command it is running, followed by whatever it would print normally.

Tail multiple files at once

The tail command can take multiple files, and it will show the output of each one. You can combine this with the -f flag, and tail will intersperse the output of each file in real time. This is incredibly handy for looking at log files. For example, we can tail both the apache and rails logs to see the requests:

==> log/production.log <==

Processing MephistoController#dispatch (for 127.0.0.1 at 2009-02-20 13:33:31) [GET]
  Parameters: {"action"=>"dispatch", "path"=>["2008", "7", "19", "capistrano-with-pairing-stations"], "controller"=>"mephisto"}
Completed in 784ms (View: 0, DB: 260) | 200 OK [http://www.pgrs.net/2008/7/19/capistrano-with-pairing-stations]

==> /var/log/apache2/access.log <==
127.0.0.1 - - [20/Feb/2009:13:33:31 -0600] "GET /2008/7/19/capistrano-with-pairing-stations HTTP/1.1" 200 16049 "http://www.pgrs.net/" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6 Ubiquity/0.1.5"

As you can see, tail prints ==> <== to show which file the output is for.

Use vim -b to show nonprintable characters

Sometimes a file will have nonprintable characters, such as windows line breaks. Most editors won’t show them, but you can use “vim -b” to see and edit them. The -b flag tells vim to use binary mode. For example, here is a file with windows line endings:

% cat foo.txt
Hello
World

% vim -b foo.txt
Hello^M
World^M
^M

As you can see, the vim binary mode can see the line endings whereas cat cannot.

** is a recursive wildcard in zsh

You can use the recursive wildcard ** in zsh to do complex matching. For example, let’s say that you want to search all ruby files in the current project for the string RAILS_ENV. Normally, you would do something like:

% find . -name '*.rb' | xargs grep RAILS_ENV
./config/environment.rb:# ENV['RAILS_ENV'] ||= 'production'
./test/test_helper.rb:ENV["RAILS_ENV"] = "test"

In zsh, you can accomplish the same with a much simpler command:

% grep RAILS_ENV **/*.rb
config/environment.rb:# ENV['RAILS_ENV'] ||= 'production'
test/test_helper.rb:ENV["RAILS_ENV"] = "test"

The wildcard **/*.rb recursively matches any files that end in .rb, so there is no need for a find command.

If there are a lot of files, you will occasionally get the error:

% grep RAILS_ENV **/*.rb
zsh: argument list too long: grep

This means that the **/*.rb match returned too many arguments to handle. In this case, you can use echo and xargs to get the job done, which is still simpler than the find command:

% echo **/*.rb | xargs grep RAILS_ENV

find -X will show bad filenames

It is pretty command to run find and then pass the arguments into xargs. However, if any filenames contain spaces or quotes, xargs will fail. You can use find -X to find any paths that will fail. find will warn on these paths and then skip them:

% find -X .
.
find: ./filename with spaces: illegal path

If you want to use xargs with these files, use the -print0 option to tell find to use a NUL character instead of a space, and xargs -0 to tell xargs to parse on NUL instead of space:

% find . -print0 | xargs -0 echo
. ./filename with spaces

cd – will return to the previous folder

Passing – into the cd command will return you to the last folder you were in:

/tmp% pwd
/tmp

/tmp% cd ~

~% cd -
/tmp

/tmp%

Use ctrl+z and kill %1 to kill a process that will not die

Sometimes, you run a command and pressing ctrl+c will not kill it. When that happened, I use to open up another terminal window to kill -9 the process until someone showed me the following trick:

% sleep 1000
^Z
zsh: suspended  sleep 1000
% kill -9 %1
%
[1]  + killed     sleep 1000

Pressing ctrl+z suspends the process and returns you to a terminal prompt. Then, kill -9 %1 sends the kill -9 signal to job #1, which is our suspended process.

pwdx shows the working directory of a process

It can be really useful to see the working directory of a running process. For example, you can see which release a ruby process is running:

% sudo pwdx 23961
23961: /var/www/myapp/releases/20081231200733

Unfortunately, I haven’t found pwdx for the mac. If anyone knows how I can install it, please let me know.

Use sh -x to debug shell scripts

If you want to know what commands a shell script runs, run it with the -x flag. For example, say we have a shell script with two echos. Compare the output with and without the -x flag:

% sh foo.sh
hello
world

% sh -x foo.sh
+ echo hello
hello
+ echo world
world

% zsh -x foo.sh
+foo.sh:1> echo hello
hello
+foo.sh:2> echo world
world

As you can see, the -x flag shows which command is being run. zsh takes it a step farther and shows the script and line number as well.

sysctl replaces /proc on macs

The /proc filesystem is a great way to find out about a linux machine. For example, you can “cat /proc/cpuinfo” to find out how many processors are on the box. However, macs don’t have /proc. You can use sysctl instead. The -a flag prints out all keys and values:

% sysctl -a
kern.ostype = Darwin
kern.osrelease = 9.6.0
kern.osrevision = 199506
kern.version = Darwin Kernel Version 9.6.0: Mon Nov 24 17:37:00 PST 2008; root:xnu-1228.9.59~1/RELEASE_I386
...

You can also just get a single value with the -n flag. For example, this command will print out the number of cpu cores:

% sysctl -n hw.ncpu
2