Blog

The GNU coreutils are a set of tools that come with every flavor of Linux. They are a useful and historical part of the Unix and Linux operating systems and you use them everyday without realizing. Commands like ls, cat, and rm are all part of the coreutils package. However there are a large number of lesser known and lesser used tools in the coreutils. I will go over some of them that I have found very useful.

The GNU coreutils are not just available for Linux. You can install them on OSX, BSD Unix, and Windows as well. In OSX, you can use brew install coreutils to install it (you must have homebrew installed). On windows you can either install cygwin , or gnuwin32. On FreeBSD you will need to have the ports collection installed. The coreutils port is in /usr/ports/sysutils/coreutils. Of course for the really brave you can always compile it from source.

Once you have the coreutils installed you can begin experimenting with some of the new commands and see what they do. The complete documentation of all the coretuils can be found at the GNU documentation page. Average Linux and Unix users will find they know most of the commands, and use them on a daily basis. However there are quite a few gems that most users probably didn’t know were there. Let’s take a look at a few.

Note: These examples are geared toward the GNU versions available on Linux. Other versions of these commands might have different options or slight incompatibilities.

expand, unexpand

expand and unexpand are a fantastic pair of tools if you ever work on code with another developer that uses a different spacing in their editor than you (like tabs instead of 2 spaces). expand will read a file and convert tab characters to the requested number of spaces. This is great for a language like Python that is dependent on a consistent spacing being used or it won’t run. unexpand does the opposite and converts the specified number of spaces into a single tab character.

# you can see this text has 6 spaces in the front and back
# of the text.

$ xxd test.txt
0000000: 2020 2020 2020 5468 6973 2069 7320 736f        This is so
0000010: 6d65 2074 6578 742e 2020 2020 2020 0a0a  me text.      ..

# convert 3 spaces to a tab
$ unexpand -t 3 test.txt | xxd
0000000: 0909 5468 6973 2069 7320 736f 6d65 2074  ..This is some t
0000010: 6578 742e 0909 0a0a                      ext.....

fold, fmt

fold, and fmt are both text formatters that will help you keep text to a specific number of columns. fold is a bit more of a hammer then fmt is because it will hard wrap lines at the specified number of columns. By hard wrap I mean it will cut whole words (unless the -s flag is specified). fmt is more intelligent, and softer with your text. Not only does it wrap lines on whole words, but it also re-spaces the lines so they don’t have a ragged edge appearance.

$ cat test.txt
This is line 1, it is 41 characters long.
This is line 2, it's shorter.
Line 3.
This is another long line that will need to be wrapped.

# Notice the ragged edges, as the original newlines are kept
$ fold -s -w 20 test.txt
This is line 1, it
is 41 characters
long.
This is line 2,
it's shorter.
Line 3.
This is another
long line that will
need to be wrapped.

# This command ignores the original newlines and wraps the text nicely
$ fmt -w 20 test.txt
This is line 1,
it is 41 characters
long.  This is line
2, it's shorter.
Line 3.  This is
another long line
that will need to
be wrapped.

shuf

shuf is a handy tool that will produce random permutations of whatever input you give it.

# Play a shuffled mp3 playlist from the current directory
for i in "$(ls -1 *.mp3 | shuf)"; do echo "$i" | xargs -I{} mpg123 {}  ; done

tac

tac is cat spelled backward. And that is exactly what it does. Running this command will print the lines of a file in reverse order.

$ cat test.txt
This is line 1
This is line 2
This is line 3
This is line 4

$ tac test.txt
This is line 4
This is line 3
This is line 2
This is line 1

This has been just a small glimpse into what the coreutils has to offer! In part 2 I will present more useful, but mostly unknown commands that can be used to solve interesting problems. Did you know you can perform a prime number factorization from the command line with a single command? Read part 2 to find out how.