Second-generation interactive shell tools04 Jan 2020
- rather than naked shell, try
tmuxfor an organized collection of shells
- rather than local file system navigation, try
fzfto fuzzily get what you want
- rather than
autojump, which remembers your favorite locations
- rather than
fd, which has a much cleaner interface
- rather than
ag, which is simpler and faster
The built-in shell environment is tremendously productive, but “second generation” tools can make it even more so.
If you want to try out these tools, you can run
docker run -it quay.io/matsen/cozy-demo
to test them out in Docker. There is some funkiness with the display on this Docker image (tmux pane splits use hashes and q’s) but it’s enough to get the idea. See the corresponding GitHub repository if you want to mimic the configuration.
When working on a modern desktop computer, it’s easy to arrange multiple windows side by side, to switch between applications, etc. On the command line this is achieved by use of a “terminal multiplexer”. This is especially lovely for working on remote machines, where one can detach and re-attach sessions with all their attendant windows.
The two most popular terminal multiplexers are GNU Screen and tmux.
This tutorial will demonstrate
tmux, which is newer and more feature-rich.
We will also use our (non-default) configuration, which is vi-keybinding oriented (emacs-style keybindings are also available).
You send commands to these programs using a “prefix” key command followed by another keystroke.
In our configuration, the prefix is
Ctrl-a, the most easily typed control combination.
For example, hitting
Ctrl-a, letting go, then typing
c opens a new terminal window.
We will abbreviate this combination
First steps in tmux
As a first step, I suggest just starting up tmux by executing
tmux and trying out the following commands.
Open a few windows, type some commands, cycle through them, open a few panes in a single window, and move around through them.
Ctrl-a cNew window
Ctrl-a <Space>Next window
Ctrl-a NUMBERMove to window number
Ctrl-a Ctrl-aMove to the previously visited window
Ctrl-a vSplit window into panes vertically
Ctrl-a sSplit window into panes horizontally
Ctrl-a <arrow>Move between panes
Ctrl-dClose a pane or window (works outside of tmux)
Ctrl-a qKill a pane or window
Hopefully it’s clear that the windows are complete environments which are indexed at the bottom, and the panes are sub-windows that have no associated indicator. Each pane has a shell running inside of it.
The following commands are also essential:
Ctrl-a ?Command help. Note that
/will allow you to search.
Ctrl-a aActually send a
Ctrl-ato the terminal (e.g. to move to the beginning of the line when editing a shell command)
Detaching and attaching a session
Tmux can keep your window configuration and all of the running processes going, even if your terminal goes away. This is called detaching and reattaching.
The polite way to detach and reattach is to (try this!)
Reattaching is also very handy if you are moving between laptops.
You can detach your remote session from one, log in on another, and then reattach on the second laptop.
Note that if you have different size terminal windows between the two machines the session can look wonky.
In that case simply use
tmux attach -d, which will “steal” the session from any other login shell and redraw the session from scratch.
If your wifi drops you might just get disconnected to a remote machine, in which case you wouldn’t have the opportunity to politely detach.
In that case, just re-connect to your remote machine, and
Also note that
tmux attach will fail if you don’t have a session open already; if that happens just enter
tmux to start a new session.
Ctrl-a <Move vertical split left
Ctrl-a >Move vertical split right
Ctrl-a +Move horizontal split up
Ctrl-a =Move horizontal split down
Note: You can hit
Ctrl-a once, and press the resize operator a number of times (or hold it) to do larger resizes.
For example, try
Naming & finding windows
Ctrl-a ,Name a window
Ctrl-a 'Present a list of windows (by name), which you can choose from by
Enterto switch to a window
Ctrl-a <numeric>Switch to a window by number.
Scrolling and copy/paste in tmux
When you have lots of output, it’s nice to be able to scroll up and down through history.
Ctrl-a [ will place you in scroll mode.
Use can now use arrow keys, PageUp/PageDown keys, or
Ctrl-d to scroll through the history.
You can search up through history with
? and down with
From this mode, you can also press
Space to enter copy-mode, arrow keys to specify a selection range, and
Enter to copy the selection.
To paste the selection, use
Note: if you are running tmux 2.1 or greater (check with
tmux -V) and you really like your mouse scroll wheel, you can try
set -g mouse on which will drop you into tmux scroll mode when you use the scroll wheel.
But I suggest keeping your fingers on the keyboard and ignoring the “rat”.
Advanced tmux tips
You can also use
l in place of the arrow keys, as in
If you want to pipe to the tmux copy/paste buffer, try this alias:
alias tc='tmux loadb -'
With this alias I can, for example,
ls /very/long/path.txt | tc and then paste rather than having to do an explicit select and copy.
There are many more commands than we have connected with command sequences.
See the tmux man page for a complete list.
Ctrl-a : will put you at a prompt that will allow you to execute an arbitrary command.
Using tmux with ssh
Ssh keys in a long-running tmux session require some care.
I use keychain to automate
ssh-agent and friends together with tmux like so:
eval `keychain --eval id_rsa` && tmux attach -d
Sometimes it seems that ssh connections inside those sessions need a little refresh. For that I use this shell command.
You can get fancy with tmux
Some people love complex tmux setups with tons of sub-panes with things running in them. I prefer to have a simple setup where things can get set up and torn down easily. If you like a fancy setup, you can use something like tmux-resurrect to maintain such setups on your local machine between reboots. There are also tmux plugin managers, etc, but just a simple configuration file is all I’ve needed.
You can also have lots of different detached sessions, give them names, etc. I used to do this, with each session corresponding to a single project. For me it ended up being too much work to manage, but perhaps it fits for you.
We all love tab completion for matching files. But sometimes it too can be painful. Consider this list of files, which is some the various posts that have appeared on fredhutch.io:
2014-04-24-command-line.md 2015-04-06-R-sequence-analysis.md 2014-04-27-terminal-multiplex.md 2015-04-23-ucsc-xena-workshop.md 2014-05-09-galaxy.md 2015-04-24-third-galaxy-101.md 2014-05-11-editing.md 2015-05-04-spring-2015-r.md 2014-05-17-git.md 2015-05-20-spring-2015-unix-hpc.md 2014-05-20-R.md 2015-06-03-summer-bioinfo.md 2014-07-13-toolbox.md 2015-08-26-data-center-tour.md 2014-07-16-aws.md 2015-08-27-cloud-computing.md 2014-08-13-synapse.md 2015-12-02-galaxy-rna-seq.md 2014-09-19-R-course.md 2016-01-21-cds-git.md 2014-10-20-introductory-r-.md 2016-02-25-immunespace.md 2014-10-22-shippable.md 2016-03-01-i-heart-pandas.md 2014-10-27-bioconductor.md 2016-03-28-gizmo-brownbag.md 2014-11-03-labkey.md 2016-06-14-scicomp-unix-hpc.md 2014-11-07-intermediate-R.md 2016-07-28-galaxy-101.md 2014-12-09-hidra.md 2016-08-23-chipseq-class.md 2014-12-15-scicomp-unix-hpc.md 2016-08-31-rnaseq-class.md 2015-02-11-scicomp-unix-hpc.md 2016-09-08-inkscape.md 2015-03-09-introductory-R.md 2016-09-27-intro-r.md 2015-03-12-rollout-galaxy-101.md 2016-10-03-introbio.md 2015-04-02-april-galaxy-101.md 2017-01-05-command-line-cozy.md
to tab-complete through these files is a pain, because I actually want to choose a file by the text portion of the descriptor, which is past the date.
What we really want is fuzzy finding, which allows us to match via partial substring matches. By this I mean that arbitrary substrings of your query string get matched with arbitrary substrings of your target strings, in an ordered fashion (see below for an example). This is available as part of the GitHub web interface for finding files, and actually even in the Chrome address bar.
fzf brings fuzzy finding to the shell.
For example, if I want the posts from 2015 that contain
galaxy, I could type
2015galaxy, and 💥:
2015-03-12-rollout-galaxy-101.md 2015-04-24-third-galaxy-101.md 2015-04-02-april-galaxy-101.md 2014-05-09-galaxy.md > 2015-12-02-galaxy-rna-seq.md 5/41 > 2015galaxy
we get a list of the files that contain
In fact, I could have typed
15axy and gotten the same result, because it matches the same set of files.
I can then type more characters to limit the search, or use the arrow keys to move between results, and then return to select.
Note that there is a “false positive” here, namely
2014-05-09-galaxy.md which is from 2014 but has a 5 in the month.
That’s just a natural consequence of the way fuzzy finding works.
Here are two very easy ways to get started with fzf from the shell:
Ctrl-tdrops you into the fzf fuzzy command completer from the shell prompt. For example,
less Ctrl-twill allow you to find the file that you want to
Ctrl-r, the usual reverse search on your history, gets replaced by the fuzzified version. This is great and allows you to be vague about what you’re looking for.
For many more uses, including vim integration, see the fzf GitHub page.
autojump is a
cd replacement that learns.
It keeps a record of where you have been in the past, and given multiple directories that match an argument, chooses the one that you have been in the most.
The autojump command is
For example, I frequently visit our data directory on the shared filesystem at
Instead of typing out that full filename I can just type
j data to
This works with partial filenames as well.
In the example Docker container I have a few directories that have cats in them, and
j cats will take you to
/root/cats/siamese-mostpopular, which is the most popular subdirectory containing
If you are in the most popular directory with a certain string, then running the same
j command will take you to the second most popular.
For example, repeating
j cats a second time would take us to
/root/cats/bengal, which is the subdirectory we’ve visited the second most.
(We could have used
cat here as the argument to
j instead of
cats but I didn’t want to confuse things with the
cat shell command.)
The information about what directories you spend the most time in is stored in
For our example Docker image it looks like this:
50.0 /root/cats/siamese-mostpopular 30.0 /root/cats/siamese-lesspopular 40.0 /root/cats/bengal 10.0 /root/work
The left-hand column here is the popularity index and the right hand is the directory.
GNU Find is incredibly powerful, but has an annoying interface.
fd command fixes that.
fd PATTERNfinds a file in the current directory whose filename matches
fd PATTERN -x COMMANDexecutes
COMMANDfor each file matching
fd .txt -x head -n 1 will perform
head -n 1 on each file matching
fd command has sensible defaults for the modern environment, such as ignoring
.git directories and files that appear in
See the fd GitHub page for more examples and detail.
ag is a faster replacement for
grep that has a nicer input and output interface.
For example, as I’m writing this I might look for instances of the word “fancy” in the repository for this blog.
ag fancy gets me:
_posts/2019-11-05-travis.md 22:It can do all sorts of fancy pipelines, but here we'll just get it to do one simple thing: run a command in a Docker container. _posts/2019-12-31-command-line-cozy.md 145:### You can get fancy with tmux 149:If you like a fancy setup, you can use something like [tmux-resurrect](https://github.com/tmux-plugins/tmux-resurrect) to maintain such setups on your local machine between reboots.
which shows clearly that this word appeared once in a post about Travis, and twice in this file (before I wrote this section).
In contrast, we can try the same thing using
grep -r fancy
_posts/2019-11-05-travis.md:It can do all sorts of fancy pipelines, but here we'll just get it to do one simple thing: run a command in a Docker container. _posts/2019-12-31-command-line-cozy.md:### You can get fancy with tmux _posts/2019-12-31-command-line-cozy.md:If you like a fancy setup, you can use something like [tmux-resurrect](https://github.com/tmux-plugins/tmux-resurrect) to maintain such setups on your local machine between reboots. _site/feed.xml:<h3 id="you-can-get-fancy-with-tmux">You can get fancy with tmux</h3> _site/feed.xml:If you like a fancy setup, you can use something like <a href="https://github.com/tmux-plugins/tmux-resurrect">tmux-resurrect</a> to maintain such setups on your local machine between reboots. _site/feed.xml:It can do all sorts of fancy pipelines, but here we’ll just get it to do one simple thing: run a command in a Docker container.</p> _site/2019/11/05/travis.html:It can do all sorts of fancy pipelines, but here we’ll just get it to do one simple thing: run a command in a Docker container.</p> _site/articles/2019/12/31/command-line-cozy.html:<h3 id="you-can-get-fancy-with-tmux">You can get fancy with tmux</h3> _site/articles/2019/12/31/command-line-cozy.html:If you like a fancy setup, you can use something like <a href="https://github.com/tmux-plugins/tmux-resurrect">tmux-resurrect</a> to maintain such setups on your local machine between reboots.
First, things aren’t as nicely organized into sections.
Second, grep looks into all of the files, including the ones that are in
The git-ignored files in
_site are auto-generated and they only add noise to my search.
ag is also very fast and has lots of nice editor integrations.
See the ag GitHub page for more details.
Conclusion and future directions
We’ve looked at a few tools that can help make it easier to interact with the shell.
Or you could just be happy with the defaults. Every minute you spend tweaking your configuration is a minute you aren’t doing creative and beautiful things.