UK-BLAST workshop 2016
These exercises were developed for Dr. Mark Farman at the University of Kentucky for Undergraduate students participating in UK-BLAST. If you are trying to learn how to run applications from the command line but are feeling lost, this guide is for you.
UK-BLAST is a bioinformatics working group for undergraduate researchers. In small labs, students often tackle the bioinformatician role in with little to no technical guidance. The group may not have anyone with a computer science background, or the task may be beyond the skills of current personnel. In these cases, a working group headed by a technical mentor (myself) allows students to bypass some of the pitfalls and headaches associated with learning computer science all alone.
PATH: A guide to finding one’s way
Written by Bradford Condon, PhD. Farman Labratory, University of Kentucky, 2016
Background: What is a PATH?
In computing, a PATH is the absolute name of a file or directory, specifiying its unique location within a file system. When you navigate to a file on your computer, locate a song in your music library, or navigate to a website, you do so using this absolute file structure.
Computer file structures
Before we dive in to the command line, let’s start with something more familiar.
This is a folder on my computer called “Home.” I keep all sorts of stuff in here: my movies, my music, my photos. I have a folder for each. Within each of those folders, I organize my things by category. For example, in my pictures folder I’ve organized my photos by food, vacation, and work photos, and within my vacation photos, I keep each trip in a separate folder. In short, we use nested folder structures to organize our files.
Command line file structure works the same way. To indicate that a folder exists within another folder, we use the / symbol. The location of my new orleans folder, above, is therefore Home/pictures/vacation/new_orleans.
Tip: Avoid adding spaces to files and folder names. Use underscores (_) instead. This is called snake_case. Alternatively, you can omit the spaces and capitalize each new word: this is called camelCase.
Let’s see this in action on the command line.
To begin, you must first access the command line. On Mac, this is through the Terminal: on Windows, you need to use a program like PuTTY.
Open up your Terminal (press CMD-SPACE top open spotlight and type terminal) and you should be greeted by a command prompt like this:
Where are we? What files and folders are located where we are? The word immediately preceeding the $ symbol is the name of my current working directory. The next section will explain some simple commands to look around your file structure.
PWD: print working directory
Our first essential command is PWD. PWD stands for print working directory.
- [ ] Type in PWD or pwd (it isnt case sensitive, files and directories are) and make note of where you are located.
Bradfords-MacBook-Pro:Documents chet$ pwd /Users/chet/Desktop/Documents Bradfords-MacBook-Pro:Documents chet$
This is called our starting directory or HOME directory, and is where we will begin every time we open a new terminal.
This is an essential command to learn! The dollar sign $ is called the prompt. It lets you know that your computer is ready to receive input commands. Notice how in the first line I typed ‘pwd’ and hit enter. The computer processed the command and gave me an output on the second line. The third line shows, again, my current working directory and the prompt, letting me know it is ready receive another command.
The value returned by PWD on my computer was:
This is where my terminal is currently located, and is referred to as my working directory. Yours will look different but it will mean the same thing: There are a series of folders within folders telling you how to find your current working directory. on OSX, the /Users/ directory contains the home directory for all the users of that computer.
- [ ] Write the full PATH pwd returns for you below.
PATH to my home directory:
Let’s go a few quick commands so we can practice navigating our file structure.
- LS - lists files and folders in your current directory
- CD - Changes directory.
- MKDIR- Creates a directory.
[ ] Try each of the above commands yourself and see what happens. Write what happens for each in the box below.
$ ls $ cd $ mkdir
What did mkdir do? It gave us a usage message, which means we didn’t use the program properly. We tried to create a directory without specifying the name of the directory.
[ ] Let’s create a directory called test:
$ mkdir test
- [ ] Try the excercises below.
Which of the above commands will let us check if our newly made directory exists? Execute this command.
What is the full path of this new folder?
Create another folder called test. What happens?
MKDIR failed because we cannot have two files with the same name in the same location.
If you ever need guidance figuring out how to run a program, type the program name with the help option/flag for documentation (-h after the command: see the code box below). As you use more and more elaborate programs, this will become very useful!
$ cd -h $ ls -h $ mkdir -h
CD: change directories
CD changes the directory we are in. If we only type in
It will automatically move us to our home directory.
To navigate into our new test folder, type issue the following two commands.
1 2 $ cd test $ ls
this will move us into the test folder, and list the files in this folder. We just made it, so it’s empty.
[ ] Next, make a new folder in this directory and use ls to see if the command was successful.
$ mkdir test $ ls
We can have two folders named test in separate locations, but we cannot have two folders named test in the same location. This is because each folder has its own ABSOLUTE PATH.
- [ ] use pwd and cd to answer the question below.
What is the full PATH of the first test directory we made? The second?
- [ ] While located in the this second test folder, issue these three commands:
$ pwd $ cd $ pwd
As you can see, the cd command returns us to our home directory when we dont supply an argument (write something after the command name).
If we want to navigate back , we could type
$ cd test $ cd test $ pwd
Alternatively: we can navigate there in one jump by typing
$ cd test/test
This command changes directory from our current directory to a folder called test, and then to the next folder inside of that. What’s important to understand is that we can do this from anywhere on the computer by specifying the full path.
Even more importantly, we can issue a command on files in other directories by giving their full PATH! The below command would list the contents of my test folder no matter what my current working directory is.
$ ls /Users/chet/Desktop/Documents/test/test
- [ ] Try listing the contents of a folder that is not your current working directory.
Relative and absolute paths
- [ ] Try the following example below (on a mac).
$ cd /usr $ pwd $ ls $ cd /Users/_yourNameHere_/Desktop/Documents/test/test $ pwd
Notice that by typing
cd /FOLDER, we are telling cd the absolute path of a file. before, we typed
cd FOLDER (without the backslash first). This specified the relative path. The absolute path of a file never changes. The relative path is really the path from our current working directory.
- What is the absolute path of our second test folder?
- What is the relative path of our test folder from the home directory?
Below are two very useful shortcuts.
$ cd .. ##One directory up. Very useful for quickly navigating directories. $ cd ../anotherfolder ##navigates one folder backwards, and then into another folder $ cd ~ #~ is a shortcut to the home directory. It's not useful here because cd also returns us to our home directory but.... $ cd ~/Desktop/Documents ## Home directory. Very useful when specifying the ABSOLUTE PATH of a file or folder. ## The hash symbol indicates a comment. # Most programming langauge ignore everything appearing in a line after a # #(or a different character used for this purpose): it is for human eyes to "annotate" code.
From Your home directory, where does
cd ..take you?
Command line completion/autofill
The final essential tip of this lesson is command line completion or autofill.
Bradfords-MacBook-Pro:Desktop chet$ pwd /Users/chet/Desktop Bradfords-MacBook-Pro:Desktop chet$ ls $RECYCLE.BIN Documents Screen Shot 2016-02-05 at 10.44.47 AM.png Solved Tests images_for_PATH Bradfords-MacBook-Pro:Desktop chet$
As you can see, I’m currently located in /Users/chet/Desktop. If I’d like to move into my test folders located in documents I could type
cd Documents/test/test However, this is a confusing file path. It’s very easy to make mistakes typing out something like this. Command line autofill is an extremely powerful tool to get around this. To access it, start typing your command and press tab after writing a few characters. It will automatically fill in the rest of the file name if it is unambiguous.
From your home directory, navigate back to your test folder by using autocomplete.
Sometimes, we’ll have two files with similar names. Suppose we have two folders, test1 and test2. How can we quickly navigate into test2?
Make two similarly named directories in current folder. Begin using autocomplete to cd into one. What happens?
Try pressing pressing tab twice after typing in a few letters. What happens?
We can use autocomplete not only to change directories quickly, but correctly navigate confusing directories.
Note: do not experiment deleting, moving, or modifying files in your root directory.
Navigate to the root of your file structure by typing cd /
- What files are located here?
- What is the easiest way to return to your home directory?
Try navigating from your root directory to your test directory created earlier using a single command! You can make use of the shortcuts listed above, especially tab autocomplete.
Keep in mind that being familiar with the absolute PATH of files is useful for more than just navigating around your computer. We can issue commands in folders where we are not currently located using the PATH. This works with both relative, and absolute, paths.
for example, to list the files in a new folder, we can type
cd /Users/chet/Desktop mkdir test_folder ls test_folder
Make a new test folder and navigate into it. From here, how can you list the contents of your home directory using…
- Absolute path
- Relative path
How can you make a new folder in your home directory from this remote location?
As you can see below, the navigation tabs I’m using at the top of my page didn’t properly update depending on what page the user is at. While on the CV page, the CV tab should be selected, not home.
So how do we fix it?
The header is defined in the
/_includes folder in a hero-footer element.
<div class="hero-foot"> <nav class="tabs is-boxed nav-menu has-shadow"> <div class="nav-center"> <ul> <li class="is-active"><a href="/">Home</a></li> <li><a href="/about/">About</a></li> <li><a href="/cv.html">CV</a></li> </ul> </div> </nav> </div>
The list element
class="is-active" is what tells the styling to make a particular tab active.
We can test if we are on a particular page using this snippet: ``. So, let’s replace
<li class="is-active"> with
<div class="hero-foot"> <nav class="tabs is-boxed nav-menu has-shadow"> <div class="nav-center"> <ul> <li ><a href="/">Home</a></li> <li ><a href="/about/">About</a></li> <li ><a href="/cv.html">CV</a></li> </ul> </div> </nav> </div>
Reload the site and voila, our header now updates based on our page.
Liquid tags make it easy to conditionally change the class of our header tab element to active or not.
If you’d like to learn more we have been working with Liquid tags. There are a variety of handy tags to make your static page more dynamic.
I previously announced a Workshop we hosted at Floracliff on mushroom identification In October 2016. It’s been a while but I wanted to share a little bit about the experience and post materials.
First of all, a big thank you to Beverly and Josie at Floracliff. It was a joy working with you all. Same goes to our wonderful participants, we couldn’t have asked for a better audience.
I’m sharing photos from the workshop and material below. Special thanks to Kevin Lutes for taking professional photographs and for permission to link them here.
Hope to see you all again soon!
Group photo of those who were able to attend the day 2 hike. We could not have asked for better students.
Dr. Bradford Condon doing his best to make sense of a field guide.
Identifying our haul.
Below are links to slides for presentations. If you really need one of the other presentations let me know and I can share them.
Knitting R Markdown to Jekyll
This is a test of the Knittr package and Jekyll.
It looks like the original package for using knitr with Jekyll here has fallen out of favor for Hugo, a GO-based static web-site generator. Pretty cool, but let’s see how easy it is to knit into Jekyll without using any packages.
Load in some libraries and data
Do some plots
Behold my beautiul plot!
Next we need to get our RMD to MD, with the image paths sorted. To do this I’ll stand on the shoulders of chepec who has implemented a solution. I created a
_knitr directory in my Jekyll site, and I included a
knitr folder in my
assets folder. Make sure you update the necessary paths in the
Now its a matter of simply sourcing the file and running
KnitPost. Make sure that your post titles match Jekyll requirements (start with
YYYY-MM-DD format date). The result is a very rapid R-studio to web post!
Switching to Jekyll
This weekend I switched my site from a Wordpress site hosted by bluehost to a Jekyll site hosted on githubpages. I’m excited about this for plenty of reasons.
- Free hostings on GithubPages
- Manage all aspects of the site, including layout and posts, on github
- Write in Markdown
- Use R plugins to script in R Markdown (I’ll share my experience with this later!)
- An excuse to practice CSS and Ruby
It’s pretty win-win.
If you’re looking to try this out for yourself, you’ll want Ruby/RVM, Jekyll, and github set up.
Migrating my old Wordpress posts
Ben Balter has created a Wordpress plugin that exports existing posts to Jekyll Markdown.
It’s simple: download and enable the plugin, then use the
export to Jekyll command on the toolbar.
All of my Wordpress posts are now neatly in my
_posts folder, and all associated assets are in
wp-content/uploads/[date]/. I need to cull some of the images to cut down on space, and I have to redo the formatting of my old Wordpress posts. For example, the MD headers have more info than we would possibly care about (such as review requests) and there are some wonky tags in the pages. All in all, though, this works extremely well.
Working with drafts
I suggest reading through the Jekyll docs, but the simple gist is that we’ll write our posts directly in Markdown as drafts in
jekyll serve --drafts
We write posts in the
_drafts folder, and mark them as
date: unpublished in the header (note that having
unpublished posts in _
drafts folder will mess with running the –draft server. I recommend just commenting the line out until moving to
--- layout: post title: "Moving to Jekyll" <!-- date: unpublished --> --- # Introduction This is my post that I get to write in *Markdown*!
When we’re ready to publish the post, we move it to the
_publish folder (and uncomment the date). Our plugin will fix the date and move it to the
Configuring the site layout
Hooboy. As someone who has almost zero experience doing this, I’d recommend starting with an existing Jekyll theme if you feel overwhelmed. If not, pick a CSS framework and mess around, there’s no better way to learn. You can always use this site as a starting point if you’re totally lost.
I’m really hopeful that The joy of writing in Markdown (and R Markdown) will get me back in the blogging game.