Data Carpentry Genomics at NMNH

Jun 12-13, 2018

9:00 - 4:30 pm

Instructors: Rebecca Dikow, Vanessa Gonzalez, Matthew Kweskin, Mike Trizna

Helpers: Mirian Tsuchiya, Mauro Lepore

Data Carpentry for Genomics at NMNH

General Information

Data Carpentry develops and teaches workshops on the fundamental data skills needed to conduct research. Its target audience is researchers who have little to no prior computational experience, and its lessons are domain specific, building on learners' existing knowledge to enable them to quickly apply skills learned to their own research. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.

For more information on what we teach and why, please see our paper "Good Enough Practices for Scientific Computing".

Who: The workshop is for Smithsonian badge holders at any SI Unit working with research data. You don't need to have any previous knowledge of the tools that will be presented at the workshop.

Where: 10th St NW and Constitution Ave NW, Washington DC. Get directions with OpenStreetMap or Google Maps.

Room: WG 33

When: Jun 12-13, 2018. Attendees must attend entire workshop. Add to your Google Calendar.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. This computer must be able to connect to the Smithsonian's SI-Staff wifi network which requires an SI network account. They should have a few specific software packages installed (listed below). They are also required to abide by Data Carpentry's Code of Conduct.

Accessibility: We are committed to making this workshop accessible to everybody. The workshop organizers have checked that:

Materials will be provided in advance of the workshop and large-print handouts are available if needed by notifying the organizers in advance. If we can help making learning easier for you (e.g. sign-language interpreters, lactation facilities) please get in touch (using contact details below) and we will attempt to provide them.

Contact: Please email for more information.


Please enter your information in the embedded Google Form below...


Please be sure to complete these surveys before and after the workshop.

Pre-workshop Survey

Post-workshop Survey


Note: people who have don't currently have a Hydra Comptuer Cluster account are required to take an introductory lesson on Hydra on Monday, June 11th at 1pm.

June 12

Before starting Pre-workshop survey
Morning Lesson 1: Project organization and management
AfternoonLesson 2: Introduction to the command line

June 13

Morning Lesson 3: Data wrangling and processing
AfternoonLesson 4: Data analysis and visualization in Python
Evening Post-workshop survey

We will use this collaborative document for chatting, taking notes, and sharing URLs and bits of code.

Workshop Lessons:


To participate in a Data Carpentry workshop, you will need access to the software described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

Text Editor

When you're writing code, it's nice to have a text editor that is optimized for writing code, with features like automatic color-coding of key words. The default text editor on macOS and Linux is usually set to Vim, which is not famous for being intuitive. If you accidentally find yourself stuck in it, try typing the escape key, followed by :q! (colon, lower-case 'q', exclamation mark), then hitting Return to return to the shell.


Video Tutorial

nano is a basic editor and the default that instructors use in the workshop. To install it, download the Data Carpentry Windows installer and double click on the file to run it. This installer requires an active internet connection.

Others editors that you can use are Notepad++ or Sublime Text. Be aware that you must add its installation directory to your system path. Please ask your instructor to help you do this.


nano is a basic editor and the default that instructors use in the workshop. See the Git installation video tutorial for an example on how to open nano. It should be pre-installed.

Others editors that you can use are BBEdit (free version is fine) or Sublime Text.


nano is a basic editor and the default that instructors use in the workshop. It should be pre-installed.

Others editors that you can use are Gedit, Kate or Sublime Text.

Terminal Emulator

We'll be doing much of the work in this workshop using the Smithsonian High-Performance Computing Cluster (Hydra). The connection to Hydra is through a terminal emulator on your computer you will use to log into the system. Mac and Linux systems have a terminal emulator built-in. In Windows you'll need to download a program that allows connections via ssh. The built-in Windows command prompt does not have that capability.


The Windows terminal emulator we recommend is PuTTY. The installer for PuTTY can be found on the Putty download page. For most Windows systems (64-bit) you'll want "putty-64bit-0.XX-installer.msi". If you don't have admin privileges to your Windows system, you can download "putty.exe" which can be run directly without admin rights.


The Mac Terinal program can be found in the Utilities directory inside of your computer's Applications directory.


Look for a program called "Terminal" in your Linux system.

File Transfer Program

To assist transferring of files to the Hydra cluster, we suggest installing a GUI based file transfer program that will transfer using the scp or sftp protocols. The program that is approved for use on Smithsonian computers is Filezilla.


  1. Go to the FileZilla download page
  2. For most Windows systems (64-bit), choose the download "FileZilla_3.XX.0_win64-setup.exe" (XX will be the value of the current release).
  3. Run the installer which will require you having administrative access to your computer.
  4. If you don't have an admin account on your computer: download and unzip the files "". The .exe files in this can be run without running an installer first.
  5. Please also install pscp, part of the PuTTY package. If you instlaled the full PuTTY installer (.msi file), pscp will be installed too. If you weren't able to run the full PuTTY installer because you don't have admin rights, make sure to also download pscp.exe. </ol> </p> </div>


    1. Go to the FileZilla download page
    2. Choose the download " " (XX will be the value of the current release).
    3. Extract the downloaded file by double clicking it if it wasn't automatically extracted.
    4. [Optional] Move the FileZilla icon to your Applications folder.


  6. Go to the FileZilla download page
  7. Choose the download "FileZilla_3.XX.0_x86_64-linux-gnu.tar.bz2 " (XX will be the value of the current release).
  8. Extract the downloaded file.
  9. Run FileZilla by starting the filezilla executable in the bin directory.
</div> </div>


Python is a popular language for research computing, and great for general-purpose programming as well. Installing all of its research packages individually can be a bit difficult, so we recommend Anaconda, an all-in-one installer.

Regardless of how you choose to install it, please make sure you install Python version 3.x (e.g., 3.6 is fine).

We will teach Python using the Jupyter notebook, a programming environment that runs in a web browser. For this to work you will need a reasonably up-to-date browser. The current versions of the Chrome, Safari and Firefox browsers are all supported (some older browsers, including Internet Explorer version 9 and below, are not).


Video Tutorial
  1. Open with your web browser.
  2. Download the Python 3 installer for Windows.
  3. Install Python 3 using all of the defaults for installation except make sure to check Make Anaconda the default Python.


Video Tutorial
  1. Open with your web browser.
  2. Download the Python 3 installer for OS X.
  3. Install Python 3 using all of the defaults for installation.


  1. Open with your web browser.
  2. Download the Python 3 installer for Linux.
    (The installation requires using the shell. If you aren't comfortable doing the installation yourself stop here and request help at the workshop.)
  3. Open a terminal window.
  4. Type
    bash Anaconda3-
    and then press tab. The name of the file you just downloaded should appear. If it does not, navigate to the folder where you downloaded the file, for example with:
    cd Downloads
    Then, try again.
  5. Press enter. You will follow the text-only prompts. To move through the text, press the space key. Type yes and press enter to approve the license. Press enter to approve the default location for the files. Type yes and press enter to prepend Anaconda to your PATH (this makes the Anaconda distribution the default Python).
  6. Close the terminal window.