SUBSCRIBE: guys asked for a new LittleBigSnake.io video and here it is! This time the Happy Hamburger Snake eats so much he transfo. Company History. We have been keeping and breeding reptiles since 1985. Since that time, our operation has expanded from a husband and wife team operating out of their home basement to one of the Worlds largest breeding facilities with a youthful, energetic, dedicated management team and a full time staff of animal care technicians.
Here we provide a short tutorial that guides you through the main features of Snakemake.Note that this is not suited to learn Snakemake from scratch, rather to give a first impression.To really learn Snakemake (starting from something simple, and extending towards advanced features), use the main Snakemake Tutorial.
This document shows all steps performed in the official Snakemake live demo,such that it becomes possible to follow them at your own pace.Solutions to each step can be found at the bottom of this document.
The examples presented in this tutorial come from Bioinformatics.However, Snakemake is a general-purpose workflow management system for any discipline.For an explanation of the steps you will perform here, have a look at Background.More thorough explanations are provided in the full Snakemake Tutorial.
Prerequisites¶
First, install Snakemake via Conda, as outlined in Installation via Conda/Mamba.The minimal version of Snakemake is sufficient for this demo.
Second, download and unpack the test data needed for this example fromhere,e.g., via
Step 1¶
First, create an empty workflow in the current directory with:
Once a Snakefile is present, you can perform a dry run of Snakemakewith:
Since the Snakefile is empty, it will report that nothing has to bedone. In the next steps, we will gradually fill the Snakefile with anexample analysis workflow.
Step 2¶
The data folder in your working directory looks as follows:
You will create a workflow that maps the sequencing samples in thedata/samples
folder to the reference genome data/genome.fa
.Then, you will call genomic variants over the mapped samples, and createan example plot.
First, create a rule called bwa
, with input files
data/genome.fa
data/samples/A.fastq
and output file
mapped/A.bam
To generate output from input, use the shell command
Providing a shell command is not enough to run your workflow on anunprepared system. For reproducibility, you also have to provide therequired software stack and define the desired version. This can be donewith the Conda package manager, which is directlyintegrated with Snakemake: add a directiveconda:'envs/mapping.yaml'
that points to a Conda environmentdefinition,with the following content
Upon execution, Snakemake will automatically create that environment,and execute the shell command within.
Now, test your workflow by simulating the creation of the filemapped/A.bam
via
to perform a dry-run and
Box cat bash (itch) mac os. to perform the actual execution.
Step 3¶
Now, generalize the rule bwa
by replacing the concrete sample nameA
with a wildcard {sample}
in input and output file the rulebwa
. This way, Snakemake can apply the rule to map any of the threeavailable samples to the reference genome.
Test this by creating the file mapped/B.bam
.
Step 4¶
Next, create a rule sort
that sorts the obtained .bam
file bygenomic coordinate. The rule should have the input file
mapped/{sample}.bam
and the output file
mapped/{sample}.sorted.bam
and uses the shell command
to perform the sorting. Moreover, use the same conda:
directive asfor the previous rule.
Test your workflow with
and
Step 5¶
Now, we aggregate over all samples to perform a joint calling of genomicvariants. First, we define a variable
at the top of the Snakefile
. This serves as a definition of thesamples over which we would want to aggregate. In real life, you wouldwant to use an external sample sheet or a configfilefor things like this.
For aggregation over many files, Snakemake provides the helper functionexpand
(see thedocs).Create a rule call
with input files
fa='data/genome.fa'
bam=expand('mapped/{sample}.sorted.bam',sample=samples)
output file
'calls/all.vcf'
and shell command
Further, define a new conda environment file with the following content:
Snakebutt Mac Os Catalina
Step 6¶
Finally, we strive to calculate some exemplary statistics. This time, wedon't use a shell command, but rather employ Snakemake's ability tointegrate with scripting languages like R and Python.
First, we create a rule stats
with input file
'calls/all.vcf'
and output file
'plots/quals.svg'
.
Instead of a shell command, we write
and create the corresponding script and its containing folder in ourworking directory with
We open the script in the editor and add the following content
As you can see, instead of writing a command line parser for passingparameters like input and output files, you have direct access to theproperties of the rule via a magic snakemake
object, that Snakemakeautomatically inserts into the script before executing the rule.
Finally, we have to define a conda environment for the rule, sayenvs/stats.yaml
, that provides the required Python packages toexecute the script:
Make sure to test your workflow with
Step 7¶
So far, we have always specified a target file at the command line wheninvoking Snakemake. When no target file is specified, Snakemake tries toexecute the first rule in the Snakefile
. We can use this property todefine default target files.
At the top of your Snakefile
define a rule all
, with input files
'calls/all.vcf'
'plots/quals.svg'
and neither a shell command nor output files. This rule simply serves asan indicator of what shall be collected as results.
Step 8¶
As a last step, we strive to annotate our workflow with some additionalinformation.
Automatic reports¶
Snakemake can automatically create HTML reports with
Such a report contains runtime statistics, a visualization of theworkflow topology, used software and data provenance information.
In addition, you can mark any output file generated in your workflow forinclusion into the report. It will be encoded directly into the report,such that it can be, e.g., emailed as a self-contained document. Thereader (e.g., a collaborator of yours) can at any time download theenclosed results from the report for further use, e.g., in a manuscriptyou write together. In this example, please mark the output file'plots/quals.svg'
for inclusion by replacing it withreport('plots/quals.svg',caption='report/calling.rst')
and adding afile report/calling.rst
, containing some description of the outputfile. This description will be presented as caption in the resultingreport.
Threads¶
The first rule bwa
can in theory use multiple threads. You can makeSnakemake aware of this, such that the information can be used forscheduling. Add a directive threads:8
to the rule and alter theshell command to
This passes the threads defined in the rule as a command line argumentto the bwa
process.
Temporary files¶
The output of the bwa
rule becomes superfluous once the sortedversion of the .bam
file is generated by the rule sort
.Snakemake can automatically delete the superfluous output once it is notneeded anymore. For this, mark the output as temporary by replacing'mapped/{sample}.bam'
in the rule bwa
withtemp('mapped/{sample}.bam')
.
Solutions¶
Only read this if you have a problem with one of the steps.
Step 2¶
The rule should look like this:
Step 3¶
The rule should look like this:
Step 4¶
The rule should look like this:
Step 5¶
The rule should look like this:
Step 6¶
Snakebutt Mac Os Update
The rule should look like this:
Step 7¶
The rule should look like this:
It has to appear as first rule in the Snakefile
.
Step 8¶
The complete workflow should look like this:
Contents:
Stopping a Program (try in order)
+ Q | quit program |
+ . | interrupt |
+ Option + Esc | force quit |
+ Shift + Option + Esc | force quit, Cocoa & Panther only |
reset | |
power button for over 5 seconds | |
+ Control + | restart immediately, no chance to save, may corrupt disk |
Back to top.
Booting (hold down while..)
click or hold | eject CD |
C | boot from CD/DVD |
D | start from hard disk's first partition; if the first install disk is in the drive, start diagnostic tests |
N | start from the network server |
Option + N | start the default boot image from a NetBoot server |
+ Option + N + V | reset NVRAM |
+ Option + O + F | bring up Open Firmware |
+ Option + P + R | reset PRAM |
R | force PowerBook screen reset |
+ S | boot single user moder (OSX) |
T | put the Mac into FireWire target disk mode |
+ V | boot verbose: show messages while booting (OS X) |
X | start in OS X (if OS 9 is on the same disk) |
Option | show icons of all start-up disks or partitions and lets you selectolder machines: restart in OS 9 |
Shift | boot with no extensions (OS9); holding Shift down after the list ofservices begins to appear disables auto-login, holding it down afteryou've logged on disables login items |
+ Option | rebuild desktop (OS9?) |
+ Option + Shift + Delete | bypass startup checks; start up from an external drive, if present;otherwise, start up in Safe mode |
eject CD (may not work on USB); F12 is used if there is no dedicated ejectkey |
Back to top.
Login
Shift + Return | don't run startup items; while showing user list, Option + Enter on aname switches to prompt for name mode, Back goes back to name listmode |
in name mode.. | You can enter these special names:
|
Back to top.
Other
+ Shift + 3 | snapshot screen to file/desktop (which depends on version) |
+ Control + Shift + 3 | snapshot screen to clipboard |
+ Shift + 4 then click/drag | snapshot screen selection to file/desktop (which depends on version) |
+ Shift + 4 then hold Control | snapshot screen selection to clipboard |
+ Shift + 4 then Space | snapshot screen selection to desktop |
+ Shift + CAPS + 4 | snapshot window to file |
+ Shift + CAPS + 4 and then click/drag | snapshot window selection to file |
+ Control + Shift + 4 then click/drag | snapshot screen selection to clipboard |
+ Control + Shift + CAPS + 4 | snapshot window to clipboard |
+ Control + Shift + CAPS + 4 then click/drag | snapshot window selection to clipboard |
+ Shift + Q | log out |
eject CD/DVD | |
reset used for firmware upgrades | |
reset | |
sleep on/off; restart if pressed for over 5 seconds |
Back to top.
Finder
Option + click on minimize | minimize all Finder windows |
Option + double-click | open selected folder and close parent window |
+ drag file or folder location | move to a new volume without copying |
Option + drag file or folder location | copy file or folder to a new location |
+ Option + drag file or folder location | make an alias in a new location |
+ uparrow | opens the parent folder |
+ Option + uparrow | open parent folder and close the current window |
+ Option + Shift + uparrow | selects the Desktop |
+ downarrow | moves down one level |
+ rightarrow | expand folder (List view) |
+ Option + rightarrow | expand folder and nested subfolders (List view) |
+ leftarrow | collapse folder (List view) |
Tab | selects the next item alphabetically |
Shift + Tab | selects the previous item alphabetically |
+ Option + 8 | toggle zoom |
+ Option + Control + 8 | toggle white-on-black mode |
+ Option + = | zoom in |
+ Option + - | zoom out |
+ Shift + A | opens your Applications folder |
+ Option + D | toggle the Dock between show and hide |
+ Shift + H | opens your Home folder |
+ L | create an alias |
+ Option + O | open selected folder and close parent window |
+ R | when an alias is selected, jump to its target |
+ T | move an item to the sidebar |
+ Option + W | close all Finder windows |
+ Shift + Delete | empty trash |
+ Delete | move the selected item(s) to the trash |
+ Option + Shift + Delete | empty trash immediately without any warning dialog |
Back to top.
Dashboard
F12 | toggle Dashboard activation (if you don't have a dedicated ejectbutton, F12 will eject the CD) |
F12 and hold | shows Dashboard while key is held down (if you don't have a dedicatedeject button, F12 will eject the CD) |
+ = (after Dashboard activated) | toggle Widget bar |
+ R (after Widget selected) | reload Widget |
Back to top.
Spotlight
+ click | opens the file in a new Finder window |
+ Space | opens the Spotlight menu |
+ Option + Space | opens the Spotlight window |
+ Return | opens the Top Hit item |
arrow keys | navigate the list of results |
Return | opens the selected search result |
Back to top.
General Keystrokes
Snakebutt Mac Os Download
+ Tab | switches to the next open program; press ONCE and release the z key toswitch to the program used most recently |
+ Shift + Tab | switches to the previous open program |
+ Option + D | hides/shows the Dock |
+ Option + H | hides all windows except the one you're in |
+ ` | cycle through application or Finder windows (if more than one open) |
+ Shift + ` | cycle through application or Finder windows (if more than one open), in reverse order |
+ ~ | switches to the next open window in this program |
click on Dock icon (Application) | shows applications's windows |
Control + click on Dock icon (Folder) | shows folder's contents; also Force Quit option |
+ click on Dock icon | reveals the original item in the Finder |
+ Option + click on Dock icon | switches to this program and hides all others |
+ click window toolbar button | cycle through available window views |
+ click window title | show path to folder or document |
Option + click yellow button | dock all this application's open windows (some applications) |
Option + click on another application's icon | switches to this program and hides all others |
Option + empty trash | empties the trash without asking 'are you sure?' and also emptieslocked files |
Back to top.
On Keyboards Without a Power Key
Snakebutt Mac Os X
Control + | brings up the dialog box for shutdown, sleep, or restart |
+ Control + | forces a restart |
+ Option + | puts the Mac to sleep |
+ Control + Option + | shuts down |