Friday, May 1, 2015

I am using modelOmatic to determine which models best fit the sequences, and it include comparisons of codon models. Installing and running modelOmatic. For awhile I thought I was having Yosemite issues but because they are all binaries, everything seems to checking out okay. Lots of "duh" moments were had.

To install on Mac OSX Yosemite:
[1] Download the binaries and move to where you store your programs.
mv ~/Downloads/modelomatic_OSX /usr/local/bin/
[2] Set modelOmatic into your path.
vi .bash_profile  #in your home directory, this opens up your bash profile
#press "i" to edit and then paste this into your .bash_profile
export PATH=/usr/local/bin/modelomatic_OSX:$PATH 
#press ESC, then type "wq!"; restart the terminal
[3] ModelOmatic requires a tree file to run. It can make a NJ tree if you have bioNJ set up. Download the binaries for bioNJ here.
[4] Move the bioNJ into where you store your programs. Set your path. You have to rename your path in lower case so that modelOmatic can read it.
mv ~/Downloads/BIONJ/ /usr/local/bin/ 
mv /usr/local/bin/BIONJ/BIONJ /usr/local/bin/BIONJ/bionj

#add to your path variable
vi .bash_profile
export PATH=/usr/local/bin/BIONJ/BIONJ:$PATH
export PATH=/usr/local/bin/BIONJ:$PATH

The format for input 
./ModelOMatic <data> bionj <output> <genetic_code> fast
I am running it as modelomatic_OSX because it is in my path. 
Using the bionj option tell modelOmatic to make its own tree using bioNJ.
Specify in <output> slot where to put your output folder.
The <genetic_code> has been tricky. It can choke and tell you that you have stop codons if you don't use the correct number:
     0:  Universal code
     1:  Vertebrate mt
     2:  Yeast mt
     3:  Mould mt
     4:  Invertebrate mt
     5:  Ciliate nuclear
     6:  Echinoderm mt
     7:  Euplotid mt
     8:  Alternative yeast nuclear
     9:  Ascidian mt
     10: Blepharisma nuclear
     11: Everything codes (64 character state-space)
My adviser was having trouble but then realized she was using cytb so it actually should have been "1" instead of "0". 
ModelOmatic uses .phylip file. Allegedly can use other formats but I have not tried. However, it actually seems ridiculously flexible in reading a .phylip file.
OR_alignments loloyohe$ modelomatic_OSX Phyllos_OR6_Funct_tAlign.phylip bionj test.txt 1

---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: <Phyllos_OR6_Funct_tAlign.phylip>: 35 sequences of length 729 (DataMatrix: 35 x 570)
Checking for sparse (>85% gaps) sequences
Starting with 35 ... after removal there are 35 sequences
Creating start tree ... 
Assertion failed: (0), function Error, file tools.cxx, line 235.
Unrecognised codon: AGA for data codon position 241 in sequence 1...Abort trap: 6
We had stop codons. Okay, so I translated the sequences, looked for where the stop codons were, and then I replaced them in the original .phylip file using "---".  Note in the future that all .phylip sequence files in /Volumes/Yango/Hayden_ORs/OR_alignments have the stop codon replaced, while the .nex files have the original sequence alignments. Lets try to get it running...
modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR4_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR4_Funct_tAlign_mOm_results.txt 0 fast
---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR4_Funct_tAlign.phylip>: 127 sequences of length 753 (DataMatrix: 127 x 666)
Checking for sparse (>85% gaps) sequences
Starting with 127 ... after removal there are 127 sequences
Creating start tree ...  estimated using bionj (1.15646s)
Optimisation settings: fast 
Scanning for modelomatic.ini file ... done
Working with genetic code: Universal
>>> Doing model analysis <<< 
RY Done  (3.56514s)
NT Done  (10.6517s)
AA Done  (71.2184s).........
Codon Done  (232.733s)

Outputting results to <Phyllos_OR4_Funct_tAlign_mOm_results.txt>
Successful exit

####now for the others
modelomatic_OSX /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR6_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR6_Funct_tAlign_mOm_results.txt 0 fast

modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR10_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR10_Funct_tAlign_mOm_results.txt 0 fast

modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR51_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR51_Funct_tAlign_mOm_results.txt 0 fast


Had a hiccup with this data:
modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR2_13_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/model_o_matic/Phyllos_OR2_13_Funct_tAlign_mOm_results.txt 0 normal

---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR2_13_Funct_tAlign.phylip>: 264 sequences of length 1530 (DataMatrix: 264 x 943)
Checking for sparse (>85% gaps) sequences
Starting with 264 ... after removal there are 264 sequences
Creating start tree ...  estimated using bionj (4.68751s)
Optimisation settings: normal 
Scanning for modelomatic.ini file ... done
Working with genetic code: Universal
>>> Doing model analysis <<< 

Broken Prob(): -0.000104812

modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR137_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR137_Funct_tAlign_mOm_results.txt 0 fast

---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR137_Funct_tAlign.phylip>: 452 sequences of length 804 (DataMatrix: 452 x 766)
Checking for sparse (>85% gaps) sequences
Starting with 452 ... after removal there are 452 sequences
Creating start tree ...  estimated using bionj (4.7425s)
Optimisation settings: fast 
Scanning for modelomatic.ini file ... done
Working with genetic code: Universal
>>> Doing model analysis <<< 
RY Done  (8.05337s)
NT Done  (44.7082s)

Broken Prob(): -40.8111

modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR52_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/Phyllos_OR52_Funct_tAlign_mOm_results.txt 0 fast

---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR51_Funct_tAlign.phylip>: 152 sequences of length 750 (DataMatrix: 152 x 712)
Checking for sparse (>85% gaps) sequences
Starting with 152 ... after removal there are 152 sequences
Creating start tree ...  estimated using bionj (1.32059s)
Optimisation settings: fast 
Scanning for modelomatic.ini file ... done
Working with genetic code: Universal
>>> Doing model analysis <<< 
RY Done  (4.69988s)
NT Done  (13.8241s)
AA Done  (85.6409s).........
...
Broken Prob(): -2.64535e+41
I think the problem is in running the "fast" option with large datasets. When I use "trimfast" and "normal", they both work. 
modelomatic_OSX  /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR51_Funct_tAlign.phylip bionj /Volumes/Yango/Hayden_batORs/model_o_matic/Phyllos_OR51_Funct_tAlign_mOm_results.txt 0 trimfast

---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR51_Funct_tAlign.phylip>: 152 sequences of length 750 (DataMatrix: 152 x 712)
Checking for sparse (>85% gaps) sequences
Starting with 152 ... after removal there are 152 sequences
Creating start tree ...  estimated using bionj (1.58694s)
Optimisation settings: fast trim=10
TRIMMING: Tree contains 152 sequences (>TrimTree=10) ...
          Loose optimisation of start tree under JC ... done
          Obtaining greedy start tree with 10 sequences ... done
          Reinitialising objects for trimmed data ... done
          New files available: data = </Volumes/Yango/Hayden_batORs/model_o_matic/Phyllos_OR51_Funct_tAlign_mOm_results.txt.trim.data>; tree = </Volumes/Yango/Hayden_batORs/model_o_matic/Phyllos_OR51_Funct_tAlign_mOm_results.txt.trim.tree>
Scanning for modelomatic.ini file ... done
Working with genetic code: Universal
>>> Doing model analysis <<< 
RY Done  (1.08313s)
NT Done  (1.18703s)
AA Done  (6.64464s).........
Codon Done  (24.4936s)

Outputting results to </Volumes/Yango/Hayden_batORs/model_o_matic/Phyllos_OR51_Funct_tAlign_mOm_results.txt>
Successful exit

However, word of caution. The "trimfast" option gives a vastly different result:


Normal it is! With fewer parameters, its best just to wait it out.

Monday, April 27, 2015

Run modelomatic:

1) Download the binary from Google Code website.

2) Move to local bin

mv ~/Downloads/modelomatic_OSX /usr/local/bin

3) Give yourself permission to run it.

sudo chmod +x /usr/local/bin/modelomatic_OSX

4) To run the binary, use the command "xcrun"

xcrun modelomatic_OSX /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR6_Funct_tAlign.fasta

But I get this error:
---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR6_Funct_tAlign.fasta>
Segmentation fault: 11

5) Okay I guess I need to make it executable (also make sure it is added to the path)


chmod a+x /usr/local/bin/modelomatic_OSX

6) Now run it
modelomatic_OSX /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR10_Funct_tAlign.fasta

Nope, still get same error.

modelomatic_OSX /Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR6_Funct_tAlign.fasta

But I get this error:
---------------------------------------------------------------------------------------------
  ModelOMatic (v1.01 (release)).
A program for choosing substitutions models for phylogenetic inference.
Written by Simon Whelan.
Contributions from James Allen, Ben Blackburne and David Talavera.
---------------------------------------------------------------------------------------------
Data: </Volumes/Yango/Hayden_batORs/OR_alignments/Phyllos_OR6_Funct_tAlign.fasta>
Segmentation fault: 11

installing garli on yosemite

sudo gem update --system
brew install open-mpi --cc=gcc-4.9 --build-from-source
brew remove ncl garli
brew install ncl garli --cc=gcc-4.9 --build-from-source

Friday, January 23, 2015

Compilation function of each BBmap shell scripts:

addadapters.sh
Randomly adds adapters to a file, or grades a trimmed file.

bbcountunique.sh
Generates a kmer uniqueness histogram, binned by file position.
There are 3 columns for single reads, 6 columns for paired:
count      number of reads or pairs processed
r1_first   percent unique 1st kmer of read 1
r1_rand    percent unique random kmer of read 1
r2_first   percent unique 1st kmer of read 2
r2_rand    percent unique random kmer of read 2
pair       percent unique concatenated kmer from read 1 and 2

bbduk.sh
Compares reads to the kmers in a reference dataset, optionally 
allowing an edit distance. Splits the reads into two outputs - those that 
match the reference, and those that don't. Can also trim (remove) the matching 
parts of the reads rather than binning the reads. Can use for quality trimming.

bbduk2.sh
Compares reads to the kmers in a reference dataset, optionally 
allowing an edit distance. Splits the reads into two outputs - those that 
match the reference, and those that don't. Can also trim (remove) the matching 
parts of the reads rather than binning the reads.

bbest.sh
Calculates EST (expressed sequence tags) capture by an assembly from a sam file.
Designed to use BBMap output generated with these flags: k=13 maxindel=100000 custom tag ordered

bbfakereads.sh
Generates fake read pairs from ends of contigs or single reads.

bbmap.sh
Fast and accurate short-read aligner for DNA and RNA.

bbmapskimmer.sh
Fast and accurate short-read aligner for DNA and RNA. (not sure difference from above)

bbmask.sh
Masks sequences of low-complexity, or containing repeat kmers, or covered by mapped reads.

bbmerge.sh
Merges paired reads into single reads by overlap detection.
With sufficient coverage, can also merge nonoverlapping reads using gapped kmers.

bbmergegapped.sh
Merges paired reads into single reads by overlap detection.
With sufficient coverage, can also merge nonoverlapping reads using gapped kmers.

bbnorm.sh
Normalizes read depth based on kmer counts.
Can also error-correct, bin reads by kmer depth, and generate a kmer depth histogram.

bbqc.sh
Performs quality-trimming; artifact, human, and phiX removal; adapter-trimming; error-correction and normalization. Designed for Illumina fragment libraries only.

bbrename.sh
Renames reads to <prefix>_<number> where you specify the prefix and the numbers are ordered.

bbsplit.sh
Maps reads to multiple references simultaneously.
Outputs reads to a file for the reference they best match, with multiple options for dealing with ambiguous mappings.

bbsplitpairs.sh
Separates paired reads into files of 'good' pairs and 'good' singletons by removing 'bad' reads that are shorter than a min length.
Designed to handle situations where reads become too short to be useful after trimming.  This program also optionally performs quality trimming.

bbwrap.sh
Wrapper for BBMap to allow multiple input and output files for the same reference.

calcmem.sh
????

calctruequality.sh
Calculates the observed quality scores from a sam file.

callpeaks.sh
No description--but appears related to keeping only certain reads within a certain region of histogram.

countbarcodes.sh
Counts the number of reads with each barcode.

countgc.sh
Counts GC content of reads or scaffolds.

crosscontaminate.sh
Generates synthetic cross-contaminated files from clean files.
Intended for use with synthetic reads generated by RandomReads.

cutprimers.sh
Cuts out sequences corresponding to primers identified in sam files.

decontaminate.sh
Decontaminates multiplexed assemblies via normalization and mapping.

dedupe.sh
Accepts one or more files containing sets of sequences (reads or scaffolds).
Removes duplicate sequences, which may be specified to be exact matches, subsequences, or sequences within some percent identity. Can also find overlapping sequences and group them into clusters.

dedupe2.sh
Accepts one or more files containing sets of sequences (reads or scaffolds).
Removes duplicate sequences, which may be specified to be exact matches, subsequences, or sequences within some percent identity.Can also find overlapping sequences and group them into clusters. (Not sure difference between dedupe.sh)

demuxbyname.sh
Demultiplexes reads based on their name (suffix or prefix) into multiple files.

ecc.sh
Corrects substitution errors in reads using kmer depth information.
Can also normalize and/or bin reads by kmer depth.

filterbarcodes.sh
Filters barcodes by quality, and generates quality histograms.

filterbycoverage.sh
Filters an assembly by contig coverage.

filterbyname.sh
Filters reads by name.

getreads.sh
Gets reads by number. The first read (or pair) has ID 0, the second read (or pair) has ID 1, etc.

grademerge.sh
Grades correctness of merging synthetic reads with headers generated by RandomReads and re-headered by RenameReads

gradesam.sh
Grades mapping correctness of a sam file of synthetic reads with headers generated by RandomReads3.java

idmatrix.sh
Generates an identity matrix via all-to-all alignment.

khist.sh
Generates a histogram of kmer counts for the input reads or assemblies.
Can also normalize, error-correct, and/or bin reads by kmer depth.

kmercount.sh
--counts kmers?

kmercountexact.sh
Counts the number of unique kmers in a file.

makechimeras.sh
Makes chimeric PacBio reads from nonchimeric reads.

mapPacBio.sh
Fast and accurate short-read aligner for DNA and RNA.

mapPacBio8k.sh
Description:  Fast and accurate short-read aligner for DNA and RNA.To index:   bbmap.sh ref=<reference fasta>
To map:     bbmap.sh in=<reads> out=<output sam>
To map without writing an index:
    bbmap.sh ref=<reference fasta> in=<reads> out=<output sam> nodisk

mapnt.sh
Maps sequences to the nt database.

matrixtocolumns.sh
Turns identity matrices into 2-column format for plotting.

mergeOTUs.sh
Merges coverage stats lines (from pileup) for the same OTU,
according to some custom naming scheme.

megebarcodes.sh
Concatenates barcodes and quality onto read names.

msa.sh
Aligns a query sequence to reference sequences.
Outputs the best matching position per reference sequence.
If there are multiple queries, only the best-matching query will be used.

phylip2fasta.sh
Calculates per-scaffold coverage information from an unsorted sam file.

pileup.sh
Calculates per-scaffold coverage information from an unsorted sam file.

printtime.sh
Prints time elapsed since last called on the same file.

randomreads.sh
Generates random synthetic reads from a reference genome.  Read names indicate their genomic origin. Allows precise customization of things like insert size and synthetic mutation type, sizes, and rates. Read names generated by this program are used by MakeRocCure (samtoroc.sh) and GradeSamFile (gradesam.sh). They can also be used by BBMap (bbmap.sh) and BBMerge (bbmerge.sh) to automatically calculate true and false positive rates, if the flag 'parsecustom' is used.

readlength.sh
Generates a length histogram of input reads.

reformat.sh
Reformats reads to change ASCII quality encoding, interleaving, file format, or compression format.

removehuman.sh
Removes all reads that map to the human genome with at least 95% identity after quality trimming.

removesmartbell.sh
Remove Smart Bell adapters from PacBio reads

repair.sh
Re-pairs reads that became disordered or had some mates eliminated. (not repair)

rqcfilter.sh
Performs quality-trimming, artifact removal, linker-trimming, adapter trimming, and spike-in removal using BBDukF. Performs human contaminant removal using BBMap.

samtoroc.sh
Creates a ROC curve from a sam file of synthetic reads with headers generated by RandomReads3.java

seal.sh
Performs high-speed alignment-free sequence quantification,
by counting the number of long kmers that match between a read and
a set of reference sequences.  Designed for RNA-seq with alternative splicing.

shuffle.sh
Reorders reads randomly.

stats.sh
Generates basic assembly statistics such as scaffold count, N50, L50, GC content, gap percent, etc.

staswraapper.sh
Runs stats.sh on multiple assemblies to produce one ouput line per file.

synthmda.sh
Generates synthetic reads following an MDA-amplified singe cell's coverage distribution.

testformat.sh
Tests the format of a sequence file based on name and contents.

textfile.sh
Translates nucleotide sequences to all 6 amino acid frames.





Thursday, January 22, 2015

Intention is to extend the ORA pipeline to scan for all chemosensory related genes.

Take 15 random V1Rs from the Young, et al (2010) paper.
-->Should you only take functional genes as training sequences?

At first I was aligning the sequences with ClustalOmega and trying to put the sequences in .msf format, as many HMMER tutorials show. However, I could not for the life of me figure out what was wrong and kept getting the following error:
hmmbuild vno.hmm vno_trainerSeqs.msf
Alignment input open failed.
   couldn't determine alignment input format
   while reading file vno_trainerSeqs.msf

So I gave up on .msf and tried PHYLIP--seemed to have worked fine

head vno_trainerSeqs.msf
 15  384
v1r-10    ----MLKLVIIENMAEIMLFSLDLLLFSTDIL----CFNFPSKMIKLPGF
v1r-06    ---------------------------------------MMNKNSRLYTD
v1r-04    ----------------------------------------------MAVD
v1r-12    -------------------------------------------------M
v1r-01    -------------------------------------MSAHGNSLKTTEE
v1r-14    ---------------------------------------------MLTYD
v1r-15    ---------------------------------------------MSSHK
v1r-03    ---------------------------------------------MDITE
v1r-02    -------------------------------------------MKMTSSN
v1r-08    ---------------------------------------------MSSAK
v1r-11    MVGDTLKLLSPL-MTRY----FFLLFYSTDSSDLNENQHPLDFDEMAFGK
v1r-05    ---------------------------------------------MDARN
v1r-13    ---------------------------------------------MASKD
v1r-07    -------MIHVD-RDSY----PL-----AGFSSSEDKYLSLTTDRKASRE
v1r-09    ---------------------------------------------MATGD

ITIQIFFYPQASFGISANTILLLFHIFTFVFSHRSKSIDMIISHLSLIHI
SNIRNTFFAEIGIGVSANSLLLLFNIFKLICGQRSRLTDLPIGLLSLINL
VAQGVSFLYQTGLGILGNSLLITLYLTSFLLGSKLKPTDLTIIHLALVHT
LSFKKAFYFQAGIGISANIFLLLWHIFTFFKDHKPKNHDLIICHLAFAHI
VALQILLLCQFGVGTVANVFLFVHNFSPVLTGSKQRPRQVILSHMAVANA
DFMCIFHKLQTIIGLFGNSFLLYLYILKLIINQRITLIDKICINLVFSNI
VGLEIVYLTLLLFGILGNMFLIYLQSLKFITDHRKRVINLIIINLALAHT
LSFGIAIVMQFSIGVSVNVFVFLFYAQIISTSYKASFSDLILAHLAFANT
LVVGILLFSQIVMGMLGNSSILFYYVILIFTGKHLTPKDLIIEHLTFANC
WETRIILVAQMGVGILGNTSLFCLCNFTLFTGQKVRCTDIILSQLALANS
VKSGISFLIQTGVGILGNSFLLCFYNLILFTGHKLRPTDLILSQLALANS
LGVGITYLLQSVVGLLGNISLIFYYLIIYYKKHKIKPMDLILMHLIIVNI
FAIGMIL-SQIMVGFLGNFFLLYHYSFLHFTRGMLQSTDLTLKHLTIANS
LVVGIILSLQTTFGILGNFSLLYHYLFLYFTGSRLRSTDLIVKNLIVANL
LVVGMVFLSQTILGILGNFSLLCHYLLLHFTGCRARCTDLILRHLTIANS

Alrighty. Will this now work with ORA?
It appears so! Actually is not too difficult.
[1] Put the alignment file of the training receptors and run "hmmbuild" to make the HMMER probability file.
hmmbuild vno.hmm vno_trainerSeqs.phy
# hmmbuild :: profile HMM construction from multiple sequence alignments
# HMMER 3.1b1 (May 2013); http://hmmer.org/
# Copyright (C) 2013 Howard Hughes Medical Institute.
# Freely distributed under the GNU General Public License (GPLv3).
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# input alignment file:             vno_trainerSeqs.phy
# output HMM file:                  vno.hmm
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# idx name                  nseq  alen  mlen eff_nseq re/pos description
#---- -------------------- ----- ----- ----- -------- ------ -----------
1     vno_trainerSeqs         15   384   315     2.03  0.590 
[3] In the "scripts" folder of the ORA pipeline, append the vno.hmm text to the or.hmm text. Make sure there is a new line after the last "//" or it will not press properly.

[4] Delete "or.hmm.h3f", "or.hmm.h3i", "or.hmm.h3m", and "or.hmm.h3p". Then repress the new or.hmm:
hmmpress or.hmm
Looks like with V2R, the ORA is picking up things like glutamate receptors (GRM3), and GABA receptors. Wonder why....will need to think about and retrain. I'm assuming the introns are throwing it off.


Troubleshooting fastx-toolkit. Trying to resolve this error. I have uninstalled the newer versions of both libgtextutils and fastx_toolkit and am reverting to hopefully a more stable version. 
The "Abort trap: 6" was never really resolved for me but after running this version and compiling from source, the error does go away (can't say it is resolved). However, a new suite of errors with the older version pops up. Curiously, this is accounted for in the newer version. The errors are similar to the installation of phylobayes. I have troubleshooted them here and have a successful installation. Now hopefully I can move on with my life. I am going to try something new.

Anyways, here is the installation error workaround:

cd /usr/local/bin/
wget http://cancan.cshl.edu/labmembers/gordon/files/libgtextutils-0.6.tar.bz2 #note this is not latest version
tar -xjf libgtextutils-0.6.tar.bz2
cd libgtextutils-0.6
./configure
make
sudo make install

wget http://cancan.cshl.edu/labmembers/gordon/files/fastx_toolkit-0.0.12.tar.bz2 #note this is not latest version
tar -xjf fastx_toolkit-0.0.12.tar.bz2  
cd fastx_toolkit-0.0.12
./configure
make
g++ -DHAVE_CONFIG_H -I. -I../..   -I../libfastx   -g -O2 -Wall -Wextra -Wformat-nonliteral -Wformat-security -Wswitch-default -Wswitch-enum -Wunused-parameter -Wfloat-equal -Werror -DDEBUG -g -O1 -DDEBUG -g -O1 -MT fastx_collapser.o -MD -MP -MF .deps/fastx_collapser.Tpo -c -o fastx_collapser.o fastx_collapser.cpp
fastx_collapser.cpp:50:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
         ^
1 error generated.
make[3]: *** [fastx_collapser.o] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
cd src/
cd fastx_collapser
vi fastx_collapser.cpp
#press i to edit in vi
#original
#include <tr/unordered_map>
#new
#include <unordered_map>
#save and exit, press ESC and then type ":wq!"

cd ../..
make
fastx_collapser.cpp:51:6: error: no member named 'tr1' in namespace 'std'
std::tr1::unordered_map<string,size_t> collapsed_sequences;
~~~~~^
1 error generated.
make[3]: *** [fastx_collapser.o] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
cd src/fastx_collapser
vi fastx_collapser.cpp
#original
std::tr1::unordered_map<string,size_t> collapsed_sequences;
#new
std::unordered_map<string,size_t> collapsed_sequences;
sudo make install

Wednesday, January 21, 2015

Downloading new transcriptome data from BGI!

#Enter the ftp address in terminal window
ftp 128.120.88.244
Connected to 128.120.88.244.

220 (vsFTPd 2.2.2)

#input given username and password
#copy from ftp into your current directory
get DR_086_MO_E_1.fq.gz
get DR_086_MO_E_2.fq.gz

#or we can get multiple files
mget DR_086_VN*.fq.gz

#or we can get all the files!
mget *.fq.gz
exit

Now we can unzip the data, concatenate the two files for each species, and run the quality control filter