Biological file format & Data submission

View Github Repository Open presentation in a new window

yumyai

See all presentation from yumyai

Biological file format & Data submission

0 0

biological-fileformat-2013

Presentation for biological file format

On Github yumyai / biological-fileformat-2013

Biological file format & Data submission

Preecha Patumcharoenpol

Goal

Understanding concept of file format! Having an experience with commonly used file format and tools
  • Preparation, Validation, Conversion

Life cycle of data

File format

A standard way that information is encoded for storage in a computer.

syn6|YP_007452940.1  syn6|YP_007452940.1  100.00  230  0    0   1   230  1   230  3e-163    453
syn6|YP_007452940.1  cya5|YP_001805484.1  68.56   229  72   0   2   230  35  263  5e-107    312
syn6|YP_007452938.1  syn6|YP_007452938.1  100.00  230  0    0   1   230  1   230  2e-172    477
syn6|YP_007452938.1  cya5|YP_001801448.1  75.59   213  51   1   19  230  11  223  2e-118    339
syn6|YP_007452938.1  apc1|SPLC1_S550780   53.37   208  75   3   23  230  19  204  1e-63     199
syn6|YP_007452938.1  apc1|SPLC1_S040720   68.69   99   31   0   29  127  8   106  1e-47     155
syn6|YP_007452938.1  ecol|NP_416530.1     39.38   193  111  3   38  230  17  203  2e-38     134
syn6|YP_007452938.1  cya5|YP_001806258.1  46.94   98   51   1   33  129  23  120  2e-24   95.9
syn6|YP_007452936.1  syn6|YP_007452936.1  100.00  317  0    0   1   317  1   317  0.0       644
syn6|YP_007452936.1  cya5|YP_001802341.1  54.57   317  137  1   1   317  1   310  2e-125    364
syn6|YP_007452936.1  apc1|SPLC1_S510950   48.28   319  161  4   1   316  1   318  2e-108    321
syn6|YP_007452936.1  ecol|NP_418077.1     24.59   122  79   2   1   115  1   116  7e-05   42.7
syn6|YP_007452936.1  ecol|NP_418089.1     22.07   290  188  12  13  280  6   279  9e-04   39.3
syn6|YP_007452934.1  syn6|YP_007452934.1  100.00  603  0    0   1   603  1   603  0.0      1229
syn6|YP_007452934.1  cya5|YP_001806132.1  88.23   603  71   0   1   603  1   603  0.0      1085

Examples

  • Common: Delimited (Tabular), XML, JSON
  • Sequence: FASTA, GENBANK, EMBL, GCC, GFF, BED
  • Alignment: CIGAR, SAM, PSL, BAM, BLAST
  • Data: SBML, KGML

http://xkcd.com/927/

IUPAC Nucleic acid code

Code Description A Adenine C Cytosine G Gwuanine T Thymine U Uracil N Any base (A, C, G, T, or U)

IUPAC Amino acid code

1-letter 3-letter description A Ala Alanine R Arg Arginine N Asn Asparagine D Asp Aspartic acid C Cys Cysteine Q Gln Glutamine E Glu Glutamic acid G Gly Glycine H His Histidine I Ile Isoleucine L Leu Leucine K Lys Lysine M Met Methionine F Phe Phenylalanine P Pro Proline S Ser Serine T Thr Threonine W Trp Tryptophan Y Tyr Tyrosine V Val Valine B Asx Aspartic acid or Asparagine Z Glx Glutamine or Glutamic acid X Xaa Any amino acid

FASTA

>SPLC1_S230110 putative signaling protein with GGDEF and EAL domain protein [Arthrospira platensis C1]
MLSLVAKIIQNLVRDTDLLARLGGDEFVIVLEDLEATNEATRVAERILESLRSSPLQVGK
RDVFVNSSIGIVVRTNRHEKAEDLLRDADLAMYRAKHEGRGRYAIFDPLMHFQAVQQMHL
ENDLRKAIENNQLVLYYQPIVNIKNQRIQGLEALVRWQHPERGLLAPGHFINIAENTGLI
IPIGRWLLHTACQQLAEWENQFPHHFLKMSVNLSVKQLDIFLLEQLDEVLNNYNLKQNSL
VLEITESMLVANIEKTCDLLNQIKAKGIGLSIDDFGTGYSSLSYLHQLPVNSLKIDRSFV
SPANLSDRHQVIAKSIIALSKLLKLHVIAEGVETPEQFHWLKKLGCEAAQGYLFSRPVPA
SDITEL
          
>gi|493673229|ref|WP_006623555.1| MULTISPECIES: diguanylate cyclase [Arthrospira]
MLSLVAKIIQNLVRDTDLLARLGGDEFVIVLEDLEATNEATRVAERILESLRSSPLQVGKRDVFVNSSIG
IVVRTNRHEKAEDLLRDADLAMYRAKHEGRGRYAIFDPLMHFQAVQQMHLENDLRKAIENNQLVLYYQPI
VNIKNQRIQGLEALVRWQHPERGLLAPGHFINIAENTGLIIPIGRWLLHTACQQLAEWENQFPHHFLKMS
VNLSVKQLDIFLLEQLDEVLNNYNLKQNSLVLEITESMLVANIEKTCDLLNQIKAKGIGLSIDDFGTGYS
SLSYLHQLPVNSLKIDRSFVSPANLSDRHQVIAKSIIALSKLLKLHVIAEGVETPEQFHWLKKLGCEAAQ
GYLFSRPVPASDITEL

Multi-FASTA

>gi|459201371|ref|YP_007507330.1| 3-hydroxypropionic acid resistance peptide [Escherichia coli str. K-12 substr. MG1655]
MKPALRDFIAIVQERLASVTA
>gi|459201369|ref|NP_414883.5| 2-hydroxy-6-ketonona-2,4-dienedioic acid hydrolase [Escherichia coli str. K-12 substr. MG1655]
MSYQPQTEAATSRFLNVEEAGKTLRIHFNDCGQGDETVVLLHGSGPGATGWANFSRNIDP
LVEAGYRVILLDCPGWGKSDSVVNSGSRSDLNARILKSVVDQLDIAKIHLLGNSMGGHSS
VAFTLKWPERVGKLVLMGGGTGGMSLFTPMPTEGIKRLNQLYRQPTIENLKLMMDIFVFD
TSDLTDALFEARLNNMLSRRDHLENFVKSLEANPKQFPDFGPRLAEIKAQTLIVWGRNDR
FVPMDAGLRLLSGIAGSELHIFRDCGHWAQWEHADAFNQLVLNFLARP
>gi|459201370|ref|YP_007507329.1| Mn(2)-response protein, MntR-repressed [Escherichia coli str. K-12 substr. MG1655]
MNEFKRCMRVFSHSPFKVRLMLLSMLCDMVNNKPQQDKPSDK

GENBANK

LOCUS       WP_006623555             366 aa            linear   BCT 08-MAY-2013
DEFINITION  MULTISPECIES: diguanylate cyclase [Arthrospira].
ACCESSION   WP_006623555
VERSION     WP_006623555.1  GI:493673229
KEYWORDS    RefSeq.
SOURCE      Arthrospira
  ORGANISM  Arthrospira
            Bacteria; Cyanobacteria; Oscillatoriophycideae; Oscillatoriales.
COMMENT     REFSEQ: This record represents a single, non-redundant, protein
            sequence which may be annotated on many different RefSeq genomes
            from the same, or different, species.
FEATURES             Location/Qualifiers
     source          1..366
                     /organism="Arthrospira"
                     /db_xref="taxon:35823"
     Protein         1..366
                     /product="diguanylate cyclase"
                     /calculated_mol_wt=41355
     Region          <2..104
                     /region_name="GGDEF"
                     /note="Diguanylate-cyclase (DGC) or GGDEF domain; cd01949"
                     /db_xref="CDD:143635"
     Site            order(17,46)
                     /site_type="other"
                     /note="I-site"
                     /db_xref="CDD:143635"
     Site            order(21,23..26)
                     /site_type="active"
                     /db_xref="CDD:143635"
     Site            25
                     /site_type="metal-binding"
                     /note="metal binding site [ion binding]"
                     /db_xref="CDD:143635"
     Region          122..362
                     /region_name="EAL"
                     /note="EAL domain. This domain is found in diverse
                     bacterial signaling proteins. It is called EAL after its
                     conserved residues and is also known as domain of unknown
                     function 2 (DUF2).  The EAL domain has been shown to
                     stimulate degradation of a second...; cd01948"
                     /db_xref="CDD:238923"
ORIGIN      
        1 mlslvakiiq nlvrdtdlla rlggdefviv ledleatnea trvaeriles lrssplqvgk
       61 rdvfvnssig ivvrtnrhek aedllrdadl amyrakhegr gryaifdplm hfqavqqmhl
      121 endlrkaien nqlvlyyqpi vniknqriqg lealvrwqhp ergllapghf iniaentgli
      181 ipigrwllht acqqlaewen qfphhflkms vnlsvkqldi flleqldevl nnynlkqnsl
      241 vleitesmlv aniektcdll nqikakgigl siddfgtgys slsylhqlpv nslkidrsfv
      301 spanlsdrhq viaksiials kllklhviae gvetpeqfhw lkklgceaaq gylfsrpvpa
      361 sditel
//

GCG

ID   AB000263 standard; RNA; PRI; 368 BP.
XX
AC   AB000263;
XX
DE   Homo sapiens mRNA for prepro cortistatin like peptide, complete cds.
XX
SQ   Sequence 368 BP;
AB000263  Length: 368  Check: 4514  ..
       1  acaagatgcc attgtccccc ggcctcctgc tgctgctgct ctccggggcc acggccaccg
      61  ctgccctgcc cctggagggt ggccccaccg gccgagacag cgagcatatg caggaagcgg
     121  caggaataag gaaaagcagc ctcctgactt tcctcgcttg gtggtttgag tggacctccc
     181  aggccagtgc cgggcccctc ataggagagg aagctcggga ggtggccagg cggcaggaag
     241  gcgcaccccc ccagcaatcc gcgcgccggg acagaatgcc ctgcaggaac ttcttctgga
     301  agaccttctc ctcctgcaaa taaaacctca cccatgaatg ctcacgcaag tttaattaca
     361  gacctgaa

GFF

##gff-version 3
0421  .      contig           1      26153  .  .  .  ID=0421;Name=0421
0421  maker  gene             33     1875   .  +  .  ID=maker-0421-snap-gene-0.22;Name=maker-0421-snap-gene-0.22
0421  maker  mRNA             33     1875   .  +  .  ID=maker-0421-snap-gene-0.22-mRNA-1;Parent=maker-0421-snap-gene-0.22;Name=maker-0421-snap-gene-0.22-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.5|0.33|1|1|1|3|105|532
0421  maker  exon             33     371    .  +  .  ID=maker-0421-snap-gene-0.22-mRNA-1:exon:46666;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  exon             438    752    .  +  .  ID=maker-0421-snap-gene-0.22-mRNA-1:exon:46667;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  exon             826    1875   .  +  .  ID=maker-0421-snap-gene-0.22-mRNA-1:exon:46668;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  CDS              33     371    .  +  0  ID=maker-0421-snap-gene-0.22-mRNA-1:cds;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  CDS              438    752    .  +  0  ID=maker-0421-snap-gene-0.22-mRNA-1:cds;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  CDS              826    1770   .  +  0  ID=maker-0421-snap-gene-0.22-mRNA-1:cds;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  three_prime_UTR  1771   1875   .  +  .  ID=maker-0421-snap-gene-0.22-mRNA-1:three_prime_utr;Parent=maker-0421-snap-gene-0.22-mRNA-1
0421  maker  gene             18315  21039  .  +  .  ID=maker-0421-snap-gene-0.23;Name=maker-0421-snap-gene-0.23
0421  maker  mRNA             18315  21039  .  +  .  ID=maker-0421-snap-gene-0.23-mRNA-1;Parent=maker-0421-snap-gene-0.23;Name=maker-0421-snap-gene-0.23-mRNA-1;_AED=0.36;_eAED=0.37;_QI=0|0|0|0.6|0.5|0.8|5|0|798
0421  maker  exon             18315  19733  .  +  .  ID=maker-0421-snap-gene-0.23-mRNA-1:exon:46669;Parent=maker-0421-snap-gene-0.23-mRNA-1
0421  maker  exon             19801  19916  .  +  .  ID=maker-0421-snap-gene-0.23-mRNA-1:exon:46670;Parent=maker-0421-snap-gene-0.23-mRNA-1
0421  maker  exon             19982  20174  .  +  .  ID=maker-0421-snap-gene-0.23-mRNA-1:exon:46671;Parent=maker-0421-snap-gene-0.23-mRNA-1
0421  maker  exon             20243  20731  .  +  .  ID=maker-0421-snap-gene-0.23-mRNA-1:exon:46672;Parent=maker-0421-snap-gene-0.23-mRNA-1

Common format

  • Delimited (Tabular)
  • JSON
  • XML

Tabular format (Tab-delimited, CSV)

syn6|YP_007452940.1  syn6|YP_007452940.1  100.00  230  0    0   1   230  1   230  3e-163    453
syn6|YP_007452940.1  cya5|YP_001805484.1  68.56   229  72   0   2   230  35  263  5e-107    312
syn6|YP_007452938.1  syn6|YP_007452938.1  100.00  230  0    0   1   230  1   230  2e-172    477
syn6|YP_007452938.1  cya5|YP_001801448.1  75.59   213  51   1   19  230  11  223  2e-118    339
syn6|YP_007452938.1  apc1|SPLC1_S550780   53.37   208  75   3   23  230  19  204  1e-63     199
syn6|YP_007452938.1  apc1|SPLC1_S040720   68.69   99   31   0   29  127  8   106  1e-47     155
syn6|YP_007452938.1  ecol|NP_416530.1     39.38   193  111  3   38  230  17  203  2e-38     134
syn6|YP_007452938.1  cya5|YP_001806258.1  46.94   98   51   1   33  129  23  120  2e-24   95.9
syn6|YP_007452936.1  syn6|YP_007452936.1  100.00  317  0    0   1   317  1   317  0.0       644
syn6|YP_007452936.1  cya5|YP_001802341.1  54.57   317  137  1   1   317  1   310  2e-125    364
syn6|YP_007452936.1  apc1|SPLC1_S510950   48.28   319  161  4   1   316  1   318  2e-108    321
syn6|YP_007452936.1  ecol|NP_418077.1     24.59   122  79   2   1   115  1   116  7e-05   42.7
syn6|YP_007452936.1  ecol|NP_418089.1     22.07   290  188  12  13  280  6   279  9e-04   39.3
syn6|YP_007452934.1  syn6|YP_007452934.1  100.00  603  0    0   1   603  1   603  0.0      1229
syn6|YP_007452934.1  cya5|YP_001806132.1  88.23   603  71   0   1   603  1   603  0.0      1085

JSON

XML

Extensible Markup Language

FORMULA: C20H21N7O7

CHARGE: -2

FORMULA: C27H52O5

CHARGE: 0

FORMULA: C31H60O5

CHARGE: 0

FORMULA: C31H56O5

CHARGE: 0

FORMULA: C35H68O5

CHARGE: 0

FORMULA: C35H64O5

CHARGE: 0

FORMULA: C39H76O5

CHARGE: 0

FORMULA: C39H72O5

CHARGE: 0

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

GENE_ASSOCIATION:

SUBSYSTEM:

EC Number:

FLUX_VALUE

Tools

  • Your editor of choice.
  • Your programming language of choice.
  • Google.

Exercise

Download goo.gl/YTHDdT 

            curl -L goo.gl/YTHDdT > NC_005213.gbk

Excercise

Readseq

  • http://www.ebi.ac.uk/cgi-bin/readseq.cgi (EMBL-EBI)
  • http://www-bimas.cit.nih.gov/molbio/readseq/ (NIH)