Tests Overview

This benchmark contains multiple data sets. Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

50 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
D1D_melanogaster5500
D2D_melanogaster5500
D3D_melanogaster5500
D4D_melanogaster5500
H1Grch375500
H2Grch375500
M1M_musculus5500
M2M_musculus5500
M3M_musculus5500
M4M_musculus5500
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accessionmain
Report timestampTue Oct 6 15:47:20 2015
Framework Version1.1b-public-vevev
Framework Working Directory/project/teaser
Framework Command Line./teaser.py paper
Benchmark Configuration
description: Examples showcasing mapper differences across datasets
include: [base.yaml, base_mappers.yaml, base_ngm.yaml]
report: {directory: /home/CIBIV/moritz/public_html/paper_benchmarks, name: main}
sub_max_memory: 17000
teaser:
  dwgsim_path: software/dwgsim
  mason_path: software/mason
  reference_directory: /scratch/mate/refs
  test_directory: /scratch/mate/tests_paper
  tests:
    D1_n: {order: 3, paired: false, platform: illumina, read_length: 150, reference: /scratch/mate/refs/d_melanogaster.fasta,
      simulator: mason, title: D1}
    D2_n: {mutation_rate: 0.05, order: 5, paired: false, platform: illumina, read_length: 150,
      reference: /scratch/mate/refs/d_melanogaster.fasta, simulator: mason, title: D2}
    D3_n: {error_rate_mult: 2, order: 7, paired: false, platform: ion_torrent, read_length: 200,
      reference: /scratch/mate/refs/d_melanogaster.fasta, simulator: dwgsim, title: D3}
    D4_n: {order: 9, paired: false, platform: illumina, read_length: 22, reference: /scratch/mate/refs/d_melanogaster.fasta,
      simulator: mason, title: D4}
    H1_n: {mutation_indel_frac: 0.3, mutation_rate: 0.001, order: 1, paired: true,
      platform: illumina, read_length: 100, reference: /scratch/mate/refs/GRCh37.fa,
      simulator: dwgsim, title: H1}
    H2_n: {mutation_indel_frac: 0.7, mutation_rate: 0.001, order: 2, paired: false,
      platform: illumina, read_length: 100, reference: /scratch/mate/refs/GRCh37.fa,
      simulator: dwgsim, title: H2}
    M1_n: {order: 4, paired: false, platform: illumina, read_length: 150, reference: /scratch/mate/refs/m_musculus.fasta,
      simulator: mason, title: M1}
    M2_n: {mutation_rate: 0.05, order: 6, paired: false, platform: illumina, read_length: 150,
      reference: /scratch/mate/refs/m_musculus.fasta, simulator: mason, title: M2}
    M3_n: {error_rate_mult: 2, order: 8, paired: false, platform: ion_torrent, read_length: 200,
      reference: /scratch/mate/refs/m_musculus.fasta, simulator: dwgsim, title: M3}
    M4_n: {order: 10, paired: false, platform: illumina, read_length: 22, reference: /scratch/mate/refs/m_musculus.fasta,
      simulator: mason, title: M4}
test_directories: [tests_base, /scratch/mate/tests_paper]
test_mappers: [ngm, bowtie2, bwa, bwamem, bwasw]
threads: 4
title: Teaser Example Dataset Evaluation
Log
[15:47:16] [Main        ] framework cmd:  ./teaser.py paper
[15:47:16] [Main        ] framework hash: vevev
[15:47:16] [Main        ] deployment mode: devel (True)
[15:47:16] [Main        ] Teaser Example Dataset Evaluation - "Examples showcasing mapper differences across datasets"

[15:47:17] [Main        ] Using Teaser for data set creation
[15:47:18] [Teaser      ] Init. Creating 10 test datasets.
[15:47:18] [Teaser      ] Data set H1_n already exists, not creating.
[15:47:18] [Teaser      ] Data set H2_n already exists, not creating.
[15:47:18] [Teaser      ] Data set D1_n already exists, not creating.
[15:47:18] [Teaser      ] Data set M1_n already exists, not creating.
[15:47:18] [Teaser      ] Data set D2_n already exists, not creating.
[15:47:18] [Teaser      ] Data set M2_n already exists, not creating.
[15:47:18] [Teaser      ] Data set D3_n already exists, not creating.
[15:47:18] [Teaser      ] Data set M3_n already exists, not creating.
[15:47:18] [Teaser      ] Data set D4_n already exists, not creating.
[15:47:18] [Teaser      ] Data set M4_n already exists, not creating.
[15:47:18] [Main        ] Data set creation completed
[15:47:18] [Main        ] 
[15:47:18] [Load H1_n] Loading cached version of H1_n /project/teaser/cache/dev_cca299c02fcf620dea5c5ea7ceb21764.test
[15:47:18] [Load H2_n] Loading cached version of H2_n /project/teaser/cache/dev_74d2b798367d6f426f678df7c75fffeb.test
[15:47:18] [Load D1_n] Loading cached version of D1_n /project/teaser/cache/dev_e525765f6e09f10291b168afbc3f62a3.test
[15:47:18] [Load M1_n] Loading cached version of M1_n /project/teaser/cache/dev_54f653a0038ef37d27e2294e518e25e9.test
[15:47:18] [Load D2_n] Loading cached version of D2_n /project/teaser/cache/dev_14cb9e9467ae19b1332406ce4dbc7964.test
[15:47:19] [Load M2_n] Loading cached version of M2_n /project/teaser/cache/dev_4701577f0e3fdac2e0cba09a16901ce1.test
[15:47:19] [Load D3_n] Loading cached version of D3_n /project/teaser/cache/dev_b2b9e907a17a100d3f57042fae8f72fa.test
[15:47:19] [Load M3_n] Loading cached version of M3_n /project/teaser/cache/dev_69dfe4bff6abfb0146f2339e99d9041b.test
[15:47:19] [Load D4_n] Loading cached version of D4_n /project/teaser/cache/dev_89b3226f0f42dcfba89be33b207fc21e.test
[15:47:19] [Load M4_n] Loading cached version of M4_n /project/teaser/cache/dev_9239ebd1ee5b38b6582fdb1b9ebaff6a.test
[15:47:19] [Load H1_n] Loading cached version of H1_n /project/teaser/cache/dev_7ae5e511948ca67a75432ecb432cc00b.test
[15:47:19] [Load H2_n] Loading cached version of H2_n /project/teaser/cache/dev_2e7215f6834caad8a6bc04d2509e8330.test
[15:47:19] [Load D1_n] Loading cached version of D1_n /project/teaser/cache/dev_6ce3250281640ea491cf43dac43f8279.test
[15:47:19] [Load M1_n] Loading cached version of M1_n /project/teaser/cache/dev_1a61b5134623503d28ac63ba46a65262.test
[15:47:19] [Load D2_n] Loading cached version of D2_n /project/teaser/cache/dev_c21cbbf8399d0ed202e67f7c8f83eb6b.test
[15:47:19] [Load M2_n] Loading cached version of M2_n /project/teaser/cache/dev_f3183ea8c9274445f8a3122fb877ddbc.test
[15:47:19] [Load D3_n] Loading cached version of D3_n /project/teaser/cache/dev_b31c1fafbdffd09060e34d6ce010714c.test
[15:47:19] [Load M3_n] Loading cached version of M3_n /project/teaser/cache/dev_d9a82d05dde31c9aacf7f395dbbc9047.test
[15:47:19] [Load D4_n] Loading cached version of D4_n /project/teaser/cache/dev_04a5d1821208ca104e48fe2d2bf392a7.test
[15:47:19] [Load M4_n] Loading cached version of M4_n /project/teaser/cache/dev_111788b39477cb1b234e8808d8794152.test
[15:47:19] [Load H1_n] Loading cached version of H1_n /project/teaser/cache/dev_3e4a7ecb924d078c36900df48aa452e8.test
[15:47:19] [Load H2_n] Loading cached version of H2_n /project/teaser/cache/dev_13108935e7dedf3ddfc2599cd68ec6b7.test
[15:47:19] [Load D1_n] Loading cached version of D1_n /project/teaser/cache/dev_ed967a97032f5518b25fa6a245f7720a.test
[15:47:19] [Load M1_n] Loading cached version of M1_n /project/teaser/cache/dev_2683cf8823d91178b5c82625866590d2.test
[15:47:19] [Load D2_n] Loading cached version of D2_n /project/teaser/cache/dev_98272e98e9efbe92112c0738b35e17d3.test
[15:47:19] [Load M2_n] Loading cached version of M2_n /project/teaser/cache/dev_c8b676755e4f8bb4adc98b0f0eed62ac.test
[15:47:19] [Load D3_n] Loading cached version of D3_n /project/teaser/cache/dev_8e0fd8cb08244389184d53126a0fa3f5.test
[15:47:19] [Load M3_n] Loading cached version of M3_n /project/teaser/cache/dev_ef5219bd1505f35e002a75a9f6500cf4.test
[15:47:19] [Load D4_n] Loading cached version of D4_n /project/teaser/cache/dev_f237777af29a7481d677f4b4e3d9ed7f.test
[15:47:19] [Load M4_n] Loading cached version of M4_n /project/teaser/cache/dev_b45f802ae19832e519e0aada5c53feda.test
[15:47:19] [Load H1_n] Loading cached version of H1_n /project/teaser/cache/dev_ad272b8a5fa20a27866c27fcf3b3f782.test
[15:47:19] [Load H2_n] Loading cached version of H2_n /project/teaser/cache/dev_47262d6a264ff8d89d6d47feaf8c6919.test
[15:47:19] [Load D1_n] Loading cached version of D1_n /project/teaser/cache/dev_f0c7eeb174faacfad8a8b8dae2b1e97d.test
[15:47:19] [Load M1_n] Loading cached version of M1_n /project/teaser/cache/dev_2821cb2a8e6e18e165168c09bf3263cb.test
[15:47:19] [Load D2_n] Loading cached version of D2_n /project/teaser/cache/dev_5770fb30900dfbcaa62fe6762445f889.test
[15:47:19] [Load M2_n] Loading cached version of M2_n /project/teaser/cache/dev_68702983ef9b76b2fd1693138f380e37.test
[15:47:19] [Load D3_n] Loading cached version of D3_n /project/teaser/cache/dev_29c367627a76dc1bb0f2ea11862c22f1.test
[15:47:19] [Load M3_n] Loading cached version of M3_n /project/teaser/cache/dev_fb1d761bf78af9fbaf444a57baf8bc83.test
[15:47:19] [Load D4_n] Loading cached version of D4_n /project/teaser/cache/dev_4bdb7ecf3f72111f6faf068091a32b9b.test
[15:47:19] [Load M4_n] Loading cached version of M4_n /project/teaser/cache/dev_1b073781c435501b5d08e0f681fdab63.test
[15:47:19] [Load H1_n] Loading cached version of H1_n /project/teaser/cache/dev_253122cda578d66995928fd923180d35.test
[15:47:19] [Load H2_n] Loading cached version of H2_n /project/teaser/cache/dev_7a1a2526bc2a4a4339cdcf1e3d9a59b0.test
[15:47:19] [Load D1_n] Loading cached version of D1_n /project/teaser/cache/dev_b956b5e7c2e9090d7acb3384e1c35988.test
[15:47:19] [Load M1_n] Loading cached version of M1_n /project/teaser/cache/dev_44edbcc405ca01aff6fc132cb5e2b4c5.test
[15:47:19] [Load D2_n] Loading cached version of D2_n /project/teaser/cache/dev_dbf7839caa47f8f030f6fd323dd88a83.test
[15:47:19] [Load M2_n] Loading cached version of M2_n /project/teaser/cache/dev_d2e3d30634ca7e142c4c0bd803430e7a.test
[15:47:19] [Load D3_n] Loading cached version of D3_n /project/teaser/cache/dev_b411c89cb194ed88fa35a1de15e7c633.test
[15:47:19] [Load M3_n] Loading cached version of M3_n /project/teaser/cache/dev_832ab95dd355583e9c1f068745b217a0.test
[15:47:20] [Load D4_n] Loading cached version of D4_n /project/teaser/cache/dev_dd3913a0bbe0eeab0e497c951ac53dc3.test
[15:47:20] [Load M4_n] Loading cached version of M4_n /project/teaser/cache/dev_4fae1631c660b863b27dd1aeeabff0a5.test
[15:47:20] [ngm         ] Skip: D1_n (already run)

[15:47:20] [ngm         ] Skip: D2_n (already run)

[15:47:20] [ngm         ] Skip: D3_n (already run)

[15:47:20] [ngm         ] Skip: D4_n (already run)

[15:47:20] [ngm         ] Skip: H1_n (already run)

[15:47:20] [ngm         ] Skip: H2_n (already run)

[15:47:20] [ngm         ] Skip: M1_n (already run)

[15:47:20] [ngm         ] Skip: M2_n (already run)

[15:47:20] [ngm         ] Skip: M3_n (already run)

[15:47:20] [ngm         ] Skip: M4_n (already run)

[15:47:20] [bowtie2     ] Skip: D1_n (already run)

[15:47:20] [bowtie2     ] Skip: D2_n (already run)

[15:47:20] [bowtie2     ] Skip: D3_n (already run)

[15:47:20] [bowtie2     ] Skip: D4_n (already run)

[15:47:20] [bowtie2     ] Skip: H1_n (already run)

[15:47:20] [bowtie2     ] Skip: H2_n (already run)

[15:47:20] [bowtie2     ] Skip: M1_n (already run)

[15:47:20] [bowtie2     ] Skip: M2_n (already run)

[15:47:20] [bowtie2     ] Skip: M3_n (already run)

[15:47:20] [bowtie2     ] Skip: M4_n (already run)

[15:47:20] [bwa         ] Skip: D1_n (already run)

[15:47:20] [bwa         ] Skip: D2_n (already run)

[15:47:20] [bwa         ] Skip: D3_n (already run)

[15:47:20] [bwa         ] Skip: D4_n (already run)

[15:47:20] [bwa         ] Skip: H1_n (already run)

[15:47:20] [bwa         ] Skip: H2_n (already run)

[15:47:20] [bwa         ] Skip: M1_n (already run)

[15:47:20] [bwa         ] Skip: M2_n (already run)

[15:47:20] [bwa         ] Skip: M3_n (already run)

[15:47:20] [bwa         ] Skip: M4_n (already run)

[15:47:20] [bwamem      ] Skip: D1_n (already run)

[15:47:20] [bwamem      ] Skip: D2_n (already run)

[15:47:20] [bwamem      ] Skip: D3_n (already run)

[15:47:20] [bwamem      ] Skip: D4_n (already run)

[15:47:20] [bwamem      ] Skip: H1_n (already run)

[15:47:20] [bwamem      ] Skip: H2_n (already run)

[15:47:20] [bwamem      ] Skip: M1_n (already run)

[15:47:20] [bwamem      ] Skip: M2_n (already run)

[15:47:20] [bwamem      ] Skip: M3_n (already run)

[15:47:20] [bwamem      ] Skip: M4_n (already run)

[15:47:20] [bwasw       ] Skip: D1_n (already run)

[15:47:20] [bwasw       ] Skip: D2_n (already run)

[15:47:20] [bwasw       ] Skip: D3_n (already run)

[15:47:20] [bwasw       ] Skip: D4_n (already run)

[15:47:20] [bwasw       ] Skip: H1_n (already run)

[15:47:20] [bwasw       ] Skip: H2_n (already run)

[15:47:20] [bwasw       ] Skip: M1_n (already run)

[15:47:20] [bwasw       ] Skip: M2_n (already run)

[15:47:20] [bwasw       ] Skip: M3_n (already run)

[15:47:20] [bwasw       ] Skip: M4_n (already run)

[15:47:20] [Main        ] Generate HTML report -> /home/CIBIV/moritz/public_html/paper_benchmarks/main...