Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
Saccharomyces_cerevisiaeSaccharomyces_cerevisiae5500
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accessionb0bfc21db96b7744c4389e9b4b565dc9
Report timestampThu Sep 6 23:58:09 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/b0bfc21db96b7744c4389e9b4b565dc9.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 75}
include: [base_teaser.yaml]
meta_timestamp: 1536270601.450902
report: {name: b0bfc21db96b7744c4389e9b4b565dc9}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    b0bfc21db96b7744c4389e9b4b565dc9: {coverage: 1.0, error_rate_mult: 1.0, insert_size: 100,
      insert_size_error: 50, mutation_indel_avg_len: 1.0, mutation_indel_frac: 0.3,
      mutation_rate: 0.02, paired: true, platform: !!python/unicode 'illumina', read_length: 150,
      reference: !!python/unicode 'Saccharomyces_cerevisiae.fasta', title: !!python/unicode 'Saccharomyces_cerevisiae',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, ngm]
test_parameters: []
threads: 4
Log
[23:50:05] [Main        ] framework cmd:  ./teaser.py setups_generated/b0bfc21db96b7744c4389e9b4b565dc9.yaml -mcpu
[23:50:05] [Main        ] framework hash: vevev
[23:50:05] [Main        ] deployment mode: devel (True)
[23:50:05] [Main        ] Test Runner Setup - ""

[23:50:05] [Main        ] Using Teaser for data set creation
[23:50:05] [Teaser      ] Init. Creating 1 test datasets.
[23:50:05] [Teaser      ] 
[23:50:05] [Teaser      ] Creating test b0bfc21db96b7744c4389e9b4b565dc9
[23:50:05] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9
[23:50:05] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9
[23:50:05] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[23:50:05] [Teaser      ] csample /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta 1500 0.500000
[23:50:05] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[23:50:05] [Teaser      ] Sampling as contig: 4052 regions of size 1500 (pad 300), totalling 6078552 base pairs
[23:50:05] [Teaser      ] Sampling 4052 regions
[23:50:07] [Teaser      ] 10%
[23:50:07] [Teaser      ] 20%
[23:50:08] [Teaser      ] 30%
[23:50:08] [Teaser      ] 40%
[23:50:09] [Teaser      ] 50%
[23:50:09] [Teaser      ] 60%
[23:50:10] [Teaser      ] 70%
[23:50:11] [Teaser      ] 80%
[23:50:12] [Teaser      ] 90%
[23:50:12] [Teaser      ] 100%
[23:50:13] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9
[23:50:13] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 150 -N 50000 --mp  --ll 100 --le 50  --hi 0.006000  --hs 0.014000  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta
[23:50:13] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 150 -N 50000 --mp  --ll 100 --le 50  --hi 0.006000  --hs 0.014000  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta
[23:52:23] [Teaser      ] move /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta_1.fastq -> ./reads1.fastq
[23:52:23] [Teaser      ] move /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta_2.fastq -> ./reads2.fastq
[23:52:23] [Teaser      ] move /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta.fastq.sam -> ./mapping_comparison.sam
[23:52:23] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9
[23:52:23] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[23:52:23] [Teaser      ] Translating SAM file coordinates (as contig)...
[23:52:23] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[23:52:48] [Teaser      ] remove /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta
[23:52:48] [Teaser      ] remove /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.1500.300.fasta.index
[23:52:48] [Teaser      ] remove mapping_comparison_unfixed.sam
[23:52:48] [Teaser      ] Took 162 seconds for generating b0bfc21db96b7744c4389e9b4b565dc9
[23:52:48] [Main        ] Data set creation completed
[23:52:48] [Main        ] 
[23:52:53] [bowtie2     ] b0bfc21db96b7744c4389e9b4b565dc9 (base: tests_base/base_mapping)
[23:52:53] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[23:52:53] [init        ] Reads: /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/reads1.fastq
[23:52:53] [init        ] Output:/project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/out_bowtie2.sam
[23:52:53] [map         ] Command(pre): 
[23:52:53] [map         ] Base run time file not existing, performing base run
[23:52:53] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[23:53:00] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bt -1 reads1.fastq -2 reads2.fastq -S out_bowtie2.sam
[23:53:00] [map         ]    Mapping 32.00MB to 12.00MB with bowtie2...
[23:53:18] [map         ]    Took 17.308 (wall), 43.740 (CPU) seconds, initialization time: 0.644 (wall), 0.284 (CPU) seconds.
[23:53:18] [map         ] Command(post): 
[23:53:18] [sort_prepare] Sorting...
[23:53:36] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[23:53:55] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
98584,1186,230,0,0,100000,0,0,0.988112659116,0.9976723946,0.992869516175

[23:53:56] [bwa         ] b0bfc21db96b7744c4389e9b4b565dc9 (base: tests_base/base_mapping)
[23:53:56] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[23:53:56] [init        ] Reads: /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/reads1.fastq
[23:53:56] [init        ] Output:/project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/out_bwa.sam
[23:53:56] [map         ] Command(pre): 
[23:53:56] [map         ] Base run time file not existing, performing base run
[23:53:56] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[23:54:01] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads1.fastq  -t 4 > out_bwa.sam_1.sai; /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads2.fastq  -t 4 > out_bwa.sam_2.sai; /project/teaser/genometeaser/software/bwa sampe /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta out_bwa.sam_1.sai out_bwa.sam_2.sai reads1.fastq reads2.fastq > out_bwa.sam
[23:54:01] [map         ]    Mapping 32.00MB to 12.00MB with bwa...
[23:54:40] [map         ]    Took 38.495 (wall), 64.012 (CPU) seconds, initialization time: 0.977 (wall), 0.052 (CPU) seconds.
[23:54:40] [map         ] Command(post): 
[23:54:40] [sort_prepare] Sorting...
[23:54:47] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[23:54:57] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
66856,950,32194,0,0,100000,0,0,0.985989440462,0.674972236244,0.801361653162

[23:54:58] [bwamem      ] b0bfc21db96b7744c4389e9b4b565dc9 (base: tests_base/base_mapping)
[23:54:58] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[23:54:58] [init        ] Reads: /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/reads1.fastq
[23:54:58] [init        ] Output:/project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/out_bwamem.sam
[23:54:58] [map         ] Command(pre): 
[23:54:58] [map         ] Base run time file not existing, performing base run
[23:54:58] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[23:55:00] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads1.fastq reads2.fastq  -t 4 > out_bwamem.sam
[23:55:00] [map         ]    Mapping 32.00MB to 12.00MB with bwamem...
[23:55:21] [map         ]    Took 20.529 (wall), 31.716 (CPU) seconds, initialization time: 0.148 (wall), 0.036 (CPU) seconds.
[23:55:21] [map         ] Command(post): 
[23:55:21] [sort_prepare] Sorting...
[23:55:30] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[23:55:41] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
98913,1087,0,0,0,100000,0,0,0.98913,1.0,0.994535299352

[23:55:41] [bwasw       ] b0bfc21db96b7744c4389e9b4b565dc9 (base: tests_base/base_mapping)
[23:55:41] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[23:55:41] [init        ] Reads: /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/reads1.fastq
[23:55:41] [init        ] Output:/project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/out_bwasw.sam
[23:55:41] [map         ] Command(pre): 
[23:55:41] [map         ] Base run time file not existing, performing base run
[23:55:41] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwasw.sam
[23:55:43] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads1.fastq reads2.fastq  -t 4 > out_bwasw.sam
[23:55:43] [map         ]    Mapping 32.00MB to 12.00MB with bwasw...
[23:56:21] [map         ]    Took 37.480 (wall), 94.112 (CPU) seconds, initialization time: 0.105 (wall), 0.036 (CPU) seconds.
[23:56:21] [map         ] Command(post): 
[23:56:21] [sort_prepare] Sorting...
[23:56:30] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[23:56:43] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
98748,1248,4,0,4,100000,0,0,0.98751950078,0.999959494491,0.99370056554

[23:56:43] [ngm         ] b0bfc21db96b7744c4389e9b4b565dc9 (base: tests_base/base_mapping)
[23:56:43] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[23:56:43] [init        ] Reads: /project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/reads1.fastq
[23:56:43] [init        ] Output:/project/teaser/genometeaser/tests_generated/b0bfc21db96b7744c4389e9b4b565dc9/out_ngm.sam
[23:56:43] [map         ] Command(pre): /project/teaser/genometeaser/software/ngm/ngm --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta 
[23:57:19] [map         ] Base run time file not existing, performing base run
[23:57:19] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -t 4 --qry reads_base.fastq  --no-progress
[23:57:35] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --qry2 reads2.fastq --output out_ngm.sam --qry1 reads1.fastq --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -t 4  --no-progress
[23:57:35] [map         ]    Mapping 32.00MB to 12.00MB with ngm...
[23:57:52] [map         ]    Took 16.474 (wall), 21.884 (CPU) seconds, initialization time: 5.591 (wall), 4.764 (CPU) seconds.
[23:57:52] [map         ] Command(post): 
[23:57:53] [sort_prepare] Sorting...
[23:58:00] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[23:58:09] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
98825,1175,0,0,0,100000,0,0,0.98825,1.0,0.994090280397