Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

7 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
Saccharomyces_cerevisiaeSaccharomyces_cerevisiae7700
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession37932f46d81271bed2cae5ca41c9d972
Report timestampMon Sep 3 13:55:20 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/37932f46d81271bed2cae5ca41c9d972.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 25}
include: [base_teaser.yaml]
meta_timestamp: 1535974743.074496
report: {name: 37932f46d81271bed2cae5ca41c9d972}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    37932f46d81271bed2cae5ca41c9d972: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 1.0,
      mutation_indel_frac: 0.3, mutation_rate: 0.001, paired: false, platform: !!python/unicode 'illumina',
      read_length: 50, reference: !!python/unicode 'Saccharomyces_cerevisiae.fasta',
      title: !!python/unicode 'Saccharomyces_cerevisiae', type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie, bowtie2, bwa, bwamem, bwasw, ngm, segemehl]
test_parameters: []
threads: 4
Log
[13:39:06] [Main        ] framework cmd:  ./teaser.py setups_generated/37932f46d81271bed2cae5ca41c9d972.yaml -mcpu
[13:39:06] [Main        ] framework hash: vevev
[13:39:06] [Main        ] deployment mode: devel (True)
[13:39:06] [Main        ] Test Runner Setup - ""

[13:39:07] [Main        ] Using Teaser for data set creation
[13:39:07] [Teaser      ] Init. Creating 1 test datasets.
[13:39:07] [Teaser      ] 
[13:39:07] [Teaser      ] Creating test 37932f46d81271bed2cae5ca41c9d972
[13:39:07] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972
[13:39:07] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972
[13:39:07] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[13:39:07] [Teaser      ] csample /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta 500 0.500000
[13:39:07] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[13:39:07] [Teaser      ] Sampling as contig: 12157 regions of size 500 (pad 100), totalling 6078552 base pairs
[13:39:07] [Teaser      ] Sampling 12157 regions
[13:39:11] [Teaser      ] 10%
[13:39:12] [Teaser      ] 20%
[13:39:13] [Teaser      ] 30%
[13:39:15] [Teaser      ] 40%
[13:39:16] [Teaser      ] 50%
[13:39:18] [Teaser      ] 60%
[13:39:20] [Teaser      ] 70%
[13:39:22] [Teaser      ] 80%
[13:39:25] [Teaser      ] 90%
[13:39:28] [Teaser      ] 100%
[13:39:29] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972
[13:39:29] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 50 -N 300000  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta
[13:39:29] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 50 -N 300000  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta
[13:40:24] [Teaser      ] move /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta.fastq -> ./reads.fastq
[13:40:24] [Teaser      ] move /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta.fastq.sam -> ./mapping_comparison.sam
[13:40:24] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972
[13:40:24] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[13:40:24] [Teaser      ] Translating SAM file coordinates (as contig)...
[13:40:24] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.teaser.findex
[13:41:22] [Teaser      ] remove /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta
[13:41:22] [Teaser      ] remove /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.sampled.2.500.100.fasta.index
[13:41:22] [Teaser      ] remove mapping_comparison_unfixed.sam
[13:41:22] [Teaser      ] Took 134 seconds for generating 37932f46d81271bed2cae5ca41c9d972
[13:41:22] [Main        ] Data set creation completed
[13:41:22] [Main        ] 
[13:41:26] [bowtie      ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:41:26] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:41:26] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:41:26] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_bowtie.sam
[13:41:26] [map         ] Command(pre): /project/teaser/genometeaser/software/bowtie-build /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bo
[13:42:15] [map         ] Base run time file not existing, performing base run
[13:42:15] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie --threads 4  -S /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bo reads_base.fastq out_bowtie.sam
[13:42:17] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie --threads 4  -S /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bo reads.fastq out_bowtie.sam
[13:42:17] [map         ]    Mapping 36.00MB to 12.00MB with bowtie...
[13:42:33] [map         ]    Took 15.277 (wall), 25.076 (CPU) seconds, initialization time: 0.702 (wall), 0.052 (CPU) seconds.
[13:42:33] [map         ] Command(post): 
[13:42:34] [sort_prepare] Sorting...
[13:43:08] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:43:41] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
267013,4743,28244,0,0,300000,0,0,0.982546843492,0.904340963974,0.941823203348

[13:43:41] [bowtie2     ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:43:41] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:43:41] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:43:41] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_bowtie2.sam
[13:43:41] [map         ] Command(pre): 
[13:43:41] [map         ] Base run time file not existing, performing base run
[13:43:41] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[13:43:47] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta_bt -U reads.fastq -S out_bowtie2.sam
[13:43:47] [map         ]    Mapping 36.00MB to 12.00MB with bowtie2...
[13:44:06] [map         ]    Took 18.752 (wall), 38.164 (CPU) seconds, initialization time: 0.628 (wall), 0.456 (CPU) seconds.
[13:44:06] [map         ] Command(post): 
[13:44:06] [sort_prepare] Sorting...
[13:44:21] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:44:56] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
292548,5100,2352,0,0,300000,0,0,0.982865666828,0.992024415056,0.987423803641

[13:44:56] [bwa         ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:44:56] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:44:56] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:44:56] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_bwa.sam
[13:44:56] [map         ] Command(pre): 
[13:44:57] [map         ] Base run time file not existing, performing base run
[13:44:57] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[13:45:01] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta out_bwa.sam.bwa reads.fastq > out_bwa.sam
[13:45:01] [map         ]    Mapping 36.00MB to 12.00MB with bwa...
[13:45:37] [map         ]    Took 35.945 (wall), 44.500 (CPU) seconds, initialization time: 0.231 (wall), 0.100 (CPU) seconds.
[13:45:37] [map         ] Command(post): 
[13:45:37] [sort_prepare] Sorting...
[13:45:50] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:46:24] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
292517,5068,2415,0,0,300000,0,0,0.982969571719,0.991811671843,0.987370826491

[13:46:24] [bwamem      ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:46:24] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:46:24] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:46:24] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_bwamem.sam
[13:46:24] [map         ] Command(pre): 
[13:46:24] [map         ] Base run time file not existing, performing base run
[13:46:24] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[13:46:26] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads.fastq  -t 4 > out_bwamem.sam
[13:46:26] [map         ]    Mapping 36.00MB to 12.00MB with bwamem...
[13:46:55] [map         ]    Took 27.947 (wall), 37.348 (CPU) seconds, initialization time: 0.367 (wall), 0.060 (CPU) seconds.
[13:46:55] [map         ] Command(post): 
[13:46:55] [sort_prepare] Sorting...
[13:47:09] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:47:44] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
294488,5152,360,0,0,300000,0,0,0.982806033907,0.998779031908,0.990728155993

[13:47:45] [bwasw       ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:47:45] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:47:45] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:47:45] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_bwasw.sam
[13:47:45] [map         ] Command(pre): 
[13:47:45] [map         ] Base run time file not existing, performing base run
[13:47:45] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads_base.fastq  -t 4 > out_bwasw.sam
[13:47:47] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta reads.fastq  -t 4 > out_bwasw.sam
[13:47:47] [map         ]    Mapping 36.00MB to 12.00MB with bwasw...
[13:48:33] [map         ]    Took 45.102 (wall), 102.680 (CPU) seconds, initialization time: 0.489 (wall), 0.072 (CPU) seconds.
[13:48:33] [map         ] Command(post): 
[13:48:33] [sort_prepare] Sorting...
[13:48:47] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:49:20] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
280571,4843,14586,0,0,300000,0,0,0.983031666281,0.950582232507,0.966534670178

[13:49:20] [ngm         ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:49:20] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:49:20] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:49:20] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_ngm.sam
[13:49:20] [map         ] Command(pre): /project/teaser/genometeaser/software/ngm/ngm --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta 
[13:50:00] [map         ] Base run time file not existing, performing base run
[13:50:00] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -t 4 --qry reads_base.fastq  --no-progress
[13:50:17] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -t 4 --qry reads.fastq  --no-progress
[13:50:17] [map         ]    Mapping 36.00MB to 12.00MB with ngm...
[13:50:44] [map         ]    Took 25.160 (wall), 26.764 (CPU) seconds, initialization time: 5.365 (wall), 4.972 (CPU) seconds.
[13:50:44] [map         ] Command(post): 
[13:50:44] [sort_prepare] Sorting...
[13:50:56] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:51:30] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
294839,5148,13,0,0,300000,0,0,0.982839256368,0.999955910084,0.991323702716

[13:51:30] [segemehl    ] 37932f46d81271bed2cae5ca41c9d972 (base: tests_base/base_mapping)
[13:51:30] [init        ] Ref:   /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta
[13:51:30] [init        ] Reads: /project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/reads.fastq
[13:51:30] [init        ] Output:/project/teaser/genometeaser/tests_generated/37932f46d81271bed2cae5ca41c9d972/out_segemehl.sam
[13:51:31] [map         ] Command(pre): 
[13:51:31] [map         ] Base run time file not existing, performing base run
[13:51:31] [map         ] Command(pre-b): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.idx -d /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -q reads_base.fastq > out_segemehl.sam
[13:51:53] [map         ] Command(main): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta.idx -d /project/teaser/genometeaser/references/Saccharomyces_cerevisiae.fasta -q reads.fastq > out_segemehl.sam
[13:51:53] [map         ]    Mapping 36.00MB to 12.00MB with segemehl...
[13:54:32] [map         ]    Took 157.673 (wall), 129.564 (CPU) seconds, initialization time: 1.732 (wall), 0.968 (CPU) seconds.
[13:54:32] [map         ] Command(post): 
[13:54:32] [sort_prepare] Sorting...
[13:54:46] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:55:20] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
294799,5193,0,8,0,300000,0,0,0.982689538388,1.0,0.991269202123