Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
Arabidopsis_thalianaArabidopsis_thaliana5500
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession0542e1134664dca94011ee7ee39d31ff
Report timestampThu Jul 13 13:05:57 2017
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/0542e1134664dca94011ee7ee39d31ff.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 25}
include: [base_teaser.yaml]
meta_timestamp: 1499942626.859486
report: {name: 0542e1134664dca94011ee7ee39d31ff}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    0542e1134664dca94011ee7ee39d31ff: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 1.0,
      mutation_indel_frac: 0.3, mutation_rate: 0.001, paired: false, platform: !!python/unicode 'illumina',
      read_length: 50, reference: !!python/unicode 'Arabidopsis_thaliana.fas', title: !!python/unicode 'Arabidopsis_thaliana',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, ngm]
test_parameters: []
threads: 4
Log
[12:43:51] [Main        ] framework cmd:  ./teaser.py setups_generated/0542e1134664dca94011ee7ee39d31ff.yaml -mcpu
[12:43:51] [Main        ] framework hash: vevev
[12:43:51] [Main        ] deployment mode: devel (True)
[12:43:51] [Main        ] Test Runner Setup - ""

[12:43:52] [Main        ] Using Teaser for data set creation
[12:43:52] [Teaser      ] Init. Creating 1 test datasets.
[12:43:52] [Teaser      ] 
[12:43:52] [Teaser      ] Creating test 0542e1134664dca94011ee7ee39d31ff
[12:43:52] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff
[12:43:52] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff
[12:43:52] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.teaser.findex
[12:43:52] [Teaser      ] csample /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas 500 0.250000
[12:43:52] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.teaser.findex
[12:43:52] [Teaser      ] Sampling as contig: 59833 regions of size 500 (pad 100), totalling 29916937 base pairs
[12:43:52] [Teaser      ] Sampling 59833 regions
[12:44:11] [Teaser      ] 10%
[12:44:17] [Teaser      ] 20%
[12:44:24] [Teaser      ] 30%
[12:44:31] [Teaser      ] 40%
[12:44:38] [Teaser      ] 50%
[12:44:46] [Teaser      ] 60%
[12:44:53] [Teaser      ] 70%
[12:45:01] [Teaser      ] 80%
[12:45:10] [Teaser      ] 90%
[12:45:18] [Teaser      ] 100%
[12:45:21] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff
[12:45:21] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 50 -N 598338  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta
[12:45:21] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 50 -N 598338  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta
[12:46:54] [Teaser      ] move /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta.fastq -> ./reads.fastq
[12:46:54] [Teaser      ] move /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta.fastq.sam -> ./mapping_comparison.sam
[12:46:54] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff
[12:46:54] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[12:46:54] [Teaser      ] Translating SAM file coordinates (as contig)...
[12:46:54] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.teaser.findex
[12:48:39] [Teaser      ] remove /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta
[12:48:39] [Teaser      ] remove /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas.sampled.4.500.100.fasta.index
[12:48:39] [Teaser      ] remove mapping_comparison_unfixed.sam
[12:48:40] [Teaser      ] Took 287 seconds for generating 0542e1134664dca94011ee7ee39d31ff
[12:48:40] [Main        ] Data set creation completed
[12:48:40] [Main        ] 
[12:48:42] [bowtie2     ] 0542e1134664dca94011ee7ee39d31ff (base: tests_base/base_mapping)
[12:48:42] [init        ] Ref:   /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas
[12:48:42] [init        ] Reads: /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/reads.fastq
[12:48:42] [init        ] Output:/project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/out_bowtie2.sam
[12:48:42] [map         ] Command(pre): 
[12:48:42] [map         ] Base run time file not existing, performing base run
[12:48:42] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas_bt -U reads_base.fastq -S out_bowtie2.sam
[12:49:05] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas_bt -U reads.fastq -S out_bowtie2.sam
[12:49:05] [map         ]    Mapping 71.00MB to 121.00MB with bowtie2...
[12:49:44] [map         ]    Took 38.144 (wall), 93.832 (CPU) seconds, initialization time: 6.512 (wall), 6.428 (CPU) seconds.
[12:49:44] [map         ] Command(post): 
[12:49:44] [sort_prepare] Sorting...
[12:50:24] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[12:51:21] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
568580,25005,4753,0,0,598338,0,0,0.957874609365,0.991709878901,0.974498636579

[12:51:21] [bwa         ] 0542e1134664dca94011ee7ee39d31ff (base: tests_base/base_mapping)
[12:51:21] [init        ] Ref:   /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas
[12:51:21] [init        ] Reads: /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/reads.fastq
[12:51:21] [init        ] Output:/project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/out_bwa.sam
[12:51:21] [map         ] Command(pre): 
[12:51:21] [map         ] Base run time file not existing, performing base run
[12:51:21] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[12:52:01] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas out_bwa.sam.bwa reads.fastq > out_bwa.sam
[12:52:01] [map         ]    Mapping 71.00MB to 121.00MB with bwa...
[12:53:33] [map         ]    Took 91.952 (wall), 144.740 (CPU) seconds, initialization time: 15.709 (wall), 15.356 (CPU) seconds.
[12:53:33] [map         ] Command(post): 
[12:53:34] [sort_prepare] Sorting...
[12:53:57] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[12:54:52] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
568662,24909,4767,0,0,598338,0,0,0.958035348762,0.991686852252,0.974570694087

[12:54:52] [bwamem      ] 0542e1134664dca94011ee7ee39d31ff (base: tests_base/base_mapping)
[12:54:52] [init        ] Ref:   /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas
[12:54:52] [init        ] Reads: /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/reads.fastq
[12:54:52] [init        ] Output:/project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/out_bwamem.sam
[12:54:52] [map         ] Command(pre): 
[12:54:53] [map         ] Base run time file not existing, performing base run
[12:54:53] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads_base.fastq  -t 4 > out_bwamem.sam
[12:55:12] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads.fastq  -t 4 > out_bwamem.sam
[12:55:12] [map         ]    Mapping 71.00MB to 121.00MB with bwamem...
[12:55:57] [map         ]    Took 44.254 (wall), 102.212 (CPU) seconds, initialization time: 8.528 (wall), 8.312 (CPU) seconds.
[12:55:57] [map         ] Command(post): 
[12:55:57] [sort_prepare] Sorting...
[12:56:19] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[12:57:13] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
572389,25221,728,0,0,598338,0,0,0.957796890949,0.998729753262,0.977835140045

[12:57:13] [bwasw       ] 0542e1134664dca94011ee7ee39d31ff (base: tests_base/base_mapping)
[12:57:13] [init        ] Ref:   /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas
[12:57:13] [init        ] Reads: /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/reads.fastq
[12:57:13] [init        ] Output:/project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/out_bwasw.sam
[12:57:13] [map         ] Command(pre): 
[12:57:13] [map         ] Base run time file not existing, performing base run
[12:57:13] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads_base.fastq  -t 4 > out_bwasw.sam
[12:57:35] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas reads.fastq  -t 4 > out_bwasw.sam
[12:57:35] [map         ]    Mapping 71.00MB to 121.00MB with bwasw...
[12:59:20] [map         ]    Took 104.139 (wall), 341.236 (CPU) seconds, initialization time: 10.256 (wall), 10.020 (CPU) seconds.
[12:59:20] [map         ] Command(post): 
[12:59:20] [sort_prepare] Sorting...
[12:59:36] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:00:30] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
538644,24196,35498,0,1,598338,0,0,0.957010873428,0.938172089831,0.94749784957

[13:00:30] [ngm         ] 0542e1134664dca94011ee7ee39d31ff (base: tests_base/base_mapping)
[13:00:30] [init        ] Ref:   /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas
[13:00:30] [init        ] Reads: /project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/reads.fastq
[13:00:30] [init        ] Output:/project/teaser/genometeaser/tests_generated/0542e1134664dca94011ee7ee39d31ff/out_ngm.sam
[13:00:30] [map         ] Command(pre): /project/teaser/genometeaser/software/ngm/ngm --ref /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas 
[13:01:36] [map         ] Base run time file not existing, performing base run
[13:01:36] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas -t 4 --qry reads_base.fastq  --no-progress
[13:03:39] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/Arabidopsis_thaliana.fas -t 4 --qry reads.fastq  --no-progress
[13:03:39] [map         ]    Mapping 71.00MB to 121.00MB with ngm...
[13:05:11] [map         ]    Took 90.280 (wall), 162.000 (CPU) seconds, initialization time: 59.824 (wall), 101.632 (CPU) seconds.
[13:05:11] [map         ] Command(post): 
[13:05:11] [sort_prepare] Sorting...
[13:05:28] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 25)... 
[13:05:57] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
572838,25348,152,0,0,598338,0,0,0.957625220249,0.999734724864,0.978227012849