Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

7 tests executed, 1 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
H_sapiensH_sapiens7601
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accessiona5e600139d12b867a2e96eee861838f1
Report timestampSat Sep 9 18:26:34 2017
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/a5e600139d12b867a2e96eee861838f1.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 37}
include: [base_teaser.yaml]
meta_timestamp: 1504968691.184985
report: {name: a5e600139d12b867a2e96eee861838f1}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    a5e600139d12b867a2e96eee861838f1: {coverage: 1.0, error_rate_mult: 1.0, insert_size: 450,
      insert_size_error: 100, mutation_indel_avg_len: 1.0, mutation_indel_frac: 0.3,
      mutation_rate: 0.001, paired: true, platform: !!python/unicode 'illumina', read_length: 75,
      reference: !!python/unicode 'H_sapiens.fa', title: !!python/unicode 'H_sapiens',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, ngm, segemehl, stampy]
test_parameters: []
threads: 4
Log
[16:51:35] [Main        ] framework cmd:  ./teaser.py setups_generated/a5e600139d12b867a2e96eee861838f1.yaml -mcpu
[16:51:35] [Main        ] framework hash: vevev
[16:51:35] [Main        ] deployment mode: devel (True)
[16:51:35] [Main        ] Test Runner Setup - ""

[16:51:35] [Main        ] Using Teaser for data set creation
[16:51:36] [Teaser      ] Init. Creating 1 test datasets.
[16:51:36] [Teaser      ] 
[16:51:36] [Teaser      ] Creating test a5e600139d12b867a2e96eee861838f1
[16:51:36] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1
[16:51:36] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1
[16:51:36] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[16:51:36] [Teaser      ] csample /project/teaser/genometeaser/references/H_sapiens.fa 5500 0.010000
[16:51:36] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[16:51:36] [Teaser      ] Sampling as contig: 5703 regions of size 5500 (pad 1100), totalling 31371612 base pairs
[16:51:36] [Teaser      ] Sampling 5703 regions
[16:52:40] [Teaser      ] 10%
[16:53:33] [Teaser      ] 20%
[16:54:16] [Teaser      ] 30%
[16:54:50] [Teaser      ] 40%
[16:55:17] [Teaser      ] 50%
[16:55:40] [Teaser      ] 60%
[16:55:59] [Teaser      ] 70%
[16:56:13] [Teaser      ] 80%
[16:56:23] [Teaser      ] 90%
[16:56:33] [Teaser      ] 100%
[16:56:33] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1
[16:56:33] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 75 -N 209144 --mp  --ll 450 --le 100  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta
[16:56:33] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 75 -N 209144 --mp  --ll 450 --le 100  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta
[16:59:03] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta_1.fastq -> ./reads1.fastq
[16:59:03] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta_2.fastq -> ./reads2.fastq
[16:59:03] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta.fastq.sam -> ./mapping_comparison.sam
[16:59:03] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1
[16:59:03] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[16:59:03] [Teaser      ] Translating SAM file coordinates (as contig)...
[16:59:03] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[16:59:56] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta
[16:59:56] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.5500.1100.fasta.index
[16:59:56] [Teaser      ] remove mapping_comparison_unfixed.sam
[16:59:56] [Teaser      ] Took 500 seconds for generating a5e600139d12b867a2e96eee861838f1
[16:59:56] [Main        ] Data set creation completed
[16:59:56] [Main        ] 
[16:59:59] [bowtie2     ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[16:59:59] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[16:59:59] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[16:59:59] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_bowtie2.sam
[16:59:59] [map         ] Command(pre): 
[16:59:59] [map         ] Base run time file not existing, performing base run
[16:59:59] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -U reads_base.fastq -S out_bowtie2.sam
[17:05:53] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -1 reads1.fastq -2 reads2.fastq -S out_bowtie2.sam
[17:05:53] [map         ]    Mapping 71.00MB to 3199.00MB with bowtie2...
[17:07:50] [map         ]    Took 116.638 (wall), 271.616 (CPU) seconds, initialization time: 61.216 (wall), 60.548 (CPU) seconds.
[17:07:50] [map         ] Command(post): 
[17:07:50] [sort_prepare] Sorting...
[17:08:24] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[17:09:04] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
403897,13958,433,0,0,418288,0,0,0.966596068014,0.998929092573,0.982496640051

[17:09:04] [bwa         ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[17:09:05] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[17:09:05] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[17:09:05] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_bwa.sam
[17:09:05] [map         ] Command(pre): 
[17:09:05] [map         ] Base run time file not existing, performing base run
[17:09:05] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/H_sapiens.fa out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[17:20:23] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/H_sapiens.fa reads1.fastq  -t 4 > out_bwa.sam_1.sai; /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/H_sapiens.fa reads2.fastq  -t 4 > out_bwa.sam_2.sai; /project/teaser/genometeaser/software/bwa sampe /project/teaser/genometeaser/references/H_sapiens.fa out_bwa.sam_1.sai out_bwa.sam_2.sai reads1.fastq reads2.fastq > out_bwa.sam
[17:20:23] [map         ]    Mapping 71.00MB to 3199.00MB with bwa...
[17:27:11] [map         ]    Took 408.349 (wall), 627.988 (CPU) seconds, initialization time: 152.793 (wall), 150.816 (CPU) seconds.
[17:27:11] [map         ] Command(post): 
[17:27:11] [sort_prepare] Sorting...
[17:27:26] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[17:28:05] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
407116,10743,429,0,0,418288,0,0,0.974290370675,0.998947355507,0.98646480996

[17:28:05] [bwamem      ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[17:28:05] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[17:28:05] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[17:28:05] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_bwamem.sam
[17:28:05] [map         ] Command(pre): 
[17:28:05] [map         ] Base run time file not existing, performing base run
[17:28:05] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwamem.sam
[17:31:37] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads1.fastq reads2.fastq  -t 4 > out_bwamem.sam
[17:31:37] [map         ]    Mapping 71.00MB to 3199.00MB with bwamem...
[17:34:08] [map         ]    Took 150.257 (wall), 303.992 (CPU) seconds, initialization time: 115.818 (wall), 114.624 (CPU) seconds.
[17:34:08] [map         ] Command(post): 
[17:34:08] [sort_prepare] Sorting...
[17:34:23] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[17:34:58] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
407694,10594,0,0,0,418288,0,0,0.974672952607,1.0,0.98717405464

[17:34:58] [bwasw       ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[17:34:58] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[17:34:58] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[17:34:58] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_bwasw.sam
[17:34:58] [map         ] Command(pre): 
[17:34:58] [map         ] Base run time file not existing, performing base run
[17:34:58] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwasw.sam
[17:38:13] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/H_sapiens.fa reads1.fastq reads2.fastq  -t 4 > out_bwasw.sam
[17:38:13] [map         ]    Mapping 71.00MB to 3199.00MB with bwasw...
[17:42:33] [map         ]    Took 259.559 (wall), 484.252 (CPU) seconds, initialization time: 95.640 (wall), 94.564 (CPU) seconds.
[17:42:33] [map         ] Command(post): 
[17:42:33] [sort_prepare] Sorting...
[17:42:46] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[17:43:24] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
402895,15246,144,3,60,418288,0,0,0.963538614965,0.999642714477,0.981258676539

[17:43:25] [ngm         ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[17:43:25] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[17:43:25] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[17:43:25] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_ngm.sam
[17:43:25] [map         ] Command(pre): 
[17:43:25] [map         ] Base run time file not existing, performing base run
[17:43:25] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads_base.fastq  --no-progress
[17:53:59] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --qry2 reads2.fastq --output out_ngm.sam --qry1 reads1.fastq --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4  --no-progress
[17:53:59] [map         ]    Mapping 71.00MB to 3199.00MB with ngm...
[17:56:47] [map         ]    Took 167.560 (wall), 290.956 (CPU) seconds, initialization time: 117.920 (wall), 127.392 (CPU) seconds.
[17:56:47] [map         ] Command(post): 
[17:56:47] [sort_prepare] Sorting...
[17:57:04] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[17:57:43] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
396864,18757,2667,0,0,418288,0,0,0.954869941605,0.993324673179,0.973717785149

[17:57:43] [segemehl    ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[17:57:43] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[17:57:43] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[17:57:43] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_segemehl.sam
[17:57:43] [map         ] Command(pre): 
[17:57:43] [map         ] Base run time file not existing, performing base run
[17:57:43] [map         ] Command(pre-b): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/H_sapiens.fa.idx -d /project/teaser/genometeaser/references/H_sapiens.fa -q reads_base.fastq > out_segemehl.sam
[18:06:49] [map         ] Command(main): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/H_sapiens.fa.idx -d /project/teaser/genometeaser/references/H_sapiens.fa -q reads1.fastq -p reads2.fastq > out_segemehl.sam
[18:06:49] [map         ]    Mapping 71.00MB to 3199.00MB with segemehl...
[18:11:26] [map         ]    Took 276.397 (wall), 275.208 (CPU) seconds, initialization time: 198.811 (wall), 197.584 (CPU) seconds.
[18:11:26] [map         ] Command(post): 
[18:11:26] [sort_prepare] Sorting...
[18:11:27] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[18:11:47] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
0,0,0,418288,0,418288,0,0,0,0,0
[18:11:47] [calc_stats  ] Unexpectedly reached end of mapper output at sorted_out_segemehl.sam:read.000000000
[18:11:47] [calc_stats  ] None

[18:11:47] [segemehl    ] Errors in test, not caching!

[18:11:47] [stampy      ] a5e600139d12b867a2e96eee861838f1 (base: tests_base/base_mapping)
[18:11:47] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[18:11:47] [init        ] Reads: /project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/reads1.fastq
[18:11:47] [init        ] Output:/project/teaser/genometeaser/tests_generated/a5e600139d12b867a2e96eee861838f1/out_stampy.sam
[18:11:47] [map         ] Command(pre): 
[18:11:47] [map         ] Base run time file not existing, performing base run
[18:11:47] [map         ] Command(pre-b): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/H_sapiens.fa_o -h /project/teaser/genometeaser/references/H_sapiens.fa_o -M reads_base.fastq -o out_stampy.sam
[18:16:03] [map         ] Command(main): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/H_sapiens.fa_o -h /project/teaser/genometeaser/references/H_sapiens.fa_o -o out_stampy.sam -M reads1.fastq reads2.fastq
[18:16:03] [map         ]    Mapping 71.00MB to 3199.00MB with stampy...
[18:25:42] [map         ]    Took 578.654 (wall), 2405.568 (CPU) seconds, initialization time: 2.153 (wall), 1.800 (CPU) seconds.
[18:25:42] [map         ] Command(post): 
[18:25:42] [sort_prepare] Sorting...
[18:25:55] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 37)... 
[18:26:34] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
400780,17505,3,0,0,418288,0,0,0.95815054329,0.999992514653,0.978624485391