Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

6 tests executed, 1 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
H_sapiensH_sapiens6501
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession913dd6b6c8f0bd5901b65d962c7cd698
Report timestampMon Dec 3 22:58:41 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/913dd6b6c8f0bd5901b65d962c7cd698.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 18}
include: [base_teaser.yaml]
meta_timestamp: 1543870517.533435
report: {name: 913dd6b6c8f0bd5901b65d962c7cd698}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    913dd6b6c8f0bd5901b65d962c7cd698: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 0.0,
      mutation_indel_frac: 0.0, mutation_rate: 0.0, paired: false, platform: !!python/unicode 'illumina',
      read_length: 36, reference: !!python/unicode 'H_sapiens.fa', title: !!python/unicode 'H_sapiens',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie, bowtie2, bwa, bwamem, bwasw, ngm]
test_parameters: []
threads: 4
Log
[21:55:21] [Main        ] framework cmd:  ./teaser.py setups_generated/913dd6b6c8f0bd5901b65d962c7cd698.yaml -mcpu
[21:55:21] [Main        ] framework hash: vevev
[21:55:21] [Main        ] deployment mode: devel (True)
[21:55:21] [Main        ] Test Runner Setup - ""

[21:55:22] [Main        ] Using Teaser for data set creation
[21:55:22] [Teaser      ] Init. Creating 1 test datasets.
[21:55:22] [Teaser      ] 
[21:55:22] [Teaser      ] Creating test 913dd6b6c8f0bd5901b65d962c7cd698
[21:55:22] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698
[21:55:22] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698
[21:55:22] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[21:55:22] [Teaser      ] csample /project/teaser/genometeaser/references/H_sapiens.fa 360 0.010000
[21:55:22] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[21:55:22] [Teaser      ] Sampling as contig: 87143 regions of size 360 (pad 72), totalling 31371612 base pairs
[21:55:22] [Teaser      ] Sampling 87143 regions
[22:00:53] [Teaser      ] 10%
[22:01:14] [Teaser      ] 20%
[22:01:25] [Teaser      ] 30%
[22:01:37] [Teaser      ] 40%
[22:01:46] [Teaser      ] 50%
[22:01:54] [Teaser      ] 60%
[22:02:04] [Teaser      ] 70%
[22:02:15] [Teaser      ] 80%
[22:02:26] [Teaser      ] 90%
[22:02:38] [Teaser      ] 100%
[22:02:40] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698
[22:02:40] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 36 -N 871433  --ll 500 --le 50  --hi 0.000000  --hs 0.000000  --hm 1 --hM 2 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta
[22:02:40] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 36 -N 871433  --ll 500 --le 50  --hi 0.000000  --hs 0.000000  --hm 1 --hM 2 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta
[22:04:45] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta.fastq -> ./reads.fastq
[22:04:45] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta.fastq.sam -> ./mapping_comparison.sam
[22:04:45] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698
[22:04:45] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[22:04:45] [Teaser      ] Translating SAM file coordinates (as contig)...
[22:04:45] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[22:06:50] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta
[22:06:50] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.360.72.fasta.index
[22:06:50] [Teaser      ] remove mapping_comparison_unfixed.sam
[22:06:50] [Teaser      ] Took 687 seconds for generating 913dd6b6c8f0bd5901b65d962c7cd698
[22:06:50] [Main        ] Data set creation completed
[22:06:50] [Main        ] 
[22:06:53] [bowtie      ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:06:53] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:06:53] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:06:53] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_bowtie.sam
[22:06:53] [map         ] Command(pre): 
[22:06:53] [map         ] Base run time file not existing, performing base run
[22:06:53] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie --threads 4  -S /project/teaser/genometeaser/references/H_sapiens.fa_bo reads_base.fastq out_bowtie.sam
[22:10:21] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie --threads 4  -S /project/teaser/genometeaser/references/H_sapiens.fa_bo reads.fastq out_bowtie.sam
[22:10:21] [map         ]    Mapping 80.00MB to 3199.00MB with bowtie...
[22:10:46] [map         ]    Took 24.427 (wall), 47.760 (CPU) seconds, initialization time: 2.897 (wall), 2.828 (CPU) seconds.
[22:10:46] [map         ] Command(post): 
[22:10:46] [sort_prepare] Sorting...
[22:11:51] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:12:39] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
722662,103851,44920,0,0,871433,7,0,0.874350433689,0.941478565157,0.906673692597

[22:12:39] [bowtie2     ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:12:39] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:12:39] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:12:39] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_bowtie2.sam
[22:12:39] [map         ] Command(pre): 
[22:12:39] [map         ] Base run time file not existing, performing base run
[22:12:39] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -U reads_base.fastq -S out_bowtie2.sam
[22:17:35] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -U reads.fastq -S out_bowtie2.sam
[22:17:35] [map         ]    Mapping 80.00MB to 3199.00MB with bowtie2...
[22:18:14] [map         ]    Took 38.097 (wall), 87.880 (CPU) seconds, initialization time: 4.141 (wall), 4.168 (CPU) seconds.
[22:18:14] [map         ] Command(post): 
[22:18:14] [sort_prepare] Sorting...
[22:18:38] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:19:48] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
752015,102175,17243,0,0,871433,7,0,0.880383755371,0.977584893495,0.926441746209

[22:19:49] [bwa         ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:19:49] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:19:49] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:19:49] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_bwa.sam
[22:19:49] [map         ] Command(pre): 
[22:19:49] [map         ] Base run time file not existing, performing base run
[22:19:49] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/H_sapiens.fa out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[22:27:50] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/H_sapiens.fa reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/H_sapiens.fa out_bwa.sam.bwa reads.fastq > out_bwa.sam
[22:27:50] [map         ]    Mapping 80.00MB to 3199.00MB with bwa...
[22:29:33] [map         ]    Took 102.903 (wall), 189.460 (CPU) seconds, initialization time: 8.349 (wall), 8.188 (CPU) seconds.
[22:29:33] [map         ] Command(post): 
[22:29:33] [sort_prepare] Sorting...
[22:29:56] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:31:00] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
764237,103275,3921,0,0,871433,3,0,0.88095265541,0.994895581378,0.934463553162

[22:31:00] [bwamem      ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:31:00] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:31:00] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:31:00] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_bwamem.sam
[22:31:00] [map         ] Command(pre): 
[22:31:00] [map         ] Base run time file not existing, performing base run
[22:31:00] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwamem.sam
[22:31:13] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads.fastq  -t 4 > out_bwamem.sam
[22:31:13] [map         ]    Mapping 80.00MB to 3199.00MB with bwamem...
[22:32:42] [map         ]    Took 88.719 (wall), 278.484 (CPU) seconds, initialization time: 5.118 (wall), 5.040 (CPU) seconds.
[22:32:42] [map         ] Command(post): 
[22:32:48] [sort_prepare] Sorting...
[22:33:18] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:33:57] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
724871,99908,46654,0,0,871433,4,0,0.878866944963,0.939530151324,0.908186661187

[22:34:10] [bwasw       ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:34:16] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:34:16] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:34:16] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_bwasw.sam
[22:34:22] [map         ] Command(pre): 
[22:34:23] [map         ] Base run time file not existing, performing base run
[22:34:23] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwasw.sam
[22:34:57] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/H_sapiens.fa reads.fastq  -t 4 > out_bwasw.sam
[22:34:57] [map         ]    Mapping 80.00MB to 3199.00MB with bwasw...
[22:36:20] [map         ]    Took 81.769 (wall), 179.204 (CPU) seconds, initialization time: 15.465 (wall), 3.392 (CPU) seconds.
[22:36:20] [map         ] Command(post): 
[22:36:24] [sort_prepare] Sorting...
[22:36:49] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:37:48] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
654958,95986,120489,0,0,871433,6,0,0.872179550006,0.844619941788,0.858178540099

[22:37:53] [ngm         ] 913dd6b6c8f0bd5901b65d962c7cd698 (base: tests_base/base_mapping)
[22:37:55] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[22:37:55] [init        ] Reads: /project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/reads.fastq
[22:37:55] [init        ] Output:/project/teaser/genometeaser/tests_generated/913dd6b6c8f0bd5901b65d962c7cd698/out_ngm.sam
[22:37:58] [map         ] Command(pre): 
[22:37:59] [map         ] Base run time file not existing, performing base run
[22:37:59] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads_base.fastq  --no-progress
[22:53:01] [map         ] Subprocess exceeded maximum allowed runtime: /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads_base.fastq  --no-progress at None:None
[22:53:01] [map         ] None

[22:55:35] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads.fastq  --no-progress
[22:55:36] [map         ]    Mapping 80.00MB to 3199.00MB with ngm...
[22:57:05] [map         ]    Took 88.059 (wall), 146.880 (CPU) seconds, initialization time: 146.945 (wall), 12.272 (CPU) seconds.
[22:57:05] [map         ] Command(post): 
[22:57:13] [sort_prepare] Sorting...
[22:57:49] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 18)... 
[22:58:33] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
754180,89364,27889,0,0,871433,2,0,0.894061246361,0.964339463653,0.927871516776
[22:58:41] [ngm         ] Errors in test, not caching!