Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 2 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
M_musculusM_musculus5314
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession43bdd5cc22aa7c6d1994fe8ebc8d72d4
Report timestampMon Aug 27 11:11:09 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 12}
include: [base_teaser.yaml]
meta_timestamp: 1535354025.830284
report: {name: 43bdd5cc22aa7c6d1994fe8ebc8d72d4}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    43bdd5cc22aa7c6d1994fe8ebc8d72d4: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 1.0,
      mutation_indel_frac: 0.3, mutation_rate: 0.001, paired: false, platform: !!python/unicode 'illumina',
      read_length: 25, reference: !!python/unicode 'M_musculus.fasta', title: !!python/unicode 'M_musculus',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, ngm]
test_parameters: []
threads: 4
Log
[09:13:48] [Main        ] framework cmd:  ./teaser.py setups_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4.yaml -mcpu
[09:13:48] [Main        ] framework hash: vevev
[09:13:48] [Main        ] deployment mode: devel (True)
[09:13:48] [Main        ] Test Runner Setup - ""

[09:13:48] [Main        ] Using Teaser for data set creation
[09:13:48] [Teaser      ] Init. Creating 1 test datasets.
[09:13:48] [Teaser      ] 
[09:13:48] [Teaser      ] Creating test 43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:13:48] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:13:48] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:13:48] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/M_musculus.fasta.teaser.findex
[09:13:48] [Teaser      ] csample /project/teaser/genometeaser/references/M_musculus.fasta 250 0.010000
[09:13:48] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/M_musculus.fasta.teaser.findex
[09:13:48] [Teaser      ] Sampling as contig: 193537 regions of size 250 (pad 50), totalling 48384255 base pairs
[09:13:48] [Teaser      ] Sampling 193537 regions
[09:21:55] [Teaser      ] 10%
[09:22:37] [Teaser      ] 20%
[09:23:20] [Teaser      ] 30%
[09:24:01] [Teaser      ] 40%
[09:24:44] [Teaser      ] 50%
[09:25:28] [Teaser      ] 60%
[09:26:10] [Teaser      ] 70%
[09:26:55] [Teaser      ] 80%
[09:27:42] [Teaser      ] 90%
[09:28:26] [Teaser      ] 100%
[09:28:36] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:28:36] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 25 -N 1935370  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta
[09:28:36] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 25 -N 1935370  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta
[09:32:44] [Teaser      ] move /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta.fastq -> ./reads.fastq
[09:32:44] [Teaser      ] move /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta.fastq.sam -> ./mapping_comparison.sam
[09:32:44] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:32:44] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[09:32:44] [Teaser      ] Translating SAM file coordinates (as contig)...
[09:32:44] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/M_musculus.fasta.teaser.findex
[09:38:21] [Teaser      ] remove /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta
[09:38:21] [Teaser      ] remove /project/teaser/genometeaser/references/M_musculus.fasta.sampled.100.250.50.fasta.index
[09:38:21] [Teaser      ] remove mapping_comparison_unfixed.sam
[09:38:21] [Teaser      ] Took 1473 seconds for generating 43bdd5cc22aa7c6d1994fe8ebc8d72d4
[09:38:21] [Main        ] Data set creation completed
[09:38:21] [Main        ] 
[09:38:23] [bowtie2     ] 43bdd5cc22aa7c6d1994fe8ebc8d72d4 (base: tests_base/base_mapping)
[09:38:23] [init        ] Ref:   /project/teaser/genometeaser/references/M_musculus.fasta
[09:38:23] [init        ] Reads: /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/reads.fastq
[09:38:23] [init        ] Output:/project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/out_bowtie2.sam
[09:38:23] [map         ] Command(pre): 
[09:38:23] [map         ] Base run time file not existing, performing base run
[09:38:23] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/M_musculus.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[09:43:57] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/M_musculus.fasta_bt -U reads.fastq -S out_bowtie2.sam
[09:43:57] [map         ]    Mapping 135.00MB to 4919.00MB with bowtie2...
[09:46:40] [map         ]    Took 161.813 (wall), 533.240 (CPU) seconds, initialization time: 62.499 (wall), 61.984 (CPU) seconds.
[09:46:40] [map         ] Command(post): 
[09:46:40] [sort_prepare] Sorting...
[09:48:38] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 12)... 
[09:51:15] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
1457479,393399,84492,0,0,1935370,6,0,0.78745276566,0.945205195169,0.859147577744

[09:51:15] [bwa         ] 43bdd5cc22aa7c6d1994fe8ebc8d72d4 (base: tests_base/base_mapping)
[09:51:15] [init        ] Ref:   /project/teaser/genometeaser/references/M_musculus.fasta
[09:51:15] [init        ] Reads: /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/reads.fastq
[09:51:15] [init        ] Output:/project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/out_bwa.sam
[09:51:15] [map         ] Command(pre): 
[09:51:15] [map         ] Base run time file not existing, performing base run
[09:51:15] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/M_musculus.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/M_musculus.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[10:09:00] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/M_musculus.fasta reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/M_musculus.fasta out_bwa.sam.bwa reads.fastq > out_bwa.sam
[10:09:00] [map         ]    Mapping 135.00MB to 4919.00MB with bwa...
[10:24:00] [map         ] Subprocess exceeded maximum allowed runtime: /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/M_musculus.fasta reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/M_musculus.fasta out_bwa.sam.bwa reads.fastq > out_bwa.sam at None:None
[10:24:00] [map         ] None

[10:24:00] [map         ]    Took 0.000 (wall), 0.000 (CPU) seconds, initialization time: 252.508 (wall), 249.644 (CPU) seconds.
[10:24:00] [map         ] Command(post): 
[10:24:00] [sort_prepare] Sorting...
[10:24:26] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 12)... 
[10:26:18] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
613095,165893,3191,1153191,0,1935370,2,0,0.787040365192,0.994822209169,0.878816633865
[10:26:18] [calc_stats  ] Unexpectedly reached end of mapper output at sorted_out_bwa.sam:read.000782179
[10:26:18] [calc_time   ] Runtime < Init runtime, using unadjusted runtime as mapping time at None:None
[10:26:18] [calc_time   ] None

[10:26:18] [calc_time   ] 	*** [INTERNAL] Error occured executing pipeline main on test 43bdd5cc22aa7c6d1994fe8ebc8d72d4: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[10:26:18] [calc_time   ] Error occured executing pipeline main: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'
 at Test 43bdd5cc22aa7c6d1994fe8ebc8d72d4:main
[10:26:18] [calc_time   ] Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[10:26:22]  -> Errors in test, not caching!

[10:26:22] [bwamem      ] 43bdd5cc22aa7c6d1994fe8ebc8d72d4 (base: tests_base/base_mapping)
[10:26:22] [init        ] Ref:   /project/teaser/genometeaser/references/M_musculus.fasta
[10:26:22] [init        ] Reads: /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/reads.fastq
[10:26:22] [init        ] Output:/project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/out_bwamem.sam
[10:26:22] [map         ] Command(pre): 
[10:26:22] [map         ] Base run time file not existing, performing base run
[10:26:22] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/M_musculus.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[10:29:28] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/M_musculus.fasta reads.fastq  -t 4 > out_bwamem.sam
[10:29:28] [map         ]    Mapping 135.00MB to 4919.00MB with bwamem...
[10:37:21] [map         ]    Took 473.092 (wall), 1358.804 (CPU) seconds, initialization time: 92.215 (wall), 91.584 (CPU) seconds.
[10:37:21] [map         ] Command(post): 
[10:37:22] [sort_prepare] Sorting...
[10:37:22] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 12)... 
[10:38:52] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
0,0,0,1935370,0,1935370,0,0,0,0,0
[10:38:52] [calc_stats  ] Unexpectedly reached end of mapper output at sorted_out_bwamem.sam:read.000000000
[10:38:52] [calc_stats  ] None

[10:38:52] [bwamem      ] Errors in test, not caching!

[10:38:52] [bwasw       ] 43bdd5cc22aa7c6d1994fe8ebc8d72d4 (base: tests_base/base_mapping)
[10:38:52] [init        ] Ref:   /project/teaser/genometeaser/references/M_musculus.fasta
[10:38:52] [init        ] Reads: /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/reads.fastq
[10:38:52] [init        ] Output:/project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/out_bwasw.sam
[10:38:52] [map         ] Command(pre): 
[10:38:53] [map         ] Base run time file not existing, performing base run
[10:38:53] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/M_musculus.fasta reads_base.fastq  -t 4 > out_bwasw.sam
[10:42:01] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/M_musculus.fasta reads.fastq  -t 4 > out_bwasw.sam
[10:42:01] [map         ]    Mapping 135.00MB to 4919.00MB with bwasw...
[10:44:09] [map         ]    Took 127.673 (wall), 123.872 (CPU) seconds, initialization time: 92.839 (wall), 92.388 (CPU) seconds.
[10:44:09] [map         ] Command(post): 
[10:44:10] [sort_prepare] Sorting...
[10:44:54] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 12)... 
[10:46:52] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
0,0,1935370,0,0,1935370,0,0,0,0,0

[10:46:52] [ngm         ] 43bdd5cc22aa7c6d1994fe8ebc8d72d4 (base: tests_base/base_mapping)
[10:46:52] [init        ] Ref:   /project/teaser/genometeaser/references/M_musculus.fasta
[10:46:52] [init        ] Reads: /project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/reads.fastq
[10:46:52] [init        ] Output:/project/teaser/genometeaser/tests_generated/43bdd5cc22aa7c6d1994fe8ebc8d72d4/out_ngm.sam
[10:46:52] [map         ] Command(pre): /project/teaser/genometeaser/software/ngm/ngm --ref /project/teaser/genometeaser/references/M_musculus.fasta 
[10:56:55] [map         ] Base run time file not existing, performing base run
[10:56:55] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/M_musculus.fasta -t 4 --qry reads_base.fastq  --no-progress
[11:02:21] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/M_musculus.fasta -t 4 --qry reads.fastq  --no-progress
[11:02:21] [map         ]    Mapping 135.00MB to 4919.00MB with ngm...
[11:07:33] [map         ]    Took 312.071 (wall), 657.036 (CPU) seconds, initialization time: 162.160 (wall), 175.696 (CPU) seconds.
[11:07:33] [map         ] Command(post): 
[11:07:33] [sort_prepare] Sorting...
[11:08:32] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 12)... 
[11:11:08] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
1450546,378398,106426,0,0,1935370,11,0,0.793105748454,0.931645527344,0.856811568863