Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

7 tests executed, 1 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
D_melanogasterD_melanogaster7603
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession4cb6b18a4da1297d4cf7e8f8ca244517
Report timestampThu Nov 8 01:15:18 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/4cb6b18a4da1297d4cf7e8f8ca244517.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 50}
include: [base_teaser.yaml]
meta_timestamp: 1541633671.462516
report: {name: 4cb6b18a4da1297d4cf7e8f8ca244517}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    4cb6b18a4da1297d4cf7e8f8ca244517: {coverage: 1.0, error_rate_mult: 1.0, insert_size: 100,
      insert_size_error: 50, mutation_indel_avg_len: 1.0, mutation_indel_frac: 0.3,
      mutation_rate: 0.001, paired: true, platform: !!python/unicode 'illumina', read_length: 101,
      reference: !!python/unicode 'D_melanogaster.fasta', title: !!python/unicode 'D_melanogaster',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, cushaw3, segemehl, stampy]
test_parameters: []
threads: 4
Log
[00:34:33] [Main        ] framework cmd:  ./teaser.py setups_generated/4cb6b18a4da1297d4cf7e8f8ca244517.yaml -mcpu
[00:34:33] [Main        ] framework hash: vevev
[00:34:33] [Main        ] deployment mode: devel (True)
[00:34:33] [Main        ] Test Runner Setup - ""

[00:34:33] [Main        ] Using Teaser for data set creation
[00:34:34] [Teaser      ] Init. Creating 1 test datasets.
[00:34:34] [Teaser      ] 
[00:34:34] [Teaser      ] Creating test 4cb6b18a4da1297d4cf7e8f8ca244517
[00:34:34] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517
[00:34:34] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517
[00:34:34] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[00:34:34] [Teaser      ] csample /project/teaser/genometeaser/references/D_melanogaster.fasta 1500 0.250000
[00:34:34] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[00:34:34] [Teaser      ] Sampling as contig: 23954 regions of size 1500 (pad 300), totalling 35931498 base pairs
[00:34:34] [Teaser      ] Sampling 23954 regions
[00:34:53] [Teaser      ] 10%
[00:34:58] [Teaser      ] 20%
[00:35:02] [Teaser      ] 30%
[00:35:07] [Teaser      ] 40%
[00:35:11] [Teaser      ] 50%
[00:35:16] [Teaser      ] 60%
[00:35:21] [Teaser      ] 70%
[00:35:26] [Teaser      ] 80%
[00:35:31] [Teaser      ] 90%
[00:35:36] [Teaser      ] 100%
[00:35:37] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517
[00:35:37] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 101 -N 177878 --mp  --ll 100 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta
[00:35:37] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 101 -N 177878 --mp  --ll 100 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta
[00:38:45] [Teaser      ] move /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta_1.fastq -> ./reads1.fastq
[00:38:46] [Teaser      ] move /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta_2.fastq -> ./reads2.fastq
[00:38:46] [Teaser      ] move /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta.fastq.sam -> ./mapping_comparison.sam
[00:38:46] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517
[00:38:46] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[00:38:46] [Teaser      ] Translating SAM file coordinates (as contig)...
[00:38:46] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[00:39:42] [Teaser      ] remove /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta
[00:39:42] [Teaser      ] remove /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.1500.300.fasta.index
[00:39:42] [Teaser      ] remove mapping_comparison_unfixed.sam
[00:39:42] [Teaser      ] Took 308 seconds for generating 4cb6b18a4da1297d4cf7e8f8ca244517
[00:39:42] [Main        ] Data set creation completed
[00:39:42] [Main        ] 
[00:39:43] [bowtie2     ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:39:43] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:39:43] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:39:43] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_bowtie2.sam
[00:39:43] [map         ] Command(pre): 
[00:39:43] [map         ] Base run time file not existing, performing base run
[00:39:44] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/D_melanogaster.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[00:40:02] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/D_melanogaster.fasta_bt -1 reads1.fastq -2 reads2.fastq -S out_bowtie2.sam
[00:40:02] [map         ]    Mapping 78.00MB to 146.00MB with bowtie2...
[00:40:44] [map         ]    Took 41.760 (wall), 124.988 (CPU) seconds, initialization time: 0.569 (wall), 0.576 (CPU) seconds.
[00:40:44] [map         ] Command(post): 
[00:40:44] [sort_prepare] Sorting...
[00:41:16] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[00:41:52] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
322274,33459,23,0,0,355756,0,0,0.905943502571,0.999928637251,0.950618704187

[00:41:52] [bwa         ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:41:52] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:41:52] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:41:52] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_bwa.sam
[00:41:52] [map         ] Command(pre): 
[00:41:52] [map         ] Base run time file not existing, performing base run
[00:41:52] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/D_melanogaster.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/D_melanogaster.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[00:42:15] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/D_melanogaster.fasta reads1.fastq  -t 4 > out_bwa.sam_1.sai; /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/D_melanogaster.fasta reads2.fastq  -t 4 > out_bwa.sam_2.sai; /project/teaser/genometeaser/software/bwa sampe /project/teaser/genometeaser/references/D_melanogaster.fasta out_bwa.sam_1.sai out_bwa.sam_2.sai reads1.fastq reads2.fastq > out_bwa.sam
[00:42:15] [map         ]    Mapping 78.00MB to 146.00MB with bwa...
[00:43:23] [map         ]    Took 66.664 (wall), 106.500 (CPU) seconds, initialization time: 0.505 (wall), 0.376 (CPU) seconds.
[00:43:23] [map         ] Command(post): 
[00:43:23] [sort_prepare] Sorting...
[00:43:37] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[00:44:12] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
317423,32935,5398,0,0,355756,0,0,0.905996152507,0.983278659071,0.943056750136

[00:44:12] [bwamem      ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:44:12] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:44:12] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:44:12] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_bwamem.sam
[00:44:12] [map         ] Command(pre): 
[00:44:12] [map         ] Base run time file not existing, performing base run
[00:44:12] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/D_melanogaster.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[00:44:15] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/D_melanogaster.fasta reads1.fastq reads2.fastq  -t 4 > out_bwamem.sam
[00:44:15] [map         ]    Mapping 78.00MB to 146.00MB with bwamem...
[00:44:47] [map         ]    Took 31.964 (wall), 84.772 (CPU) seconds, initialization time: 0.303 (wall), 0.244 (CPU) seconds.
[00:44:47] [map         ] Command(post): 
[00:44:47] [sort_prepare] Sorting...
[00:45:02] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[00:45:36] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
324280,31476,0,0,0,355756,0,0,0.911523628554,1.0,0.95371421513

[00:45:37] [bwasw       ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:45:37] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:45:37] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:45:37] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_bwasw.sam
[00:45:37] [map         ] Command(pre): 
[00:45:37] [map         ] Base run time file not existing, performing base run
[00:45:37] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/D_melanogaster.fasta reads_base.fastq  -t 4 > out_bwasw.sam
[00:45:39] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/D_melanogaster.fasta reads1.fastq reads2.fastq  -t 4 > out_bwasw.sam
[00:45:39] [map         ]    Mapping 78.00MB to 146.00MB with bwasw...
[00:46:45] [map         ]    Took 66.005 (wall), 208.488 (CPU) seconds, initialization time: 0.298 (wall), 0.236 (CPU) seconds.
[00:46:45] [map         ] Command(post): 
[00:46:45] [sort_prepare] Sorting...
[00:47:02] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[00:47:36] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
321547,34204,5,0,29,355756,0,0,0.903854100199,0.999984450415,0.949492324705

[00:47:36] [cushaw3     ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:47:36] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:47:36] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:47:36] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_cushaw3.sam
[00:47:36] [map         ] Command(pre): 
[00:47:36] [map         ] Base run time file not existing, performing base run
[00:47:36] [map         ] Command(pre-b): /project/teaser/genometeaser/software/cushaw3/cushaw3 align -r /project/teaser/genometeaser/references/D_melanogaster.fasta_cushaw3 -f reads_base.fastq -o out_cushaw3.sam -t 4 -multi 1
[00:47:52] [map         ] Command(main): /project/teaser/genometeaser/software/cushaw3/cushaw3 align -r /project/teaser/genometeaser/references/D_melanogaster.fasta_cushaw3 -q reads1.fastq reads2.fastq -o out_cushaw3.sam -t 4 -multi 1
[00:47:52] [map         ]    Mapping 78.00MB to 146.00MB with cushaw3...
[00:48:58] [map         ]    Took 65.936 (wall), 244.868 (CPU) seconds, initialization time: 0.249 (wall), 0.192 (CPU) seconds.
[00:48:58] [map         ] Command(post): 
[00:48:59] [sort_prepare] Sorting...
[00:49:14] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[00:49:48] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
323927,31829,0,0,0,355756,0,0,0.910531375437,1.0,0.953170816395

[00:49:48] [segemehl    ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[00:49:48] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[00:49:48] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[00:49:48] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_segemehl.sam
[00:49:48] [map         ] Command(pre): 
[00:49:48] [map         ] Base run time file not existing, performing base run
[00:49:48] [map         ] Command(pre-b): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/D_melanogaster.fasta.idx -d /project/teaser/genometeaser/references/D_melanogaster.fasta -q reads_base.fastq > out_segemehl.sam
[00:53:13] [map         ] Command(main): /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/D_melanogaster.fasta.idx -d /project/teaser/genometeaser/references/D_melanogaster.fasta -q reads1.fastq -p reads2.fastq > out_segemehl.sam
[00:53:13] [map         ]    Mapping 78.00MB to 146.00MB with segemehl...
[01:08:14] [map         ] Subprocess exceeded maximum allowed runtime: /project/teaser/genometeaser/software/segemehl.x  -r 1 -i /project/teaser/genometeaser/references/D_melanogaster.fasta.idx -d /project/teaser/genometeaser/references/D_melanogaster.fasta -q reads1.fastq -p reads2.fastq > out_segemehl.sam at None:None
[01:08:14] [map         ] None

[01:08:14] [map         ]    Took 0.000 (wall), 0.000 (CPU) seconds, initialization time: 11.338 (wall), 11.272 (CPU) seconds.
[01:08:14] [map         ] Command(post): 
[01:08:15] [sort_prepare] Sorting...
[01:08:26] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[01:08:59] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
312041,30395,0,106,0,342543,0,0,0.911238888435,1.0,0.953558337421
[01:08:59] [calc_time   ] Runtime < Init runtime, using unadjusted runtime as mapping time at None:None
[01:08:59] [calc_time   ] None

[01:08:59] [calc_time   ] 	*** [INTERNAL] Error occured executing pipeline main on test 4cb6b18a4da1297d4cf7e8f8ca244517: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[01:08:59] [calc_time   ] Error occured executing pipeline main: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'
 at Test 4cb6b18a4da1297d4cf7e8f8ca244517:main
[01:08:59] [calc_time   ] Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[01:09:00]  -> Errors in test, not caching!

[01:09:00] [stampy      ] 4cb6b18a4da1297d4cf7e8f8ca244517 (base: tests_base/base_mapping)
[01:09:00] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[01:09:00] [init        ] Reads: /project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/reads1.fastq
[01:09:00] [init        ] Output:/project/teaser/genometeaser/tests_generated/4cb6b18a4da1297d4cf7e8f8ca244517/out_stampy.sam
[01:09:00] [map         ] Command(pre): 
[01:09:00] [map         ] Base run time file not existing, performing base run
[01:09:00] [map         ] Command(pre-b): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/D_melanogaster.fasta_o -h /project/teaser/genometeaser/references/D_melanogaster.fasta_o -M reads_base.fastq -o out_stampy.sam
[01:09:29] [map         ] Command(main): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/D_melanogaster.fasta_o -h /project/teaser/genometeaser/references/D_melanogaster.fasta_o -o out_stampy.sam -M reads1.fastq reads2.fastq
[01:09:29] [map         ]    Mapping 78.00MB to 146.00MB with stampy...
[01:14:26] [map         ]    Took 296.893 (wall), 1248.792 (CPU) seconds, initialization time: 0.637 (wall), 0.456 (CPU) seconds.
[01:14:26] [map         ] Command(post): 
[01:14:26] [sort_prepare] Sorting...
[01:14:43] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[01:15:17] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
323232,32524,0,0,0,355756,0,0,0.908577789271,1.0,0.952099300724