Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 1 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
D_melanogasterD_melanogaster5415
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession9df2b6f30934a6f088552bf04d77f27b
Report timestampThu Nov 1 23:15:38 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/9df2b6f30934a6f088552bf04d77f27b.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 30}
include: [base_teaser.yaml]
meta_timestamp: 1541106790.826083
report: {name: 9df2b6f30934a6f088552bf04d77f27b}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    9df2b6f30934a6f088552bf04d77f27b: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 1.0,
      mutation_indel_frac: 0.3, mutation_rate: 0.001, paired: false, platform: !!python/unicode 'illumina',
      read_length: 60, reference: !!python/unicode 'D_melanogaster.fasta', title: !!python/unicode 'D_melanogaster',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, ngm, ngm_0_4_11]
test_parameters: []
threads: 4
Log
[22:13:13] [Main        ] framework cmd:  ./teaser.py setups_generated/9df2b6f30934a6f088552bf04d77f27b.yaml -mcpu
[22:13:13] [Main        ] framework hash: vevev
[22:13:13] [Main        ] deployment mode: devel (True)
[22:13:13] [Main        ] Test Runner Setup - ""

[22:13:13] [Main        ] Using Teaser for data set creation
[22:13:13] [Teaser      ] Init. Creating 1 test datasets.
[22:13:14] [Teaser      ] 
[22:13:14] [Teaser      ] Creating test 9df2b6f30934a6f088552bf04d77f27b
[22:13:14] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b
[22:13:14] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b
[22:13:14] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[22:13:14] [Teaser      ] csample /project/teaser/genometeaser/references/D_melanogaster.fasta 600 0.250000
[22:13:14] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[22:13:14] [Teaser      ] Sampling as contig: 59885 regions of size 600 (pad 120), totalling 35931498 base pairs
[22:13:14] [Teaser      ] Sampling 59885 regions
[22:13:42] [Teaser      ] 10%
[22:13:54] [Teaser      ] 20%
[22:14:06] [Teaser      ] 30%
[22:14:18] [Teaser      ] 40%
[22:14:30] [Teaser      ] 50%
[22:14:43] [Teaser      ] 60%
[22:14:55] [Teaser      ] 70%
[22:15:07] [Teaser      ] 80%
[22:15:21] [Teaser      ] 90%
[22:15:34] [Teaser      ] 100%
[22:15:36] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b
[22:15:36] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 60 -N 598858  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta
[22:15:36] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 60 -N 598858  --ll 500 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta
[22:18:10] [Teaser      ] move /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta.fastq -> ./reads.fastq
[22:18:10] [Teaser      ] move /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta.fastq.sam -> ./mapping_comparison.sam
[22:18:10] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b
[22:18:10] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[22:18:10] [Teaser      ] Translating SAM file coordinates (as contig)...
[22:18:10] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/D_melanogaster.fasta.teaser.findex
[22:19:43] [Teaser      ] remove /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta
[22:19:43] [Teaser      ] remove /project/teaser/genometeaser/references/D_melanogaster.fasta.sampled.4.600.120.fasta.index
[22:19:44] [Teaser      ] remove mapping_comparison_unfixed.sam
[22:19:44] [Teaser      ] Took 390 seconds for generating 9df2b6f30934a6f088552bf04d77f27b
[22:19:44] [Main        ] Data set creation completed
[22:19:44] [Main        ] 
[22:19:46] [bowtie2     ] 9df2b6f30934a6f088552bf04d77f27b (base: tests_base/base_mapping)
[22:19:46] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[22:19:46] [init        ] Reads: /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/reads.fastq
[22:19:46] [init        ] Output:/project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/out_bowtie2.sam
[22:19:46] [map         ] Command(pre): 
[22:19:46] [map         ] Base run time file not existing, performing base run
[22:19:46] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/D_melanogaster.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[22:20:05] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/D_melanogaster.fasta_bt -U reads.fastq -S out_bowtie2.sam
[22:20:05] [map         ]    Mapping 83.00MB to 146.00MB with bowtie2...
[22:20:37] [map         ]    Took 31.198 (wall), 74.648 (CPU) seconds, initialization time: 0.568 (wall), 0.592 (CPU) seconds.
[22:20:37] [map         ] Command(post): 
[22:20:37] [sort_prepare] Sorting...
[22:21:24] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 30)... 
[22:22:19] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
528412,68538,1908,0,0,598858,0,0,0.885186364017,0.996402172273,0.937507429453

[22:22:19] [bwa         ] 9df2b6f30934a6f088552bf04d77f27b (base: tests_base/base_mapping)
[22:22:19] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[22:22:19] [init        ] Reads: /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/reads.fastq
[22:22:19] [init        ] Output:/project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/out_bwa.sam
[22:22:19] [map         ] Command(pre): 
[22:22:19] [map         ] Base run time file not existing, performing base run
[22:22:19] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/D_melanogaster.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/D_melanogaster.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[22:22:43] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/D_melanogaster.fasta reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/D_melanogaster.fasta out_bwa.sam.bwa reads.fastq > out_bwa.sam
[22:22:43] [map         ]    Mapping 83.00MB to 146.00MB with bwa...
[22:23:43] [map         ]    Took 58.975 (wall), 109.824 (CPU) seconds, initialization time: 0.552 (wall), 0.376 (CPU) seconds.
[22:23:43] [map         ] Command(post): 
[22:23:44] [sort_prepare] Sorting...
[22:24:01] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 30)... 
[22:24:53] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
524782,67902,6174,0,0,598858,0,0,0.885433046953,0.988371917824,0.934074970631

[22:24:53] [bwamem      ] 9df2b6f30934a6f088552bf04d77f27b (base: tests_base/base_mapping)
[22:24:54] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[22:24:54] [init        ] Reads: /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/reads.fastq
[22:24:54] [init        ] Output:/project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/out_bwamem.sam
[22:24:54] [map         ] Command(pre): 
[22:24:54] [map         ] Base run time file not existing, performing base run
[22:24:54] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/D_melanogaster.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[22:24:56] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/D_melanogaster.fasta reads.fastq  -t 4 > out_bwamem.sam
[22:24:56] [map         ]    Mapping 83.00MB to 146.00MB with bwamem...
[22:25:30] [map         ]    Took 33.065 (wall), 86.152 (CPU) seconds, initialization time: 0.326 (wall), 0.240 (CPU) seconds.
[22:25:30] [map         ] Command(post): 
[22:25:30] [sort_prepare] Sorting...
[22:25:46] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 30)... 
[22:26:37] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
530286,68470,102,0,0,598858,0,0,0.885646239871,0.999807687957,0.939270810455

[22:26:38] [ngm         ] 9df2b6f30934a6f088552bf04d77f27b (base: tests_base/base_mapping)
[22:26:38] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[22:26:38] [init        ] Reads: /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/reads.fastq
[22:26:38] [init        ] Output:/project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/out_ngm.sam
[22:26:38] [map         ] Command(pre): 
[22:26:38] [map         ] Base run time file not existing, performing base run
[22:26:38] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads_base.fastq  --no-progress
[22:27:41] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads.fastq  --no-progress
[22:27:41] [map         ]    Mapping 83.00MB to 146.00MB with ngm...
[22:28:13] [map         ]    Took 31.315 (wall), 58.544 (CPU) seconds, initialization time: 3.228 (wall), 3.360 (CPU) seconds.
[22:28:13] [map         ] Command(post): 
[22:28:13] [sort_prepare] Sorting...
[22:28:32] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 30)... 
[22:29:24] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
529945,68237,676,0,0,598858,0,0,0.885926022515,0.998726021021,0.938950374866

[22:29:24] [ngm_0_4_11  ] 9df2b6f30934a6f088552bf04d77f27b (base: tests_base/base_mapping)
[22:29:25] [init        ] Ref:   /project/teaser/genometeaser/references/D_melanogaster.fasta
[22:29:25] [init        ] Reads: /project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/reads.fastq
[22:29:25] [init        ] Output:/project/teaser/genometeaser/tests_generated/9df2b6f30934a6f088552bf04d77f27b/out_ngm_0_4_11.sam
[22:29:25] [map         ] Command(pre): 
[22:29:25] [map         ] Base run time file not existing, performing base run
[22:29:25] [map         ] Command(pre-b): /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads_base.fastq  --no-progress
[22:44:26] [map         ] Subprocess exceeded maximum allowed runtime: /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads_base.fastq  --no-progress at None:None
[22:44:26] [map         ] None

[22:59:27] [map         ] Subprocess exceeded maximum allowed runtime: /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads_base.fastq  --no-progress at None:None
[22:59:27] [map         ] None

[22:59:27] [map         ] Command(main): /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads.fastq  --no-progress
[22:59:27] [map         ]    Mapping 83.00MB to 146.00MB with ngm_0_4_11...
[23:14:27] [map         ] Subprocess exceeded maximum allowed runtime: /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/D_melanogaster.fasta -t 4 --qry reads.fastq  --no-progress at None:None
[23:14:27] [map         ] None

[23:14:27] [map         ]    Took 0.000 (wall), 0.000 (CPU) seconds, initialization time: 0.000 (wall), 0.000 (CPU) seconds.
[23:14:27] [map         ] Command(post): 
[23:14:27] [sort_prepare] Sorting...
[23:14:46] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 30)... 
[23:15:38] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
528485,68247,173,1953,0,598858,0,0,0.885632076041,0.9996727563,0.939203298412
[23:15:38] [calc_stats  ] Unexpectedly reached end of mapper output at sorted_out_ngm_0_4_11.sam:read.000596905
[23:15:38] [calc_time   ] Runtime < Init runtime, using unadjusted runtime as mapping time at None:None
[23:15:38] [calc_time   ] None

[23:15:38] [calc_time   ] 	*** [INTERNAL] Error occured executing pipeline main on test 9df2b6f30934a6f088552bf04d77f27b: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[23:15:38] [calc_time   ] Error occured executing pipeline main: 'memory', traceback: Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'
 at Test 9df2b6f30934a6f088552bf04d77f27b:main
[23:15:38] [calc_time   ] Traceback (most recent call last):
  File "/project/teaser/genometeaser/lib/test.py", line 432, in executePipeline
    result = script_locals[script_name](self, *args)
  File "tests_base/base_mapping/calc_time.py", line 42, in calc_time
    stats_out.memory = mapper_result["memory"]
KeyError: 'memory'

[23:15:38]  -> Errors in test, not caching!