Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
H_sapiensH_sapiens5500
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession6996f5893fc23a4f84e9e00364871d38
Report timestampFri Jul 13 16:26:44 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/6996f5893fc23a4f84e9e00364871d38.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 50}
include: [base_teaser.yaml]
meta_timestamp: 1530733947.382113
report: {name: 6996f5893fc23a4f84e9e00364871d38}
sub_max_memory: 12000
sub_max_runtime: 900
teaser:
  tests:
    6996f5893fc23a4f84e9e00364871d38: {coverage: 1.0, error_rate_mult: 1.0, insert_size: 100,
      insert_size_error: 50, mutation_indel_avg_len: 1.0, mutation_indel_frac: 0.3,
      mutation_rate: 0.001, paired: true, platform: !!python/unicode 'illumina', read_length: 100,
      reference: !!python/unicode 'H_sapiens.fa', title: !!python/unicode 'H_sapiens',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwamem, ngm, ngm_0_4_11, stampy]
test_parameters: []
threads: 4
Log
[15:51:02] [Main        ] framework cmd:  ./teaser.py setups_generated/6996f5893fc23a4f84e9e00364871d38.yaml -mcpu
[15:51:02] [Main        ] framework hash: vevev
[15:51:02] [Main        ] deployment mode: devel (True)
[15:51:02] [Main        ] Test Runner Setup - ""

[15:51:02] [Main        ] Using Teaser for data set creation
[15:51:02] [Teaser      ] Init. Creating 1 test datasets.
[15:51:02] [Teaser      ] 
[15:51:02] [Teaser      ] Creating test 6996f5893fc23a4f84e9e00364871d38
[15:51:02] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38
[15:51:02] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38
[15:51:02] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[15:51:02] [Teaser      ] csample /project/teaser/genometeaser/references/H_sapiens.fa 1500 0.010000
[15:51:02] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[15:51:02] [Teaser      ] Sampling as contig: 20914 regions of size 1500 (pad 300), totalling 31371612 base pairs
[15:51:02] [Teaser      ] Sampling 20914 regions
[15:51:04] [Teaser      ] 10%
[15:51:07] [Teaser      ] 20%
[15:51:09] [Teaser      ] 30%
[15:51:11] [Teaser      ] 40%
[15:51:14] [Teaser      ] 50%
[15:51:16] [Teaser      ] 60%
[15:51:18] [Teaser      ] 70%
[15:51:21] [Teaser      ] 80%
[15:51:23] [Teaser      ] 90%
[15:51:26] [Teaser      ] 100%
[15:51:29] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38
[15:51:29] [Teaser      ] 	Simulator cmd: /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 100 -N 156858 --mp  --ll 100 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta
[15:51:29] [Teaser      ] subprogram /project/teaser/genometeaser/software/mason illumina --read-name-prefix read --sq -n 100 -N 156858 --mp  --ll 100 --le 50  --hi 0.000300  --hs 0.000700  --hm 1 --hM 4 --source-no-N  /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta
[15:54:12] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta_1.fastq -> ./reads1.fastq
[15:54:12] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta_2.fastq -> ./reads2.fastq
[15:54:12] [Teaser      ] move /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta.fastq.sam -> ./mapping_comparison.sam
[15:54:12] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38
[15:54:12] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[15:54:12] [Teaser      ] Translating SAM file coordinates (as contig)...
[15:54:12] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/H_sapiens.fa.teaser.findex
[15:55:01] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta
[15:55:01] [Teaser      ] remove /project/teaser/genometeaser/references/H_sapiens.fa.sampled.100.1500.300.fasta.index
[15:55:01] [Teaser      ] remove mapping_comparison_unfixed.sam
[15:55:01] [Teaser      ] Took 238 seconds for generating 6996f5893fc23a4f84e9e00364871d38
[15:55:01] [Main        ] Data set creation completed
[15:55:01] [Main        ] 
[15:55:01] [bowtie2     ] 6996f5893fc23a4f84e9e00364871d38 (base: tests_base/base_mapping)
[15:55:01] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[15:55:01] [init        ] Reads: /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/reads1.fastq
[15:55:01] [init        ] Output:/project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/out_bowtie2.sam
[15:55:01] [map         ] Command(pre): 
[15:55:02] [map         ] Base run time file not existing, performing base run
[15:55:02] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -U reads_base.fastq -S out_bowtie2.sam
[15:55:10] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/H_sapiens.fa_bt -1 reads1.fastq -2 reads2.fastq -S out_bowtie2.sam
[15:55:10] [map         ]    Mapping 69.00MB to 3199.00MB with bowtie2...
[15:55:57] [map         ]    Took 46.877 (wall), 151.372 (CPU) seconds, initialization time: 3.418 (wall), 3.464 (CPU) seconds.
[15:55:57] [map         ] Command(post): 
[15:55:57] [sort_prepare] Sorting...
[15:56:29] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[15:57:01] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
302773,10753,190,0,0,313716,0,0,0.965703003898,0.999372860712,0.982249480526

[15:57:01] [bwamem      ] 6996f5893fc23a4f84e9e00364871d38 (base: tests_base/base_mapping)
[15:57:01] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[15:57:01] [init        ] Reads: /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/reads1.fastq
[15:57:01] [init        ] Output:/project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/out_bwamem.sam
[15:57:01] [map         ] Command(pre): 
[15:57:01] [map         ] Base run time file not existing, performing base run
[15:57:01] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads_base.fastq  -t 4 > out_bwamem.sam
[15:57:11] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/H_sapiens.fa reads1.fastq reads2.fastq  -t 4 > out_bwamem.sam
[15:57:11] [map         ]    Mapping 69.00MB to 3199.00MB with bwamem...
[15:57:56] [map         ]    Took 43.324 (wall), 124.268 (CPU) seconds, initialization time: 4.644 (wall), 4.560 (CPU) seconds.
[15:57:56] [map         ] Command(post): 
[15:57:56] [sort_prepare] Sorting...
[15:58:10] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[15:58:40] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
304489,9227,0,0,0,313716,0,0,0.970588047788,1.0,0.985074530293

[15:58:40] [ngm         ] 6996f5893fc23a4f84e9e00364871d38 (base: tests_base/base_mapping)
[15:58:40] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[15:58:40] [init        ] Reads: /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/reads1.fastq
[15:58:40] [init        ] Output:/project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/out_ngm.sam
[15:58:40] [map         ] Command(pre): 
[15:58:40] [map         ] Base run time file not existing, performing base run
[15:58:40] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads_base.fastq  --no-progress
[15:58:56] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --qry2 reads2.fastq --output out_ngm.sam --qry1 reads1.fastq --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4  --no-progress
[15:58:56] [map         ]    Mapping 69.00MB to 3199.00MB with ngm...
[15:59:40] [map         ]    Took 43.961 (wall), 105.972 (CPU) seconds, initialization time: 7.619 (wall), 7.816 (CPU) seconds.
[15:59:40] [map         ] Command(post): 
[15:59:41] [sort_prepare] Sorting...
[15:59:55] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[16:00:26] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
300919,11826,971,0,0,313716,0,0,0.962186445826,0.996783596674,0.979179513044

[16:00:26] [ngm_0_4_11  ] 6996f5893fc23a4f84e9e00364871d38 (base: tests_base/base_mapping)
[16:00:26] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[16:00:26] [init        ] Reads: /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/reads1.fastq
[16:00:26] [init        ] Output:/project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/out_ngm_0_4_11.sam
[16:00:26] [map         ] Command(pre): 
[16:00:26] [map         ] Base run time file not existing, performing base run
[16:00:26] [map         ] Command(pre-b): /software/ngm/bin-linux/ngm-0.4.11 --output out_ngm_0_4_11.sam --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4 --qry reads_base.fastq  --no-progress
[16:08:10] [map         ] Command(main): /software/ngm/bin-linux/ngm-0.4.11 --qry2 reads2.fastq --output out_ngm_0_4_11.sam --qry1 reads1.fastq --ref /project/teaser/genometeaser/references/H_sapiens.fa -t 4  --no-progress
[16:08:10] [map         ]    Mapping 69.00MB to 3199.00MB with ngm_0_4_11...
[16:08:52] [map         ]    Took 41.080 (wall), 110.656 (CPU) seconds, initialization time: 7.549 (wall), 7.824 (CPU) seconds.
[16:08:52] [map         ] Command(post): 
[16:08:52] [sort_prepare] Sorting...
[16:09:06] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[16:09:37] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
300315,12518,883,0,0,313716,0,0,0.959985039941,0.997068373628,0.978175368996

[16:09:37] [stampy      ] 6996f5893fc23a4f84e9e00364871d38 (base: tests_base/base_mapping)
[16:09:37] [init        ] Ref:   /project/teaser/genometeaser/references/H_sapiens.fa
[16:09:37] [init        ] Reads: /project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/reads1.fastq
[16:09:37] [init        ] Output:/project/teaser/genometeaser/tests_generated/6996f5893fc23a4f84e9e00364871d38/out_stampy.sam
[16:09:37] [map         ] Command(pre): 
[16:09:37] [map         ] Base run time file not existing, performing base run
[16:09:37] [map         ] Command(pre-b): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/H_sapiens.fa_o -h /project/teaser/genometeaser/references/H_sapiens.fa_o -M reads_base.fastq -o out_stampy.sam
[16:13:51] [map         ] Command(main): /project/teaser/genometeaser/software/stampy  -t 4 -g /project/teaser/genometeaser/references/H_sapiens.fa_o -h /project/teaser/genometeaser/references/H_sapiens.fa_o -o out_stampy.sam -M reads1.fastq reads2.fastq
[16:13:51] [map         ]    Mapping 69.00MB to 3199.00MB with stampy...
[16:25:59] [map         ]    Took 727.787 (wall), 2987.788 (CPU) seconds, initialization time: 1.760 (wall), 1.404 (CPU) seconds.
[16:25:59] [map         ] Command(post): 
[16:25:59] [sort_prepare] Sorting...
[16:26:13] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 50)... 
[16:26:44] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
300728,12987,1,0,0,313716,0,0,0.958602553273,0.999996674747,0.978862190859