Tests Overview

Use the links in the list below to view results for an individual data set, or navigate this page to compare mapper performance across all data sets.

5 tests executed, 0 fail, 0 aborted, 0 errors total.
StateNameReferenceExecutedOKWarningsErrors
E_coliE_coli5500
Errors and Warnings
No problems were encountered.
Results: Correctly Mapped Reads
Results: Correct/s
Results: Precision
Results: Recall
Results: F-Measure
Raw Results
Setup

This section contains information about the benchmarking process itself and can be shared in order to reproduce results and diagnose problems.

Teaser Accession30952ba36f06792420203f74b83db277
Report timestampMon Aug 6 14:55:04 2018
Framework Version1.2-public-vevev
Framework Working Directory/project/teaser/genometeaser
Framework Command Line./teaser.py setups_generated/30952ba36f06792420203f74b83db277.yaml -mcpu
Benchmark Configuration
evaluation: {pos_threshold: 75}
include: [base_teaser.yaml]
meta_timestamp: 1533559684.60224
report: {name: 30952ba36f06792420203f74b83db277}
sub_max_memory: 14000
sub_max_runtime: 900
teaser:
  tests:
    30952ba36f06792420203f74b83db277: {coverage: 1.0, error_rate_mult: 1.0, mutation_indel_avg_len: 1.0,
      mutation_indel_frac: 0.3, mutation_rate: 0.001, paired: false, platform: !!python/unicode 'ion_torrent',
      read_length: 150, reference: !!python/unicode 'E_coli.fasta', title: !!python/unicode 'E_coli',
      type: !!python/unicode 'simulated_teaser'}
test_mappers: [bowtie2, bwa, bwamem, bwasw, ngm]
test_parameters: []
threads: 4
Log
[14:48:08] [Main        ] framework cmd:  ./teaser.py setups_generated/30952ba36f06792420203f74b83db277.yaml -mcpu
[14:48:08] [Main        ] framework hash: vevev
[14:48:08] [Main        ] deployment mode: devel (True)
[14:48:08] [Main        ] Test Runner Setup - ""

[14:48:08] [Main        ] Using Teaser for data set creation
[14:48:08] [Teaser      ] Init. Creating 1 test datasets.
[14:48:08] [Teaser      ] 
[14:48:08] [Teaser      ] Creating test 30952ba36f06792420203f74b83db277
[14:48:08] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277
[14:48:08] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277
[14:48:08] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/E_coli.fasta.teaser.findex
[14:48:08] [Teaser      ] csample /project/teaser/genometeaser/references/E_coli.fasta 1500 0.500000
[14:48:08] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/E_coli.fasta.teaser.findex
[14:48:08] [Teaser      ] Sampling as contig: 1710 regions of size 1500 (pad 300), totalling 2566034 base pairs
[14:48:08] [Teaser      ] Sampling 1710 regions
[14:48:09] [Teaser      ] 10%
[14:48:09] [Teaser      ] 20%
[14:48:09] [Teaser      ] 30%
[14:48:09] [Teaser      ] 40%
[14:48:10] [Teaser      ] 50%
[14:48:10] [Teaser      ] 60%
[14:48:10] [Teaser      ] 70%
[14:48:10] [Teaser      ] 80%
[14:48:10] [Teaser      ] 90%
[14:48:10] [Teaser      ] 100%
[14:48:11] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277
[14:48:11] [Teaser      ] subprogram /project/teaser/genometeaser/software/dwgsim -c 2 -1 150 -2 0 -N 100000 -f TACGTACGTCTGAGCATCGATCGATGTACAGC  -d 500 -s 50  -r 0.001000  -R 0.300000  -y 0  /project/teaser/genometeaser/references/E_coli.fasta.sampled.2.1500.300.fasta dwgout
[14:48:56] [Teaser      ] move enc_dwgout.bwa.read1.fastq -> reads.fastq
[14:48:56] [Teaser      ] move enc_dwgout.bwa.read1.fastq.sam -> mapping_comparison.sam
[14:48:56] [Teaser      ] remove dwgout.bwa.read1.fastq.sam
[14:48:56] [Teaser      ] remove dwgout.mutations.txt
[14:48:56] [Teaser      ] remove dwgout.mutations.vcf
[14:48:56] [Teaser      ] remove dwgout.bfast.fastq
[14:48:56] [Teaser      ] remove dwgout.bwa.read1.fastq
[14:48:56] [Teaser      ] remove dwgout.bwa.read2.fastq
[14:48:56] [Teaser      ] change directory /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277
[14:48:56] [Teaser      ] move mapping_comparison.sam -> mapping_comparison_unfixed.sam
[14:48:56] [Teaser      ] Translating SAM file coordinates (as contig)...
[14:48:56] [Teaser      ] Loaded index from /project/teaser/genometeaser/references/E_coli.fasta.teaser.findex
[14:49:12] [Teaser      ] remove /project/teaser/genometeaser/references/E_coli.fasta.sampled.2.1500.300.fasta
[14:49:12] [Teaser      ] remove /project/teaser/genometeaser/references/E_coli.fasta.sampled.2.1500.300.fasta.index
[14:49:12] [Teaser      ] remove mapping_comparison_unfixed.sam
[14:49:12] [Teaser      ] Took 64 seconds for generating 30952ba36f06792420203f74b83db277
[14:49:12] [Main        ] Data set creation completed
[14:49:12] [Main        ] 
[14:49:14] [bowtie2     ] 30952ba36f06792420203f74b83db277 (base: tests_base/base_mapping)
[14:49:14] [init        ] Ref:   /project/teaser/genometeaser/references/E_coli.fasta
[14:49:14] [init        ] Reads: /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/reads.fastq
[14:49:14] [init        ] Output:/project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/out_bowtie2.sam
[14:49:14] [map         ] Command(pre): 
[14:49:14] [map         ] Base run time file not existing, performing base run
[14:49:14] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/E_coli.fasta_bt -U reads_base.fastq -S out_bowtie2.sam
[14:49:18] [map         ] Command(main): /project/teaser/genometeaser/software/bowtie2/bowtie2  -p 4 -x /project/teaser/genometeaser/references/E_coli.fasta_bt -U reads.fastq -S out_bowtie2.sam
[14:49:18] [map         ]    Mapping 37.00MB to 5.00MB with bowtie2...
[14:49:38] [map         ]    Took 19.303 (wall), 38.784 (CPU) seconds, initialization time: 0.847 (wall), 0.796 (CPU) seconds.
[14:49:38] [map         ] Command(post): 
[14:49:38] [sort_prepare] Sorting...
[14:50:01] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[14:50:11] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
93130,3555,3315,0,0,100000,0,0,0.963231111341,0.965628078179,0.964428105421

[14:50:12] [bwa         ] 30952ba36f06792420203f74b83db277 (base: tests_base/base_mapping)
[14:50:12] [init        ] Ref:   /project/teaser/genometeaser/references/E_coli.fasta
[14:50:12] [init        ] Reads: /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/reads.fastq
[14:50:12] [init        ] Output:/project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/out_bwa.sam
[14:50:12] [map         ] Command(pre): 
[14:50:12] [map         ] Base run time file not existing, performing base run
[14:50:12] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/E_coli.fasta reads_base.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/E_coli.fasta out_bwa.sam.bwa reads_base.fastq > out_bwa.sam
[14:50:16] [map         ] Command(main): /project/teaser/genometeaser/software/bwa aln /project/teaser/genometeaser/references/E_coli.fasta reads.fastq  -t 4 > out_bwa.sam.bwa; /project/teaser/genometeaser/software/bwa samse /project/teaser/genometeaser/references/E_coli.fasta out_bwa.sam.bwa reads.fastq > out_bwa.sam
[14:50:16] [map         ]    Mapping 37.00MB to 5.00MB with bwa...
[14:50:33] [map         ]    Took 16.096 (wall), 33.972 (CPU) seconds, initialization time: 0.673 (wall), 0.036 (CPU) seconds.
[14:50:33] [map         ] Command(post): 
[14:50:33] [sort_prepare] Sorting...
[14:50:42] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[14:50:52] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
1548,51,98401,0,0,100000,0,0,0.968105065666,0.0154878988284,0.0304880450624

[14:50:52] [bwamem      ] 30952ba36f06792420203f74b83db277 (base: tests_base/base_mapping)
[14:50:52] [init        ] Ref:   /project/teaser/genometeaser/references/E_coli.fasta
[14:50:52] [init        ] Reads: /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/reads.fastq
[14:50:52] [init        ] Output:/project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/out_bwamem.sam
[14:50:52] [map         ] Command(pre): 
[14:50:52] [map         ] Base run time file not existing, performing base run
[14:50:52] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/E_coli.fasta reads_base.fastq  -t 4 > out_bwamem.sam
[14:50:54] [map         ] Command(main): /project/teaser/genometeaser/software/bwa mem /project/teaser/genometeaser/references/E_coli.fasta reads.fastq  -t 4 > out_bwamem.sam
[14:50:54] [map         ]    Mapping 37.00MB to 5.00MB with bwamem...
[14:51:23] [map         ]    Took 28.189 (wall), 61.192 (CPU) seconds, initialization time: 0.445 (wall), 0.040 (CPU) seconds.
[14:51:23] [map         ] Command(post): 
[14:51:23] [sort_prepare] Sorting...
[14:51:35] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[14:51:45] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
96327,3662,11,0,0,100000,0,0,0.963375971357,0.99988581868,0.981291416871

[14:51:45] [bwasw       ] 30952ba36f06792420203f74b83db277 (base: tests_base/base_mapping)
[14:51:45] [init        ] Ref:   /project/teaser/genometeaser/references/E_coli.fasta
[14:51:45] [init        ] Reads: /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/reads.fastq
[14:51:45] [init        ] Output:/project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/out_bwasw.sam
[14:51:45] [map         ] Command(pre): 
[14:51:45] [map         ] Base run time file not existing, performing base run
[14:51:45] [map         ] Command(pre-b): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/E_coli.fasta reads_base.fastq  -t 4 > out_bwasw.sam
[14:51:48] [map         ] Command(main): /project/teaser/genometeaser/software/bwa bwasw /project/teaser/genometeaser/references/E_coli.fasta reads.fastq  -t 4 > out_bwasw.sam
[14:51:48] [map         ]    Mapping 37.00MB to 5.00MB with bwasw...
[14:52:34] [map         ]    Took 45.206 (wall), 120.964 (CPU) seconds, initialization time: 0.116 (wall), 0.040 (CPU) seconds.
[14:52:34] [map         ] Command(post): 
[14:52:34] [sort_prepare] Sorting...
[14:52:48] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[14:52:56] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
87000,3439,9532,29,106,100000,0,0,0.961974369465,0.901255542204,0.93062560504

[14:52:56] [ngm         ] 30952ba36f06792420203f74b83db277 (base: tests_base/base_mapping)
[14:52:56] [init        ] Ref:   /project/teaser/genometeaser/references/E_coli.fasta
[14:52:56] [init        ] Reads: /project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/reads.fastq
[14:52:56] [init        ] Output:/project/teaser/genometeaser/tests_generated/30952ba36f06792420203f74b83db277/out_ngm.sam
[14:52:56] [map         ] Command(pre): 
[14:52:56] [map         ] Base run time file not existing, performing base run
[14:52:56] [map         ] Command(pre-b): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/E_coli.fasta -t 4 --qry reads_base.fastq  --no-progress
[14:53:57] [map         ] Command(main): /project/teaser/genometeaser/software/ngm/ngm --output out_ngm.sam --ref /project/teaser/genometeaser/references/E_coli.fasta -t 4 --qry reads.fastq  --no-progress
[14:53:57] [map         ]    Mapping 37.00MB to 5.00MB with ngm...
[14:54:43] [map         ]    Took 44.886 (wall), 70.544 (CPU) seconds, initialization time: 11.459 (wall), 10.896 (CPU) seconds.
[14:54:43] [map         ] Command(post): 
[14:54:44] [sort_prepare] Sorting...
[14:54:58] [calc_stats  ] Compute mapping statistics (lib.evaluator.ThresholdBasedEvaluator, threshold 75)... 
[14:55:04] [calc_stats  ] correct,wrong,not_mapped,not_found,not_found_comparison,total,reverse,secondary,precision,recall,fmeasure
96408,3592,0,0,0,100000,0,0,0.96408,1.0,0.981711539245