img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%31.77437%
ghc-7.0.1-O100.04189%10.66215%31.90185%
ghc-7.0.1-O2101.82857%6.20750%32.59242%
ghc-7.0.1-Odph98.85555%6.88909%31.54693%
ghc-7.2.195.60913%7.77134%31.39737%
ghc-7.2.1-O96.59010%9.17158%32.00604%
ghc-7.2.1-O2100.42109%9.16102%32.32774%
ghc-7.2.1-Odph97.89637%6.95620%31.92913%
ghc-7.4.198.92658%7.23461%31.74735%
ghc-7.4.1-O97.90996%8.80652%31.56250%
ghc-7.4.1-O2102.58436%9.20558%32.02430%
ghc-7.4.1-Odph99.76810%7.21172%30.62098%
ghc-7.6.197.12532%9.09215%32.18546%
ghc-7.6.1-O96.32214%8.60332%31.44726%
ghc-7.6.1-O298.96920%8.66016%31.10561%
ghc-7.6.1-Odph102.08499%9.96587%31.89511%
ghc-7.8.191.00639%9.38120%32.14248%
ghc-7.8.1-O90.79321%7.87171%32.49100%
ghc-7.8.1-O294.74204%8.40692%32.49410%
ghc-7.8.1-Odph91.61459%7.82993%31.89070%
ghc-7.10.188.30637%6.84869%32.47733%
ghc-7.10.1-O88.93338%6.85823%33.18151%
ghc-7.10.1-O291.39313%9.52952%31.37100%
ghc-7.10.1-Odph87.57061%5.86825%30.79975%
ghc-8.0.186.28850%7.22753%29.78081%
ghc-8.0.1-O88.97120%9.27539%32.35346%
ghc-8.0.1-O289.85905%7.38254%33.84309%
ghc-8.0.1-Odph88.30389%8.19417%30.91933%
csv

Seperated by entropy

The following shows the summary including only the lower or the upper half of programs (per category), regarding the entropy of their files. This entropy is measured by taking the length of the gnu zipped program code length. Programs with lower entropy should be simpler than programs with higher entropy. If the number of programs is uneven in a category, then one program belongs to the upper and the lower half.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%32.46509%
ghc-7.0.1-O100.53914%10.30462%31.60902%
ghc-7.0.1-O2101.54222%4.80776%33.04887%
ghc-7.0.1-Odph99.96774%5.48886%31.59900%
ghc-7.2.196.75591%8.07066%31.40944%
ghc-7.2.1-O95.31435%7.36260%32.47248%
ghc-7.2.1-O2102.11391%9.24332%32.34621%
ghc-7.2.1-Odph97.77215%7.60660%32.41686%
ghc-7.4.1101.80286%5.50888%32.06409%
ghc-7.4.1-O98.75005%9.52125%31.57692%
ghc-7.4.1-O2102.86729%10.30664%31.42115%
ghc-7.4.1-Odph99.80426%8.02102%30.23118%
ghc-7.6.198.02536%9.67216%31.99079%
ghc-7.6.1-O96.01105%8.24692%31.26373%
ghc-7.6.1-O297.05945%7.97829%30.68206%
ghc-7.6.1-Odph100.78976%10.78593%31.88667%
ghc-7.8.192.61708%8.86822%32.60894%
ghc-7.8.1-O91.53116%7.49199%32.42684%
ghc-7.8.1-O293.80619%8.86581%31.84071%
ghc-7.8.1-Odph91.61995%8.31247%31.73866%
ghc-7.10.188.53216%7.44303%32.61741%
ghc-7.10.1-O88.48066%7.54029%33.06967%
ghc-7.10.1-O291.43961%10.20475%31.32061%
ghc-7.10.1-Odph88.76449%5.56074%31.30010%
ghc-8.0.185.72401%6.04061%30.10305%
ghc-8.0.1-O89.20008%9.02408%31.99629%
ghc-8.0.1-O289.90233%7.59979%33.01201%
ghc-8.0.1-Odph89.53821%7.50459%32.04996%
csv

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%31.42120%
ghc-7.0.1-O101.11389%11.88628%31.30545%
ghc-7.0.1-O2102.47407%6.32824%32.74917%
ghc-7.0.1-Odph99.05201%6.76376%31.49845%
ghc-7.2.194.90127%8.07789%31.01048%
ghc-7.2.1-O95.87930%10.20552%31.32434%
ghc-7.2.1-O2100.01713%10.18241%31.81912%
ghc-7.2.1-Odph98.39323%7.09667%32.07693%
ghc-7.4.197.78494%7.50261%31.59356%
ghc-7.4.1-O97.49340%9.46805%32.02862%
ghc-7.4.1-O2103.37299%10.31992%32.39521%
ghc-7.4.1-Odph99.40196%6.70803%30.16692%
ghc-7.6.195.77771%9.42596%31.66034%
ghc-7.6.1-O97.91861%8.79133%31.42874%
ghc-7.6.1-O299.19824%9.39351%31.22737%
ghc-7.6.1-Odph102.65902%11.13961%31.89777%
ghc-7.8.189.59495%7.86116%31.67866%
ghc-7.8.1-O91.02216%8.84286%32.58714%
ghc-7.8.1-O294.46654%9.23272%32.90291%
ghc-7.8.1-Odph91.22364%8.54666%31.58804%
ghc-7.10.187.92368%7.53207%31.45904%
ghc-7.10.1-O88.90962%6.95691%33.10144%
ghc-7.10.1-O291.15555%9.28409%31.91864%
ghc-7.10.1-Odph86.96904%6.39421%30.35765%
ghc-8.0.187.27450%7.94757%30.11444%
ghc-8.0.1-O88.37279%10.48464%32.20874%
ghc-8.0.1-O289.43955%7.97975%33.94090%
ghc-8.0.1-Odph88.16986%8.60559%30.48067%
csv

binarytrees

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%29.95552%
ghc-7.0.1-O101.20418%0.00000%25.21016%
ghc-7.0.1-O2105.30165%0.00000%36.43757%
ghc-7.0.1-Odph104.81685%0.00000%27.19765%
ghc-7.2.194.66915%0.00000%28.24595%
ghc-7.2.1-O98.45374%0.00000%31.42705%
ghc-7.2.1-O299.64699%0.00000%30.73143%
ghc-7.2.1-Odph112.18010%0.00000%33.49761%
ghc-7.4.1100.49272%0.00000%31.22645%
ghc-7.4.1-O104.84360%0.00000%35.06487%
ghc-7.4.1-O2107.62022%0.00000%29.18355%
ghc-7.4.1-Odph102.82748%0.00000%31.65362%
ghc-7.6.199.31271%0.00000%33.30389%
ghc-7.6.1-O99.41556%0.00000%28.26589%
ghc-7.6.1-O299.66217%0.00000%31.59125%
ghc-7.6.1-Odph103.84934%0.00000%33.23542%
ghc-7.8.197.84555%0.00000%33.13202%
ghc-7.8.1-O95.87440%0.00000%35.48068%
ghc-7.8.1-O296.14808%0.00000%28.36841%
ghc-7.8.1-Odph93.56782%0.00000%28.21792%
ghc-7.10.195.24569%0.00000%33.51413%
ghc-7.10.1-O91.35771%0.00000%29.52032%
ghc-7.10.1-O290.92354%0.00000%34.44998%
ghc-7.10.1-Odph92.09941%0.00000%29.59519%
ghc-8.0.187.96819%0.00000%30.09090%
ghc-8.0.1-O83.61854%0.00000%24.05684%
ghc-8.0.1-O295.63228%0.00000%30.73975%
ghc-8.0.1-Odph82.98671%0.00000%24.60636%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.08263120.50122%100.00000%29.95552%36.09677%29.95552%1.11376
ghc-7.0.1-O 35 1.09566121.95227%101.20418%25.21016%30.74436%25.51373%1.10239
ghc-7.0.1-O2 35 1.14002126.88977%105.30165%36.43757%46.23555%38.36937%0.87407
ghc-7.0.1-Odph 35 1.13477126.30558%104.81685%27.19765%34.35215%28.50772%1.12130
ghc-7.2.1 35 1.02491114.07748%94.66915%28.24595%32.22226%26.74020%0.88178
ghc-7.2.1-O 35 1.06589118.63795%98.45374%31.42705%37.28440%30.94110%0.85825
ghc-7.2.1-O2 35 1.07880120.07583%99.64699%30.73143%36.90102%30.62294%1.09701
ghc-7.2.1-Odph 35 1.21449135.17839%112.18010%33.49761%45.28154%37.57766%1.12171
ghc-7.4.1 35 1.08796121.09495%100.49272%31.22645%37.81366%31.38031%1.09386
ghc-7.4.1-O 35 1.13506126.33781%104.84360%35.06487%44.30019%36.76327%0.89076
ghc-7.4.1-O2 35 1.16512129.68368%107.62022%29.18355%37.84630%31.40740%1.11356
ghc-7.4.1-Odph 35 1.11324123.90837%102.82748%31.65362%39.22148%32.54862%1.11092
ghc-7.6.1 35 1.07518119.67302%99.31271%33.30389%39.85577%33.07500%0.84170
ghc-7.6.1-O 35 1.07630119.79695%99.41556%28.26589%33.86167%28.10069%1.10112
ghc-7.6.1-O2 35 1.07897120.09413%99.66217%31.59125%37.93923%31.48452%0.88186
ghc-7.6.1-Odph 35 1.12430125.13972%103.84934%33.23542%41.59071%34.51477%1.04924
ghc-7.8.1 35 1.05930117.90508%97.84555%33.13202%39.06433%32.41821%0.82438
ghc-7.8.1-O 35 1.03796115.52982%95.87440%35.48068%40.99077%34.01689%0.80108
ghc-7.8.1-O2 35 1.04092115.85961%96.14808%28.36841%32.86753%27.27568%0.83093
ghc-7.8.1-Odph 35 1.01299112.75036%93.56782%28.21792%31.81580%26.40289%0.84334
ghc-7.10.1 35 1.03115114.77221%95.24569%33.51413%38.46491%31.92076%0.81679
ghc-7.10.1-O 35 0.98906110.08715%91.35771%29.52032%32.49808%26.96909%0.81584
ghc-7.10.1-O2 35 0.98436109.56397%90.92354%34.44998%37.74476%31.32314%0.81164
ghc-7.10.1-Odph 35 0.99709110.98091%92.09941%29.59519%32.84501%27.25699%0.80061
ghc-8.0.1 35 0.95237106.00274%87.96819%30.09090%31.89718%26.47042%0.79830
ghc-8.0.1-O 35 0.90528100.76136%83.61854%24.05684%24.24000%20.11598%0.79029
ghc-8.0.1-O2 35 1.03534115.23807%95.63228%30.73975%35.42390%29.39713%0.87083
ghc-8.0.1-Odph 35 0.89844100.00000%82.98671%24.60636%24.60636%20.42001%0.79971
csv

fannkuchredux

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%30.47978%
ghc-7.0.1-O98.47057%1.07061%31.88537%
ghc-7.0.1-O298.32552%5.13072%30.82895%
ghc-7.0.1-Odph97.28145%2.43634%31.09590%
ghc-7.2.196.76096%0.70093%31.12282%
ghc-7.2.1-O93.80854%3.34210%31.28382%
ghc-7.2.1-O297.78677%1.73122%31.94244%
ghc-7.2.1-Odph97.20847%2.40259%30.10119%
ghc-7.4.197.42420%5.24672%31.69053%
ghc-7.4.1-O91.34942%0.87872%28.79508%
ghc-7.4.1-O299.00519%0.65523%34.35702%
ghc-7.4.1-Odph103.00431%1.30667%31.71672%
ghc-7.6.196.94506%3.88168%34.69201%
ghc-7.6.1-O97.35541%2.19136%29.71110%
ghc-7.6.1-O2104.66406%5.29875%32.94518%
ghc-7.6.1-Odph102.54495%0.30967%31.31270%
ghc-7.8.182.85228%1.27524%30.49133%
ghc-7.8.1-O86.03096%0.97758%30.36340%
ghc-7.8.1-O296.40861%6.63535%33.18742%
ghc-7.8.1-Odph89.11369%2.42003%32.11994%
ghc-7.10.187.71063%1.59617%33.16460%
ghc-7.10.1-O94.32159%0.74229%37.22456%
ghc-7.10.1-O294.18314%1.56415%34.95632%
ghc-7.10.1-Odph85.73790%5.08319%30.17935%
ghc-8.0.179.69573%2.68507%24.61164%
ghc-8.0.1-O88.30055%1.42043%33.44760%
ghc-8.0.1-O293.68741%3.95777%34.72589%
ghc-8.0.1-Odph78.90487%4.93024%22.07422%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%31.34680%
ghc-7.0.1-O99.54118%0.00000%35.16045%
ghc-7.0.1-O293.19480%0.00000%32.33677%
ghc-7.0.1-Odph99.71780%0.00000%32.38670%
ghc-7.2.196.06003%0.00000%31.22562%
ghc-7.2.1-O97.15063%0.00000%33.84651%
ghc-7.2.1-O299.51799%0.00000%31.84564%
ghc-7.2.1-Odph94.80588%0.00000%32.49103%
ghc-7.4.1102.67092%0.00000%31.25505%
ghc-7.4.1-O92.22815%0.00000%27.67406%
ghc-7.4.1-O299.66043%0.00000%32.23872%
ghc-7.4.1-Odph101.69764%0.00000%31.21016%
ghc-7.6.1100.82675%0.00000%34.45584%
ghc-7.6.1-O95.16405%0.00000%27.82153%
ghc-7.6.1-O299.36531%0.00000%32.88708%
ghc-7.6.1-Odph102.23528%0.00000%32.33481%
ghc-7.8.181.57704%0.00000%30.72742%
ghc-7.8.1-O87.00854%0.00000%33.16892%
ghc-7.8.1-O289.77325%0.00000%30.07654%
ghc-7.8.1-Odph86.69366%0.00000%32.58525%
ghc-7.10.186.11446%0.00000%34.92862%
ghc-7.10.1-O93.57930%0.00000%38.63293%
ghc-7.10.1-O295.74729%0.00000%37.71823%
ghc-7.10.1-Odph90.82109%0.00000%33.82343%
ghc-8.0.182.38080%0.00000%29.30449%
ghc-8.0.1-O89.72097%0.00000%33.33746%
ghc-8.0.1-O297.64518%0.00000%37.28060%
ghc-8.0.1-Odph83.83511%0.00000%30.52124%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.15460122.58351%100.00000%31.34680%38.42601%31.34680%1.10952
ghc-7.0.1-O 35 1.14931122.02107%99.54118%35.16045%42.90315%34.99912%1.11444
ghc-7.0.1-O2 35 1.07603114.24146%93.19480%32.33677%36.94200%30.13619%0.83401
ghc-7.0.1-Odph 35 1.15134122.23758%99.71780%32.38670%39.58872%32.29531%1.09917
ghc-7.2.1 35 1.10911117.75376%96.06003%31.22562%36.76934%29.99534%0.99825
ghc-7.2.1-O 35 1.12170119.09066%97.15063%33.84651%40.30804%32.88210%1.09248
ghc-7.2.1-O2 35 1.14904121.99265%99.51799%31.84564%38.84934%31.69214%1.11996
ghc-7.2.1-Odph 35 1.09463116.21638%94.80588%32.49103%37.75990%30.80341%1.09522
ghc-7.4.1 35 1.18544125.85763%102.67092%31.25505%39.33686%32.08985%1.12471
ghc-7.4.1-O 35 1.06487113.05650%92.22815%27.67406%31.28732%25.52327%1.10151
ghc-7.4.1-O2 35 1.15068122.16725%99.66043%32.23872%39.38516%32.12924%1.09939
ghc-7.4.1-Odph 35 1.17420124.66454%101.69764%31.21016%38.90800%31.74000%1.12329
ghc-7.6.1 35 1.16415123.59697%100.82675%34.45584%42.58637%34.74070%1.10700
ghc-7.6.1-O 35 1.09877116.65543%95.16405%27.82153%32.45533%26.47609%1.10350
ghc-7.6.1-O2 35 1.14727121.80548%99.36531%32.88708%40.05827%32.67835%1.11750
ghc-7.6.1-Odph 35 1.18041125.32360%102.23528%32.33481%40.52314%33.05758%1.12459
ghc-7.8.1 35 0.94189100.00000%81.57704%30.72742%30.72742%25.06652%0.79494
ghc-7.8.1-O 35 1.00460106.65812%87.00854%33.16892%35.37735%28.85979%0.79656
ghc-7.8.1-O2 35 1.03652110.04721%89.77325%30.07654%33.09840%27.00069%0.81797
ghc-7.8.1-Odph 35 1.00097106.27213%86.69366%32.58525%34.62904%28.24935%0.80411
ghc-7.10.1 35 0.99428105.56213%86.11446%34.92862%36.87140%30.07859%0.79746
ghc-7.10.1-O 35 1.08047114.71279%93.57930%38.63293%44.31692%36.15243%0.80696
ghc-7.10.1-O2 35 1.10550117.37039%95.74729%37.71823%44.27003%36.11418%0.83139
ghc-7.10.1-Odph 35 1.04862111.33168%90.82109%33.82343%37.65620%30.71881%0.84093
ghc-8.0.1 35 0.95117100.98528%82.38080%29.30449%29.59322%24.14127%0.81967
ghc-8.0.1-O 35 1.03592109.98312%89.72097%33.33746%36.66558%29.91070%0.83509
ghc-8.0.1-O2 35 1.12741119.69689%97.64518%37.28060%44.62372%36.40271%0.87155
ghc-8.0.1-Odph 35 0.96796102.76802%83.83511%30.52124%31.36607%25.58751%0.80364
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%29.61275%
ghc-7.0.1-O97.39997%0.00000%28.61030%
ghc-7.0.1-O2103.45624%0.00000%29.32112%
ghc-7.0.1-Odph94.84511%0.00000%29.80509%
ghc-7.2.197.46188%0.00000%31.02003%
ghc-7.2.1-O90.46644%0.00000%28.72112%
ghc-7.2.1-O296.05555%0.00000%32.03924%
ghc-7.2.1-Odph99.61106%0.00000%27.71136%
ghc-7.4.192.17748%0.00000%32.12602%
ghc-7.4.1-O90.47070%0.00000%29.91610%
ghc-7.4.1-O298.34996%0.00000%36.47531%
ghc-7.4.1-Odph104.31098%0.00000%32.22329%
ghc-7.6.193.06338%0.00000%34.92817%
ghc-7.6.1-O99.54678%0.00000%31.60068%
ghc-7.6.1-O2109.96281%0.00000%33.00329%
ghc-7.6.1-Odph102.85461%0.00000%30.29059%
ghc-7.8.184.12752%0.00000%30.25524%
ghc-7.8.1-O85.05338%0.00000%27.55789%
ghc-7.8.1-O2103.04396%0.00000%36.29830%
ghc-7.8.1-Odph91.53372%0.00000%31.65463%
ghc-7.10.189.30681%0.00000%31.40057%
ghc-7.10.1-O95.06388%0.00000%35.81620%
ghc-7.10.1-O292.61899%0.00000%32.19440%
ghc-7.10.1-Odph80.65471%0.00000%26.53526%
ghc-8.0.177.01065%0.00000%19.91878%
ghc-8.0.1-O86.88012%0.00000%33.55774%
ghc-8.0.1-O289.72964%0.00000%32.17117%
ghc-8.0.1-Odph73.97464%0.00000%13.62721%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.15090135.18147%100.00000%29.61275%40.03095%29.61275%1.11064
ghc-7.0.1-O 35 1.12098131.66670%97.39997%28.61030%37.67024%27.86642%1.10538
ghc-7.0.1-O2 35 1.19068139.85366%103.45624%29.32112%41.00666%30.33453%1.13242
ghc-7.0.1-Odph 35 1.09157128.21301%94.84511%29.80509%38.21400%28.26867%1.10128
ghc-7.2.1 35 1.12169131.75040%97.46188%31.02003%40.86902%30.23271%1.09078
ghc-7.2.1-O 35 1.04118122.29386%90.46644%28.72112%35.12417%25.98298%0.87724
ghc-7.2.1-O2 35 1.10550129.84930%96.05555%32.03924%41.60272%30.77546%0.88965
ghc-7.2.1-Odph 35 1.14642134.65570%99.61106%27.71136%37.31492%27.60358%1.12176
ghc-7.4.1 35 1.06087124.60687%92.17748%32.12602%40.03122%29.61295%0.86679
ghc-7.4.1-O 35 1.04123122.29962%90.47070%29.91610%36.58727%27.06530%0.87135
ghc-7.4.1-O2 35 1.13191132.95092%98.34996%36.47531%48.49427%35.87346%1.11707
ghc-7.4.1-Odph 35 1.20051141.00912%104.31098%32.22329%45.43778%33.61243%1.12855
ghc-7.6.1 35 1.07107125.80444%93.06338%34.92817%43.94119%32.50534%0.80405
ghc-7.6.1-O 35 1.14568134.56879%99.54678%31.60068%42.52465%31.45746%1.10133
ghc-7.6.1-O2 35 1.26556148.64934%109.96281%33.00329%49.05917%36.29134%1.15062
ghc-7.6.1-Odph 35 1.18375139.04038%102.85461%30.29059%42.11615%31.15527%1.11904
ghc-7.8.1 35 0.96822113.72482%84.12752%30.25524%34.40772%25.45299%0.79826
ghc-7.8.1-O 35 0.97888114.97641%85.05338%27.55789%31.68507%23.43892%0.80129
ghc-7.8.1-O2 35 1.18593139.29634%103.04396%36.29830%50.56220%37.40321%1.10754
ghc-7.8.1-Odph 35 1.05346123.73663%91.53372%31.65463%39.16837%28.97466%0.82152
ghc-7.10.1 35 1.02783120.72625%89.30681%31.40057%37.90874%28.04285%0.81117
ghc-7.10.1-O 35 1.09409128.50875%95.06388%35.81620%46.02695%34.04827%0.85302
ghc-7.10.1-O2 35 1.06595125.20371%92.61899%32.19440%40.30858%29.81813%0.81393
ghc-7.10.1-Odph 35 0.92825109.03022%80.65471%26.53526%28.93146%21.40194%0.80593
ghc-8.0.1 35 0.88631104.10413%77.01065%19.91878%20.73628%15.33959%0.80124
ghc-8.0.1-O 35 0.99990117.44583%86.88012%33.55774%39.41217%29.15501%0.81948
ghc-8.0.1-O2 35 1.03270121.29785%89.72964%32.17117%39.02294%28.86708%0.82237
ghc-8.0.1-Odph 35 0.85137100.00000%73.97464%13.62721%13.62721%10.08068%0.80072
csv

fasta

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%24.89461%
ghc-7.0.1-O114.07164%0.00000%28.59609%
ghc-7.0.1-O2102.74506%0.00000%30.58117%
ghc-7.0.1-Odph101.10633%0.00000%33.78541%
ghc-7.2.1103.01327%0.00000%31.27044%
ghc-7.2.1-O104.52957%0.00000%32.66546%
ghc-7.2.1-O2107.16817%0.00000%30.24731%
ghc-7.2.1-Odph108.05043%0.00000%33.21763%
ghc-7.4.1103.92786%0.00000%29.69301%
ghc-7.4.1-O104.79273%0.00000%35.26769%
ghc-7.4.1-O2117.13384%0.00000%31.80458%
ghc-7.4.1-Odph101.52031%0.00000%28.80456%
ghc-7.6.1103.33047%0.00000%29.12939%
ghc-7.6.1-O106.60454%0.00000%32.15865%
ghc-7.6.1-O2114.60715%0.00000%30.86491%
ghc-7.6.1-Odph119.26793%0.00000%34.82185%
ghc-7.8.190.62252%0.00000%25.29351%
ghc-7.8.1-O102.02279%0.00000%35.59746%
ghc-7.8.1-O2107.95912%0.00000%36.58237%
ghc-7.8.1-Odph95.29430%0.00000%34.99351%
ghc-7.10.192.45713%0.00000%29.72405%
ghc-7.10.1-O99.64392%0.00000%35.46523%
ghc-7.10.1-O2108.26974%0.00000%33.83147%
ghc-7.10.1-Odph92.87876%0.00000%28.77783%
ghc-8.0.193.89088%0.00000%29.92746%
ghc-8.0.1-O101.83895%0.00000%36.51756%
ghc-8.0.1-O299.81546%0.00000%35.05254%
ghc-8.0.1-Odph100.91042%0.00000%33.86998%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.04487110.34785%100.00000%24.89461%27.47067%24.89461%1.11245
ghc-7.0.1-O 35 1.19190125.87560%114.07164%28.59609%35.99550%32.62003%1.12357
ghc-7.0.1-O2 35 1.07355113.37696%102.74506%30.58117%34.67200%31.42064%0.84519
ghc-7.0.1-Odph 35 1.05643111.56866%101.10633%33.78541%37.69393%34.15919%0.82601
ghc-7.2.1 35 1.07636113.67293%103.01327%31.27044%35.54603%32.21270%0.89008
ghc-7.2.1-O 35 1.09220115.34613%104.52957%32.66546%37.67835%34.14507%0.83143
ghc-7.2.1-O2 35 1.11977118.25777%107.16817%30.24731%35.76979%32.41549%1.11846
ghc-7.2.1-Odph 35 1.12899119.23132%108.05043%33.21763%39.60582%35.89180%1.04352
ghc-7.4.1 35 1.08591114.68216%103.92786%29.69301%34.05259%30.85931%1.11477
ghc-7.4.1-O 35 1.09495115.63653%104.79273%35.26769%40.78233%36.95797%0.82187
ghc-7.4.1-O2 35 1.22390129.25468%117.13384%31.80458%41.10891%37.25393%1.12247
ghc-7.4.1-Odph 35 1.06076112.02548%101.52031%28.80456%32.26845%29.24248%1.12601
ghc-7.6.1 35 1.07967114.02295%103.33047%29.12939%33.21419%30.09954%1.10039
ghc-7.6.1-O 35 1.11388117.63582%106.60454%32.15865%37.83009%34.28258%1.11089
ghc-7.6.1-O2 35 1.19750126.46652%114.60715%30.86491%39.03378%35.37340%1.11732
ghc-7.6.1-Odph 35 1.24620131.60960%119.26793%34.82185%45.82889%41.53130%1.13089
ghc-7.8.1 35 0.94689100.00000%90.62252%25.29351%25.29351%22.92162%0.80644
ghc-7.8.1-O 35 1.06601112.57995%102.02279%35.59746%40.07561%36.31752%0.80680
ghc-7.8.1-O2 35 1.12803119.13057%107.95912%36.58237%43.58079%39.49401%0.84473
ghc-7.8.1-Odph 35 0.99570105.15521%95.29430%34.99351%36.79750%33.34682%0.81179
ghc-7.10.1 35 0.96606102.02446%92.45713%29.72405%30.32580%27.48200%0.80232
ghc-7.10.1-O 35 1.04115109.95492%99.64392%35.46523%38.99577%35.33895%0.81317
ghc-7.10.1-O2 35 1.13128119.47333%108.26974%33.83147%40.41958%36.62924%0.91807
ghc-7.10.1-Odph 35 0.97046102.48971%92.87876%28.77783%29.49431%26.72849%0.82330
ghc-8.0.1 35 0.98104103.60657%93.89088%29.92746%31.00681%28.09915%0.80803
ghc-8.0.1-O 35 1.06409112.37709%101.83895%36.51756%41.03736%37.18909%0.81815
ghc-8.0.1-O2 35 1.04294110.14421%99.81546%35.05254%38.60835%34.98786%0.81196
ghc-8.0.1-Odph 35 1.05438111.35248%100.91042%33.86998%37.71507%34.17835%0.84721
csv

mandelbrot

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%36.08479%
ghc-7.0.1-O91.23989%0.00000%35.96103%
ghc-7.0.1-O298.69402%0.00000%30.10267%
ghc-7.0.1-Odph95.22671%0.00000%36.07082%
ghc-7.2.184.05400%0.00000%32.16359%
ghc-7.2.1-O86.22506%0.00000%30.23818%
ghc-7.2.1-O290.27293%0.00000%33.72538%
ghc-7.2.1-Odph89.67647%0.00000%37.44806%
ghc-7.4.193.79593%0.00000%33.78458%
ghc-7.4.1-O90.04421%0.00000%26.90627%
ghc-7.4.1-O296.31171%0.00000%32.65722%
ghc-7.4.1-Odph84.17654%0.00000%30.17751%
ghc-7.6.189.02875%0.00000%33.10827%
ghc-7.6.1-O92.29837%0.00000%34.97432%
ghc-7.6.1-O286.71071%0.00000%31.55884%
ghc-7.6.1-Odph85.45351%0.00000%30.23507%
ghc-7.8.185.29356%0.00000%34.95585%
ghc-7.8.1-O77.29752%0.00000%28.21880%
ghc-7.8.1-O291.21751%0.00000%35.07194%
ghc-7.8.1-Odph81.12180%0.00000%30.60264%
ghc-7.10.185.41727%0.00000%34.25707%
ghc-7.10.1-O81.61488%0.00000%36.07284%
ghc-7.10.1-O283.04379%0.00000%33.64421%
ghc-7.10.1-Odph77.53920%0.00000%28.92424%
ghc-8.0.180.54579%0.00000%32.58415%
ghc-8.0.1-O79.15707%0.00000%29.77058%
ghc-8.0.1-O289.64226%0.00000%34.92297%
ghc-8.0.1-Odph75.98359%0.00000%25.41410%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.23441131.60736%100.00000%36.08479%47.49023%36.08479%1.12519
ghc-7.0.1-O 35 1.12627120.07841%91.23989%35.96103%43.18143%32.81080%0.90682
ghc-7.0.1-O2 35 1.21828129.88859%98.69402%30.10267%39.09993%29.70953%1.12925
ghc-7.0.1-Odph 35 1.17548125.32536%95.22671%36.07082%45.20588%34.34905%1.09389
ghc-7.2.1 35 1.03757110.62125%84.05400%32.16359%35.57977%27.03478%0.87627
ghc-7.2.1-O 35 1.06437113.47853%86.22506%30.23818%34.31384%26.07289%1.10369
ghc-7.2.1-O2 35 1.11433118.80582%90.27293%33.72538%40.06772%30.44489%0.87262
ghc-7.2.1-Odph 35 1.10697118.02083%89.67647%37.44806%44.19651%33.58209%0.85105
ghc-7.4.1 35 1.15782123.44234%93.79593%33.78458%41.70448%31.68856%1.09612
ghc-7.4.1-O 35 1.11151118.50480%90.04421%26.90627%31.88522%24.22754%1.10881
ghc-7.4.1-O2 35 1.18888126.75329%96.31171%32.65722%41.39410%31.45273%1.13147
ghc-7.4.1-Odph 35 1.03908110.78252%84.17654%30.17751%33.43141%25.40238%0.86506
ghc-7.6.1 35 1.09898117.16839%89.02875%33.10827%38.79243%29.47588%0.85829
ghc-7.6.1-O 35 1.13934121.47144%92.29837%34.97432%42.48382%32.28073%0.86647
ghc-7.6.1-O2 35 1.07036114.11768%86.71071%31.55884%36.01422%27.36490%0.91837
ghc-7.6.1-Odph 35 1.05484112.46311%85.45351%30.23507%34.00331%25.83693%0.89855
ghc-7.8.1 35 1.05287112.25261%85.29356%34.95585%39.23885%29.81509%0.88867
ghc-7.8.1-O 35 0.95417101.72923%77.29752%28.21880%28.70677%21.81243%0.79989
ghc-7.8.1-O2 35 1.12599120.04895%91.21751%35.07194%42.10350%31.99175%0.90648
ghc-7.8.1-Odph 35 1.00137106.76226%81.12180%30.60264%32.67208%24.82542%0.82536
ghc-7.10.1 35 1.05440112.41541%85.41727%34.25707%38.51022%29.26145%0.81207
ghc-7.10.1-O 35 1.00746107.41119%81.61488%36.07284%38.74626%29.44080%0.80266
ghc-7.10.1-O2 35 1.02510109.29175%83.04379%33.64421%36.77034%27.93943%0.82191
ghc-7.10.1-Odph 35 0.95715102.04729%77.53920%28.92424%29.51641%22.42763%0.80416
ghc-8.0.1 35 0.99426106.00419%80.54579%32.58415%34.54056%26.24516%0.81656
ghc-8.0.1-O 35 0.97712104.17653%79.15707%29.77058%31.01395%23.56552%0.79546
ghc-8.0.1-O2 35 1.10655117.97581%89.64226%34.92297%41.20066%31.30574%0.86452
ghc-8.0.1-Odph 35 0.93795100.00000%75.98359%25.41410%25.41410%19.31055%0.80954
csv

meteor

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%32.14312%
ghc-7.0.1-O97.32510%9.20866%34.31306%
ghc-7.0.1-O2101.92331%7.70290%32.46043%
ghc-7.0.1-Odph97.50357%9.45062%31.96256%
ghc-7.2.194.74451%7.02051%32.70094%
ghc-7.2.1-O102.69604%8.76272%33.37199%
ghc-7.2.1-O299.79400%7.19459%32.47946%
ghc-7.2.1-Odph96.68966%5.78003%30.54581%
ghc-7.4.197.82894%9.07180%31.87233%
ghc-7.4.1-O100.69428%4.55237%32.29952%
ghc-7.4.1-O2103.20681%4.12504%31.78179%
ghc-7.4.1-Odph100.82711%7.73650%31.66758%
ghc-7.6.197.39871%7.49069%33.24767%
ghc-7.6.1-O94.23205%9.28866%31.78347%
ghc-7.6.1-O298.35934%8.28085%29.95130%
ghc-7.6.1-Odph103.59112%5.43168%32.26865%
ghc-7.8.194.62429%11.40569%32.56947%
ghc-7.8.1-O91.63214%6.78040%33.26033%
ghc-7.8.1-O296.12845%4.23796%32.52497%
ghc-7.8.1-Odph92.36370%5.83361%31.85362%
ghc-7.10.188.95471%4.20853%34.11502%
ghc-7.10.1-O87.21171%4.49601%31.80322%
ghc-7.10.1-O292.42291%10.09156%29.33242%
ghc-7.10.1-Odph88.13520%5.00437%31.66909%
ghc-8.0.189.73014%7.15566%30.55810%
ghc-8.0.1-O91.48252%8.23662%33.41117%
ghc-8.0.1-O289.81523%5.54614%35.71417%
ghc-8.0.1-Odph91.09146%6.32728%33.79178%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%28.59016%
ghc-7.0.1-O113.08639%0.00000%35.26870%
ghc-7.0.1-O2113.96930%0.00000%32.09249%
ghc-7.0.1-Odph106.74440%0.00000%32.27713%
ghc-7.2.192.72104%0.00000%28.28032%
ghc-7.2.1-O118.66854%0.00000%29.36954%
ghc-7.2.1-O2101.55591%0.00000%35.05605%
ghc-7.2.1-Odph102.52033%0.00000%30.45471%
ghc-7.4.189.70239%0.00000%26.44000%
ghc-7.4.1-O101.83405%0.00000%32.96814%
ghc-7.4.1-O2106.42293%0.00000%35.68584%
ghc-7.4.1-Odph97.12859%0.00000%28.98000%
ghc-7.6.1101.81054%0.00000%32.96728%
ghc-7.6.1-O107.89994%0.00000%35.23809%
ghc-7.6.1-O2111.63165%0.00000%30.95863%
ghc-7.6.1-Odph112.43082%0.00000%33.37022%
ghc-7.8.197.38781%0.00000%31.79310%
ghc-7.8.1-O100.08510%0.00000%34.97302%
ghc-7.8.1-O2100.49174%0.00000%33.55091%
ghc-7.8.1-Odph98.90913%0.00000%33.44592%
ghc-7.10.191.62455%0.00000%32.44183%
ghc-7.10.1-O89.12701%0.00000%27.54821%
ghc-7.10.1-O298.80098%0.00000%32.26499%
ghc-7.10.1-Odph90.70520%0.00000%28.54105%
ghc-8.0.1101.28594%0.00000%34.25579%
ghc-8.0.1-O101.10284%0.00000%34.14986%
ghc-8.0.1-O297.79672%0.00000%38.08881%
ghc-8.0.1-Odph95.89815%0.00000%33.74967%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.04740112.19943%100.00000%28.59016%32.07800%28.59016%0.81829
ghc-7.0.1-O 35 1.18447126.88228%113.08639%35.26870%44.74973%39.88409%1.10907
ghc-7.0.1-O2 35 1.19371127.87290%113.96930%32.09249%41.03760%36.57559%1.12111
ghc-7.0.1-Odph 35 1.11804119.76661%106.74440%32.27713%38.65723%34.45403%0.91123
ghc-7.2.1 35 0.97116104.03248%92.72104%28.28032%29.42072%26.22181%0.80607
ghc-7.2.1-O 35 1.24293133.14543%118.66854%29.36954%39.10420%34.85241%1.12402
ghc-7.2.1-O2 35 1.06370113.94515%101.55591%35.05605%39.94467%35.60149%0.84292
ghc-7.2.1-Odph 35 1.07380115.02723%102.52033%30.45471%35.03121%31.22227%0.97649
ghc-7.4.1 35 0.93954100.64557%89.70239%26.44000%26.61069%23.71731%0.80290
ghc-7.4.1-O 35 1.06661114.25722%101.83405%32.96814%37.66848%33.57279%0.86275
ghc-7.4.1-O2 35 1.11467119.40592%106.42293%35.68584%42.61101%37.97792%0.84417
ghc-7.4.1-Odph 35 1.01732108.97773%97.12859%28.98000%31.58174%28.14786%0.83745
ghc-7.6.1 35 1.06636114.23085%101.81054%32.96728%37.65881%33.56417%0.83647
ghc-7.6.1-O 35 1.13014121.06312%107.89994%35.23809%42.66034%38.02188%0.88735
ghc-7.6.1-O2 35 1.16923125.25008%111.63165%30.95863%38.77570%34.55963%1.12639
ghc-7.6.1-Odph 35 1.17760126.14674%112.43082%33.37022%42.09545%37.51841%1.12612
ghc-7.8.1 35 1.02004109.26857%97.38781%31.79310%34.73987%30.96260%0.82510
ghc-7.8.1-O 35 1.04829112.29491%100.08510%34.97302%39.27293%35.00279%0.81315
ghc-7.8.1-O2 35 1.05255112.75116%100.49174%33.55091%37.82905%33.71590%0.82475
ghc-7.8.1-Odph 35 1.03597110.97548%98.90913%33.44592%37.11677%33.08107%0.80867
ghc-7.10.1 35 0.95968102.80222%91.62455%32.44183%33.35093%29.72468%0.80356
ghc-7.10.1-O 35 0.93352100.00000%89.12701%27.54821%27.54821%24.55289%0.80306
ghc-7.10.1-O2 35 1.03484110.85414%98.80098%32.26499%35.76707%31.87812%0.81575
ghc-7.10.1-Odph 35 0.95005101.77072%90.70520%28.54105%29.04643%25.88822%0.81519
ghc-8.0.1 35 1.06087113.64225%101.28594%34.25579%38.92904%34.69629%0.83386
ghc-8.0.1-O 35 1.05895113.43681%101.10284%34.14986%38.73851%34.52647%0.81070
ghc-8.0.1-O2 35 1.02432109.72737%97.79672%38.08881%41.79384%37.24960%0.81023
ghc-8.0.1-Odph 35 1.00444107.59718%95.89815%33.74967%36.31370%32.36531%0.81149
csv

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%34.40434%
ghc-7.0.1-O99.00789%0.00000%35.03263%
ghc-7.0.1-O298.88764%0.00000%34.53739%
ghc-7.0.1-Odph102.03404%0.00000%32.75886%
ghc-7.2.197.97895%0.00000%33.61782%
ghc-7.2.1-O101.69100%0.00000%35.23869%
ghc-7.2.1-O2104.90579%0.00000%27.41434%
ghc-7.2.1-Odph94.21343%0.00000%27.71199%
ghc-7.4.1107.78009%0.00000%33.88804%
ghc-7.4.1-O102.94591%0.00000%34.51802%
ghc-7.4.1-O2109.25383%0.00000%33.15484%
ghc-7.4.1-Odph108.23584%0.00000%29.60692%
ghc-7.6.193.65606%0.00000%35.31173%
ghc-7.6.1-O100.79210%0.00000%27.63544%
ghc-7.6.1-O290.50240%0.00000%25.99531%
ghc-7.6.1-Odph103.32360%0.00000%32.54027%
ghc-7.8.194.78010%0.00000%31.00164%
ghc-7.8.1-O95.13247%0.00000%32.40228%
ghc-7.8.1-O292.89381%0.00000%31.58922%
ghc-7.8.1-Odph86.50247%0.00000%27.61678%
ghc-7.10.188.78752%0.00000%33.25821%
ghc-7.10.1-O86.33669%0.00000%32.45710%
ghc-7.10.1-O2100.21102%0.00000%33.32120%
ghc-7.10.1-Odph92.65125%0.00000%34.48809%
ghc-8.0.194.52629%0.00000%29.88742%
ghc-8.0.1-O96.49107%0.00000%34.81148%
ghc-8.0.1-O293.53448%0.00000%37.63121%
ghc-8.0.1-Odph94.44658%0.00000%34.51113%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.09146115.82561%100.00000%34.40434%39.84903%34.40434%0.85137
ghc-7.0.1-O 35 1.08063114.67649%99.00789%35.03263%40.17419%34.68507%0.82683
ghc-7.0.1-O2 35 1.07932114.53722%98.88764%34.53739%39.55817%34.15321%0.83089
ghc-7.0.1-Odph 35 1.11366118.18155%102.03404%32.75886%38.71493%33.42519%0.84577
ghc-7.2.1 35 1.06940113.48471%97.97895%33.61782%38.15108%32.93838%0.84986
ghc-7.2.1-O 35 1.10992117.78422%101.69100%35.23869%41.50562%35.83458%0.86978
ghc-7.2.1-O2 35 1.14501121.50777%104.90579%27.41434%33.31055%28.75923%1.12000
ghc-7.2.1-Odph 35 1.02830109.12328%94.21343%27.71199%30.24023%26.10842%0.89404
ghc-7.4.1 35 1.17638124.83695%107.78009%33.88804%42.30480%36.52457%1.11521
ghc-7.4.1-O 35 1.12361119.23773%102.94591%34.51802%41.15850%35.53489%1.10816
ghc-7.4.1-O2 35 1.19246126.54392%109.25383%33.15484%41.95543%36.22293%1.12034
ghc-7.4.1-Odph 35 1.18135125.36482%108.23584%29.60692%37.11666%32.04530%1.12418
ghc-7.6.1 35 1.02222108.47770%93.65606%35.31173%38.30535%33.07158%0.81290
ghc-7.6.1-O 35 1.10011116.74307%100.79210%27.63544%32.26246%27.85434%1.12385
ghc-7.6.1-O2 35 0.98780104.82495%90.50240%25.99531%27.24957%23.52638%0.85615
ghc-7.6.1-Odph 35 1.12774119.67519%103.32360%32.54027%38.94263%33.62177%0.93358
ghc-7.8.1 35 1.03449109.77963%94.78010%31.00164%34.03349%29.38339%0.83034
ghc-7.8.1-O 35 1.03833110.18776%95.13247%32.40228%35.70335%30.82509%0.81234
ghc-7.8.1-O2 35 1.01390107.59482%92.89381%31.58922%33.98836%29.34443%0.81094
ghc-7.8.1-Odph 35 0.94414100.19202%86.50247%27.61678%27.66981%23.88920%0.80917
ghc-7.10.1 35 0.96908102.83869%88.78752%33.25821%34.20231%29.52914%0.82482
ghc-7.10.1-O 35 0.94233100.00000%86.33669%32.45710%32.45710%28.02239%0.79307
ghc-7.10.1-O2 35 1.09376116.07003%100.21102%33.32120%38.67592%33.39151%0.85338
ghc-7.10.1-Odph 35 1.01125107.31388%92.65125%34.48809%37.01050%31.95364%0.81677
ghc-8.0.1 35 1.03172109.48565%94.52629%29.88742%32.72243%28.25147%0.86283
ghc-8.0.1-O 35 1.05316111.76137%96.49107%34.81148%38.90579%33.58997%0.82281
ghc-8.0.1-O2 35 1.02089108.33689%93.53448%37.63121%40.76848%35.19816%0.81183
ghc-8.0.1-Odph 35 1.03085109.39333%94.44658%34.51113%37.75287%32.59458%0.81398
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%32.72120%
ghc-7.0.1-O94.20384%0.00000%32.49569%
ghc-7.0.1-O2106.03093%0.00000%31.51223%
ghc-7.0.1-Odph106.22750%0.00000%32.57561%
ghc-7.2.1106.18306%0.00000%33.80942%
ghc-7.2.1-O103.41386%0.00000%34.34614%
ghc-7.2.1-O2107.33866%0.00000%32.08673%
ghc-7.2.1-Odph104.42865%0.00000%30.78532%
ghc-7.4.1108.28709%0.00000%31.94221%
ghc-7.4.1-O106.65906%0.00000%33.33628%
ghc-7.4.1-O2102.29025%0.00000%27.97212%
ghc-7.4.1-Odph111.11197%0.00000%31.91495%
ghc-7.6.1109.31169%0.00000%34.31933%
ghc-7.6.1-O94.06182%0.00000%35.15661%
ghc-7.6.1-O2104.27087%0.00000%30.98471%
ghc-7.6.1-Odph102.98440%0.00000%29.58875%
ghc-7.8.1111.14858%0.00000%32.91399%
ghc-7.8.1-O93.49040%0.00000%34.58205%
ghc-7.8.1-O296.24984%0.00000%28.43690%
ghc-7.8.1-Odph96.98239%0.00000%32.29801%
ghc-7.10.194.32042%0.00000%35.93337%
ghc-7.10.1-O93.66338%0.00000%34.97327%
ghc-7.10.1-O2102.59266%0.00000%30.47783%
ghc-7.10.1-Odph92.17855%0.00000%32.06975%
ghc-8.0.184.85136%0.00000%26.15003%
ghc-8.0.1-O92.39662%0.00000%31.08983%
ghc-8.0.1-O289.27372%0.00000%28.91480%
ghc-8.0.1-Odph98.11113%0.00000%31.89338%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.08219117.85315%100.00000%32.72120%38.56297%32.72120%0.88192
ghc-7.0.1-O 35 1.01946111.02219%94.20384%32.49569%36.07743%30.61219%0.81540
ghc-7.0.1-O2 35 1.14745124.96079%106.03093%31.51223%39.37793%33.41271%1.11888
ghc-7.0.1-Odph 35 1.14958125.19245%106.22750%32.57561%40.78220%34.60426%1.11861
ghc-7.2.1 35 1.14910125.14008%106.18306%33.80942%42.30913%35.89987%0.91125
ghc-7.2.1-O 35 1.11913121.87648%103.41386%34.34614%41.85986%35.51866%0.87494
ghc-7.2.1-O2 35 1.16161126.50199%107.33866%32.08673%40.59036%34.44147%1.12034
ghc-7.2.1-Odph 35 1.13011123.07245%104.42865%30.78532%37.88825%32.14869%1.11568
ghc-7.4.1 35 1.17187127.61975%108.28709%31.94221%40.76456%34.58929%1.11573
ghc-7.4.1-O 35 1.15425125.70106%106.65906%33.33628%41.90406%35.55616%1.11126
ghc-7.4.1-O2 35 1.10697120.55228%102.29025%27.97212%33.72103%28.61276%1.11700
ghc-7.4.1-Odph 35 1.20244130.94895%111.11197%31.91495%41.79229%35.46132%1.13222
ghc-7.6.1 35 1.18296128.82727%109.31169%34.31933%44.21266%37.51504%1.10659
ghc-7.6.1-O 35 1.01793110.85482%94.06182%35.15661%38.97279%33.06894%0.82153
ghc-7.6.1-O2 35 1.12841122.88650%104.27087%30.98471%38.07603%32.30803%1.11366
ghc-7.6.1-Odph 35 1.11448121.37036%102.98440%29.58875%35.91197%30.47179%1.11665
ghc-7.8.1 35 1.20284130.99210%111.14858%32.91399%43.11473%36.58344%1.11721
ghc-7.8.1-O 35 1.01174110.18138%93.49040%34.58205%38.10298%32.33090%0.79889
ghc-7.8.1-O2 35 1.04160113.43347%96.24984%28.43690%32.25697%27.37048%0.89119
ghc-7.8.1-Odph 35 1.04953114.29680%96.98239%32.29801%36.91559%31.32338%0.82566
ghc-7.10.1 35 1.02072111.15959%94.32042%35.93337%39.94339%33.89251%0.81952
ghc-7.10.1-O 35 1.01361110.38525%93.66338%34.97327%38.60533%32.75715%0.80117
ghc-7.10.1-O2 35 1.11025120.90868%102.59266%30.47783%36.85034%31.26802%1.10970
ghc-7.10.1-Odph 35 0.99755108.63532%92.17855%32.06975%34.83907%29.56143%0.81427
ghc-8.0.1 35 0.91825100.00000%84.85136%26.15003%26.15003%22.18866%0.80699
ghc-8.0.1-O 35 0.99991108.89232%92.39662%31.08983%33.85443%28.72595%0.80890
ghc-8.0.1-O2 35 0.96611105.21189%89.27372%28.91480%30.42180%25.81331%0.81856
ghc-8.0.1-Odph 35 1.06175115.62706%98.11113%31.89338%36.87737%31.29095%0.82540
csv

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%34.78683%
ghc-7.0.1-O95.66064%0.00000%34.01341%
ghc-7.0.1-O299.80499%0.00000%32.36080%
ghc-7.0.1-Odph88.65674%0.00000%30.16333%
ghc-7.2.191.66288%0.00000%33.02602%
ghc-7.2.1-O96.31386%0.00000%34.64251%
ghc-7.2.1-O298.44610%0.00000%38.13702%
ghc-7.2.1-Odph89.48597%0.00000%31.03299%
ghc-7.4.197.23807%0.00000%33.58278%
ghc-7.4.1-O99.00820%0.00000%29.01605%
ghc-7.4.1-O297.91615%0.00000%32.15296%
ghc-7.4.1-Odph90.15603%0.00000%33.27836%
ghc-7.6.194.71361%0.00000%33.03238%
ghc-7.6.1-O83.77581%0.00000%31.54892%
ghc-7.6.1-O290.98105%0.00000%28.22744%
ghc-7.6.1-Odph95.29495%0.00000%33.73515%
ghc-7.8.194.40796%0.00000%37.42419%
ghc-7.8.1-O89.59112%0.00000%28.76062%
ghc-7.8.1-O2100.95799%0.00000%34.88081%
ghc-7.8.1-Odph95.07721%0.00000%33.81546%
ghc-7.10.188.31115%0.00000%36.75290%
ghc-7.10.1-O79.79504%0.00000%26.73900%
ghc-7.10.1-O278.21524%0.00000%20.44056%
ghc-7.10.1-Odph85.72788%0.00000%30.92709%
ghc-8.0.181.77333%0.00000%30.55169%
ghc-8.0.1-O90.78017%0.00000%34.08021%
ghc-8.0.1-O286.85326%0.00000%34.35588%
ghc-8.0.1-Odph84.30577%0.00000%34.72995%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.15004127.85231%100.00000%34.78683%44.47577%34.78683%0.88151
ghc-7.0.1-O 35 1.10013122.30434%95.66064%34.01341%41.59988%32.53744%0.87586
ghc-7.0.1-O2 35 1.14780127.60299%99.80499%32.36080%41.29335%32.29769%1.13588
ghc-7.0.1-Odph 35 1.01959113.34970%88.65674%30.16333%34.19004%26.74183%0.86448
ghc-7.2.1 35 1.05416117.19312%91.66288%33.02602%38.70422%30.27260%0.86848
ghc-7.2.1-O 35 1.10765123.13950%96.31386%34.64251%42.65861%33.36554%0.97675
ghc-7.2.1-O2 35 1.13217125.86562%98.44610%38.13702%48.00139%37.54441%0.88035
ghc-7.2.1-Odph 35 1.02912114.40988%89.48597%31.03299%35.50481%27.77017%0.83872
ghc-7.4.1 35 1.11828124.32112%97.23807%33.58278%41.75048%32.65524%1.08939
ghc-7.4.1-O 35 1.13863126.58427%99.00820%29.01605%36.72976%28.72827%1.09497
ghc-7.4.1-O2 35 1.12607125.18806%97.91615%32.15296%40.25166%31.48294%1.12220
ghc-7.4.1-Odph 35 1.03683115.26657%90.15603%33.27836%38.35882%30.00245%0.81511
ghc-7.6.1 35 1.08924121.09355%94.71361%33.03238%40.00008%31.28616%0.86831
ghc-7.6.1-O 35 0.96345107.10932%83.77581%31.54892%33.79183%26.43037%0.79703
ghc-7.6.1-O2 35 1.04632116.32138%90.98105%28.22744%32.83455%25.68162%0.90436
ghc-7.6.1-Odph 35 1.09593121.83680%95.29495%33.73515%41.10183%32.14790%0.87504
ghc-7.8.1 35 1.08573120.70276%94.40796%37.42419%45.17203%35.33141%0.80035
ghc-7.8.1-O 35 1.03033114.54433%89.59112%28.76062%32.94366%25.76696%1.03170
ghc-7.8.1-O2 35 1.16106129.07712%100.95799%34.88081%45.02314%35.21496%1.10388
ghc-7.8.1-Odph 35 1.09343121.55841%95.07721%33.81546%41.10554%32.15080%0.88246
ghc-7.10.1 35 1.01561112.90785%88.31115%36.75290%41.49691%32.45691%0.80372
ghc-7.10.1-O 35 0.91767102.01981%79.79504%26.73900%27.27908%21.33640%0.79001
ghc-7.10.1-O2 35 0.89951100.00000%78.21524%20.44056%20.44056%15.98763%0.81183
ghc-7.10.1-Odph 35 0.98590109.60508%85.72788%30.92709%33.89766%26.51314%0.80991
ghc-8.0.1 35 0.94043104.54910%81.77333%30.55169%31.94152%24.98314%0.79364
ghc-8.0.1-O 35 1.04401116.06455%90.78017%34.08021%39.55505%30.93808%0.85300
ghc-8.0.1-O2 35 0.99885111.04390%86.85326%34.35588%38.15010%29.83920%0.80791
ghc-8.0.1-Odph 35 0.96955107.78688%84.30577%34.72995%37.43433%29.27935%0.79692
csv

Program: '5'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%30.21309%
ghc-7.0.1-O84.66674%0.00000%34.75489%
ghc-7.0.1-O290.92367%0.00000%31.79924%
ghc-7.0.1-Odph83.85515%0.00000%32.03786%
ghc-7.2.185.17660%0.00000%34.77111%
ghc-7.2.1-O93.39292%0.00000%33.26308%
ghc-7.2.1-O286.72352%0.00000%29.70316%
ghc-7.2.1-Odph92.79991%0.00000%32.74404%
ghc-7.4.186.13707%0.00000%33.50862%
ghc-7.4.1-O93.02420%0.00000%31.65913%
ghc-7.4.1-O2100.15086%0.00000%29.94316%
ghc-7.4.1-Odph97.50311%0.00000%34.55766%
ghc-7.6.187.50166%0.00000%30.60760%
ghc-7.6.1-O84.63056%0.00000%29.33830%
ghc-7.6.1-O294.41073%0.00000%33.59040%
ghc-7.6.1-Odph103.92182%0.00000%32.10887%
ghc-7.8.175.39701%0.00000%29.71445%
ghc-7.8.1-O79.86161%0.00000%35.58368%
ghc-7.8.1-O290.04888%0.00000%34.16703%
ghc-7.8.1-Odph84.34728%0.00000%32.09193%
ghc-7.10.181.72991%0.00000%32.18877%
ghc-7.10.1-O87.13644%0.00000%37.29851%
ghc-7.10.1-O282.29463%0.00000%30.15751%
ghc-7.10.1-Odph79.41309%0.00000%32.31947%
ghc-8.0.186.21380%0.00000%31.94557%
ghc-8.0.1-O76.64188%0.00000%32.92448%
ghc-8.0.1-O281.61799%0.00000%39.58016%
ghc-8.0.1-Odph82.69565%0.00000%34.07476%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.27062132.63126%100.00000%30.21309%40.07201%30.21309%1.14428
ghc-7.0.1-O 35 1.07579112.29457%84.66674%34.75489%39.02785%29.42583%0.80818
ghc-7.0.1-O2 35 1.15529120.59321%90.92367%31.79924%38.34772%28.91303%1.11730
ghc-7.0.1-Odph 35 1.06548111.21815%83.85515%32.03786%35.63191%26.86539%0.88107
ghc-7.2.1 35 1.08227112.97080%85.17660%34.77111%39.28120%29.61685%0.82956
ghc-7.2.1-O 35 1.18667123.86821%93.39292%33.26308%41.20238%31.06536%1.10689
ghc-7.2.1-O2 35 1.10192115.02250%86.72352%29.70316%34.16532%25.75963%1.10307
ghc-7.2.1-Odph 35 1.17913123.08170%92.79991%32.74404%40.30191%30.38644%1.12946
ghc-7.4.1 35 1.09447114.24469%86.13707%33.50862%38.28181%28.86334%0.84766
ghc-7.4.1-O 35 1.18198123.37917%93.02420%31.65913%39.06077%29.45065%1.12758
ghc-7.4.1-O2 35 1.27253132.83136%100.15086%29.94316%39.77391%29.98833%1.15663
ghc-7.4.1-Odph 35 1.23889129.31960%97.50311%34.55766%44.68983%33.69479%1.12756
ghc-7.6.1 35 1.11181116.05455%87.50166%30.60760%35.52152%26.78216%1.12092
ghc-7.6.1-O 35 1.07533112.24657%84.63056%29.33830%32.93124%24.82917%0.91357
ghc-7.6.1-O2 35 1.19960125.21814%94.41073%33.59040%42.06127%31.71294%1.13075
ghc-7.6.1-Odph 35 1.32045137.83282%103.92182%32.10887%44.25656%33.36812%1.15169
ghc-7.8.1 35 0.95801100.00000%75.39701%29.71445%29.71445%22.40380%0.81546
ghc-7.8.1-O 35 1.01474105.92147%79.86161%35.58368%37.69076%28.41770%0.81186
ghc-7.8.1-O2 35 1.14418119.43297%90.04888%34.16703%40.80669%30.76702%0.90859
ghc-7.8.1-Odph 35 1.07173111.87086%84.34728%32.09193%35.90152%27.06867%0.83645
ghc-7.10.1 35 1.03847108.39942%81.72991%32.18877%34.89244%26.30786%0.82843
ghc-7.10.1-O 35 1.10717115.57016%87.13644%37.29851%43.10595%32.50059%0.85641
ghc-7.10.1-O2 35 1.04565109.14840%82.29463%30.15751%32.91644%24.81801%0.83261
ghc-7.10.1-Odph 35 1.00904105.32659%79.41309%32.31947%34.04099%25.66589%0.82219
ghc-8.0.1 35 1.09545114.34645%86.21380%31.94557%36.52863%27.54149%1.09928
ghc-8.0.1-O 35 0.97383101.65110%76.64188%32.92448%33.46809%25.23394%0.80350
ghc-8.0.1-O2 35 1.03705108.25097%81.61799%39.58016%42.84590%32.30453%0.81846
ghc-8.0.1-Odph 35 1.05075109.68028%82.69565%34.07476%37.37329%28.17834%0.83293
csv

nbody

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%32.35604%
ghc-7.0.1-O91.68312%0.00000%29.20049%
ghc-7.0.1-O2105.40036%0.00000%34.27704%
ghc-7.0.1-Odph101.58350%0.00000%31.40450%
ghc-7.2.194.12598%0.00000%33.06852%
ghc-7.2.1-O85.51331%0.00000%30.55286%
ghc-7.2.1-O296.47827%0.00000%33.56285%
ghc-7.2.1-Odph94.74920%0.00000%31.96635%
ghc-7.4.195.15024%0.00000%30.89414%
ghc-7.4.1-O96.69694%0.00000%31.75353%
ghc-7.4.1-O294.36869%0.00000%31.45682%
ghc-7.4.1-Odph94.31131%0.00000%27.00146%
ghc-7.6.185.09949%0.00000%23.32735%
ghc-7.6.1-O97.27000%0.00000%37.57154%
ghc-7.6.1-O292.28945%0.00000%31.67165%
ghc-7.6.1-Odph86.20550%0.00000%29.76676%
ghc-7.8.189.11934%0.00000%32.36162%
ghc-7.8.1-O91.56061%0.00000%29.41792%
ghc-7.8.1-O284.40017%0.00000%30.25581%
ghc-7.8.1-Odph95.22249%0.00000%33.62820%
ghc-7.10.190.48099%0.00000%35.66726%
ghc-7.10.1-O89.27728%0.00000%30.83099%
ghc-7.10.1-O291.10096%0.00000%35.69370%
ghc-7.10.1-Odph89.19386%0.00000%31.37737%
ghc-8.0.182.27209%0.00000%29.50815%
ghc-8.0.1-O79.77533%0.00000%28.69860%
ghc-8.0.1-O282.91612%0.00000%33.49080%
ghc-8.0.1-Odph87.92003%0.00000%32.76407%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.17498125.35204%100.00000%32.35604%40.55895%32.35604%1.11147
ghc-7.0.1-O 35 1.07726114.92666%91.68312%29.20049%33.55915%26.77192%1.10793
ghc-7.0.1-O2 35 1.23844132.12151%105.40036%34.27704%45.28734%36.12812%1.13762
ghc-7.0.1-Odph 35 1.19359127.33700%101.58350%31.40450%39.98954%31.90179%1.12616
ghc-7.2.1 35 1.10596117.98883%94.12598%33.06852%39.01717%31.12607%0.89803
ghc-7.2.1-O 35 1.00477107.19268%85.51331%30.55286%32.75043%26.12676%0.81122
ghc-7.2.1-O2 35 1.13360120.93748%96.47827%33.56285%40.59007%32.38086%0.92874
ghc-7.2.1-Odph 35 1.11329118.77006%94.74920%31.96635%37.96645%30.28786%1.09412
ghc-7.4.1 35 1.11800119.27276%95.15024%30.89414%36.84830%29.39585%1.03519
ghc-7.4.1-O 35 1.13617121.21159%96.69694%31.75353%38.48896%30.70469%1.11326
ghc-7.4.1-O2 35 1.10882118.29308%94.36869%31.45682%37.21124%29.68539%0.90191
ghc-7.4.1-Odph 35 1.10814118.22115%94.31131%27.00146%31.92143%25.46543%1.11581
ghc-7.6.1 35 0.99990106.67395%85.09949%23.32735%24.88421%19.85146%0.87465
ghc-7.6.1-O 35 1.14291121.92993%97.27000%37.57154%45.81095%36.54583%0.80878
ghc-7.6.1-O2 35 1.08438115.68671%92.28945%31.67165%36.63989%29.22959%0.91116
ghc-7.6.1-Odph 35 1.01290108.06036%86.20550%29.76676%32.16606%25.66058%0.88586
ghc-7.8.1 35 1.04714111.71291%89.11934%32.36162%36.15210%28.84046%0.85178
ghc-7.8.1-O 35 1.07582114.77310%91.56061%29.41792%33.76386%26.93523%1.09466
ghc-7.8.1-O2 35 0.99169105.79733%84.40017%30.25581%32.00984%25.53595%0.80667
ghc-7.8.1-Odph 35 1.11885119.36334%95.22249%33.62820%40.13974%32.02161%1.03150
ghc-7.10.1 35 1.06314113.41976%90.48099%35.66726%40.45373%32.27209%0.80397
ghc-7.10.1-O 35 1.04899111.91089%89.27728%30.83099%34.50324%27.52507%0.87034
ghc-7.10.1-O2 35 1.07042114.19691%91.10096%35.69370%40.76110%32.51730%0.83231
ghc-7.10.1-Odph 35 1.04801111.80633%89.19386%31.37737%35.08188%27.98669%0.81570
ghc-8.0.1 35 0.96668103.12975%82.27209%29.50815%30.43168%24.27697%0.81684
ghc-8.0.1-O 35 0.93735100.00000%79.77533%28.69860%28.69860%22.89440%0.79815
ghc-8.0.1-O2 35 0.97425103.93705%82.91612%33.49080%34.80935%27.76927%0.80848
ghc-8.0.1-Odph 35 1.03305110.20955%87.92003%32.76407%36.10914%28.80618%0.81517
csv

pidigits

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%36.66839%
ghc-7.0.1-O125.03306%0.00000%30.77396%
ghc-7.0.1-O2109.39256%0.00000%33.88273%
ghc-7.0.1-Odph106.47269%0.00000%28.03052%
ghc-7.2.1112.21355%0.00000%27.83671%
ghc-7.2.1-O96.99476%0.00000%25.97080%
ghc-7.2.1-O2124.63866%0.00000%33.72735%
ghc-7.2.1-Odph99.62698%0.00000%32.50994%
ghc-7.4.1110.53195%0.00000%33.32445%
ghc-7.4.1-O112.52011%0.00000%32.89627%
ghc-7.4.1-O2119.99932%0.00000%29.04257%
ghc-7.4.1-Odph107.08159%0.00000%29.09468%
ghc-7.6.1117.53009%0.00000%29.02102%
ghc-7.6.1-O108.83674%0.00000%29.11291%
ghc-7.6.1-O2101.24525%0.00000%32.59209%
ghc-7.6.1-Odph116.71352%0.00000%31.42685%
ghc-7.8.1100.90185%0.00000%34.31174%
ghc-7.8.1-O101.38031%0.00000%31.81799%
ghc-7.8.1-O2102.81380%0.00000%31.70725%
ghc-7.8.1-Odph107.97170%0.00000%34.27946%
ghc-7.10.195.46569%0.00000%30.64250%
ghc-7.10.1-O95.43335%0.00000%35.40620%
ghc-7.10.1-O289.88326%0.00000%25.15129%
ghc-7.10.1-Odph94.70517%0.00000%33.93429%
ghc-8.0.193.07368%0.00000%33.00702%
ghc-8.0.1-O102.66829%0.00000%37.15243%
ghc-8.0.1-O291.47934%0.00000%28.94096%
ghc-8.0.1-Odph96.32525%0.00000%36.95452%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.01903111.25542%100.00000%36.66839%40.79557%36.66839%0.80007
ghc-7.0.1-O 35 1.27413139.10606%125.03306%30.77396%42.80844%38.47762%1.16178
ghc-7.0.1-O2 35 1.11475121.70515%109.39256%33.88273%41.23703%37.06519%0.85989
ghc-7.0.1-Odph 35 1.08499118.45663%106.47269%28.03052%33.20401%29.84485%1.10678
ghc-7.2.1 35 1.14349124.84366%112.21355%27.83671%34.75236%31.23656%1.10999
ghc-7.2.1-O 35 0.98841107.91192%96.99476%25.97080%28.02559%25.19031%0.85696
ghc-7.2.1-O2 35 1.27011138.66727%124.63866%33.72735%46.76880%42.03732%1.15651
ghc-7.2.1-Odph 35 1.01523110.84042%99.62698%32.50994%36.03416%32.38868%0.82031
ghc-7.4.1 35 1.12636122.97279%110.53195%33.32445%40.98001%36.83417%1.10839
ghc-7.4.1-O 35 1.14662125.18472%112.52011%32.89627%41.18111%37.01492%1.10959
ghc-7.4.1-O2 35 1.22283133.50574%119.99932%29.04257%38.77350%34.85089%1.15779
ghc-7.4.1-Odph 35 1.09120119.13407%107.08159%29.09468%34.66167%31.15504%1.12031
ghc-7.6.1 35 1.19767130.75860%117.53009%29.02102%37.94747%34.10843%1.11623
ghc-7.6.1-O 35 1.10908121.08677%108.83674%29.11291%35.25188%31.68554%1.09919
ghc-7.6.1-O2 35 1.03172112.64082%101.24525%32.59209%36.71200%32.99794%0.84354
ghc-7.6.1-Odph 35 1.18935129.85011%116.71352%31.42685%40.80780%36.67938%1.10953
ghc-7.8.1 35 1.02822112.25877%100.90185%34.31174%38.51794%34.62118%0.81410
ghc-7.8.1-O 35 1.03310112.79108%101.38031%31.81799%35.88785%32.25717%0.86770
ghc-7.8.1-O2 35 1.04771114.38592%102.81380%31.70725%36.26863%32.59943%0.83080
ghc-7.8.1-Odph 35 1.10027120.12437%107.97170%34.27946%41.17798%37.01211%0.87753
ghc-7.10.1 35 0.97283106.21075%95.46569%30.64250%32.54563%29.25307%0.79690
ghc-7.10.1-O 35 0.97250106.17477%95.43335%35.40620%37.59245%33.78932%0.79706
ghc-7.10.1-O2 35 0.91594100.00000%89.88326%25.15129%25.15129%22.60680%0.80677
ghc-7.10.1-Odph 35 0.96508105.36463%94.70517%33.93429%35.75474%32.13753%0.80837
ghc-8.0.1 35 0.94845103.54951%93.07368%33.00702%34.17861%30.72085%0.78692
ghc-8.0.1-O 35 1.04622114.22404%102.66829%37.15243%42.43700%38.14376%0.80159
ghc-8.0.1-O2 30 0.93221101.77572%91.47934%28.94096%29.45487%26.47500%0.80799
ghc-8.0.1-Odph 35 0.98159107.16706%96.32525%36.95452%39.60308%35.59654%0.81148
csv

spectralnorm

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%31.43236%
ghc-7.0.1-O93.74600%0.00000%29.64629%
ghc-7.0.1-O295.97020%0.00000%34.46030%
ghc-7.0.1-Odph93.83525%0.00000%31.61665%
ghc-7.2.187.59827%0.00000%29.83029%
ghc-7.2.1-O82.85766%0.00000%35.79662%
ghc-7.2.1-O292.72557%0.00000%31.98407%
ghc-7.2.1-Odph90.50443%0.00000%33.50764%
ghc-7.4.198.15368%0.00000%31.05024%
ghc-7.4.1-O77.76157%0.00000%29.33615%
ghc-7.4.1-O284.11851%0.00000%34.54816%
ghc-7.4.1-Odph96.92386%0.00000%29.56956%
ghc-7.6.187.44398%0.00000%34.89870%
ghc-7.6.1-O81.89153%0.00000%28.39155%
ghc-7.6.1-O290.96011%0.00000%30.44734%
ghc-7.6.1-Odph92.56954%0.00000%31.18183%
ghc-7.8.180.47429%0.00000%33.96745%
ghc-7.8.1-O81.95345%0.00000%34.82168%
ghc-7.8.1-O275.64832%0.00000%31.43784%
ghc-7.8.1-Odph77.76566%0.00000%29.34938%
ghc-7.10.168.72123%0.00000%21.49603%
ghc-7.10.1-O74.10509%0.00000%30.59883%
ghc-7.10.1-O274.40859%0.00000%28.47762%
ghc-7.10.1-Odph79.84974%0.00000%29.08371%
ghc-8.0.175.95768%0.00000%30.01915%
ghc-8.0.1-O75.55379%0.00000%30.44788%
ghc-8.0.1-O272.23124%0.00000%28.79058%
ghc-8.0.1-Odph90.55752%0.00000%35.23490%
csv

Input: ''

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 35 1.28167145.51545%100.00000%31.43236%45.73894%31.43236%1.56714
ghc-7.0.1-O 35 1.20152136.41491%93.74600%29.64629%40.44196%27.79221%1.11468
ghc-7.0.1-O2 35 1.23002139.65147%95.97020%34.46030%48.12431%33.07162%1.13087
ghc-7.0.1-Odph 35 1.20266136.54479%93.83525%31.61665%43.17089%29.66757%1.11464
ghc-7.2.1 35 1.12272127.46902%87.59827%29.83029%38.02437%26.13082%1.10469
ghc-7.2.1-O 35 1.06196120.57070%82.85766%35.79662%43.16023%29.66024%0.87537
ghc-7.2.1-O2 35 1.18844134.93002%92.72557%31.98407%43.15611%29.65741%1.13787
ghc-7.2.1-Odph 35 1.15997131.69792%90.50443%33.50764%44.12887%30.32590%1.09408
ghc-7.4.1 35 1.25801142.82877%98.15368%31.05024%44.34867%30.47695%1.12084
ghc-7.4.1-O 35 0.99665113.15510%77.76157%29.33615%33.19535%22.81225%0.87104
ghc-7.4.1-O2 35 1.07812122.40543%84.11851%34.54816%42.28882%29.06139%0.84023
ghc-7.4.1-Odph 35 1.24224141.03919%96.92386%29.56956%41.70467%28.65996%1.14702
ghc-7.6.1 35 1.12074127.24450%87.44398%34.89870%44.40667%30.51681%0.85395
ghc-7.6.1-O 35 1.04958119.16483%81.89153%28.39155%33.83274%23.25027%1.09848
ghc-7.6.1-O2 35 1.16581132.36101%90.96011%30.44734%40.30041%27.69494%1.11961
ghc-7.6.1-Odph 35 1.18644134.70298%92.56954%31.18183%42.00286%28.86488%1.13330
ghc-7.8.1 35 1.03142117.10252%80.47429%33.96745%39.77675%27.33507%0.83783
ghc-7.8.1-O 35 1.05037119.25493%81.95345%34.82168%41.52657%28.53757%0.80990
ghc-7.8.1-O2 30 0.96956110.08000%75.64832%31.43784%34.60677%23.78220%0.80863
ghc-7.8.1-Odph 35 0.99670113.16105%77.76566%29.34938%33.21206%22.82374%0.81317
ghc-7.10.1 30 0.88078100.00000%68.72123%21.49603%21.49603%14.77233%0.79562
ghc-7.10.1-O 35 0.94978107.83435%74.10509%30.59883%32.99605%22.67529%0.80982
ghc-7.10.1-O2 30 0.95367108.27600%74.40859%28.47762%30.83443%21.18980%0.82012
ghc-7.10.1-Odph 30 1.02341116.19371%79.84974%29.08371%33.79344%23.22326%0.82376
ghc-8.0.1 30 0.97353110.53016%75.95768%30.01915%33.18021%22.80185%0.80680
ghc-8.0.1-O 35 0.96835109.94243%75.55379%30.44788%33.47514%23.00453%0.80552
ghc-8.0.1-O2 30 0.92577105.10761%72.23124%28.79058%30.26109%20.79579%0.81189
ghc-8.0.1-Odph 35 1.16065131.77519%90.55752%35.23490%46.43086%31.90785%1.10364
csv