img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.30829%
ghc-7.2.1102.30194%7.59007%2.62504%
ghc-7.4.1103.90692%14.32291%2.66702%
ghc-7.6.1102.20934%14.33668%2.41780%
ghc-7.8.192.93963%11.26792%2.28907%
ghc-7.10.191.47456%11.50062%2.12898%
ghc-8.0.191.90408%11.51504%2.39622%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.75090%
ghc-7.2.1101.41727%7.14227%4.28452%
ghc-7.4.1101.22068%12.27495%4.35134%
ghc-7.6.1104.27758%15.96051%4.31467%
ghc-7.8.192.15075%10.00086%4.12044%
ghc-7.10.190.10979%10.81079%3.88648%
ghc-8.0.189.72039%10.57580%3.78657%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.89026%
ghc-7.2.1103.02047%7.84923%1.96688%
ghc-7.4.1105.92648%14.77044%1.98401%
ghc-7.6.1100.19222%12.58838%1.34037%
ghc-7.8.192.93829%11.95924%1.27562%
ghc-7.10.191.72301%11.56600%1.06050%
ghc-8.0.193.04599%11.45180%2.02556%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.38207%
ghc-7.2.1103.02099%7.87548%0.58657%
ghc-7.4.1106.25250%16.10389%0.61300%
ghc-7.6.1100.86558%12.64286%0.41284%
ghc-7.8.194.22289%12.33054%0.32655%
ghc-7.10.193.44387%12.19835%0.34152%
ghc-8.0.194.31066%12.36611%0.50755%
csv

Seperated by entropy

The following shows the summary including only the lower or the upper half of programs (per category), regarding the entropy of their files. This entropy is measured by taking the length of the gnu zipped program code length. Programs with lower entropy should be simpler than programs with higher entropy. If the number of programs is uneven in a category, then one program belongs to the upper and the lower half.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.44531%
ghc-7.2.1102.46160%8.27440%2.71165%
ghc-7.4.1104.65885%15.62086%2.88342%
ghc-7.6.1100.83362%14.88492%2.50761%
ghc-7.8.190.50337%10.24628%2.50343%
ghc-7.10.189.17817%10.91333%2.35977%
ghc-8.0.189.30424%10.15526%2.65905%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%4.23939%
ghc-7.2.1101.57124%8.00739%4.80137%
ghc-7.4.1101.79286%13.91987%5.07960%
ghc-7.6.1102.75935%16.83456%4.87876%
ghc-7.8.189.85898%9.56213%4.84844%
ghc-7.10.187.67607%10.76107%4.59935%
ghc-8.0.186.78673%9.12148%4.42258%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.92577%
ghc-7.2.1103.09074%8.38882%1.89158%
ghc-7.4.1106.53951%15.69478%2.01783%
ghc-7.6.199.07821%13.08347%1.22475%
ghc-7.8.190.36884%10.51848%1.31154%
ghc-7.10.189.38837%10.45362%1.16574%
ghc-8.0.190.49705%9.89453%2.22973%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.40189%
ghc-7.2.1103.10441%8.41594%0.54641%
ghc-7.4.1106.87247%17.12624%0.61161%
ghc-7.6.199.83800%13.19959%0.40310%
ghc-7.8.191.55847%10.81525%0.34530%
ghc-7.10.191.11381%11.25271%0.35439%
ghc-8.0.191.70788%10.98049%0.56904%
csv

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.65799%
ghc-7.2.1100.21698%3.73926%3.01945%
ghc-7.4.1101.36690%11.16299%3.09508%
ghc-7.6.199.30636%11.72568%2.78821%
ghc-7.8.193.10351%12.01549%2.62534%
ghc-7.10.193.32015%11.79661%2.44468%
ghc-8.0.194.05438%11.50555%2.75416%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%4.58185%
ghc-7.2.1100.16070%4.49290%5.23248%
ghc-7.4.1100.05107%8.62075%5.37625%
ghc-7.6.1102.40505%13.81712%5.31525%
ghc-7.8.193.04347%10.57847%5.04717%
ghc-7.10.192.71726%11.01813%4.76337%
ghc-8.0.192.60172%10.42259%4.60154%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.14085%
ghc-7.2.1100.23574%2.89376%2.21703%
ghc-7.4.1102.12446%11.56244%2.24428%
ghc-7.6.196.69247%9.11697%1.52079%
ghc-7.8.192.36856%12.68300%1.43425%
ghc-7.10.192.81105%11.97548%1.19699%
ghc-8.0.194.44067%11.58955%2.30036%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.42674%
ghc-7.2.1100.27862%3.27396%0.66040%
ghc-7.4.1102.48911%13.53063%0.68705%
ghc-7.6.197.49357%9.57668%0.44557%
ghc-7.8.193.92425%13.15481%0.35668%
ghc-7.10.194.69051%12.55484%0.37994%
ghc-8.0.195.74333%12.58357%0.56885%
csv

binarytrees

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.91281%
ghc-7.2.1101.49141%0.98466%4.06499%
ghc-7.4.198.32662%3.38980%5.32279%
ghc-7.6.1100.70914%2.47235%4.33060%
ghc-7.8.1102.90049%1.85781%5.47113%
ghc-7.10.1105.49610%3.40602%4.60072%
ghc-8.0.1103.93455%2.32770%4.91232%
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 38.35232106.90502%100.00000%9.86243%10.54343%9.86243%37.63287
ghc-7.2.1 20 38.39031107.01090%100.09904%10.70472%11.45522%10.71533%36.75450
ghc-7.4.1 20 35.87514100.00000%93.54098%13.34123%13.34123%12.47952%34.24283
ghc-7.6.1 20 37.30969103.99873%97.28142%11.51040%11.97067%11.19748%35.62532
ghc-7.8.1 20 40.07256111.70008%104.48535%13.85139%15.47201%14.47267%38.47585
ghc-7.10.1 20 41.76646116.42173%108.90202%11.66053%13.57539%12.69855%41.43228
ghc-8.0.1 20 39.02290108.77421%101.74846%12.31435%13.39483%12.52966%39.63483
csv

Input: '16'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201210.12421100.00000%100.00000%1.41043%1.41043%1.41043%1210.99402
ghc-7.2.1 201236.81074102.20527%102.20527%1.00798%1.03020%1.03020%1235.23638
ghc-7.4.1 201215.88141100.47575%100.47575%1.84523%1.85401%1.85401%1215.90072
ghc-7.6.1 201232.21727101.82569%101.82569%1.25198%1.27484%1.27484%1234.12319
ghc-7.8.1 201213.67346100.29330%100.29330%1.99850%2.00436%2.00436%1213.17293
ghc-7.10.1 201220.32985100.84335%100.84335%1.69499%1.70929%1.70929%1224.70957
ghc-8.0.1 201245.17123102.89615%102.89615%1.60939%1.65600%1.65600%1244.92030
csv

Input: '20'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2034311.58701100.00000%100.00000%0.46557%0.46557%0.46557%34262.24397
ghc-7.2.1 2035056.12200102.16992%102.16992%0.48228%0.49274%0.49274%35022.52316
ghc-7.4.1 2034642.04753100.96312%100.96312%0.78191%0.78944%0.78944%34583.01815
ghc-7.6.1 2035347.90708103.02032%103.02032%0.22941%0.23634%0.23634%35342.40829
ghc-7.8.1 2035657.57070103.92283%103.92283%0.56351%0.58561%0.58561%35595.31812
ghc-7.10.1 2036625.19177106.74293%106.74293%0.44664%0.47676%0.47676%36605.66050
ghc-8.0.1 2036767.96484107.15903%107.15903%0.81324%0.87146%0.87146%36823.40996
csv

fannkuchredux

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.03203%
ghc-7.2.1112.51041%9.66064%1.42784%
ghc-7.4.1116.89544%15.39020%1.08238%
ghc-7.6.1116.50017%8.08403%1.37021%
ghc-7.8.1104.04599%7.27537%0.73484%
ghc-7.10.195.97983%11.85513%0.47796%
ghc-8.0.197.56363%14.28223%0.60405%
csv

Mean scores per input

Mean scores for input '10'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.05077%
ghc-7.2.1112.68279%9.06934%2.45990%
ghc-7.4.1116.34393%15.36066%1.93378%
ghc-7.6.1116.89595%7.66564%2.66474%
ghc-7.8.1103.61886%6.38343%1.45438%
ghc-7.10.194.61754%10.86139%1.05747%
ghc-8.0.197.19897%14.86177%1.38518%
csv

Mean scores for input '11'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.88893%
ghc-7.2.1112.52111%9.99251%1.35490%
ghc-7.4.1117.08793%15.45265%0.95471%
ghc-7.6.1116.34039%8.35011%1.11354%
ghc-7.8.1103.92548%6.99902%0.59468%
ghc-7.10.196.08612%11.97941%0.21441%
ghc-8.0.197.08592%13.80266%0.34918%
csv

Mean scores for input '12'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.15638%
ghc-7.2.1112.32733%9.89027%0.46873%
ghc-7.4.1117.25447%15.34179%0.35867%
ghc-7.6.1116.26416%8.20550%0.33234%
ghc-7.8.1104.59363%8.28021%0.15547%
ghc-7.10.197.23583%12.51845%0.16201%
ghc-8.0.198.40600%14.12408%0.07778%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.24867%
ghc-7.2.1122.16111%0.31343%0.21718%
ghc-7.4.1132.28048%0.40785%0.20455%
ghc-7.6.1124.57392%0.09058%0.25420%
ghc-7.8.196.82510%0.38317%0.29900%
ghc-7.10.184.19341%0.39718%0.22473%
ghc-8.0.183.30080%0.79403%0.24603%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202503.48546121.45178%100.00000%0.54045%0.65638%0.54045%2501.15517
ghc-7.2.1 203048.04678147.87013%121.75213%0.36594%0.54112%0.44554%3050.36551
ghc-7.4.1 203297.20528159.95757%131.70459%0.35686%0.57083%0.47000%3295.55016
ghc-7.6.1 203118.38125151.28227%124.56159%0.50150%0.75869%0.62468%3116.16849
ghc-7.8.1 202434.27482118.09416%97.23543%0.61619%0.72768%0.59915%2437.20936
ghc-7.10.1 202096.82289101.72333%83.75614%0.49655%0.50511%0.41589%2095.63720
ghc-8.0.1 202061.29991100.00000%82.33720%0.55767%0.55767%0.45917%2059.58895
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 1530706.11241120.07214%100.00000%0.13613%0.16346%0.13613%30706.25548
ghc-7.2.1 1537619.17004147.10473%122.51362%0.21582%0.31747%0.26440%37609.17131
ghc-7.4.1 1540698.05928159.14431%132.54058%0.16214%0.25803%0.21490%40701.51532
ghc-7.6.1 1538287.60742149.71856%124.69051%0.07741%0.11590%0.09653%38285.10192
ghc-7.8.1 1529762.34885116.38168%96.92646%0.16519%0.19225%0.16011%29762.19396
ghc-7.10.1 1525825.90066100.98873%84.10671%0.10505%0.10609%0.08836%25815.44647
ghc-8.0.1 1525573.05292100.00000%83.28326%0.10198%0.10198%0.08494%25565.27857
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 15412152.43840118.64940%100.00000%0.06942%0.08237%0.06942%412163.40982
ghc-7.2.1 15503722.80348145.01045%122.21760%0.06978%0.10118%0.08528%503760.91091
ghc-7.4.1 15546498.71142157.32467%132.59626%0.09465%0.14891%0.12550%546681.87645
ghc-7.6.1 15513004.76650147.68252%124.46967%0.18370%0.27129%0.22865%512838.21907
ghc-7.8.1 15396958.11528114.27530%96.31342%0.11563%0.13214%0.11137%396804.24874
ghc-7.10.1 15349164.75066100.51667%84.71738%0.07258%0.07296%0.06149%349111.22867
ghc-8.0.1 15347370.00173100.00000%84.28192%0.07845%0.07845%0.06612%347399.15443
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.81539%
ghc-7.2.1102.85970%0.53429%2.63850%
ghc-7.4.1101.51041%0.38956%1.96022%
ghc-7.6.1108.42641%0.56912%2.48621%
ghc-7.8.1111.26688%1.19704%1.17069%
ghc-7.10.1107.76624%1.75818%0.73120%
ghc-8.0.1111.82647%0.69031%0.96206%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 321.58670100.00000%100.00000%3.56109%3.56109%3.56109%319.53677
ghc-7.2.1 20 333.20707103.61345%103.61345%4.55385%4.71840%4.71840%336.62854
ghc-7.4.1 20 324.74879100.98328%100.98328%3.51070%3.54522%3.54522%323.59020
ghc-7.6.1 20 351.27013109.23030%109.23030%4.82798%5.27361%5.27361%345.39428
ghc-7.8.1 20 353.75273110.00229%110.00229%2.29258%2.52189%2.52189%351.11784
ghc-7.10.1 20 339.20621105.47893%105.47893%1.61839%1.70706%1.70706%337.95770
ghc-8.0.1 20 360.37244112.06074%112.06074%2.21270%2.47956%2.47956%360.51566
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 203793.57391100.00000%100.00000%1.64173%1.64173%1.64173%3791.08808
ghc-7.2.1 203889.49820102.52860%102.52860%2.49398%2.55704%2.55704%3864.02972
ghc-7.4.1 203855.60924101.63527%101.63527%1.74728%1.77585%1.77585%3821.35717
ghc-7.6.1 204096.69101107.99028%107.99028%2.14966%2.32143%2.32143%4068.30002
ghc-7.8.1 204208.00304110.92450%110.92450%1.02417%1.13606%1.13606%4209.26865
ghc-7.10.1 204099.54584108.06553%108.06553%0.32377%0.34989%0.34989%4096.71620
ghc-8.0.1 204206.64025110.88858%110.88858%0.59637%0.66131%0.66131%4199.07843
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2050479.87221100.00000%100.00000%0.24334%0.24334%0.24334%50448.76199
ghc-7.2.1 2051710.09792102.43706%102.43706%0.86768%0.88883%0.88883%51514.31738
ghc-7.4.1 2051445.39025101.91268%101.91268%0.62268%0.63459%0.63459%51359.65878
ghc-7.6.1 2054547.87184108.05866%108.05866%0.48098%0.51974%0.51974%54465.82117
ghc-7.8.1 2056978.56685112.87383%112.87383%0.19530%0.22045%0.22045%56991.26216
ghc-7.10.1 2055403.81611109.75427%109.75427%0.25144%0.27596%0.27596%55379.15549
ghc-8.0.1 2056805.04342112.53009%112.53009%0.07712%0.08678%0.08678%56806.86937
csv

fasta

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.95626%
ghc-7.2.193.37099%0.19465%2.71268%
ghc-7.4.193.43008%1.10228%2.26506%
ghc-7.6.194.38767%1.39835%3.05816%
ghc-7.8.189.01118%1.00314%2.61800%
ghc-7.10.189.26454%0.16403%1.92268%
ghc-8.0.191.34790%0.11464%2.11360%
csv

Input: '250000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 110.18863112.30561%100.00000%7.47960%8.40002%7.47960%107.25109
ghc-7.2.1 20 102.59331104.56436%93.10698%6.41522%6.70803%5.97302%98.42570
ghc-7.4.1 20 103.76467105.75823%94.17003%5.77550%6.10807%5.43879%99.96251
ghc-7.6.1 20 106.16939108.20915%96.35240%8.46724%9.16233%8.15839%104.30498
ghc-7.8.1 20 99.61858101.53248%90.40731%6.06827%6.16126%5.48616%100.32330
ghc-7.10.1 20 98.11498100.00000%89.04275%4.53635%4.53635%4.03929%96.70019
ghc-8.0.1 20 100.47685102.40724%91.18622%5.15477%5.27886%4.70044%100.04431
csv

Input: '2500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201019.96911112.95453%100.00000%1.09440%1.23617%1.09440%1017.01817
ghc-7.2.1 20 954.39027105.69213%93.57051%1.51464%1.60086%1.41726%950.51772
ghc-7.4.1 20 961.30399106.45777%94.24834%0.83057%0.88420%0.78280%959.97220
ghc-7.6.1 20 950.71838105.28549%93.21051%0.51296%0.54007%0.47813%949.96928
ghc-7.8.1 20 902.99088100.00000%88.53120%1.64703%1.64703%1.45814%898.51654
ghc-7.10.1 20 911.00156100.88713%89.31658%1.07320%1.08272%0.95854%908.48066
ghc-8.0.1 20 932.65157103.28472%91.43920%0.99971%1.03255%0.91413%933.30467
csv

Input: '25000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2010058.14882113.51377%100.00000%0.29477%0.33461%0.29477%10061.89841
ghc-7.2.1 209397.87916106.06213%93.43548%0.20817%0.22079%0.19451%9397.40480
ghc-7.4.1 209240.61020104.28723%91.87188%0.18911%0.19721%0.17374%9236.88142
ghc-7.6.1 209414.43857106.24902%93.60011%0.19429%0.20643%0.18185%9415.93525
ghc-7.8.1 208860.73001100.00000%88.09504%0.13869%0.13869%0.12218%8860.59964
ghc-7.10.1 208995.43468101.52024%89.43430%0.15849%0.16090%0.14174%8991.31078
ghc-8.0.1 209194.98762103.77235%91.41829%0.18633%0.19336%0.17034%9189.88698
csv

mandelbrot

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.13758%
ghc-7.2.1103.94810%4.38210%5.37977%
ghc-7.4.1100.11180%1.76739%5.70749%
ghc-7.6.1100.29097%1.30693%2.14085%
ghc-7.8.196.95556%1.23381%2.05561%
ghc-7.10.197.33622%2.06216%2.61690%
ghc-8.0.197.43275%0.65183%2.11098%
csv

Input: '1000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 167.74479102.51789%100.00000%4.22982%4.33633%4.22982%164.87336
ghc-7.2.1 20 184.72573112.89586%110.12308%11.30530%12.76322%12.44975%174.65467
ghc-7.4.1 20 171.93583105.07926%102.49846%12.99776%13.65794%13.32250%163.59584
ghc-7.6.1 20 170.57432104.24716%101.68681%5.19471%5.41533%5.28233%167.73297
ghc-7.8.1 20 164.32584100.42838%97.96182%5.12799%5.14995%5.02347%161.91270
ghc-7.10.1 20 167.35323102.27859%99.76658%7.03733%7.19769%7.02091%163.12538
ghc-8.0.1 20 163.62490100.00000%97.54395%3.12818%3.12818%3.05135%163.90855
csv

Input: '4000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202592.41145105.56869%100.00000%4.47867%4.72807%4.47867%2539.18195
ghc-7.2.1 202626.50185106.95692%101.31501%4.19026%4.48178%4.24537%2571.86304
ghc-7.4.1 202547.70524103.74815%98.27550%3.48029%3.61074%3.42027%2513.26784
ghc-7.6.1 202554.66172104.03143%98.54384%0.78030%0.81176%0.76894%2554.96271
ghc-7.8.1 202468.44033100.52031%95.21792%0.68288%0.68644%0.65023%2468.92198
ghc-7.10.1 202455.66328100.00000%94.72506%0.44440%0.44440%0.42096%2459.36265
ghc-8.0.1 202503.87165101.96315%96.58465%2.89329%2.95009%2.79448%2486.35923
csv

Input: '16000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2040110.62745102.54621%100.00000%0.70424%0.72218%0.70424%40053.32028
ghc-7.2.1 2040273.56469102.96277%100.40622%0.64373%0.66280%0.64635%40320.61008
ghc-7.4.1 2039934.72323102.09650%99.56145%0.64443%0.65794%0.64161%39842.33462
ghc-7.6.1 2040368.24173103.20482%100.64226%0.44755%0.46189%0.45042%40393.71248
ghc-7.8.1 2039182.84981100.17427%97.68695%0.35595%0.35657%0.34772%39134.96325
ghc-7.10.1 2039114.68558100.00000%97.51701%0.36895%0.36895%0.35979%39074.84667
ghc-8.0.1 2039376.45986100.66925%98.16964%0.31146%0.31355%0.30576%39358.26855
csv

meteor

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.36709%
ghc-7.2.198.29047%1.84800%1.47987%
ghc-7.4.196.15394%5.71356%1.17390%
ghc-7.6.1110.99281%18.87196%1.16218%
ghc-7.8.190.32382%5.39231%1.35837%
ghc-7.10.186.64621%7.01461%1.37480%
ghc-8.0.185.46355%6.68365%1.49354%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.30817%
ghc-7.2.1101.77802%0.00000%1.08851%
ghc-7.4.199.65533%0.00000%1.29852%
ghc-7.6.1123.13403%0.00000%0.83240%
ghc-7.8.193.72321%0.00000%1.18933%
ghc-7.10.191.00968%0.00000%1.30089%
ghc-8.0.190.73076%0.00000%1.28063%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 761.51110110.21621%100.00000%1.30817%1.44181%1.30817%759.70832
ghc-7.2.1 20 775.05095112.17588%101.77802%1.08851%1.22105%1.10787%771.83809
ghc-7.4.1 20 758.88642109.83633%99.65533%1.29852%1.42625%1.29405%762.25197
ghc-7.6.1 20 937.67929135.71366%123.13403%0.83240%1.12968%1.02497%938.73615
ghc-7.8.1 20 713.71268103.29817%93.72321%1.18933%1.22856%1.11468%712.71044
ghc-7.10.1 20 693.04884100.30742%91.00968%1.30089%1.30489%1.18394%691.67955
ghc-8.0.1 20 690.92477100.00000%90.73076%1.28063%1.28063%1.16192%692.38374
csv

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.62597%
ghc-7.2.197.28834%0.00000%1.26021%
ghc-7.4.1100.15917%0.00000%1.16078%
ghc-7.6.1129.18292%0.00000%0.89910%
ghc-7.8.191.96176%0.00000%1.52535%
ghc-7.10.183.54486%0.00000%1.56762%
ghc-8.0.182.87020%0.00000%2.02833%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 556.26650120.67064%100.00000%1.62597%1.96207%1.62597%553.77396
ghc-7.2.1 20 541.18246117.39847%97.28834%1.26021%1.47947%1.22604%540.46302
ghc-7.4.1 20 557.15189120.86271%100.15917%1.16078%1.40295%1.16262%554.36827
ghc-7.6.1 20 718.60129155.88585%129.18292%0.89910%1.40157%1.16149%719.00965
ghc-7.8.1 20 511.55247110.97085%91.96176%1.52535%1.69269%1.40274%508.94619
ghc-7.10.1 20 464.73207100.81412%83.54486%1.56762%1.58039%1.30967%463.76079
ghc-8.0.1 20 460.97916100.00000%82.87020%2.02833%2.02833%1.68088%460.24932
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.88735%
ghc-7.2.198.48638%0.00000%2.31612%
ghc-7.4.198.76065%0.00000%1.86536%
ghc-7.6.1125.34710%0.00000%1.55439%
ghc-7.8.190.10430%0.00000%1.23062%
ghc-7.10.179.12171%0.00000%1.55248%
ghc-8.0.178.47738%0.00000%1.66598%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 486.03006127.42525%100.00000%1.88735%2.40497%1.88735%485.90328
ghc-7.2.1 20 478.67343125.49652%98.48638%2.31612%2.90665%2.28106%477.08146
ghc-7.4.1 20 480.00644125.84600%98.76065%1.86536%2.34748%1.84224%479.21236
ghc-7.6.1 20 609.22456159.72385%125.34710%1.55439%2.48273%1.94838%608.63004
ghc-7.8.1 20 437.93399114.81563%90.10430%1.23062%1.41295%1.10885%437.14018
ghc-7.10.1 20 384.55528100.82103%79.12171%1.55248%1.56523%1.22835%385.27346
ghc-8.0.1 20 381.42368100.00000%78.47738%1.66598%1.66598%1.30742%383.36252
csv

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.51542%
ghc-7.2.196.57899%0.00000%0.69191%
ghc-7.4.184.89293%0.00000%0.58267%
ghc-7.6.181.64935%0.00000%0.88229%
ghc-7.8.180.18534%0.00000%1.24719%
ghc-7.10.181.37686%0.00000%0.84149%
ghc-8.0.179.53321%0.00000%0.98643%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201956.33802125.73363%100.00000%0.51542%0.64806%0.51542%1956.25254
ghc-7.2.1 201889.41159121.43228%96.57899%0.69191%0.84020%0.66823%1886.21970
ghc-7.4.1 201660.79263106.73896%84.89293%0.58267%0.62194%0.49465%1661.67235
ghc-7.6.1 201597.33720102.66069%81.64935%0.88229%0.90576%0.72038%1594.51293
ghc-7.8.1 201568.69627100.81994%80.18534%1.24719%1.25741%1.00006%1566.36844
ghc-7.10.1 201592.00639102.31808%81.37686%0.84149%0.86100%0.68478%1591.93319
ghc-8.0.1 201555.93850100.00000%79.53321%0.98643%0.98643%0.78454%1554.14858
csv

Program: '5'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.49854%
ghc-7.2.197.32062%0.00000%2.04263%
ghc-7.4.197.30162%0.00000%0.96217%
ghc-7.6.195.65066%0.00000%1.64270%
ghc-7.8.195.64448%0.00000%1.59937%
ghc-7.10.198.17795%0.00000%1.61150%
ghc-8.0.195.70618%0.00000%1.50635%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 491.45945104.55387%100.00000%1.49854%1.56678%1.49854%491.99374
ghc-7.2.1 20 478.29140101.75248%97.32062%2.04263%2.07842%1.98790%475.71853
ghc-7.4.1 20 478.19802101.73261%97.30162%0.96217%0.97884%0.93621%476.60458
ghc-7.6.1 20 470.08423100.00647%95.65066%1.64270%1.64281%1.57126%467.46246
ghc-7.8.1 20 470.05381100.00000%95.64448%1.59937%1.59937%1.52971%466.92835
ghc-7.10.1 20 482.50483102.64885%98.17795%1.61150%1.65418%1.58214%479.71388
ghc-8.0.1 20 470.35709100.06452%95.70618%1.50635%1.50732%1.44167%470.23685
csv

nbody

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.26612%
ghc-7.2.1100.31074%1.17282%4.17121%
ghc-7.4.190.94714%1.40987%4.71807%
ghc-7.6.177.58118%1.60574%5.21468%
ghc-7.8.169.06330%0.91181%4.46673%
ghc-7.10.170.23838%0.76977%4.33152%
ghc-8.0.173.16439%0.91208%6.14771%
csv

Input: '500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 315.94924142.26714%100.00000%4.75023%6.75802%4.75023%315.05588
ghc-7.2.1 20 322.08117145.02826%101.94079%6.32944%9.17948%6.45229%316.31996
ghc-7.4.1 20 293.26112132.05103%92.81906%6.61504%8.73523%6.14002%295.88597
ghc-7.6.1 20 252.18536113.55524%79.81832%10.23230%11.61932%8.16725%239.48244
ghc-7.8.1 20 222.08167100.00000%70.29030%9.53521%9.53521%6.70233%213.78585
ghc-7.10.1 20 224.36681101.02897%71.01356%9.30492%9.40066%6.60775%215.96249
ghc-8.0.1 20 234.76313105.71027%74.30407%8.08572%8.54744%6.00802%226.35309
csv

Input: '5000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202979.82190146.82912%100.00000%4.61471%6.77573%4.61471%2909.28541
ghc-7.2.1 202972.70456146.47842%99.76115%4.53032%6.63594%4.51950%2948.62822
ghc-7.4.1 202699.89096133.03568%90.60578%5.72972%7.62257%5.19145%2644.19287
ghc-7.6.1 202288.51091112.76515%76.80026%4.48989%5.06303%3.44825%2238.89082
ghc-7.8.1 202029.44880100.00000%68.10638%3.23000%3.23000%2.19983%2001.91780
ghc-7.10.1 202061.70487101.58940%69.18886%2.91486%2.96119%2.01676%2036.77132
ghc-8.0.1 202178.77726107.35808%73.11770%8.63628%9.27174%6.31465%2082.94856
csv

Input: '50000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2028963.45773145.36318%100.00000%0.43343%0.63005%0.43343%28978.99009
ghc-7.2.1 2028740.52126144.24430%99.23028%1.65385%2.38559%1.64112%28552.32061
ghc-7.4.1 2025898.13637129.97880%89.41659%1.80944%2.35189%1.61794%25745.26650
ghc-7.6.1 2022048.42252110.65768%76.12497%0.92185%1.02010%0.70176%21978.41017
ghc-7.8.1 2019924.89223100.00000%68.79321%0.63499%0.63499%0.43683%19882.29380
ghc-7.10.1 2020422.91656102.49951%70.51270%0.77478%0.79415%0.54632%20396.78589
ghc-8.0.1 2020874.37002104.76528%72.07140%1.72113%1.80315%1.24044%20715.88275
csv

pidigits

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.70774%
ghc-7.2.198.54942%1.92771%2.64379%
ghc-7.4.198.62290%2.35990%2.86949%
ghc-7.6.199.74541%1.10341%2.78360%
ghc-7.8.197.60828%2.01359%2.87014%
ghc-7.10.1100.58415%1.82029%2.91370%
ghc-8.0.1100.37179%1.48274%3.38979%
csv

Input: '2000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 82.34128105.52891%100.00000%9.03866%9.53840%9.03866%83.05856
ghc-7.2.1 20 78.91149101.13328%95.83466%6.07145%6.14026%5.81855%78.91800
ghc-7.4.1 20 78.48239100.58334%95.31354%6.16628%6.20225%5.87730%76.07850
ghc-7.6.1 20 80.84804103.61516%98.18651%6.76273%7.00722%6.64009%80.34261
ghc-7.8.1 20 78.02723100.00000%94.76076%6.75507%6.75507%6.40116%78.11925
ghc-7.10.1 20 80.70261103.42877%98.00989%6.69768%6.92733%6.56439%81.68970
ghc-8.0.1 20 80.95793103.75600%98.31997%8.36037%8.67439%8.21991%80.07200
csv

Input: '6000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 630.66846100.95401%100.00000%1.34035%1.35314%1.34035%627.69463
ghc-7.2.1 20 631.44269101.07794%100.12276%1.30231%1.31634%1.30391%629.92145
ghc-7.4.1 20 634.77679101.61164%100.65142%1.82255%1.85192%1.83442%632.55678
ghc-7.6.1 20 634.35963101.54487%100.58528%1.05680%1.07312%1.06298%633.09057
ghc-7.8.1 20 624.70871100.00000%99.05501%1.34576%1.34576%1.33304%623.26926
ghc-7.10.1 20 642.40298102.83240%101.86065%1.46196%1.50337%1.48916%639.42276
ghc-8.0.1 20 637.12106101.98690%101.02314%1.11121%1.13329%1.12258%637.16974
csv

Input: '10000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201770.58549101.00084%100.00000%0.74421%0.75166%0.74421%1766.62487
ghc-7.2.1 201765.11167100.68859%99.69085%0.55763%0.56147%0.55590%1761.98785
ghc-7.4.1 201768.88113100.90362%99.90374%0.61965%0.62525%0.61905%1767.44872
ghc-7.6.1 201778.80892101.46993%100.46445%0.53128%0.53909%0.53375%1780.45505
ghc-7.8.1 151753.04038100.00000%99.00908%0.50959%0.50959%0.50454%1749.35109
ghc-7.10.1 201803.90624102.90158%101.88191%0.58147%0.59834%0.59241%1801.76206
ghc-8.0.1 201801.96490102.79084%101.77226%0.69779%0.71726%0.71016%1799.59760
csv

spectralnorm

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.99042%
ghc-7.2.1102.40979%1.67539%1.08085%
ghc-7.4.1128.94756%3.97030%0.77699%
ghc-7.6.197.32091%2.89297%1.16682%
ghc-7.8.184.24592%2.40954%0.91241%
ghc-7.10.184.96472%2.00915%0.94737%
ghc-8.0.184.58816%2.29291%0.79172%
csv

Input: '500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 370.54190123.15665%100.00000%2.46399%3.03457%2.46399%367.90456
ghc-7.2.1 20 372.40203123.77491%100.50200%2.55341%3.16048%2.56623%371.79499
ghc-7.4.1 20 459.72849152.79952%124.06923%1.93453%2.95596%2.40016%456.40333
ghc-7.6.1 20 346.55717115.18487%93.52712%2.78290%3.20548%2.60277%345.52699
ghc-7.8.1 20 300.87038100.00000%81.19740%2.52711%2.52711%2.05195%299.49466
ghc-7.10.1 20 304.68313101.26724%82.22636%2.29852%2.32765%1.88999%303.79164
ghc-8.0.1 20 302.16451100.43013%81.54665%1.94401%1.95237%1.58527%300.13998
csv

Input: '3000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2012414.30163118.41105%100.00000%0.40570%0.48040%0.40570%12402.24871
ghc-7.2.1 2012680.81880120.95316%102.14686%0.47972%0.58023%0.49001%12687.64511
ghc-7.4.1 2016011.86179152.72557%128.97916%0.25433%0.38843%0.32804%16007.53888
ghc-7.6.1 2012152.53511115.91425%97.89141%0.40392%0.46820%0.39541%12150.19512
ghc-7.8.1 1510484.07371100.00000%84.45158%0.11140%0.11140%0.09408%10481.87485
ghc-7.10.1 2010636.24398101.45144%85.67734%0.46573%0.47249%0.39903%10657.01215
ghc-8.0.1 1510568.94531100.80953%85.13524%0.25626%0.25833%0.21816%10572.73348
csv

Input: '5500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2040670.68900114.95515%100.00000%0.10157%0.11676%0.10157%40687.36732
ghc-7.2.1 2042533.61581120.22069%104.58051%0.20944%0.25179%0.21903%42521.24431
ghc-7.4.1 1554415.05652153.80342%133.79428%0.14210%0.21856%0.19013%54398.64822
ghc-7.6.1 2040892.01815115.58073%100.54420%0.31365%0.36252%0.31536%40897.17312
ghc-7.8.1 1535419.61041100.11305%87.08879%0.09873%0.09884%0.08598%35422.41567
ghc-7.10.1 1535379.61456100.00000%86.99045%0.07785%0.07785%0.06772%35381.73301
ghc-8.0.1 1535417.09438100.10594%87.08260%0.17488%0.17507%0.15229%35407.89545
csv