img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.92850%
ghc-7.2.1105.65882%7.85792%1.89867%
ghc-7.4.1116.18607%37.91242%2.20063%
ghc-7.6.1110.23298%13.75032%1.65221%
ghc-7.8.1100.30660%11.94336%2.06364%
ghc-7.10.197.90140%14.09735%2.49993%
ghc-8.0.198.63334%12.98415%2.79906%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.50132%
ghc-7.2.1103.54977%7.07586%2.26578%
ghc-7.4.1110.18198%31.27774%2.95761%
ghc-7.6.1110.77406%14.82526%2.37830%
ghc-7.8.199.54353%8.86703%3.08019%
ghc-7.10.195.32047%12.04376%3.72033%
ghc-8.0.196.29778%11.57217%3.29419%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.95199%
ghc-7.2.1107.21359%7.51261%2.18936%
ghc-7.4.1119.32688%38.24026%1.67771%
ghc-7.6.1109.21701%12.17588%1.40906%
ghc-7.8.199.25324%12.63514%2.03932%
ghc-7.10.197.76098%13.22402%2.44787%
ghc-8.0.199.02549%12.35648%3.06658%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.97418%
ghc-7.2.1107.53126%8.53706%1.01145%
ghc-7.4.1122.80189%45.23423%1.49346%
ghc-7.6.1110.36970%13.35847%0.71547%
ghc-7.8.1102.59993%14.91165%0.43606%
ghc-7.10.1102.23583%16.73916%0.56885%
ghc-8.0.1102.03647%14.84044%1.72696%
csv

Seperated by entropy

The following shows the summary including only the lower or the upper half of programs (per category), regarding the entropy of their files. This entropy is measured by taking the length of the gnu zipped program code length. Programs with lower entropy should be simpler than programs with higher entropy. If the number of programs is uneven in a category, then one program belongs to the upper and the lower half.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.02304%
ghc-7.2.1105.92132%8.06775%1.98484%
ghc-7.4.1119.46086%40.89839%2.36774%
ghc-7.6.1110.40346%14.53665%1.57907%
ghc-7.8.198.77526%11.95412%2.27793%
ghc-7.10.196.87902%14.76562%2.78254%
ghc-8.0.197.34580%13.15522%3.11644%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.73625%
ghc-7.2.1104.17644%7.10132%2.60600%
ghc-7.4.1113.31685%35.00019%3.36267%
ghc-7.6.1111.30259%15.63671%2.51330%
ghc-7.8.198.63905%8.95376%3.63980%
ghc-7.10.194.53205%12.84992%4.33767%
ghc-8.0.195.37800%11.93338%3.91418%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.96548%
ghc-7.2.1106.48482%7.76230%2.14503%
ghc-7.4.1121.90640%40.22423%1.67007%
ghc-7.6.1108.85114%12.97536%1.13169%
ghc-7.8.197.04415%11.97552%2.17810%
ghc-7.10.196.07323%13.30672%2.73466%
ghc-8.0.196.89032%11.74830%3.19168%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.06173%
ghc-7.2.1107.85049%9.08173%0.93729%
ghc-7.4.1125.79246%47.61190%1.64407%
ghc-7.6.1110.67130%14.25531%0.69185%
ghc-7.8.1100.70095%15.00911%0.43223%
ghc-7.10.1101.03762%17.57107%0.60880%
ghc-8.0.1100.61243%15.34530%1.90159%
csv

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.26667%
ghc-7.2.1104.41482%5.81806%2.22387%
ghc-7.4.1116.84249%40.71826%2.55937%
ghc-7.6.1107.72154%10.81919%1.90520%
ghc-7.8.1101.40979%12.55080%2.40035%
ghc-7.10.1100.69643%13.88567%2.88655%
ghc-8.0.1101.80811%12.00922%3.28602%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.13188%
ghc-7.2.1103.21265%4.96083%2.81014%
ghc-7.4.1112.56569%33.93383%3.63477%
ghc-7.6.1109.51935%11.81691%2.91716%
ghc-7.8.1101.65029%8.30412%3.81713%
ghc-7.10.199.07318%11.17258%4.55908%
ghc-8.0.1100.44457%9.88870%4.06954%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.20815%
ghc-7.2.1105.11307%5.40415%2.47564%
ghc-7.4.1117.98618%40.70430%1.88848%
ghc-7.6.1105.78230%8.66331%1.58374%
ghc-7.8.199.33448%13.50558%2.29822%
ghc-7.10.199.34593%13.40747%2.75326%
ghc-8.0.1101.13385%11.78699%3.49041%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.08917%
ghc-7.2.1105.43396%6.93570%1.13458%
ghc-7.4.1121.80851%48.27573%1.69396%
ghc-7.6.1107.09247%10.86367%0.78102%
ghc-7.8.1103.14154%15.86745%0.47851%
ghc-7.10.1104.36585%16.85030%0.63053%
ghc-8.0.1104.43028%14.34777%1.96230%
csv

binarytrees

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%3.70164%
ghc-7.2.1104.41942%0.37948%3.15209%
ghc-7.4.1103.68813%2.11783%4.96596%
ghc-7.6.1103.59008%3.49307%2.89808%
ghc-7.8.1101.81349%6.63143%3.29762%
ghc-7.10.1107.75544%9.45853%4.67857%
ghc-8.0.1107.04177%6.83214%3.47905%
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 35.58488101.36534%100.00000%9.32676%9.45410%9.32676%34.26758
ghc-7.2.1 20 36.99127105.37152%103.95222%7.77222%8.18971%8.07940%35.19761
ghc-7.4.1 20 35.90469102.27634%100.89873%11.82767%12.09691%11.93397%35.61000
ghc-7.6.1 20 35.10557100.00000%98.65305%7.74059%7.74059%7.63633%34.08987
ghc-7.8.1 20 36.86751105.01897%103.60442%6.97816%7.32839%7.22968%36.88943
ghc-7.10.1 20 37.97593108.17637%106.71929%10.79466%11.67727%11.51998%36.61204
ghc-8.0.1 20 39.46879112.42887%110.91451%8.53589%9.59680%9.46754%39.38057
csv

Input: '16'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201160.27345107.58972%100.00000%1.41761%1.52521%1.41761%1155.18420
ghc-7.2.1 201211.60769112.34983%104.42432%1.15898%1.30212%1.21026%1207.04450
ghc-7.4.1 201230.20803114.07460%106.02742%1.90480%2.17290%2.01961%1233.32402
ghc-7.6.1 201232.27620114.26638%106.20567%0.53489%0.61120%0.56808%1233.36989
ghc-7.8.1 201078.42412100.00000%92.94569%2.33894%2.33894%2.17395%1065.17774
ghc-7.10.1 201122.26322104.06511%96.72403%2.41977%2.51814%2.34050%1119.25540
ghc-8.0.1 201130.56275104.83471%97.43934%1.04156%1.09192%1.01489%1130.61332
csv

Input: '20'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2033325.22206100.00000%100.00000%0.36054%0.36054%0.36054%33322.55649
ghc-7.2.1 2034952.06696104.88172%104.88172%0.52507%0.55070%0.55070%34939.80545
ghc-7.4.1 2034704.29598104.13823%104.13823%1.16541%1.21363%1.21363%34613.03562
ghc-7.6.1 2035295.24995105.91152%105.91152%0.41876%0.44351%0.44351%35272.26885
ghc-7.8.1 2036287.96056108.89038%108.89038%0.57575%0.62693%0.62693%36235.93834
ghc-7.10.1 2039931.27951119.82300%119.82300%0.82127%0.98408%0.98408%39912.47192
ghc-8.0.1 2037581.34251112.77147%112.77147%0.85971%0.96951%0.96951%37576.24591
csv

fannkuchredux

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.05698%
ghc-7.2.1115.48947%6.49862%1.05256%
ghc-7.4.1115.24152%13.15643%1.03095%
ghc-7.6.1121.96233%11.10299%1.60175%
ghc-7.8.1106.63555%8.03429%0.68897%
ghc-7.10.198.21061%11.56968%0.77662%
ghc-8.0.198.57662%14.30427%0.92981%
csv

Mean scores per input

Mean scores for input '10'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.89739%
ghc-7.2.1115.59786%5.14223%0.97511%
ghc-7.4.1114.92186%11.66600%1.86154%
ghc-7.6.1122.58353%9.77132%2.46810%
ghc-7.8.1105.85519%7.43629%1.11860%
ghc-7.10.197.53653%11.55975%1.74141%
ghc-8.0.197.96826%14.62944%1.35201%
csv

Mean scores for input '11'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.00825%
ghc-7.2.1117.11608%4.80110%1.34250%
ghc-7.4.1114.99099%13.72077%0.96674%
ghc-7.6.1122.51903%10.74095%1.76843%
ghc-7.8.1106.70070%8.01615%0.64744%
ghc-7.10.198.12077%11.45445%0.37527%
ghc-8.0.199.11935%14.85234%1.14533%
csv

Mean scores for input '12'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.26532%
ghc-7.2.1113.75447%8.45786%0.84005%
ghc-7.4.1115.81171%13.94380%0.26456%
ghc-7.6.1120.78441%12.52593%0.56873%
ghc-7.8.1107.35075%8.54210%0.30087%
ghc-7.10.198.97453%11.64879%0.21318%
ghc-8.0.198.64224%13.36245%0.29209%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.20722%
ghc-7.2.1121.62320%0.63597%0.20027%
ghc-7.4.1128.35171%1.31802%0.16920%
ghc-7.6.1132.97506%0.43903%0.32314%
ghc-7.8.198.63736%0.16258%0.21990%
ghc-7.10.186.65628%0.55076%0.31011%
ghc-8.0.184.29521%0.79265%0.19152%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202431.27055119.99210%100.00000%0.29348%0.35215%0.29348%2432.94804
ghc-7.2.1 202935.51816144.87856%120.74009%0.26594%0.38530%0.32110%2936.21690
ghc-7.4.1 203077.69317151.89542%126.58785%0.21535%0.32711%0.27261%3076.00295
ghc-7.6.1 203217.90461158.81536%132.35486%0.52653%0.83620%0.69688%3216.76569
ghc-7.8.1 202392.82962118.09490%98.41890%0.29377%0.34693%0.28912%2391.99346
ghc-7.10.1 202090.32813103.16534%85.97678%0.48306%0.49835%0.41532%2089.02167
ghc-8.0.1 202026.19225100.00000%83.33882%0.39517%0.39517%0.32933%2024.08741
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2029801.21325118.67040%100.00000%0.15890%0.18857%0.15890%29793.11429
ghc-7.2.1 2036332.79839144.67960%121.91718%0.18534%0.26815%0.22596%36332.88632
ghc-7.4.1 2038357.66737152.74277%128.71176%0.20229%0.30899%0.26037%38343.23472
ghc-7.6.1 2039713.09308158.14016%133.25999%0.18623%0.29451%0.24818%39720.60724
ghc-7.8.1 1529409.19100117.10934%98.68454%0.22703%0.26587%0.22404%29407.80424
ghc-7.10.1 2025827.61369102.84726%86.66632%0.31018%0.31901%0.26882%25805.31750
ghc-8.0.1 2025112.59181100.00000%84.26701%0.09978%0.09978%0.08408%25117.14842
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 15399917.26345117.26108%100.00000%0.16929%0.19851%0.16929%399884.28386
ghc-7.2.1 15488748.19064143.30749%122.21233%0.14953%0.21428%0.18274%488425.17276
ghc-7.4.1 15518914.68921152.15271%129.75551%0.08995%0.13685%0.11671%518772.51667
ghc-7.6.1 15533131.07801156.32115%133.31034%0.25666%0.40121%0.34215%533391.37097
ghc-7.8.1 15395152.86357115.86409%98.80865%0.13889%0.16093%0.13724%395130.88882
ghc-7.10.1 15349230.71670102.39911%87.32574%0.13711%0.14040%0.11973%349329.50207
ghc-8.0.1 15341048.59152100.00000%85.27979%0.07962%0.07962%0.06790%341066.42369
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.90674%
ghc-7.2.1109.35574%2.96891%1.90484%
ghc-7.4.1102.13133%0.83176%1.89270%
ghc-7.6.1110.94959%1.94917%2.88037%
ghc-7.8.1114.63373%1.06363%1.15804%
ghc-7.10.1109.76494%0.63769%1.24312%
ghc-8.0.1112.85803%0.82385%1.66810%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 315.36318100.00000%100.00000%3.50129%3.50129%3.50129%312.16577
ghc-7.2.1 20 348.33637110.45562%110.45562%1.68427%1.86038%1.86038%349.48573
ghc-7.4.1 20 325.63097103.25586%103.25586%3.50774%3.62195%3.62195%328.15997
ghc-7.6.1 20 355.76816112.81221%112.81221%4.40967%4.97465%4.97465%353.80273
ghc-7.8.1 20 357.27963113.29148%113.29148%1.94342%2.20173%2.20173%354.73545
ghc-7.10.1 20 344.04950109.09628%109.09628%2.99975%3.27262%3.27262%340.89525
ghc-8.0.1 20 355.09170112.59771%112.59771%2.30885%2.59972%2.59972%356.89238
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 203768.54381100.00000%100.00000%1.85759%1.85759%1.85759%3744.06282
ghc-7.2.1 204232.63905112.31498%112.31498%2.49967%2.80750%2.80750%4283.73190
ghc-7.4.1 203816.41247101.27022%101.27022%1.73118%1.75317%1.75317%3794.96577
ghc-7.6.1 204212.40594111.77808%111.77808%3.35063%3.74526%3.74526%4223.68652
ghc-7.8.1 204323.15482114.71685%114.71685%1.06785%1.22500%1.22500%4307.54278
ghc-7.10.1 204129.39012109.57522%109.57522%0.44036%0.48253%0.48253%4124.00906
ghc-8.0.1 204295.07310113.97169%113.97169%2.19088%2.49698%2.49698%4349.05943
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2050597.63303100.00000%100.00000%0.36135%0.36135%0.36135%50553.75893
ghc-7.2.1 2053277.59509105.29662%105.29662%1.53058%1.61165%1.61165%52895.89321
ghc-7.4.1 2051542.75317101.86791%101.86791%0.43918%0.44739%0.44739%51534.79956
ghc-7.6.1 2054776.22787108.25848%108.25848%0.88080%0.95355%0.95355%54675.61769
ghc-7.8.1 2058639.03768115.89285%115.89285%0.46285%0.53641%0.53641%58611.32413
ghc-7.10.1 2055972.78503110.62333%110.62333%0.28926%0.31999%0.31999%55968.64536
ghc-8.0.1 2056671.72519112.00470%112.00470%0.50457%0.56514%0.56514%56663.55054
csv

fasta

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.95952%
ghc-7.2.1108.77330%0.39057%1.06453%
ghc-7.4.1108.31493%0.67381%1.70203%
ghc-7.6.1109.05635%0.34602%1.15062%
ghc-7.8.1104.48956%0.75125%1.45802%
ghc-7.10.1106.76074%0.20593%1.41482%
ghc-8.0.1106.32174%0.06930%1.68939%
csv

Input: '250000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 91.58449100.00000%100.00000%2.29352%2.29352%2.29352%90.95912
ghc-7.2.1 20 99.22039108.33754%108.33754%2.37905%2.57740%2.57740%99.57854
ghc-7.4.1 20 98.67861107.74598%107.74598%3.79691%4.09101%4.09101%97.08773
ghc-7.6.1 20 99.45951108.59863%108.59863%2.63441%2.86093%2.86093%99.15499
ghc-7.8.1 20 96.66559105.54798%105.54798%3.43436%3.62490%3.62490%97.36682
ghc-7.10.1 20 97.95568106.95662%106.95662%3.35585%3.58930%3.58930%97.99731
ghc-8.0.1 20 97.40551106.35589%106.35589%4.20620%4.47354%4.47354%95.64652
csv

Input: '2500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 858.13469100.00000%100.00000%0.41239%0.41239%0.41239%858.16156
ghc-7.2.1 20 932.76868108.69724%108.69724%0.62045%0.67442%0.67442%931.40360
ghc-7.4.1 20 937.61006109.26141%109.26141%1.10705%1.20958%1.20958%934.21447
ghc-7.6.1 20 936.52815109.13533%109.13533%0.53262%0.58127%0.58127%936.66202
ghc-7.8.1 20 891.43519103.88057%103.88057%0.65011%0.67534%0.67534%890.36709
ghc-7.10.1 20 913.70888106.47616%106.47616%0.51705%0.55053%0.55053%914.66606
ghc-8.0.1 20 911.55450106.22511%106.22511%0.58894%0.62560%0.62560%910.40225
csv

Input: '25000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 208475.04354100.00000%100.00000%0.17264%0.17264%0.17264%8473.06848
ghc-7.2.1 209261.96105109.28511%109.28511%0.19407%0.21209%0.21209%9260.86345
ghc-7.4.1 209147.74124107.93740%107.93740%0.20215%0.21819%0.21819%9146.41668
ghc-7.6.1 209274.67219109.43510%109.43510%0.28483%0.31171%0.31171%9265.62100
ghc-7.8.1 208817.44514104.04012%104.04012%0.28959%0.30129%0.30129%8813.83238
ghc-7.10.1 209055.53682106.84944%106.84944%0.37158%0.39703%0.39703%9057.64455
ghc-8.0.1 209016.10850106.38421%106.38421%0.27303%0.29046%0.29046%9006.45216
csv

mandelbrot

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%2.54661%
ghc-7.2.1100.21131%0.98737%3.02578%
ghc-7.4.198.74355%2.86916%1.59479%
ghc-7.6.199.27260%2.96017%0.77758%
ghc-7.8.197.22948%2.33951%2.03798%
ghc-7.10.197.06453%3.40834%2.24056%
ghc-8.0.197.30798%2.44334%1.68998%
csv

Input: '1000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 159.75269100.00000%100.00000%1.67908%1.67908%1.67908%160.15171
ghc-7.2.1 20 161.94860101.37457%101.37457%1.85232%1.87778%1.87778%162.45140
ghc-7.4.1 20 162.77066101.88915%101.88915%2.09404%2.13360%2.13360%162.88280
ghc-7.6.1 20 164.03177102.67857%102.67857%1.60892%1.65202%1.65202%164.65697
ghc-7.8.1 20 159.92122100.10550%100.10550%2.00733%2.00945%2.00945%159.30069
ghc-7.10.1 20 161.70255101.22055%101.22055%6.05486%6.12877%6.12877%159.72087
ghc-8.0.1 20 159.89625100.08987%100.08987%1.73788%1.73944%1.73944%160.33271
csv

Input: '4000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202615.22732107.67497%100.00000%5.13554%5.52969%5.13554%2583.86733
ghc-7.2.1 202623.03610107.99648%100.29859%6.21729%6.71445%6.23585%2622.74467
ghc-7.4.1 202483.18607102.23853%94.95106%2.00638%2.05129%1.90508%2477.44161
ghc-7.6.1 202496.53979102.78833%95.46167%0.23764%0.24426%0.22685%2495.97555
ghc-7.8.1 202468.12111101.61827%94.37501%3.75606%3.81685%3.54478%2437.55969
ghc-7.10.1 202428.81629100.00000%92.87209%0.15627%0.15627%0.14513%2429.67457
ghc-8.0.1 202462.02784101.36740%94.14202%3.03969%3.08126%2.86163%2446.25796
csv

Input: '16000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2039920.97715102.98561%100.00000%0.82521%0.84984%0.82521%39966.87073
ghc-7.2.1 2039506.10113101.91534%98.96076%1.00774%1.02704%0.99727%39360.17624
ghc-7.4.1 2039677.63702102.35785%99.39045%0.68395%0.70008%0.67978%39586.76082
ghc-7.6.1 2039792.25711102.65354%99.67756%0.48620%0.49910%0.48463%39760.54546
ghc-7.8.1 2038806.36010100.11019%97.20794%0.35055%0.35094%0.34076%38795.52100
ghc-7.10.1 2038763.64690100.00000%97.10095%0.51055%0.51055%0.49575%38686.48520
ghc-8.0.1 2038999.61644100.60874%97.69204%0.29236%0.29414%0.28561%38963.47326
csv

meteor

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.63464%
ghc-7.2.197.32319%2.29835%0.69173%
ghc-7.4.195.22305%6.03449%0.82089%
ghc-7.6.1112.25907%19.69354%0.65580%
ghc-7.8.195.20011%6.69868%0.82587%
ghc-7.10.186.90409%5.61596%1.15639%
ghc-8.0.187.08449%5.49041%0.85620%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.75380%
ghc-7.2.198.99035%0.00000%0.54394%
ghc-7.4.1101.38023%0.00000%0.44849%
ghc-7.6.1120.86763%0.00000%0.54015%
ghc-7.8.199.65865%0.00000%0.57710%
ghc-7.10.189.12440%0.00000%0.64774%
ghc-8.0.189.79106%0.00000%0.62928%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 780.85071112.20272%100.00000%0.75380%0.84579%0.75380%780.59416
ghc-7.2.1 20 772.96688111.06987%98.99035%0.54394%0.60416%0.53845%772.82466
ghc-7.4.1 20 791.62828113.75138%101.38023%0.44849%0.51016%0.45468%790.22039
ghc-7.6.1 20 943.79578135.61677%120.86763%0.54015%0.73254%0.65287%944.38132
ghc-7.8.1 20 778.18528111.81972%99.65865%0.57710%0.64531%0.57513%777.64230
ghc-7.10.1 20 695.92850100.00000%89.12440%0.64774%0.64774%0.57729%694.97022
ghc-8.0.1 20 701.13414100.74801%89.79106%0.62928%0.63399%0.56504%700.97944
csv

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.61491%
ghc-7.2.199.40860%0.00000%0.65995%
ghc-7.4.1100.05284%0.00000%0.90399%
ghc-7.6.1134.55621%0.00000%0.68390%
ghc-7.8.1102.47247%0.00000%0.85097%
ghc-7.10.187.31357%0.00000%1.50369%
ghc-8.0.185.84227%0.00000%1.17327%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 560.64048116.49272%100.00000%0.61491%0.71633%0.61491%560.26343
ghc-7.2.1 20 557.32488115.80379%99.40860%0.65995%0.76424%0.65604%557.80003
ghc-7.4.1 20 560.93671116.55427%100.05284%0.90399%1.05363%0.90446%559.74270
ghc-7.6.1 20 754.37661156.74819%134.55621%0.68390%1.07200%0.92023%753.28482
ghc-7.8.1 20 574.50216119.37297%102.47247%0.85097%1.01583%0.87201%573.45458
ghc-7.10.1 20 489.51523101.71395%87.31357%1.50369%1.52946%1.31292%486.37003
ghc-8.0.1 20 481.26654100.00000%85.84227%1.17327%1.17327%1.00716%479.90835
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.59211%
ghc-7.2.194.12477%0.00000%0.81707%
ghc-7.4.195.85255%0.00000%1.34284%
ghc-7.6.1127.92030%0.00000%0.64232%
ghc-7.8.196.20611%0.00000%1.11370%
ghc-7.10.178.76652%0.00000%1.68579%
ghc-8.0.179.14858%0.00000%0.99905%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 500.82414126.95749%100.00000%0.59211%0.75173%0.59211%500.39205
ghc-7.2.1 20 471.39956119.49844%94.12477%0.81707%0.97638%0.76906%472.25430
ghc-7.4.1 20 480.05273121.69200%95.85255%1.34284%1.63412%1.28714%478.32268
ghc-7.6.1 20 640.65573162.40440%127.92030%0.64232%1.04316%0.82166%639.92574
ghc-7.8.1 20 481.82342122.14086%96.20611%1.11370%1.36028%1.07145%479.75312
ghc-7.10.1 20 394.48176100.00000%78.76652%1.68579%1.68579%1.32784%393.52752
ghc-8.0.1 20 396.39521100.48506%79.14858%0.99905%1.00390%0.79074%396.69033
csv

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.31278%
ghc-7.2.199.15565%0.00000%0.27074%
ghc-7.4.184.26846%0.00000%0.54307%
ghc-7.6.184.59416%0.00000%0.57750%
ghc-7.8.182.93804%0.00000%0.46370%
ghc-7.10.183.69097%0.00000%0.60463%
ghc-8.0.184.93811%0.00000%0.73481%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201592.66937120.57194%100.00000%0.31278%0.37712%0.31278%1592.71279
ghc-7.2.1 201579.22163119.55388%99.15565%0.27074%0.32368%0.26845%1580.43887
ghc-7.4.1 201342.11799101.60412%84.26846%0.54307%0.55178%0.45764%1340.88953
ghc-7.6.1 201347.30520101.99681%84.59416%0.57750%0.58903%0.48853%1346.43679
ghc-7.8.1 201320.92874100.00000%82.93804%0.46370%0.46370%0.38458%1320.47545
ghc-7.10.1 201332.92042100.90782%83.69097%0.60463%0.61012%0.50602%1331.92911
ghc-8.0.1 201352.78319102.41152%84.93811%0.73481%0.75253%0.62414%1355.52882
csv

Program: '5'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.89959%
ghc-7.2.194.93658%0.00000%1.16695%
ghc-7.4.194.56118%0.00000%0.86606%
ghc-7.6.193.35706%0.00000%0.83514%
ghc-7.8.194.72530%0.00000%1.12386%
ghc-7.10.195.62497%0.00000%1.34010%
ghc-8.0.195.70243%0.00000%0.74458%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 491.26064107.11563%100.00000%0.89959%0.96360%0.89959%492.14202
ghc-7.2.1 20 466.38605101.69192%94.93658%1.16695%1.18669%1.10786%464.87271
ghc-7.4.1 20 464.54187101.28981%94.56118%0.86606%0.87723%0.81895%464.12722
ghc-7.6.1 20 458.62647100.00000%93.35706%0.83514%0.83514%0.77966%459.20778
ghc-7.8.1 20 465.34809101.46560%94.72530%1.12386%1.14034%1.06458%464.62867
ghc-7.10.1 20 469.76782102.42929%95.62497%1.34010%1.37266%1.28147%468.22075
ghc-8.0.1 20 470.14838102.51226%95.70243%0.74458%0.76328%0.71258%469.24550
csv

nbody

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%5.62225%
ghc-7.2.1100.93537%2.27978%5.43464%
ghc-7.4.199.04619%1.49397%6.58438%
ghc-7.6.198.68522%4.06064%4.21458%
ghc-7.8.176.23369%5.56311%7.73495%
ghc-7.10.172.44959%2.99318%9.45056%
ghc-8.0.181.11278%5.56503%14.39544%
csv

Input: '500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 230.46101130.82430%100.00000%5.61270%7.34278%5.61270%225.80699
ghc-7.2.1 20 239.97532136.22522%104.12838%6.05435%8.24755%6.30429%248.69692
ghc-7.4.1 20 232.04463131.72326%100.68715%6.09682%8.03092%6.13871%224.72262
ghc-7.6.1 20 240.59158136.57505%104.39578%4.37306%5.97250%4.56529%243.66861
ghc-7.8.1 20 193.52271109.85577%83.97200%14.99345%16.47117%12.59030%174.59754
ghc-7.10.1 20 176.16071100.00000%76.43840%12.10915%12.10915%9.25604%167.63824
ghc-8.0.1 20 204.56904116.12637%88.76514%15.69572%18.22687%13.93233%224.69804
csv

Input: '5000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202305.81689139.50450%100.00000%5.79157%8.07950%5.79157%2276.94280
ghc-7.2.1 202281.65331138.04258%98.95206%5.86817%8.10057%5.80667%2188.85445
ghc-7.4.1 202238.32931135.42143%97.07316%5.24638%7.10473%5.09283%2168.04751
ghc-7.6.1 202221.76175134.41907%96.35465%5.47605%7.36086%5.27643%2179.34841
ghc-7.8.1 201696.94019102.66678%73.59388%7.15235%7.34309%5.26369%1638.83791
ghc-7.10.1 201652.86202100.00000%71.68228%14.46762%14.46762%10.37072%1540.63845
ghc-8.0.1 201818.80262110.03959%78.87888%16.33200%17.97167%12.88250%1581.20341
csv

Input: '50000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2022357.84453144.45003%100.00000%5.46247%7.89054%5.46247%21656.01308
ghc-7.2.1 2022296.51302144.05377%99.72568%4.38140%6.31157%4.36938%21803.90840
ghc-7.4.1 2022218.83512143.55191%99.37825%8.40995%12.07264%8.35766%21268.43523
ghc-7.6.1 2021308.19414137.66842%95.30523%2.79465%3.84735%2.66344%21139.30827
ghc-7.8.1 2015904.29624102.75481%71.13519%1.05906%1.08824%0.75337%15808.89890
ghc-7.10.1 2015477.90975100.00000%69.22809%1.77493%1.77493%1.22875%15318.80374
ghc-8.0.1 2016923.61668109.34045%75.69431%11.15859%12.20086%8.44642%15821.52174
csv

pidigits

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%1.85300%
ghc-7.2.199.73674%0.32378%1.61079%
ghc-7.4.1100.52666%0.50265%1.75933%
ghc-7.6.1100.07668%0.11452%1.69476%
ghc-7.8.199.24684%0.28899%1.80096%
ghc-7.10.1102.63792%0.74611%1.97898%
ghc-8.0.1100.77092%0.67528%1.64818%
csv

Input: '2000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 76.46476101.17405%100.00000%4.60022%4.65423%4.60022%74.75978
ghc-7.2.1 20 76.54653101.28224%100.10694%3.80506%3.85385%3.80913%76.91825
ghc-7.4.1 20 77.40649102.42010%101.23159%4.56160%4.67200%4.61778%76.41395
ghc-7.6.1 20 76.64645101.41445%100.23761%4.32173%4.38286%4.33200%75.19810
ghc-7.8.1 20 75.57745100.00000%98.83958%4.32152%4.32152%4.27138%75.24086
ghc-7.10.1 20 79.27531104.89282%103.67562%4.95526%5.19771%5.13739%79.03151
ghc-8.0.1 20 76.32640100.99097%99.81905%3.88822%3.92675%3.88118%75.39838
csv

Input: '6000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 624.27332100.68642%100.00000%0.64951%0.65397%0.64951%622.63870
ghc-7.2.1 20 620.01738100.00000%99.31826%0.76404%0.76404%0.75883%619.23738
ghc-7.4.1 15 625.85601100.94169%100.25352%0.49573%0.50040%0.49699%626.68672
ghc-7.6.1 15 624.15153100.66678%99.98049%0.53404%0.53760%0.53393%624.24838
ghc-7.8.1 20 620.65782100.10329%99.42085%0.68047%0.68117%0.67653%621.09769
ghc-7.10.1 20 636.46868102.65336%101.95353%0.62557%0.64216%0.63779%636.79603
ghc-8.0.1 20 631.64025101.87460%101.18008%0.85888%0.87498%0.86901%630.64804
csv

Input: '10000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201755.92499100.52261%100.00000%0.30925%0.31087%0.30925%1755.66508
ghc-7.2.1 201752.15023100.30652%99.78503%0.26327%0.26408%0.26270%1751.92720
ghc-7.4.1 201757.59085100.61798%100.09487%0.22065%0.22202%0.22086%1757.46961
ghc-7.6.1 201756.13481100.53462%100.01195%0.22852%0.22974%0.22854%1755.79998
ghc-7.8.1 151746.79601100.00000%99.48010%0.40088%0.40088%0.39880%1745.75631
ghc-7.10.1 151796.04101102.81916%102.28461%0.35612%0.36616%0.36426%1794.14956
ghc-8.0.1 151778.99156101.84312%101.31364%0.19746%0.20110%0.20005%1779.31469
csv

spectralnorm

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (mean(mean / mean of first impl))mean score std (std(mean / mean of first impl))mean rel std (mean(std / mean))
ghc-7.0.1100.00000%0.00000%0.78748%
ghc-7.2.1114.10819%4.27453%0.80804%
ghc-7.4.1223.62439%11.54514%1.23624%
ghc-7.6.1123.88145%5.17077%0.93924%
ghc-7.8.1118.67942%4.79176%0.86457%
ghc-7.10.1118.45061%4.43165%0.92197%
ghc-8.0.1118.60637%4.67022%0.86894%
csv

Input: '500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 151.12996100.00000%100.00000%2.03693%2.03693%2.03693%150.88186
ghc-7.2.1 20 166.90141110.43569%110.43569%2.18330%2.41114%2.41114%165.51166
ghc-7.4.1 15 323.34887213.95418%213.95418%2.24441%4.80200%4.80200%321.54607
ghc-7.6.1 20 179.90016119.03673%119.03673%2.02401%2.40932%2.40932%178.02147
ghc-7.8.1 20 172.71968114.28553%114.28553%1.94108%2.21838%2.21838%172.13319
ghc-7.10.1 20 173.13771114.56214%114.56214%1.82974%2.09619%2.09619%173.19726
ghc-8.0.1 20 173.14620114.56775%114.56775%1.77555%2.03421%2.03421%173.69433
csv

Input: '3000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 205171.83138100.00000%100.00000%0.19281%0.19281%0.19281%5170.01140
ghc-7.2.1 205781.38730111.78607%111.78607%0.20091%0.22458%0.22458%5778.23770
ghc-7.4.1 1511226.31249217.06648%217.06648%0.72786%1.57993%1.57993%11190.70875
ghc-7.6.1 206286.88847121.56020%121.56020%0.42035%0.51098%0.51098%6278.34404
ghc-7.8.1 206020.45120116.40850%116.40850%0.44175%0.51424%0.51424%6010.37609
ghc-7.10.1 206006.47182116.13820%116.13820%0.64616%0.75044%0.75044%5999.60791
ghc-8.0.1 206004.48577116.09980%116.09980%0.38090%0.44223%0.44223%5996.49143
csv

Input: '5500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 1516102.61935100.00000%100.00000%0.13271%0.13271%0.13271%16102.86637
ghc-7.2.1 1519339.69597120.10280%120.10280%0.03992%0.04795%0.04795%19340.99120
ghc-7.4.1 1538622.53472239.85250%239.85250%0.73645%1.76639%1.76639%38496.27640
ghc-7.6.1 2021102.06737131.04742%131.04742%0.37336%0.48928%0.48928%21076.15467
ghc-7.8.1 2020183.70374125.34423%125.34423%0.21089%0.26434%0.26434%20158.28584
ghc-7.10.1 1520072.15654124.65150%124.65150%0.29001%0.36150%0.36150%20039.32409
ghc-8.0.1 2020152.68039125.15157%125.15157%0.45036%0.56363%0.56363%20111.10169
csv