img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.96614%112.05777%0.87452%
ghc-7.2.1113.97375%112.16581%0.93486%
ghc-7.4.1121.19005%128.11154%1.08181%
ghc-7.6.1118.78718%117.17441%1.06743%
ghc-7.8.1108.32798%107.28820%1.04548%
ghc-7.10.1105.31931%106.04988%0.97867%
ghc-8.0.1106.02146%105.84457%1.08023%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1110.31070%110.82234%1.46929%
ghc-7.2.1112.70161%110.66582%1.61221%
ghc-7.4.1117.18948%123.77440%1.77459%
ghc-7.6.1120.46647%119.06027%1.73699%
ghc-7.8.1108.98639%107.22203%1.76990%
ghc-7.10.1104.03248%104.55556%1.87762%
ghc-8.0.1104.66830%104.60268%1.90795%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.55863%112.32872%0.94026%
ghc-7.2.1115.41981%112.06254%0.82559%
ghc-7.4.1124.19633%128.64723%0.93969%
ghc-7.6.1117.53156%114.80553%1.01027%
ghc-7.8.1107.14431%106.49884%1.27978%
ghc-7.10.1104.91285%104.83077%0.64754%
ghc-8.0.1106.61589%105.36171%0.96345%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1107.21707%113.38794%0.35004%
ghc-7.2.1114.61746%114.24049%0.43664%
ghc-7.4.1124.88635%133.09932%0.55722%
ghc-7.6.1117.34852%115.97198%0.51124%
ghc-7.8.1108.45155%107.97831%0.36305%
ghc-7.10.1107.86073%108.21133%0.51307%
ghc-8.0.1107.65419%107.43595%0.48056%
csv

Seperated by entropy

The following shows the summary including only the lower or the upper half of programs (per category), regarding the entropy of their files. This entropy is measured by taking the length of the gnu zipped program code length. Programs with lower entropy should be simpler than programs with higher entropy. If the number of programs is uneven in a category, then one program belongs to the upper and the lower half.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1110.29595%112.79117%0.86294%
ghc-7.2.1115.34895%112.95870%0.78929%
ghc-7.4.1125.10277%129.76714%1.12023%
ghc-7.6.1120.09472%118.03399%1.01947%
ghc-7.8.1107.68257%107.54194%1.05193%
ghc-7.10.1105.16607%106.36714%1.08684%
ghc-8.0.1105.46471%105.81655%1.03369%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1112.04702%111.65801%1.54176%
ghc-7.2.1114.39921%111.53277%1.53464%
ghc-7.4.1120.87160%126.21827%1.97510%
ghc-7.6.1122.12829%120.21288%1.87989%
ghc-7.8.1109.02917%107.75902%1.99619%
ghc-7.10.1104.21783%104.81700%2.23983%
ghc-8.0.1104.34918%104.43907%2.21876%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1109.83968%112.72436%0.88099%
ghc-7.2.1116.10958%112.81489%0.67429%
ghc-7.4.1127.84214%129.24784%0.96026%
ghc-7.6.1118.63811%115.62745%0.84445%
ghc-7.8.1106.01391%106.21090%1.21625%
ghc-7.10.1104.23525%104.81081%0.72870%
ghc-8.0.1105.58431%104.97935%0.86965%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.28976%114.03504%0.36892%
ghc-7.2.1115.95478%114.85435%0.35736%
ghc-7.4.1128.58979%134.21589%0.58128%
ghc-7.6.1118.68778%116.78022%0.51347%
ghc-7.8.1107.45274%108.08798%0.36433%
ghc-7.10.1107.48705%108.74022%0.57695%
ghc-8.0.1106.95776%107.72883%0.41262%
csv

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1106.50686%111.69497%1.13169%
ghc-7.2.1110.35800%109.56987%1.24419%
ghc-7.4.1118.74475%129.39811%1.39061%
ghc-7.6.1114.02687%112.65894%1.30159%
ghc-7.8.1107.21290%106.81824%1.31348%
ghc-7.10.1106.20861%106.31197%1.21093%
ghc-8.0.1107.16596%105.83235%1.38497%
csv

Summary for input no. 0

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 0.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1106.25021%108.88358%2.08254%
ghc-7.2.1108.69161%107.69216%2.41303%
ghc-7.4.1115.40862%125.11117%2.50284%
ghc-7.6.1115.72174%114.72287%2.40396%
ghc-7.8.1107.80956%106.03811%2.45377%
ghc-7.10.1104.99040%104.79168%2.73954%
ghc-8.0.1105.74361%104.75255%2.64914%
csv

Summary for input no. 1

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 1.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1107.34704%112.76819%1.27496%
ghc-7.2.1111.92317%108.88902%1.11048%
ghc-7.4.1120.75199%129.34061%1.18188%
ghc-7.6.1112.82895%109.62645%1.22320%
ghc-7.8.1105.96919%106.14623%1.64998%
ghc-7.10.1105.37981%105.00578%0.80938%
ghc-8.0.1107.59609%105.07168%1.21510%
csv

Summary for input no. 2

Mean score per implementation. Excludes all categories with less than 3 inputs. The plot shows the distribution of mean scores per category per implementation for input no. 2.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1106.03809%113.95442%0.42032%
ghc-7.2.1111.20576%112.05368%0.54113%
ghc-7.4.1121.62272%134.49395%0.70669%
ghc-7.6.1112.83398%111.96919%0.57651%
ghc-7.8.1107.61461%108.22914%0.42820%
ghc-7.10.1108.82266%108.38724%0.56439%
ghc-8.0.1108.79446%107.32042%0.62501%
csv

binarytrees

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1103.69831%103.72396%1.65583%
ghc-7.2.1106.11830%105.15943%1.28968%
ghc-7.4.1106.33483%106.14340%2.95182%
ghc-7.6.1108.01037%104.16968%1.85313%
ghc-7.8.1105.09183%103.66337%2.32185%
ghc-7.10.1111.24618%105.69735%2.23560%
ghc-8.0.1107.81631%103.42794%1.92705%
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 36.20066102.28534%100.00000%10.03459%10.26391%10.03459%34.85335
ghc-7.2.1 20 35.61208100.62231%98.37413%4.53638%4.56461%4.46262%35.46328
ghc-7.4.1 20 35.39184100.00000%97.76572%12.59612%12.59612%12.31469%33.96606
ghc-7.6.1 20 36.87765104.19818%101.87010%10.44987%10.88858%10.64529%36.31278
ghc-7.8.1 20 37.76556106.70698%104.32285%9.24146%9.86129%9.64096%36.27919
ghc-7.10.1 20 39.07529110.40763%107.94081%8.35636%9.22606%9.01992%39.49754
ghc-8.0.1 20 37.26539105.29375%102.94119%7.80938%8.22279%8.03907%37.27534
csv

Input: '16'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201167.27827109.01886%100.00000%1.57979%1.72227%1.57979%1161.52058
ghc-7.2.1 201215.58678113.53067%104.13856%0.94060%1.06786%0.97952%1216.41146
ghc-7.4.1 201234.27302115.27588%105.73940%1.82849%2.10781%1.93344%1236.54016
ghc-7.6.1 201224.48991114.36218%104.90129%0.94036%1.07541%0.98645%1226.54413
ghc-7.8.1 201070.71227100.00000%91.72725%1.95901%1.95901%1.79695%1062.86615
ghc-7.10.1 201117.54812104.37427%95.73965%1.42076%1.48291%1.36023%1114.32825
ghc-8.0.1 201127.03694105.26049%96.55255%1.35905%1.43055%1.31220%1130.47068
csv

Input: '20'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2033371.08566100.00000%100.00000%0.28638%0.28638%0.28638%33354.01282
ghc-7.2.1 2034908.69846104.60762%104.60762%0.50273%0.52590%0.52590%34893.77406
ghc-7.4.1 2034806.35150104.30093%104.30093%1.11671%1.16473%1.16473%34751.99800
ghc-7.6.1 2035287.76572105.74354%105.74354%0.64761%0.68481%0.68481%35266.02503
ghc-7.8.1 2036298.13930108.77123%108.77123%0.69140%0.75204%0.75204%36336.50885
ghc-7.10.1 2039868.79516119.47108%119.47108%0.94112%1.12436%1.12436%39772.87905
ghc-8.0.1 2037735.98821113.07989%113.07989%0.67425%0.76244%0.76244%37769.74465
csv

fannkuchredux

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.32931%108.36509%0.40694%
ghc-7.2.1124.40706%114.81352%0.62384%
ghc-7.4.1124.13898%121.52481%0.47789%
ghc-7.6.1131.06721%119.03797%0.74099%
ghc-7.8.1115.31864%100.96700%0.49040%
ghc-7.10.1105.58269%103.82149%0.35163%
ghc-8.0.1106.44080%106.45325%0.54564%
csv

Mean scores per input

Mean scores for input '10'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.99501%108.99501%1.07902%
ghc-7.2.1125.02590%114.72053%0.98354%
ghc-7.4.1124.88147%120.91520%1.33633%
ghc-7.6.1132.63024%118.42141%1.26997%
ghc-7.8.1115.28386%101.55921%0.93230%
ghc-7.10.1105.24661%102.99990%0.90927%
ghc-8.0.1106.35398%106.35398%0.81280%
csv

Mean scores for input '11'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1108.36578%108.36578%0.40678%
ghc-7.2.1125.88806%113.71511%0.59406%
ghc-7.4.1123.84573%122.10378%0.39040%
ghc-7.6.1131.21376%119.21005%0.96879%
ghc-7.8.1115.57246%100.14777%0.62852%
ghc-7.10.1105.66410%103.89621%0.19620%
ghc-8.0.1106.82917%106.82917%0.61206%
csv

Mean scores for input '12'

The plot shows the distribution of mean scores per program for each implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1107.63145%107.63145%0.15353%
ghc-7.2.1122.33495%115.76168%0.41553%
ghc-7.4.1123.69310%121.52656%0.20920%
ghc-7.6.1129.37793%119.36620%0.33069%
ghc-7.8.1115.10008%100.52825%0.20126%
ghc-7.10.1105.83824%104.41701%0.24371%
ghc-8.0.1106.14041%106.14041%0.32654%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1117.35240%101.03432%0.15795%
ghc-7.2.1142.73147%100.56144%0.14917%
ghc-7.4.1150.84650%100.25459%0.21292%
ghc-7.6.1155.96801%100.72148%0.27550%
ghc-7.8.1115.76848%100.91616%0.22233%
ghc-7.10.1101.74744%100.33109%0.24926%
ghc-8.0.1100.00000%100.00000%0.17294%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202431.19386118.79913%100.00000%0.36315%0.43142%0.36315%2429.00734
ghc-7.2.1 202935.26611143.43038%120.73353%0.32929%0.47230%0.39757%2931.91303
ghc-7.4.1 203090.19043151.00069%127.10588%0.48438%0.73142%0.61567%3091.51116
ghc-7.6.1 203214.24606157.06260%132.20855%0.35808%0.56242%0.47342%3213.10787
ghc-7.8.1 202396.04050117.08138%98.55407%0.44463%0.52058%0.43820%2394.41023
ghc-7.10.1 202091.11370102.18128%86.01180%0.43305%0.44249%0.37247%2088.34470
ghc-8.0.1 202046.47444100.00000%84.17570%0.35694%0.35694%0.30045%2045.87104
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2029805.13260117.43143%100.00000%0.11156%0.13101%0.11156%29791.61672
ghc-7.2.1 1536333.68188143.15374%121.90411%0.10364%0.14836%0.12634%36329.16055
ghc-7.4.1 2038381.04998151.22032%128.77329%0.18874%0.28541%0.24304%38351.20630
ghc-7.6.1 2039700.77472156.42000%133.20113%0.26485%0.41428%0.35278%39705.12273
ghc-7.8.1 2029376.65639115.74325%98.56241%0.21614%0.25016%0.21303%29377.21130
ghc-7.10.1 2025812.76009101.70159%86.60508%0.13585%0.13817%0.11766%25808.17040
ghc-8.0.1 2025380.88166100.00000%85.15608%0.18982%0.18982%0.16165%25371.96889
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 15399827.75930115.84528%100.00000%0.09726%0.11267%0.09726%400001.73880
ghc-7.2.1 15488776.09017141.61699%122.24666%0.09726%0.13773%0.11889%488895.68591
ghc-7.4.1 15518813.50208150.31997%129.75925%0.10559%0.15873%0.13702%518701.84984
ghc-7.6.1 15533010.96260154.43352%133.31014%0.22050%0.34052%0.29394%532818.53136
ghc-7.8.1 15395168.27336114.49525%98.83463%0.11435%0.13093%0.11302%395139.82475
ghc-7.10.1 15349837.17797101.36111%87.49697%0.26323%0.26681%0.23032%349601.14388
ghc-8.0.1 15345139.43447100.00000%86.32203%0.07634%0.07634%0.06589%345088.61839
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1100.00000%100.00000%1.04844%
ghc-7.2.1108.43521%101.94896%2.60903%
ghc-7.4.1102.16005%100.78735%1.07260%
ghc-7.6.1110.14190%101.34780%1.99297%
ghc-7.8.1114.87054%100.85059%1.08169%
ghc-7.10.1109.56251%100.80242%0.49605%
ghc-8.0.1113.29644%100.54204%1.72154%
csv

Input: '10'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 315.87206100.00000%100.00000%3.20601%3.20601%3.20601%312.19359
ghc-7.2.1 20 344.24692108.98302%108.98302%2.93765%3.20154%3.20154%345.62014
ghc-7.4.1 20 326.23332103.28021%103.28021%3.68673%3.80766%3.80766%327.84209
ghc-7.6.1 20 353.77208111.99854%111.99854%4.50402%5.04444%5.04444%354.27927
ghc-7.8.1 20 358.55880113.51394%113.51394%1.95482%2.21899%2.21899%358.58763
ghc-7.10.1 20 342.41763108.40390%108.40390%1.90921%2.06966%2.06966%341.22556
ghc-8.0.1 20 357.28824113.11170%113.11170%1.85088%2.09356%2.09356%359.27518
csv

Input: '11'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 203763.14932100.00000%100.00000%1.48320%1.48320%1.48320%3742.51215
ghc-7.2.1 204165.98633110.70478%110.70478%3.40526%3.76978%3.76978%4226.04824
ghc-7.4.1 203816.83482101.42661%101.42661%0.80754%0.81906%0.81906%3811.72234
ghc-7.6.1 204142.07500110.06938%110.06938%3.54372%3.90055%3.90055%4056.44997
ghc-7.8.1 204342.74690115.40193%115.40193%1.82775%2.10926%2.10926%4310.89535
ghc-7.10.1 204131.22300109.78100%109.78100%0.28334%0.31106%0.31106%4130.81925
ghc-8.0.1 204294.68336114.12471%114.12471%1.97349%2.25224%2.25224%4337.93303
csv

Input: '12'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2050602.08287100.00000%100.00000%0.24236%0.24236%0.24236%50572.65383
ghc-7.2.1 2053475.40772105.67827%105.67827%1.77535%1.87616%1.87616%52961.15168
ghc-7.4.1 2051504.20308101.78277%101.78277%0.41449%0.42188%0.42188%51489.50040
ghc-7.6.1 2054846.28520108.38741%108.38741%0.49595%0.53755%0.53755%54791.00900
ghc-7.8.1 2058550.70589115.70809%115.70809%0.35423%0.40987%0.40987%58597.60080
ghc-7.10.1 2055921.94299110.51312%110.51312%0.22563%0.24935%0.24935%55909.98706
ghc-8.0.1 2057007.23171112.65788%112.65788%1.39682%1.57363%1.57363%56607.78377
csv

fasta

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1100.00000%100.00000%0.67866%
ghc-7.2.1109.43856%100.76408%0.72067%
ghc-7.4.1108.09328%100.50351%0.70814%
ghc-7.6.1109.50844%100.11992%1.41708%
ghc-7.8.1103.67028%100.13668%0.71040%
ghc-7.10.1106.87410%100.55754%0.96417%
ghc-8.0.1106.73751%100.74113%0.70379%
csv

Input: '250000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 91.27777100.00000%100.00000%2.01324%2.01324%2.01324%91.57420
ghc-7.2.1 20 100.93530110.58038%110.58038%3.30667%3.65653%3.65653%101.04386
ghc-7.4.1 20 98.21063107.59535%107.59535%3.12203%3.35916%3.35916%96.80112
ghc-7.6.1 20 99.97244109.52551%109.52551%3.44869%3.77719%3.77719%98.90641
ghc-7.8.1 20 94.52937103.56231%103.56231%3.28755%3.40466%3.40466%95.03523
ghc-7.10.1 20 98.23191107.61865%107.61865%4.32257%4.65189%4.65189%98.11557
ghc-8.0.1 20 98.44565107.85283%107.85283%3.79052%4.08818%4.08818%98.23122
csv

Input: '2500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 859.72972100.00000%100.00000%0.78522%0.78522%0.78522%859.99299
ghc-7.2.1 20 933.69500108.60332%108.60332%0.53409%0.58004%0.58004%935.11088
ghc-7.4.1 20 935.82205108.85073%108.85073%0.69230%0.75357%0.75357%933.99837
ghc-7.6.1 20 942.78015109.66006%109.66006%3.17333%3.47987%3.47987%934.48460
ghc-7.8.1 20 890.49351103.57831%103.57831%0.53651%0.55570%0.55570%889.02159
ghc-7.10.1 20 912.72182106.16381%106.16381%0.51762%0.54953%0.54953%912.94257
ghc-8.0.1 20 912.11080106.09274%106.09274%0.64093%0.67998%0.67998%909.68639
csv

Input: '25000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 208481.06676100.00000%100.00000%0.19773%0.19773%0.19773%8479.72851
ghc-7.2.1 209256.36307109.14150%109.14150%0.21194%0.23131%0.23131%9259.46861
ghc-7.4.1 209145.80105107.83786%107.83786%0.16429%0.17717%0.17717%9146.63034
ghc-7.6.1 209273.19641109.33998%109.33998%0.26003%0.28431%0.28431%9273.33791
ghc-7.8.1 208809.32814103.87052%103.87052%0.20326%0.21113%0.21113%8800.69561
ghc-7.10.1 159061.57847106.84480%106.84480%0.40059%0.42801%0.42801%9051.57530
ghc-8.0.1 209013.31309106.27570%106.27570%0.14349%0.15249%0.15249%9013.75720
csv

mandelbrot

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1102.82121%101.80931%2.01078%
ghc-7.2.1103.56905%102.29185%2.05825%
ghc-7.4.1101.37577%100.99401%1.04959%
ghc-7.6.1102.89144%100.19563%0.61937%
ghc-7.8.1100.75576%100.76730%1.50605%
ghc-7.10.1100.12762%100.18053%0.64275%
ghc-8.0.1100.57807%100.32832%0.64143%
csv

Input: '1000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 160.42612100.53764%100.00000%1.75470%1.76413%1.75470%160.79570
ghc-7.2.1 20 162.42034101.78739%101.24307%1.67803%1.70802%1.69889%163.14165
ghc-7.4.1 20 159.56823100.00000%99.46524%1.89012%1.89012%1.88001%158.90612
ghc-7.6.1 20 164.59941103.15300%102.60138%2.39831%2.47392%2.46070%163.04342
ghc-7.8.1 20 160.13111100.35275%99.81610%1.61399%1.61969%1.61103%159.70962
ghc-7.10.1 20 160.17993100.38335%99.84654%1.86841%1.87558%1.86555%159.44228
ghc-8.0.1 20 159.79206100.14027%99.60476%1.89539%1.89805%1.88790%159.27346
csv

Input: '4000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202550.70415105.04810%100.00000%4.47875%4.70484%4.47875%2479.32123
ghc-7.2.1 202596.58612106.93770%101.79880%5.31514%5.68389%5.41075%2560.36736
ghc-7.4.1 202471.99218101.80643%96.91411%0.80919%0.82381%0.78422%2464.50646
ghc-7.6.1 202497.53006102.85818%97.91532%0.19427%0.19982%0.19022%2496.10362
ghc-7.8.1 202472.79424101.83946%96.94555%4.90941%4.99972%4.75945%2434.95444
ghc-7.10.1 202428.12966100.00000%95.19448%0.34502%0.34502%0.32844%2429.03788
ghc-8.0.1 202450.82369100.93463%96.08420%0.69584%0.70234%0.66859%2447.96004
csv

Input: '16000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2039853.15825102.92747%100.00000%1.03451%1.06480%1.03451%39897.39195
ghc-7.2.1 2039518.14966102.06225%99.15939%0.97764%0.99781%0.96943%39376.17946
ghc-7.4.1 2039624.03961102.33573%99.42509%0.75600%0.77365%0.75165%39554.91924
ghc-7.6.1 2039751.03939102.66373%99.74376%0.50995%0.52354%0.50865%39752.85069
ghc-7.8.1 2038752.13844100.08390%97.23731%0.43111%0.43147%0.41920%38725.08843
ghc-7.10.1 2038719.65351100.00000%97.15580%0.41191%0.41191%0.40020%38702.94552
ghc-8.0.1 2038975.56582100.66094%97.79794%0.20009%0.20142%0.19569%38993.90852
csv

meteor

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1116.78402%108.64901%0.61942%
ghc-7.2.1113.46517%106.35086%0.85065%
ghc-7.4.1110.52331%107.99125%0.65276%
ghc-7.6.1128.74043%123.07038%0.72862%
ghc-7.8.1110.86342%108.78452%0.82353%
ghc-7.10.1100.84926%100.84995%0.78100%
ghc-8.0.1101.59002%101.11321%0.96624%
csv

Program: '1'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1110.52791%100.00000%0.57403%
ghc-7.2.1111.40301%100.00000%2.66817%
ghc-7.4.1113.44337%100.00000%0.60866%
ghc-7.6.1136.10860%100.00000%0.60423%
ghc-7.8.1111.88503%100.00000%0.70272%
ghc-7.10.1100.00000%100.00000%0.67073%
ghc-8.0.1101.11977%100.00000%0.70734%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 767.67932110.52791%100.00000%0.57403%0.63446%0.57403%767.44429
ghc-7.2.1 20 773.75738111.40301%100.79174%2.66817%2.97242%2.68929%770.24378
ghc-7.4.1 20 787.92885113.44337%102.63776%0.60866%0.69048%0.62472%787.40665
ghc-7.6.1 20 945.35185136.10860%123.14411%0.60423%0.82241%0.74407%945.93377
ghc-7.8.1 20 777.10534111.88503%101.22786%0.70272%0.78623%0.71135%775.23167
ghc-7.10.1 20 694.55701100.00000%90.47489%0.67073%0.67073%0.60684%692.46854
ghc-8.0.1 20 702.33445101.11977%91.48800%0.70734%0.71526%0.64713%701.74705
csv

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1115.07278%100.00000%0.55880%
ghc-7.2.1115.96310%100.00000%0.84532%
ghc-7.4.1116.08782%100.00000%0.61556%
ghc-7.6.1156.18912%100.00000%0.68540%
ghc-7.8.1119.61876%100.00000%0.93585%
ghc-7.10.1101.69598%100.00000%1.39884%
ghc-8.0.1100.00000%100.00000%1.38922%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 555.63597115.07278%100.00000%0.55880%0.64303%0.55880%554.70634
ghc-7.2.1 20 559.93494115.96310%100.77370%0.84532%0.98026%0.85186%559.87327
ghc-7.4.1 20 560.53717116.08782%100.88209%0.61556%0.71459%0.62099%560.37564
ghc-7.6.1 20 754.16878156.18912%135.73073%0.68540%1.07053%0.93030%754.94005
ghc-7.8.1 20 577.58653119.61876%103.95053%0.93585%1.11945%0.97282%576.46288
ghc-7.10.1 20 491.04528101.69598%88.37536%1.39884%1.42256%1.23623%490.82792
ghc-8.0.1 20 482.85614100.00000%86.90153%1.38922%1.38922%1.20726%482.17158
csv

Program: '3'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1128.16177%100.00000%0.82635%
ghc-7.2.1119.85242%100.00000%0.76505%
ghc-7.4.1122.50026%100.00000%1.11886%
ghc-7.6.1163.27358%100.00000%0.90364%
ghc-7.8.1123.24776%100.00000%1.16807%
ghc-7.10.1100.00000%100.00000%1.07386%
ghc-8.0.1101.30392%100.00000%1.24197%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 502.09530128.16177%100.00000%0.82635%1.05906%0.82635%500.91612
ghc-7.2.1 20 469.54202119.85242%93.51651%0.76505%0.91693%0.71544%468.79844
ghc-7.4.1 20 479.91539122.50026%95.58253%1.11886%1.37061%1.06944%479.72938
ghc-7.6.1 20 639.65173163.27358%127.39648%0.90364%1.47540%1.15120%637.95827
ghc-7.8.1 20 482.84384123.24776%96.16578%1.16807%1.43962%1.12329%481.15285
ghc-7.10.1 20 391.76683100.00000%78.02639%1.07386%1.07386%0.83790%391.17030
ghc-8.0.1 20 396.87517101.30392%79.04379%1.24197%1.25816%0.98170%397.05996
csv

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1128.28020%100.00000%0.32302%
ghc-7.2.1119.63090%100.00000%0.29485%
ghc-7.4.1101.40363%100.00000%0.33112%
ghc-7.6.1101.88786%100.00000%0.62823%
ghc-7.8.1100.00000%100.00000%0.40693%
ghc-7.10.1100.53769%100.00000%0.32556%
ghc-8.0.1102.19500%100.00000%0.58835%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201695.07577128.28020%100.00000%0.32302%0.41437%0.32302%1694.03506
ghc-7.2.1 201580.78524119.63090%93.25750%0.29485%0.35274%0.27497%1581.02647
ghc-7.4.1 201339.93280101.40363%79.04855%0.33112%0.33577%0.26175%1339.31531
ghc-7.6.1 201346.33129101.88786%79.42602%0.62823%0.64009%0.49898%1343.57807
ghc-7.8.1 201321.38538100.00000%77.95435%0.40693%0.40693%0.31722%1320.88545
ghc-7.10.1 201328.49034100.53769%78.37351%0.32556%0.32731%0.25516%1328.43755
ghc-8.0.1 201350.38979102.19500%79.66545%0.58835%0.60127%0.46871%1350.11800
csv

Program: '5'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1103.88539%100.00000%1.06499%
ghc-7.2.1101.53269%100.00000%0.87543%
ghc-7.4.1100.81219%100.00000%0.85382%
ghc-7.6.1100.00000%100.00000%0.87347%
ghc-7.8.1101.52923%100.00000%1.21176%
ghc-7.10.1102.03071%100.00000%0.88583%
ghc-8.0.1103.36256%100.00000%1.17296%
csv

Input: '2098'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 476.24369103.88539%100.00000%1.06499%1.10637%1.06499%475.12693
ghc-7.2.1 20 465.45818101.53269%97.73530%0.87543%0.88885%0.85561%464.16712
ghc-7.4.1 20 462.15516100.81219%97.04174%0.85382%0.86075%0.82856%461.00225
ghc-7.6.1 20 458.43183100.00000%96.25993%0.87347%0.87347%0.84080%457.45233
ghc-7.8.1 20 465.44232101.52923%97.73197%1.21176%1.23029%1.18428%464.83213
ghc-7.10.1 20 467.74124102.03071%98.21469%0.88583%0.90382%0.87001%467.67641
ghc-8.0.1 20 473.84685103.36256%99.49672%1.17296%1.21240%1.16706%473.38042
csv

nbody

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1140.05540%104.39289%4.89582%
ghc-7.2.1135.08839%104.08014%4.73024%
ghc-7.4.1132.43620%103.95060%6.53814%
ghc-7.6.1132.32743%103.65914%4.53373%
ghc-7.8.1105.39281%102.65106%5.73485%
ghc-7.10.1100.00000%100.00000%9.33225%
ghc-8.0.1107.84853%102.62777%13.43539%
csv

Input: '500000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 247.26146131.97480%100.00000%4.28697%5.65772%4.28697%250.14663
ghc-7.2.1 20 241.18585128.73197%97.54284%6.12874%7.88965%5.97815%249.92065
ghc-7.4.1 20 237.66459126.85251%96.11873%10.40079%13.19367%9.99711%227.30083
ghc-7.6.1 20 236.01201125.97045%95.45038%5.58017%7.02937%5.32630%242.55714
ghc-7.8.1 20 203.33869108.53120%82.23630%13.93456%15.12334%11.45926%224.46873
ghc-7.10.1 20 187.35506100.00000%75.77204%16.23312%16.23312%12.30017%168.81993
ghc-8.0.1 20 205.60613109.74143%83.15333%14.91921%16.37256%12.40582%225.91291
csv

Input: '5000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 202394.39002142.68359%100.00000%5.25409%7.49672%5.25409%2466.69975
ghc-7.2.1 202263.47254134.88211%94.53232%5.16181%6.96236%4.87958%2193.75928
ghc-7.4.1 202205.17985131.40840%92.09777%6.89397%9.05925%6.34919%2140.60477
ghc-7.6.1 202251.20393134.15101%94.01993%6.04626%8.11113%5.68469%2199.34902
ghc-7.8.1 201777.21077105.90539%74.22395%14.49399%15.34992%10.75801%1607.52109
ghc-7.10.1 201678.11175100.00000%70.08515%12.05750%12.05750%8.45052%1551.09309
ghc-8.0.1 201845.02728109.94663%77.05626%17.38712%19.11655%13.39786%1576.66104
csv

Input: '50000000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 2022752.42366145.89304%100.00000%5.20988%7.60085%5.20988%22285.94547
ghc-7.2.1 2022141.46612141.97546%97.31476%3.34562%4.74995%3.25578%21810.65860
ghc-7.4.1 2021731.56439139.34709%95.51318%3.89786%5.43156%3.72297%21260.97830
ghc-7.6.1 2021383.56522137.11565%93.98368%2.76205%3.78721%2.59588%21165.50254
ghc-7.8.1 2015883.76295101.84983%69.81130%0.93387%0.95114%0.65194%15806.43472
ghc-7.10.1 2015595.27691100.00000%68.54337%4.15240%4.15240%2.84620%15344.47656
ghc-8.0.1 2016213.74289103.96573%71.26161%9.34927%9.72003%6.66244%15637.57100
csv

pidigits

Program: '4'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1101.04931%100.84110%0.79061%
ghc-7.2.1100.21644%100.19685%0.82340%
ghc-7.4.1101.37964%101.04311%1.06782%
ghc-7.6.1101.20275%100.89507%0.97733%
ghc-7.8.1100.00000%100.00000%1.00831%
ghc-7.10.1103.43397%101.06855%1.10791%
ghc-8.0.1102.43546%100.77258%1.00302%
csv

Input: '2000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 77.04672102.25063%100.00000%4.25092%4.34659%4.25092%76.03899
ghc-7.2.1 20 75.71761100.48674%98.27493%3.83679%3.85546%3.77060%74.64561
ghc-7.4.1 20 77.51842102.87663%100.61222%3.82396%3.93396%3.84737%77.70187
ghc-7.6.1 20 77.22228102.48362%100.22786%4.43682%4.54701%4.44693%75.17787
ghc-7.8.1 20 75.35085100.00000%97.79891%4.44502%4.44502%4.34718%75.50679
ghc-7.10.1 20 79.11616104.99704%102.68596%5.54540%5.82251%5.69435%78.19914
ghc-8.0.1 20 78.03062103.55639%101.27702%4.60960%4.77354%4.66847%77.72996
csv

Input: '6000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 15 623.21033100.52327%100.00000%0.54151%0.54434%0.54151%623.55578
ghc-7.2.1 20 620.09988100.02155%99.50090%0.63412%0.63426%0.63096%618.83594
ghc-7.4.1 20 624.31356100.70122%100.17702%0.65636%0.66096%0.65752%624.05662
ghc-7.6.1 20 623.06223100.49938%99.97623%0.55549%0.55827%0.55536%622.69254
ghc-7.8.1 20 619.96625100.00000%99.47946%0.66065%0.66065%0.65721%618.83870
ghc-7.10.1 15 635.91847102.57308%102.03914%0.69032%0.70809%0.70440%634.93058
ghc-8.0.1 15 631.68133101.88963%101.35925%0.53708%0.54723%0.54438%631.44371
csv

Input: '10000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 201756.07096100.38467%100.00000%0.21468%0.21551%0.21468%1755.45402
ghc-7.2.1 151751.81910100.14162%99.75788%0.22946%0.22978%0.22890%1751.14757
ghc-7.4.1 151759.44414100.57750%100.19209%0.48512%0.48792%0.48605%1755.85387
ghc-7.6.1 151760.49095100.63734%100.25170%0.37877%0.38119%0.37973%1758.15429
ghc-7.8.1 151749.34169100.00000%99.61680%0.34909%0.34909%0.34775%1747.26031
ghc-7.10.1 151797.43728102.74935%102.35562%0.35524%0.36501%0.36361%1796.48684
ghc-8.0.1 151782.04526101.86948%101.47911%0.40759%0.41521%0.41362%1780.77134
csv

spectralnorm

Program: '2'

The following plot shows the rel means (means / min means) per input distribution for every implementation.

img tex tex_standalone

mean score (gmean(mean / best mean))mean score std (gmean std(mean / best mean))mean rel std (gmean(std / mean))
ghc-7.0.1100.00000%100.00000%0.41970%
ghc-7.2.1113.82518%103.94922%0.23547%
ghc-7.4.1225.03082%104.58101%1.24459%
ghc-7.6.1123.82024%104.25387%0.80491%
ghc-7.8.1118.59762%104.11526%0.61499%
ghc-7.10.1118.54853%103.63540%0.69206%
ghc-8.0.1118.32304%103.98649%0.63974%
csv

Input: '500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 20 151.05247100.00000%100.00000%2.16871%2.16871%2.16871%150.44508
ghc-7.2.1 20 165.97599109.87970%109.87970%1.94951%2.14211%2.14211%164.43145
ghc-7.4.1 15 328.85562217.70953%217.70953%2.76650%6.02293%6.02293%327.06099
ghc-7.6.1 15 179.78282119.02012%119.02012%1.84879%2.20044%2.20044%179.28403
ghc-7.8.1 20 172.58790114.25692%114.25692%1.67243%1.91087%1.91087%171.57760
ghc-7.10.1 20 174.50940115.52900%115.52900%2.46996%2.85352%2.85352%174.41953
ghc-8.0.1 15 172.81121114.40476%114.40476%2.06806%2.36596%2.36596%172.80909
csv

Input: '3000'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 205174.41491100.00000%100.00000%0.23358%0.23358%0.23358%5171.07341
ghc-7.2.1 155780.95850111.72197%111.72197%0.06997%0.07818%0.07818%5782.47796
ghc-7.4.1 1511297.08810218.32590%218.32590%0.86061%1.87894%1.87894%11270.54340
ghc-7.6.1 156292.32803121.60463%121.60463%0.59380%0.72209%0.72209%6274.78677
ghc-7.8.1 156022.85608116.39685%116.39685%0.36866%0.42911%0.42911%6014.53037
ghc-7.10.1 205984.56424115.65683%115.65683%0.38025%0.43978%0.43978%5974.17594
ghc-8.0.1 205994.51166115.84907%115.84907%0.35014%0.40563%0.40563%5986.98572
csv

Input: '5500'

The following plot shows the actual distribution of the measurements for each implementation.

img tex tex_standalone

nmeanmean / best meanmean / mean of first implstd / meanstd / best meanstd / mean of first implmedian
ghc-7.0.1 1516100.62758100.00000%100.00000%0.14594%0.14594%0.14594%16094.96154
ghc-7.2.1 2019342.01767120.13207%120.13207%0.09570%0.11497%0.11497%19335.72481
ghc-7.4.1 1538599.89397239.74155%239.74155%0.80973%1.94126%1.94126%38530.39270
ghc-7.6.1 2021117.72998131.16091%131.16091%0.47503%0.62305%0.62305%21075.62582
ghc-7.8.1 1520195.14288125.43078%125.43078%0.37725%0.47319%0.47319%20153.55308
ghc-7.10.1 2020075.62455124.68846%124.68846%0.35292%0.44005%0.44005%20044.68894
ghc-8.0.1 1520123.99384124.98888%124.98888%0.36159%0.45195%0.45195%20078.72571
csv