Summary
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep '[ab]*' -R . vs. grep 'a*' -R . is too low.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The mean difference per standard deviation of etime (0.200) of grep '[ab]*' -R . vs. grep 'a*' -R . is less than 2.000.
Summary regarding etime
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(6).(000)m |
|---|
std dev | (4).(899)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
median | 3(0).(000)m |
|---|
min | 20.000m |
|---|
max | 30.000m |
|---|
n | 10 |
|---|
mean ci (lower bound) | 25.895m |
|---|
mean ci (upper bound) | 26.105m |
|---|
std dev ci (lower bound) | 9.772m |
|---|
std dev ci (upper bound) | nan |
|---|
normality probability | 1.68% |
|---|
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | |
mean | 2(5).(000)m |
|---|
std dev | (5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
median | 2(5).(000)m |
|---|
min | 20.000m |
|---|
max | 30.000m |
|---|
n | 10 |
|---|
mean ci (lower bound) | 24.893m |
|---|
mean ci (upper bound) | 25.107m |
|---|
std dev ci (lower bound) | 9.973m |
|---|
std dev ci (upper bound) | nan |
|---|
normality probability | 0.22% |
|---|
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep '[ab]*' -R . vs. grep 'a*' -R . is too low.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The mean difference per standard deviation of etime (0.200) of grep '[ab]*' -R . vs. grep 'a*' -R . is less than 2.000.
⍗ | Difference of means | ... per left mean | ... per max std dev | t test |
| (1).(000)m | (3).(846)% | (20).(000)% | 67.33% |
|---|
The geometric mean of the values of the properties of grep '[ab]*' -R . relative to the values of
grep 'a*' -R . is 1.04e+00
(geometric standard deviation is 1.00e+00)
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep '[ab]*' -R . vs. grep 'a*' -R . is too low.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The mean difference per standard deviation of etime (0.200) of grep '[ab]*' -R . vs. grep 'a*' -R . is less than 2.000.
⍗ | grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime) |
Difference of means | (1).(000)m |
|---|
... per left mean | 3.846% |
|---|
... per max std dev | 20.000% |
|---|
... ci (lower bound) | nan |
|---|
... ci (upper bound) | nan |
|---|
t | 67.33% |
|---|
min n | 10 |
|---|
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
This color means that there are severe warnings related to the corresponding data set (2 severe warning(s) and 0 warning(s)).
The simple arithmetical mean
\[ \frac{1}{n}\sum_{i=1}^{n} a_i. \]
The standard deviation relative to the mean is a measure of how big the relative variation
of data is. A small value is considered neccessary for a benchmark to be useful.
Or to quote
Gernot Heiser:
Always do several runs, and check the standard deviation. Watch out for abnormal variance.
In the sort of measurements we do, standard deviations are normally
expected to be less than 0.1%. If you see >1% this should ring alarm bells.
Standard error mean:
\[ \sigma(\overline{X}) = \frac{\sigma}{\sqrt{n}} \]
Put simply, the standard error of the sample is an estimate of how far the sample mean is
likely to be from the population mean, whereas the standard deviation of the sample is the
degree to which individuals within the sample differ from the sample mean.
(wikipedia)
The minimum value. It's a bad sign if the maximum
is far lower than the mean and you can't explain it.
The number of valid runs
or statistically spoken: the sample size.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(6).(000)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)"
'Difference of means','(1).(000)m'
'... per left mean',0.03846
'... per max std dev',0.2
't',0.6733
'min n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
This color means that there are severe warnings related to the corresponding data set (6 severe warning(s) and 0 warning(s)).
Difference between the mean of the left and the mean of the right.
It's the absolute difference and is often less important that the relative differences.
The mean difference relative to the left mean
\begin{align}
& \frac{ \overline{\text{grep '[ab]*' -R .}} - \overline{\text{grep 'a*' -R .}}}{ \overline{\text{grep '[ab]*' -R .}} } \\
&= \frac{ 0.001000 }{ 0.026000}
\end{align}
gives a number that helps to talk about the practical significance of the mean difference.
The mean difference relative to the maximum standard deviation:
\begin{align}
&\frac{
\overline{
\text{grep '[ab]*' -R .}
}
- \overline{\text{grep 'a*' -R .}}}{
\text{max}(\sigma_\text{grep '[ab]*' -R .}, \sigma_\text{grep 'a*' -R .}) } \\
= & \frac{0.0010000000000000009}{0.004999999999999999} \end{align}
It's important because, as
Gernot Heiser points out:
- Don't believe any effect that is less than a standard deviation
- Be highly suspicious if it is less than two standard deviations
Probability that the null hypothesis is not incorrect. It's the probability that the measured
values (for a given property) come out of the same population for both benchmarked programs.
Or short: That the programs have the same characteristics for a given property.
Important note: Statistical tests can only given an probability of the null hypothesis being incorrect.
But this okay, if your aim is to see whether a specific program is better (different) than another
program in some respect.
The minimum of the number of valid runs of both.
or statistically spoken: the minimum sample size.
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
"grep '[ab]*' -R .",'2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
"grep 'a*' -R .",'2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The mean difference per standard deviation of etime (0.200) of grep '[ab]*' -R . vs. grep 'a*' -R . is less than 2.000.
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep '[ab]*' -R . vs. grep 'a*' -R . is too low.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
⍗ | grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime) |
Difference of means | (1).(000)m |
|---|
... per left mean | 3.846% |
|---|
... per max std dev | 20.000% |
|---|
t | 67.33% |
|---|
min n | 10 |
|---|
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep 'a*' -R . vs. grep '[ab]*' -R . (regarding etime)\\
\midrule Difference of means & -(1).(000)m\\
... per left mean & -4.000\%\\
... per max std dev & -20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . vs. grep '[ab]*' -R . (regarding etime)\\
\midrule Difference of means & -(1).(000)m\\
... per left mean & -4.000\%\\
... per max std dev & -20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . vs. grep '[ab]*' -R . (regarding etime)"
'Difference of means','-(1).(000)m'
'... per left mean',-0.04
'... per max std dev',-0.2
't',0.6733
'min n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The mean difference relative to the left mean
\begin{align}
& \frac{ \overline{\text{grep 'a*' -R .}} - \overline{\text{grep '[ab]*' -R .}}}{ \overline{\text{grep 'a*' -R .}} } \\
&= \frac{ -0.001000 }{ 0.025000}
\end{align}
gives a number that helps to talk about the practical significance of the mean difference.
The mean difference relative to the maximum standard deviation:
\begin{align}
&\frac{
\overline{
\text{grep 'a*' -R .}
}
- \overline{\text{grep '[ab]*' -R .}}}{
\text{max}(\sigma_\text{grep 'a*' -R .}, \sigma_\text{grep '[ab]*' -R .}) } \\
= & \frac{-0.0010000000000000009}{0.004999999999999999} \end{align}
It's important because, as
Gernot Heiser points out:
- Don't believe any effect that is less than a standard deviation
- Be highly suspicious if it is less than two standard deviations
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
"grep 'a*' -R .",'2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
"grep '[ab]*' -R .",'2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep 'a*' -R . vs. grep '[ab]*' -R . is too low.
The mean difference per standard deviation of etime (-0.200) of grep 'a*' -R . vs. grep '[ab]*' -R . is less than 2.000.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
⍗ | grep 'a*' -R . vs. grep '[ab]*' -R . (regarding etime) |
Difference of means | -(1).(000)m |
|---|
... per left mean | -4.000% |
|---|
... per max std dev | -20.000% |
|---|
t | 67.33% |
|---|
min n | 10 |
|---|
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
\begin{tabular}{lrr}\toprule
vs. & grep '[ab]*' -R . & grep 'a*' -R .\\
\midrule grep '[ab]*' -R . & & 1(04).(000)\%\\
grep 'a*' -R . & (96).(154)\% & \\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrr}\toprule
vs. & grep '[ab]*' -R . & grep 'a*' -R .\\
\midrule grep '[ab]*' -R . & & 1(04).(000)\%\\
grep 'a*' -R . & (96).(154)\% & \\
\bottomrule
\end{tabular}
\end{document}
'vs.',"grep '[ab]*' -R .","grep 'a*' -R ."
"grep '[ab]*' -R .",'','1(04).(000)%'
"grep 'a*' -R .",'(96).(154)%',''
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
This color means that there are severe warnings related to the corresponding data set (2 severe warning(s) and 0 warning(s)).
Click on the cell to get more information.
Left mean relative to the right mean:
\begin{align}
& \frac{\overline{\text{left[etime]}}}{\overline{\text{right[etime]}}} \\
&= \frac{0.0260}{0.0250}
\end{align}
The maximum standard deviation of both benchmarks relative to the mean of the right one is 20.00%.
This color means that there are severe warnings related to the corresponding data set (6 severe warning(s) and 0 warning(s)).
Click on the cell to get more information.
Left mean relative to the right mean:
\begin{align}
& \frac{\overline{\text{left[etime]}}}{\overline{\text{right[etime]}}} \\
&= \frac{0.0250}{0.0260}
\end{align}
The maximum standard deviation of both benchmarks relative to the mean of the right one is 19.23%.
This color means that there are severe warnings related to the corresponding data set (6 severe warning(s) and 0 warning(s)).
Click on the cell to get more information.
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(6).(000)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | |
mean | 2(5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . (regarding etime) & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . (regarding etime) & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . (regarding etime) & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . (regarding etime) & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
"grep '[ab]*' -R . (regarding etime)",'2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
"grep 'a*' -R . (regarding etime)",'2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(6).(000)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule etime & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule etime & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
'etime','2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
⍗ | |
mean | 2(6).(000)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
std dev & (4).(899)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
median & 3(0).(000)m\\
min & 20.000m\\
max & 30.000m\\
n & 10\\
mean ci (lower bound) & 25.895m\\
mean ci (upper bound) & 26.105m\\
std dev ci (lower bound) & 9.772m\\
std dev ci (upper bound) & nan\\
normality probability & 1.68\%\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
std dev & (4).(899)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
median & 3(0).(000)m\\
min & 20.000m\\
max & 30.000m\\
n & 10\\
mean ci (lower bound) & 25.895m\\
mean ci (upper bound) & 26.105m\\
std dev ci (lower bound) & 9.772m\\
std dev ci (upper bound) & nan\\
normality probability & 1.68\%\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'std dev','(4).(899)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'median','3(0).(000)m'
'min','20.000m'
'max','30.000m'
'n',10
'mean ci (lower bound)','25.895m'
'mean ci (upper bound)','26.105m'
'std dev ci (lower bound)','9.772m'
'std dev ci (upper bound)',nan
'normality probability',0.0168
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The sample standard deviation
\[ \sigma_N = \sqrt{\frac{1}{N} \sum_{i=1}^N (x_i - \overline{x})^2} \]
In statistics, the standard deviation is a measure that is used to quantify the amount of
variation or dispersion of a set of data values. A standard deviation close to 0
indicates that the data points tend to be very close to the mean (also called the
expected value) of the set, while a high standard deviation indicates that the data
points are spread out over a wider range of values.
(
wikipedia)
The median is the value that seperates that data into two equal sizes subsets
(with the < and the > relation respectively).
As the mean and the standard deviation are already given here, the median isn't important.
The maximum value. It's a bad sign if the maximum
is far higher than the mean and you can't explain it.
The chance is \[ 1 - \alpha = 0.95 \] that the mean lies in the given interval
(assuming the data is normal distributed to a certain degree).
Quoting the
minitab blog:
If process knowledge tells you that your data should follow a normal distribution,
then run a normality test to be sure. If your Anderson-Darling Normality
Test p-value is larger than, say, an alpha level of 0.05 (here 0.05), then you can conclude
that your data follow a normal distribution and, therefore, the mean is an adequate
measure of central tendency.
The T test is robust against non normality, but that's not the case fpr statistical properties like
the given confidence intervals.
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | |
mean | 2(5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule etime & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule etime & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
'etime','2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | |
mean | 2(5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
std dev & (5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
median & 2(5).(000)m\\
min & 20.000m\\
max & 30.000m\\
n & 10\\
mean ci (lower bound) & 24.893m\\
mean ci (upper bound) & 25.107m\\
std dev ci (lower bound) & 9.973m\\
std dev ci (upper bound) & nan\\
normality probability & 0.22\%\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
std dev & (5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
median & 2(5).(000)m\\
min & 20.000m\\
max & 30.000m\\
n & 10\\
mean ci (lower bound) & 24.893m\\
mean ci (upper bound) & 25.107m\\
std dev ci (lower bound) & 9.973m\\
std dev ci (upper bound) & nan\\
normality probability & 0.22\%\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'std dev','(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'median','2(5).(000)m'
'min','20.000m'
'max','30.000m'
'n',10
'mean ci (lower bound)','24.893m'
'mean ci (upper bound)','25.107m'
'std dev ci (lower bound)','9.973m'
'std dev ci (upper bound)',nan
'normality probability',0.0022
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)"
'Difference of means','(1).(000)m'
'... per left mean',0.03846
'... per max std dev',0.2
't',0.6733
'min n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
"grep '[ab]*' -R .",'2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
"grep 'a*' -R .",'2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The used statistical significance test showed that the significance of the
difference with etime (0.673) of grep '[ab]*' -R . vs. grep 'a*' -R . is too low.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
The mean difference per standard deviation of etime (0.200) of grep '[ab]*' -R . vs. grep 'a*' -R . is less than 2.000.
⍗ | grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime) |
Difference of means | (1).(000)m |
|---|
... per left mean | 3.846% |
|---|
... per max std dev | 20.000% |
|---|
t | 67.33% |
|---|
min n | 10 |
|---|
⍗ | mean | \(\sigma\) per mean | sem | min | n |
| 2(6).(000)m | 1(8).(842)% | (1).(633)m | 20.000m | 10 |
|---|
| 2(5).(000)m | 2(0).(000)% | (1).(667)m | 20.000m | 10 |
|---|
\begin{tabular}{lrrrr}\toprule
& Difference of means & ... per left mean & ... per max std dev & t test\\
\midrule etime & (1).(000)m & (3).(846)\% & (20).(000)\% & 67.33\%\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrr}\toprule
& Difference of means & ... per left mean & ... per max std dev & t test\\
\midrule etime & (1).(000)m & (3).(846)\% & (20).(000)\% & 67.33\%\\
\bottomrule
\end{tabular}
\end{document}
'','Difference of means','... per left mean','... per max std dev','t test'
'etime','(1).(000)m','(3).(846)%','(20).(000)%',0.6733
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The mean difference relative to the left mean
gives a number that helps to talk about the practical significance of the mean difference.
A tiny difference might be cool, but irrelevant (as caching effects are probably higher, use the
\verb|temci build| if you are curious about this).
The mean difference relative to the maximum standard deviation is important,
because it puts the value info context, or as
Gernot Heiser points out:
- Don't believe any effect that is less than a standard deviation
- Be highly suspicious if it is less than two standard deviations
This color means that there are severe warnings related to the corresponding data set (6 severe warning(s) and 0 warning(s)).
Click on the cell to get more information.
Geometric mean of the means of the left relative to the means of the right:
\[\sqrt[\|properties\|]{
\prod_{p \in \text{properties}}
\frac{\overline{\text{left[p]}}}{
\overline{\text{right[p]}}}}\]
Using the more widely known would be like
lying.
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
... ci (lower bound) & nan\\
... ci (upper bound) & nan\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)\\
\midrule Difference of means & (1).(000)m\\
... per left mean & 3.846\%\\
... per max std dev & 20.000\%\\
... ci (lower bound) & nan\\
... ci (upper bound) & nan\\
t & 67.33\%\\
min n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . vs. grep 'a*' -R . (regarding etime)"
'Difference of means','(1).(000)m'
'... per left mean',0.03846
'... per max std dev',0.2
'... ci (lower bound)',nan
'... ci (upper bound)',nan
't',0.6733
'min n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The chance is \[ 1 - \alpha = 0.95 \] that the mean difference
\begin{align} &\text{grep '[ab]*' -R .} - \text{grep 'a*' -R .} \\ =& 0.0010000000000000009 \end{align}
lies in the interval \(( nan, nan)\) (assuming the data is normal
distributed to a certain degree).
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep '[ab]*' -R . (regarding etime)\\
\midrule mean & 2(6).(000)m\\
\(\sigma\) per mean & 1(8).(842)\%\\
sem & (1).(633)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep '[ab]*' -R . (regarding etime)"
'mean','2(6).(000)m'
'\\(\\sigma\\) per mean','1(8).(842)%'
'sem','(1).(633)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (18.842%) of grep '[ab]*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep '[ab]*' -R . is less than 15.
⍗ | |
mean | 2(6).(000)m |
|---|
\(\sigma\) per mean | 1(8).(842)% |
|---|
sem | (1).(633)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lr}\toprule
& grep 'a*' -R . (regarding etime)\\
\midrule mean & 2(5).(000)m\\
\(\sigma\) per mean & 2(0).(000)\%\\
sem & (1).(667)m\\
min & 20.000m\\
n & 10\\
\bottomrule
\end{tabular}
\end{document}
'',"grep 'a*' -R . (regarding etime)"
'mean','2(5).(000)m'
'\\(\\sigma\\) per mean','2(0).(000)%'
'sem','(1).(667)m'
'min','20.000m'
'n',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
The standard deviation per mean of etime (20.000%) of grep 'a*' -R . is too high. It should be ≤ 5.000%.
The number of observations of etime (10) of grep 'a*' -R . is less than 15.
⍗ | |
mean | 2(5).(000)m |
|---|
\(\sigma\) per mean | 2(0).(000)% |
|---|
sem | (1).(667)m |
|---|
min | 20.000m |
|---|
n | 10 |
|---|
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\documentclass[10pt,a4paper]{article}
\usepackage{booktabs}
\begin{document}
\begin{tabular}{lrrrrr}\toprule
& mean & \(\sigma\) per mean & sem & min & n\\
\midrule grep '[ab]*' -R . & 2(6).(000)m & 1(8).(842)\% & (1).(633)m & 20.000m & 10\\
grep 'a*' -R . & 2(5).(000)m & 2(0).(000)\% & (1).(667)m & 20.000m & 10\\
\bottomrule
\end{tabular}
\end{document}
'','mean','\\(\\sigma\\) per mean','sem','min','n'
"grep '[ab]*' -R .",'2(6).(000)m','1(8).(842)%','(1).(633)m','20.000m',10
"grep 'a*' -R .",'2(5).(000)m','2(0).(000)%','(1).(667)m','20.000m',10
Latex table (requires package booktabs)
Latex table with surrounding article environment
CSV table
