---
title: "oneway"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{oneway}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```
# 1. Overview
The `oneway` package provides an integrated workflow for one-factor experimental designs. Its
functionality covers:
* descriptive statistics;
* *a priori* test (omnibus test)
- classical ANOVA
- aligned rank transform ANOVA (`ART-ANOVA`)
- Kruskal--Wallis test
* *Post hoc* test (Multiple-comparison procedures, MCP):
- REGWQ
- Tukey-HSD
- Tukey-Kramer
- Games-Howell
- Dunn's
* Effect sizes;
* a single adaptive workflow, `pairwise_comparison()`, that selects an appropriate *a priori* +
*post hoc* protocol from the observed distributional, variance, and design characteristics.
# 2. Installation
Install the released version from [CRAN](https://cran.r-project.org/package=oneway):
```{r, eval=FALSE}
install.packages("oneway")
```
or the development version from [GitHub](https://github.com/P10911004-NPUST/oneway):
```{r, eval=FALSE}
if (!require("pak")) install.packages("pak")
pak::pak("P10911004-NPUST/oneway")
```
Then load the package:
```{r}
library(oneway)
# For reproducibility
set.seed(123)
```
# 3. Quick start
The core function of this package is `pairwise_comparison()`, which provides out-of-the-box
statistical inference for one-way designs. It expects a data frame and a two-sided formula:
```{r}
out <- pairwise_comparison(morphine, tolerance ~ grp)
```
## The function return a list which includes 5 components:
- method: the combination of a ***a priori*** and a ***post-hoc*** test;
- data: the input data;
- pre_hoc: An ANOVA-like table from the selected ***a priori*** test;
- post_hoc: pairwise comparison result from the selected ***post hoc*** test;
- summary: descriptive statistics for reporting.
> The `post_hoc` component is a data frame with at least 15 columns:
| Column name | Interpretation |
| :------ | :----------------- |
| `x1` | The name of the group being subtracted |
| `x2` | The name of the group as the subtrahend |
| `[x1 - x2]` | The difference between x1 and x2 |
| `Hedges's g` | Effect size |
| `Pvalue` | *P*-value yielded from the standard value |
| `Padj` | adjusted *P*-value according to the `p_adjust_method` argument |
| `signif` | Significance symbols. See `oneway:pval2asterisk` |
| `diff_CI` | The (1 - `alpha`) confidence interval of `[x1 - x2]` |
| `mu` | Currently not available |
| `standard_value` | The statistics of the test, generally is a symbol |
| `critical_value` | The critical value for the statistics |
| `StdErr` | The standard error used to calculate `diff_CI` |
| `method` | The name of the *post hoc* method |
| `alternative` | The direction of testing, either `less`, `greater`, or `two.sided` |
| `alpha` | Error tolerance |
> The `summary` component is a data frame with 13 columns:
| Column name | Interpretation |
| :--- | :-------------- |
| `GROUP` | The name of each groups from the independent variable |
| `CLD` | Compact letter display, see `oneway::compact_letter_display` |
| `N` | Sample size |
| `AVG` | Mean |
| `SD` | Sample's standard deviation |
| `MED` | Median |
| `MIN` | Minimum value |
| `MAX` | Maximum value |
| `CI (95%)` | Confidence interval containing the true population mean |
| `SKEW (= 0)` | Skewness, see `normality::skewness` |
| `KURT (= 3)` | Kurtosis, see `normality::kurtosis` |
| `normality` | Is the sample follows normal distribution, see `normality::is_normal` |
| `n_outliers` | Number of possible outliers, see `outlying::Grubbs_test` |
- Note that the `n_outliers` field provides only a **soft suggestion** that approximately *n*
observations may be outliers. It should therefore be interpreted as a diagnostic flag rather
than as evidence that those observations should be removed.
If you only require a plug-and-play function to analyze your data, `pairwise_comparison()`
has you covered. Feel free to skip the rest of this guide.
# 4. Interface
Most analysis functions expect a data frame and a two-sided formula:
```r
response ~ group
```
For example:
```{r}
anorexia[c(2, 19, 43, 47, 60, 71), ]
```
The `anorexia` data contain `weight_gain` as the response and `therapy` as the grouping variable:
```{r}
pairwise_comparison(anorexia, weight_gain ~ therapy)
```
The package also accepts a list of numeric vectors for several functions:
```{r}
group_data <- list(control = rnorm(20, 10, 2),
treatment_A = rnorm(20, 12, 2),
treatment_B = rnorm(20, 15, 2))
pairwise_comparison(group_data)
```
# 5. Describe the data
`describe()` is useful as a first-pass summary. This is the function used to generate the `summary`
component of the `pairwise_comparison()`.
```{r}
describe(anorexia, weight_gain ~ therapy, rounding = 2)
```
# 6. Omnibus / *a priori* test
## 6.1 Parametric
The classic ANOVA procedure for parametric analysis is `oneway_anova()`.
```{r pre-hoc}
anova_ooo <- oneway_anova(O_O_O, val ~ grp)
anova_ooo
```
The function checks normality and variance homogeneity. With `var_equal = NA` (the default), it
uses `varequal::is_var_equal` to check the variance homogeneity and decide to conduct:
* **Fisher's one-way ANOVA** when equal variances are supported;
* **Welch's ANOVA** when equal variances are not supported.
You can also force the procedure:
```{r}
fisher <- oneway_anova(O_O_O, val ~ grp, var_equal = TRUE)
welch <- oneway_anova(O_X_X, val ~ grp, var_equal = FALSE)
```
Reading the ANOVA table
The Fisher/Welch's ANOVA output includes columns such as:
* `DF`: degrees of freedom;
* `SS`: sum of squares;
* `MS`: mean square;
* `Fvalue`: observed *F* statistic;
* `Fcrit`: critical *F* value;
* `Pvalue`: omnibus *p*-value;
* `signif`: significance label;
* `p_omega2`: partial omega-squared effect size; and
* `method`: Fisher's or Welch's ANOVA.
The automatic selection is convenient for exploratory and routine analyses, but it is still
important to examine the study design and the diagnostics before interpreting the result.
## 6.2 Nonparametric
### ART-ANOVA
`oneway_art()` implements a one-way aligned-rank-transform analysis.
```{r}
art <- oneway_art(X_O_O, val ~ grp)
art
```
Conceptually, the response is aligned with respect to the group effect, ranked, and then analyzed
with the ANOVA machinery.
```{r}
attr(art, "data") |> head()
```
ART is useful when the raw response is not adequately compatible with a classical ANOVA but a
rank-based ANOVA workflow remains appropriate.
### Kruskal--Wallis test
`Kruskal_Wallis_test()` provides the classical nonparametric alternative:
```{r}
kw <- Kruskal_Wallis_test(X_X_O, val ~ grp)
kw
```
# 7. *Post hoc* analysis
`oneway` implements five multiple comparison procedures (MCPs) for *post hoc* analysis. Each
procedure returns results in a standardized format consistent with that of `pairwise_comparison()`.
Although these MCPs can be applied directly without *a priori* adjustment or protection, their
interpretation should nevertheless be informed by the outcome of the omnibus test. In particular,
evidence from the omnibus test provides an overall assessment of whether differences among the
group means warrant subsequent pairwise comparisons.
## 7.1 Parametric
### REGWQ
`REGWQ_test()` implements the Ryan--Einot--Gabriel--Welsch studentized-range procedure. REGWQ is a
stepwise studentized-range procedure intended primarily for balanced, homoscedastic normal one-way
designs. The package uses the studentized-range distribution and step-specific significance levels.
```{r}
regwq <- REGWQ_test(O_O_O, val ~ grp)
```
### Tukey HSD
`Tukey_HSD_test()` provides the classical Tukey honestly significant difference procedure. The
function is intended primarily for balanced designs with approximately normal responses and
homogeneous variances. When the design is unbalanced, use Tukey--Kramer instead.
```{r}
tukey <- Tukey_HSD_test(O_O_O, val ~ grp)
```
### Tukey--Kramer
`Tukey_Kramer_test()` extends the Tukey's comparison to unequal sample sizes while retaining the
equal-variance assumption:
```{r}
tukey_kramer <- Tukey_Kramer_test(O_O_X, val ~ grp)
```
### Games--Howell
`Games_Howell_test()` is intended for normal data with unequal variances and/or unequal sample
sizes. Each comparison uses a Welch--Satterthwaite-type degrees of freedom and the
studentized-range distribution. This makes Games--Howell the natural choice among the package's
parametric *post hoc* procedures when homoscedasticity assumption is violated.
```{r}
games_howell <- Games_Howell_test(O_X_X, val ~ grp)
```
## 7.2 Nonparametric
### Dunn's test
`Dunn_test()` is the nonparametric multiple-comparison procedure associated with Kruskal--Wallis
analysis.
```{r}
dunn <- Dunn_test(X_X_O, val ~ grp, p_adjust_method = "holm")
```
Dunn's procedure compares mean ranks. Because many pairwise comparisons are performed, the function
supports `stats::p.adjust()` methods, with `"holm"` as the default:
```{r}
dunn_bh <- Dunn_test(X_X_O, val ~ grp, p_adjust_method = "BH")
```
The available choices are those accepted by `stats::p.adjust()`.
# 8. Automatic Workflow
`pairwise_comparison()` automatically implements the conventional workflow for one-way analysis by
evaluating the relevant distributional and variance assumptions, selecting an appropriate omnibus
test, and subsequently performing pairwise comparisons when warranted.
```{r}
adaptive <- pairwise_comparison(O_O_O, val ~ grp)
```
The decision logic is:
| Data characteristics | Omnibus / pre-hoc | *Post hoc* |
| --- | --- | --- |
| Normal, equal variance, balanced | Fisher's ANOVA | REGWQ |
| Normal, equal variance, unbalanced | Fisher's ANOVA | Tukey--Kramer |
| Normal, unequal variance | Welch's ANOVA | Games--Howell |
| Non-normal, ART rank response becomes suitable | ART-ANOVA | similar to normal |
| Non-normal, ART rank response remains unsuitable | Kruskal--Wallis | Dunn |
The adaptive workflow can therefore return 7 complete protocols:
1. one-way ANOVA + REGWQ;
2. one-way ANOVA + Tukey--Kramer;
3. one-way ANOVA + Games--Howell;
4. ART-ANOVA + [REGWQ on y''];
5. ART-ANOVA + [Tukey--Kramer on y''];
6. ART-ANOVA + [Games--Howell on y'']; or
7. Kruskal--Wallis + Dunn.
The Fisher's and Welch's ANOVA procedures are conducted on the residuals rather than the raw
response.
# 9. Compact letter displays
A compact letter display is a presentation device:
* groups **sharing a letter** are **NOT** significantly different;
* groups without a common letter are significantly different.
`compact_letter_display()` can be used independently. For example:
```{r}
dunn <- Dunn_test(X_X_O, val ~ grp, verbose = FALSE)
summary_tab <- row_arrange(dunn$summary, cols = "MED")
padj_name <- "Padj (holm)"
cld <- compact_letter_display(x1 = dunn$post_hoc$x1,
x2 = dunn$post_hoc$x2,
pvalues = dunn$post_hoc[[padj_name]],
grp_names = summary_tab$GROUP,
centers = summary_tab$MED)
cld
```
The `centers` argument controls the order used to assign letters. For parametric procedures it is
typically natural to use the mean; for rank-based procedures, the median is more appropriate.
The symbols can also be customized:
```{r}
compact_letter_display(
x1 = dunn$post_hoc$x1,
x2 = dunn$post_hoc$x2,
pvalues = dunn$post_hoc[[padj_name]],
grp_names = summary_tab$GROUP,
centers = summary_tab$MED,
display_letters = as.character(1:6),
display_null_letter = "_"
)
```
For two-samples comparison, use `pval2asterisk()` to convert numeric p-values into user-defined
significance symbols.
```{r}
p <- c(0.20, 0.04, 0.008, 0.0005, 1e-6)
pval2asterisk(p)
```
The default labels are:
| Range | Label |
| :--------- | :----: |
| p > 0.055 | `ns` |
| 0.05 < p ≤ 0.055 | `.` |
| 0.01 < p ≤ 0.05 | `*` |
| 0.001 < p ≤ 0.01 | `**` |
| p ≤ 0.001 | `***` |
Custom thresholds and symbols are supported. The function expects thresholds in descending order:
```{r}
pval2asterisk(p,
break_points = c(0.05, 0.01, 0),
symbols = c("NS", "S", "HS"))
```
# 10. Effect sizes
This package provides quick inference to the effect size using only 2 of the many estimators.
For comprehensive effect size analysis, please refer to the paper of
[Lakens, 2013](https://doi.org/10.3389/fpsyg.2013.00863) and the
[effectsize](https://cran.r-project.org/package=effectsize) CRAN package.
### Hedges' *g*
`Hedges_g_s()` applies a small-sample correction to Cohen's *d*:
```{r}
g1 <- subset(O_O_O, grp == "G1")$val
g2 <- subset(O_O_O, grp == "G2")$val
Hedges_g_s(g1, g2)
```
Conventional benchmarks:
| Hedges' *g* | Effect | Practical meaning |
| --- | --- | --------- |
| 0.2 | Small | The difference is noticeable but modest. |
| 0.5 | Medium | The difference is clearly visible. |
| 0.8 | Large | The difference is substantial and practically important. |
### Partial-ω2
The classical ANOVA implementation reports partial omega-squared as `p_omega2`:
```{r}
fisher[, c("method", "Pvalue", "p_omega2")]
```
# 11. Datasets
The package includes 6 simulated datasets designed specifically for different analysis conditions:
| Dataset | Distribution | Variance | Balance |
| :---: | :---: | :---: | :---: |
| `O_O_O` | Normal | Homogeneous | Balanced |
| `O_O_X` | Normal | Homogeneous | Unbalanced |
| `O_X_X` | Normal | Heterogeneous | Unbalanced |
| `X_O_O` | Non-normal | Homogeneous | Balanced |
| `X_X_O` | Non-normal | Heterogeneous | Balanced |
| `X_X_X` | Non-normal | Heterogeneous | Unbalanced |
Each has six groups (`G1`--`G6`) and uses the common `grp` / `val` column names.
Additional datasets are:
* `anorexia`: weight gain following three therapies;
* `morphine`: morphine tolerance across five treatment groups; and
* `plasma_etching`: etch rate across plasma power levels.
# 12. Interpretation notes
### Statistical significance is not the same as practical importance
A small *p*-value indicates evidence against the null hypothesis under the specified model. Effect
sizes and confidence intervals provide complementary information about magnitude and precision.
### CLDs are a summary, not the primary inferential result
Use the pairwise table to inspect the actual differences, adjusted *p*-values, confidence intervals,
and effect sizes. Treat the CLD as a compact visualization of those decisions.
### Do not mechanically transform data merely to obtain a desired p-value
The package provides rank-based alternatives so that a scientifically defensible analysis does not
depend on forcing the response into a particular distributional shape.
# Summary
A practical way to use `oneway` is:
1. Confirm that the design is genuinely one-way:
one independent grouping variable and one quantitative response variable.
2. Inspect sample sizes, descriptive statistics, possible outliers, and distributional shape.
3. Use `oneway_anova()` when a parametric omnibus analysis is appropriate.
4. Use `oneway_art()` when an aligned-rank-transform analysis is justified.
5. Use `Kruskal_Wallis_test()` when a rank-based omnibus test is more appropriate.
6. Choose a *post hoc* procedure that matches the assumptions and balance of the design.
7. Report the estimated pairwise differences, confidence intervals, adjusted *p*-values, and
effect sizes (not only significance stars or CLDs).
8. Use `pairwise_comparison()` when you want to formalize this decision process automatically.
The package's automatic workflow is particularly useful when you analyze many similar one-way
experiments and want a consistent rule for selecting the analysis protocol.
# References
Dinno, A. (2015).
Nonparametric pairwise multiple comparisons in independent groups using Dunn’s test.
*The Stata Journal: Promoting Communications on Statistics and Stata*, 15(1), 292–300.
Elkin, L.A., Kay, M., Higgins, J.J., & Wobbrock, J.O. (2021).
An aligned rank transform procedure for multifactor contrast tests.
*Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology*, 754–768.
Hollander, M., Wolfe, D.A., & Chicken, E. (2014).
*Nonparametric statistical methods* (3rd ed.). Wiley.
Howell, D.C. (2010). *Statistical methods for psychology* (7th ed.). Cengage Learning.
Howell, D.C. (2013). *Statistical methods for psychology* (8th ed.). Cengage Learning.
Kruskal, W.H., & Wallis, W. A. (1952).
Use of ranks in one-criterion variance analysis.
*Journal of the American Statistical Association*, 47(260), 583–621.
Lakens, D. (2013).
Calculating and reporting effect sizes to facilitate cumulative science:
A practical primer for t-tests and ANOVAs.
*Frontiers in Psychology*, 4(863).
Montgomery, D.C. (2017). *Design and analysis of experiments* (9th ed.). John Wiley & Sons.
Piepho, H.P. (2004).
An algorithm for a letter-based representation of all-pairwise comparisons.
*Journal of Computational and Graphical Statistics*, 13(2), 456–466.
Piepho, H.P. (2018).
Letters in mean comparisons: What they do and don’t mean.
*Agronomy Journal*, 110(2), 431–434.
Wobbrock, J.O., Findlater, L., Gergle, D., & Higgins, J.J. (2011).
The aligned rank transform for nonparametric factorial analyses using only ANOVA procedures.
*Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 143–146.
Zar, J.H. (2014). *Biostatistical analysis* (5th ed.). Pearson.