Test and effect size details

Indrajeet Patil

2021-03-10

This vignette provides a go-to summary for which test is carried out for each function included in the package and what effect size it returns. Additionally, there are also recommendations on how to interpret those effect sizes.

Summary of tests and effect sizes

two_sample_test + oneway_anova

No. of groups: 2 => two_sample_test No. of groups: > 2 => oneway_anova

between-subjects

Following (between-subjects) tests are carried out for each type of analyses-

Type No. of groups Test Function used
Parametric > 2 Fisher’s or Welch’s one-way ANOVA stats::oneway.test
Non-parametric > 2 Kruskal–Wallis one-way ANOVA stats::kruskal.test
Robust > 2 Heteroscedastic one-way ANOVA for trimmed means WRS2::t1way
Bayes Factor > 2 Fisher’s ANOVA BayesFactor::anovaBF
Parametric 2 Student’s or Welch’s t-test stats::t.test
Non-parametric 2 Mann–Whitney U test stats::wilcox.test
Robust 2 Yuen’s test for trimmed means WRS2::yuen
Bayesian 2 Student’s t-test BayesFactor::ttestBF

Following effect sizes (and confidence intervals/CI) are available for each type of test-

Type No. of groups Effect size CI? Function used
Parametric > 2 \(\eta_{p}^2\), \(\omega_{p}^2\) Yes effectsize::omega_squared, effectsize::eta_squared
Non-parametric > 2 \(\epsilon_{ordinal}^2\) Yes effectsize::rank_epsilon_squared
Robust > 2 \(\xi\) (Explanatory measure of effect size) Yes WRS2::t1way
Bayes Factor > 2 \(R_{posterior}^2\) Yes performance::r2_bayes
Parametric 2 Cohen’s d, Hedge’s g Yes effectsize::cohens_d, effectsize::hedges_g
Non-parametric 2 r (rank-biserial correlation) Yes effectsize::rank_biserial
Robust 2 \(\xi\) (Explanatory measure of effect size) Yes WRS2::yuen.effect.ci
Bayesian 2 \(\delta_{posterior}\) Yes bayestestR::describe_posterior

within-subjects

Following (within-subjects) tests are carried out for each type of analyses-

Type No. of groups Test Function used
Parametric > 2 One-way repeated measures ANOVA afex::aov_ez
Non-parametric > 2 Friedman rank sum test stats::friedman.test
Robust > 2 Heteroscedastic one-way repeated measures ANOVA for trimmed means WRS2::rmanova
Bayes Factor > 2 One-way repeated measures ANOVA BayesFactor::anovaBF
Parametric 2 Student’s t-test stats::t.test
Non-parametric 2 Wilcoxon signed-rank test stats::wilcox.test
Robust 2 Yuen’s test on trimmed means for dependent samples WRS2::yuend
Bayesian 2 Student’s t-test BayesFactor::ttestBF

Following effect sizes (and confidence intervals/CI) are available for each type of test-

Type No. of groups Effect size CI? Function used
Parametric > 2 \(\eta_{p}^2\), \(\omega_{p}^2\) Yes effectsize::omega_squared, effectsize::eta_squared
Non-parametric > 2 \(W_{Kendall}\) (Kendall’s coefficient of concordance) Yes effectsize::kendalls_w
Robust > 2 \(\delta_{R-avg}^{AKP}\) Yes Algina-Keselman-Penfield robust standardized difference average WRS2::wmcpAKP
Bayes Factor > 2 \(R_{posterior}^2\) Yes performance::r2_bayes
Parametric 2 Cohen’s d, Hedge’s g Yes effectsize::cohens_d, effectsize::hedges_g
Non-parametric 2 r (rank-biserial correlation) Yes effectsize::rank_biserial
Robust 2 \(\delta_{R}^{AKP}\) (Algina-Keselman-Penfield robust standardized difference) Yes WRS2::dep.effect
Bayesian 2 \(\delta_{posterior}\) Yes bayestestR::describe_posterior

one_sample_test

Following tests are carried out for each type of analyses-

Type Test Function used
Parametric One-sample Student’s t-test stats::t.test
Non-parametric One-sample Wilcoxon test stats::wilcox.test
Robust Bootstrap-t method for one-sample test trimcibt (custom)
Bayesian One-sample Student’s t-test BayesFactor::ttestBF

Following effect sizes (and confidence intervals/CI) are available for each type of test-

Type Effect size CI? Function used
Parametric Cohen’s d, Hedge’s g Yes effectsize::cohens_d, effectsize::hedges_g
Non-parametric r (rank-biserial correlation) Yes effectsize::rank_biserial
Robust trimmed mean Yes trimcibt (custom)
Bayes Factor \(\delta_{posterior}\) Yes bayestestR::describe_posterior

corr_test

Following tests are carried out for each type of analyses. Additionally, the correlation coefficients (and their confidence intervals) are used as effect sizes-

Type Test CI? Function used
Parametric Pearson’s correlation coefficient Yes correlation::correlation
Non-parametric Spearman’s rank correlation coefficient Yes correlation::correlation
Robust Winsorized Pearson correlation coefficient Yes correlation::correlation
Bayesian Pearson’s correlation coefficient Yes correlation::correlation

contingency_table

Following tests are carried out for each type of analyses-

Type of data Design Test Function used
Unpaired \(n \times p\) contingency table Pearson’s \(\chi^2\) test stats::chisq.test
Paired \(n \times p\) contingency table McNemar’s \(\chi^2\) test stats::mcnemar.test
Frequency \(n \times 1\) contingency table Goodness of fit (\(\chi^2\) test) stats::chisq.test

Following effect sizes (and confidence intervals/CI) are available for each type of test-

Test Effect size CI? Function used
Pearson’s \(\chi^2\) test Cramer’s \(V\) Yes effectsize::cramers_v
McNemar’s test Cohen’s \(g\) Yes effectsize::cohens_g
Goodness of fit Cramer’s \(V\) Yes effectsize::cramers_v

meta_analysis

Type Test Effect size 95% CI available? Function used
Parametric Meta-analysis via random-effects models \(\beta\) Yes metafor::metafor
Robust Meta-analysis via robust random-effects models \(\beta\) Yes metaplus::metaplus
Bayes Meta-analysis via Bayesian random-effects models \(\beta\) Yes metaBMA::meta_random

Effect size interpretation

See effectsize’s interpretation functions to check different rules/conventions to interpret effect sizes:

https://easystats.github.io/effectsize/reference/index.html#section-interpretation

Dataframe as output

Although the primary focus of this package is to get expressions containing statistical results, one can also use it to extract dataframes containing these details.

For a more detailed summary of these dataframe: https://indrajeetpatil.github.io/statsExpressions//articles/web_only/dataframe_outputs.html

References

Suggestions

If you find any bugs or have any suggestions/remarks, please file an issue on GitHub: https://github.com/IndrajeetPatil/ggstatsplot/issues

Session Information

For details, see- https://indrajeetpatil.github.io/ggstatsplot/articles/web_only/session_info.html