library("lessR")
The Regression()
function performs multiple aspects of a complete regression analysis. Abbreviate with reg()
.
To illustrate, first read the Employee data included as part of lessR. Read into the default lessR data frame d.
<- Read("Employee") d
##
## >>> Suggestions
## Details about your data, Enter: details() for d, or details(name)
##
## Data Types
## ------------------------------------------------------------
## character: Non-numeric data values
## integer: Numeric data values, integers only
## double: Numeric data values with decimal digits
## ------------------------------------------------------------
##
## Variable Missing Unique
## Name Type Values Values Values First and last values
## ------------------------------------------------------------------------------------------
## 1 Years integer 36 1 16 7 NA 15 ... 1 2 10
## 2 Gender character 37 0 2 M M M ... F F M
## 3 Dept character 36 1 5 ADMN SALE SALE ... MKTG SALE FINC
## 4 Salary double 37 0 37 53788.26 94494.58 ... 56508.32 57562.36
## 5 JobSat character 35 2 3 med low low ... high low high
## 6 Plan integer 37 0 3 1 1 3 ... 2 2 1
## 7 Pre integer 37 0 27 82 62 96 ... 83 59 80
## 8 Post integer 37 0 22 92 74 97 ... 90 71 87
## ------------------------------------------------------------------------------------------
As an option, also read the table of variable labels. Create the table formatted as two columns. The first column is the variable name and the second column is the corresponding variable label. Not all variables need to be entered into the table. The table can be a csv
file or an Excel file.
Currently, read the label file into the l data frame. The labels are displayed on both the text and visualization output. Each displayed label consists of the variable name juxtaposed with the corresponding label.
<- rd("Employee_lbl") l
##
## >>> Suggestions
## Details about your data, Enter: details() for d, or details(name)
##
## Data Types
## ------------------------------------------------------------
## character: Non-numeric data values
## ------------------------------------------------------------
##
## Variable Missing Unique
## Name Type Values Values Values First and last values
## ------------------------------------------------------------------------------------------
## 1 label character 8 0 8 Time of Company Employment ... Test score on legal issues after instruction
## ------------------------------------------------------------------------------------------
The brief version provides just the basic analysis, what Excel provides, plus a scatterplot with the regression line, which becomes a scatterplot matrix with multiple regression. Because d is the default name of the data frame that contains the variables for analysis, the data
parameter that names the input data frame need not be specified.
reg_brief(Salary ~ Years + Pre)
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ Years + Pre, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
## Predictor Variable 1: Years
## Predictor Variable 2: Pre
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 36
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 44140.971 13666.115 3.230 0.003 16337.052 71944.891
## Years 3251.408 347.529 9.356 0.000 2544.355 3958.462
## Pre -18.265 167.652 -0.109 0.914 -359.355 322.825
##
##
## Standard deviation of residuals: 11,753.478 for 33 degrees of freedom
## 95% range of residual variation: 47,825.260 = 2 * (2.035 * 11,753.478)
##
## R-squared: 0.726 Adjusted R-squared: 0.710 PRESS R-squared: 0.659
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 43.827 df: 2 and 33 p-value: 0.000
##
##
## df Sum Sq Mean Sq F-value p-value
## Years 1 12107157290.292 12107157290.292 87.641 0.000
## Pre 1 1639658.444 1639658.444 0.012 0.914
##
## Model 2 12108796948.736 6054398474.368 43.827 0.000
## Residuals 33 4558759843.773 138144237.690
## Salary 35 16667556792.508 476215908.357
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## RESIDUALS AND INFLUENCE
##
## PREDICTION ERROR
The full output is extensive: Summary of the analysis, estimated model, fit indices, ANOVA, correlation matrix, collinearity analysis, best subset regression, residuals and influence statistics, and prediction intervals. The motivation is to provide virtually all of the information needed for a proper regression analysis.
reg(Salary ~ Years + Pre)
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ Years + Pre, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
## Predictor Variable 1: Years
## Predictor Variable 2: Pre
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 36
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 44140.971 13666.115 3.230 0.003 16337.052 71944.891
## Years 3251.408 347.529 9.356 0.000 2544.355 3958.462
## Pre -18.265 167.652 -0.109 0.914 -359.355 322.825
##
##
## Standard deviation of residuals: 11,753.478 for 33 degrees of freedom
## 95% range of residual variation: 47,825.260 = 2 * (2.035 * 11,753.478)
##
## R-squared: 0.726 Adjusted R-squared: 0.710 PRESS R-squared: 0.659
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 43.827 df: 2 and 33 p-value: 0.000
##
##
## df Sum Sq Mean Sq F-value p-value
## Years 1 12107157290.292 12107157290.292 87.641 0.000
## Pre 1 1639658.444 1639658.444 0.012 0.914
##
## Model 2 12108796948.736 6054398474.368 43.827 0.000
## Residuals 33 4558759843.773 138144237.690
## Salary 35 16667556792.508 476215908.357
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## Salary Years Pre
## Salary 1.00 0.85 0.03
## Years 0.85 1.00 0.05
## Pre 0.03 0.05 1.00
##
##
## Tolerance VIF
## Years 0.998 1.002
## Pre 0.998 1.002
##
##
## Years Pre R2adj X's
## 1 0 0.718 1
## 1 1 0.710 2
## 0 1 -0.028 1
##
## [based on Thomas Lumley's leaps function from the leaps package]
##
##
##
## RESIDUALS AND INFLUENCE
##
## Data, Fitted, Residual, Studentized Residual, Dffits, Cook's Distance
## [sorted by Cook's Distance]
## [res_rows = 20, out of 36 rows of data, or do res_rows="all"]
## -----------------------------------------------------------------------------------------
## Years Pre Salary fitted resid rstdnt dffits cooks
## Correll, Trevon 21 97 134419.230 110648.843 23770.387 2.424 1.217 0.430
## James, Leslie 18 70 122563.380 101387.773 21175.607 1.998 0.714 0.156
## Capelle, Adam 24 83 108138.430 120658.778 -12520.348 -1.211 -0.634 0.132
## Hoang, Binh 15 96 111074.860 91158.659 19916.201 1.860 0.649 0.131
## Korhalkar, Jessica 2 74 72502.500 49292.181 23210.319 2.171 0.638 0.122
## Billing, Susan 4 91 72675.260 55484.493 17190.767 1.561 0.472 0.071
## Singh, Niral 2 59 61055.440 49566.155 11489.285 1.064 0.452 0.068
## Skrotzki, Sara 18 63 91352.330 101515.627 -10163.297 -0.937 -0.397 0.053
## Saechao, Suzanne 8 98 55545.250 68362.271 -12817.021 -1.157 -0.390 0.050
## Kralik, Laura 10 74 92681.190 75303.447 17377.743 1.535 0.287 0.026
## Anastasiou, Crystal 2 59 56508.320 49566.155 6942.165 0.636 0.270 0.025
## Langston, Matthew 5 94 49188.960 58681.106 -9492.146 -0.844 -0.268 0.024
## Afshari, Anbar 6 100 69441.930 61822.925 7619.005 0.689 0.264 0.024
## Cassinelli, Anastis 10 80 57562.360 75193.857 -17631.497 -1.554 -0.265 0.022
## Osterman, Pascal 5 69 49704.790 59137.730 -9432.940 -0.826 -0.216 0.016
## Bellingar, Samantha 10 67 66337.830 75431.301 -9093.471 -0.793 -0.198 0.013
## LaRoe, Maria 10 80 61961.290 75193.857 -13232.567 -1.148 -0.195 0.013
## Ritchie, Darnell 7 82 53788.260 65403.102 -11614.842 -1.006 -0.190 0.012
## Sheppard, Cory 14 66 95027.550 88455.199 6572.351 0.579 0.176 0.011
## Downs, Deborah 7 90 57139.900 65256.982 -8117.082 -0.706 -0.174 0.010
##
##
## PREDICTION ERROR
##
## Data, Predicted, Standard Error of Forecast,
## 95% Prediction Intervals
## [sorted by lower bound of prediction interval]
## [to see all intervals do pred_rows="all"]
## ----------------------------------------------
##
## Years Pre Salary pred sf pi.lwr pi.upr width
## Hamide, Bita 1 83 51036.850 45876.388 12290.483 20871.211 70881.564 50010.352
## Singh, Niral 2 59 61055.440 49566.155 12619.291 23892.014 75240.296 51348.281
## Anastasiou, Crystal 2 59 56508.320 49566.155 12619.291 23892.014 75240.296 51348.281
## ...
## Link, Thomas 10 83 66312.890 75139.062 11933.518 50860.137 99417.987 48557.849
## LaRoe, Maria 10 80 61961.290 75193.857 11918.048 50946.405 99441.308 48494.903
## Cassinelli, Anastis 10 80 57562.360 75193.857 11918.048 50946.405 99441.308 48494.903
## ...
## Correll, Trevon 21 97 134419.230 110648.843 12881.876 84440.470 136857.217 52416.747
## Capelle, Adam 24 83 108138.430 120658.778 12955.608 94300.394 147017.161 52716.767
##
##
## ----------------------------------
## Plot 1: Distribution of Residuals
## Plot 2: Residuals vs Fitted Values
## Plot 3: ScatterPlot Matrix
## ----------------------------------
Request a briefer output with the reg_brief()
version of the function. Standardize the variables in the model by setting the rescale
parameter to "z"
. Plot the residuals as a line connecting each data point to the corresponding point on the regression line as specified with the plot_errors
parameter.
reg_brief(Salary ~ Years, rescale="z", plot_errors=TRUE)
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ Years, rescale="z", plot_errors=TRUE, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
## Predictor Variable: Years
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 36
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 42710.898 3746.753 11.399 0.000 35096.580 50325.216
## Years 3249.552 342.030 9.501 0.000 2554.463 3944.641
##
##
## Standard deviation of residuals: 11,581.425 for 34 degrees of freedom
## 95% range of residual variation: 47,072.573 = 2 * (2.032 * 11,581.425)
##
## R-squared: 0.726 Adjusted R-squared: 0.718 PRESS R-squared: 0.681
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 90.265 df: 1 and 34 p-value: 0.000
##
##
## df Sum Sq Mean Sq F-value p-value
## Model 1 12107157290.292 12107157290.292 90.265 0.000
## Residuals 34 4560399502.217 134129397.124
## Salary 35 16667556792.508 476215908.357
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## RESIDUALS AND INFLUENCE
##
## PREDICTION ERROR
Specify a cross-validation with the kfold
parameter. Here, specify three folds. The function automatically creates the training and testing data sets.
reg(Salary ~ Years, kfold=3)
## 3-FOLD CROSS-VALIDATION
##
## Model from Training Data Applied to Testing Data
## ---------------------------------- ----------------------------------
## fold n se MSE Rsq n se MSE Rsq
## 1 | 24 10653.506 113497190.591 0.813 | 12 15603.833 243479594.737 0.239
## 2 | 24 11327.650 128315647.880 0.607 | 12 14715.189 216536787.805 0.765
## 3 | 24 12286.462 150957143.289 0.737 | 12 11190.434 125225820.953 0.685
## ---------------------------------- ----------------------------------
## Mean 11422.539 130923327.253 0.719 13836.485 195080734.498 0.563
The standard output also includes \(R^2_{press}\), the value of \(R^2\) when applied to new, previously unseen data, a value comparable to the average \(R^2\) on test data.
The output of Regression()
can be stored into an R object, here named r. The output object consists of various components that together define a comprehensive regression analysis. R calls the resulting output structure a list object.
<- reg(Salary ~ Years + Pre) r
Entering the name of the object displays the full output, the default output when the output is directed to the R console instead of saving into an R object.
r
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ Years + Pre, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
## Predictor Variable 1: Years
## Predictor Variable 2: Pre
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 36
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 44140.971 13666.115 3.230 0.003 16337.052 71944.891
## Years 3251.408 347.529 9.356 0.000 2544.355 3958.462
## Pre -18.265 167.652 -0.109 0.914 -359.355 322.825
##
##
## Standard deviation of residuals: 11,753.478 for 33 degrees of freedom
## 95% range of residual variation: 47,825.260 = 2 * (2.035 * 11,753.478)
##
## R-squared: 0.726 Adjusted R-squared: 0.710 PRESS R-squared: 0.659
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 43.827 df: 2 and 33 p-value: 0.000
##
##
## df Sum Sq Mean Sq F-value p-value
## Years 1 12107157290.292 12107157290.292 87.641 0.000
## Pre 1 1639658.444 1639658.444 0.012 0.914
##
## Model 2 12108796948.736 6054398474.368 43.827 0.000
## Residuals 33 4558759843.773 138144237.690
## Salary 35 16667556792.508 476215908.357
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## Salary Years Pre
## Salary 1.00 0.85 0.03
## Years 0.85 1.00 0.05
## Pre 0.03 0.05 1.00
##
##
## Tolerance VIF
## Years 0.998 1.002
## Pre 0.998 1.002
##
##
## Years Pre R2adj X's
## 1 0 0.718 1
## 1 1 0.710 2
## 0 1 -0.028 1
##
## [based on Thomas Lumley's leaps function from the leaps package]
##
##
##
## RESIDUALS AND INFLUENCE
##
## Data, Fitted, Residual, Studentized Residual, Dffits, Cook's Distance
## [sorted by Cook's Distance]
## [res_rows = 20, out of 36 rows of data, or do res_rows="all"]
## -----------------------------------------------------------------------------------------
## Years Pre Salary fitted resid rstdnt dffits cooks
## Correll, Trevon 21 97 134419.230 110648.843 23770.387 2.424 1.217 0.430
## James, Leslie 18 70 122563.380 101387.773 21175.607 1.998 0.714 0.156
## Capelle, Adam 24 83 108138.430 120658.778 -12520.348 -1.211 -0.634 0.132
## Hoang, Binh 15 96 111074.860 91158.659 19916.201 1.860 0.649 0.131
## Korhalkar, Jessica 2 74 72502.500 49292.181 23210.319 2.171 0.638 0.122
## Billing, Susan 4 91 72675.260 55484.493 17190.767 1.561 0.472 0.071
## Singh, Niral 2 59 61055.440 49566.155 11489.285 1.064 0.452 0.068
## Skrotzki, Sara 18 63 91352.330 101515.627 -10163.297 -0.937 -0.397 0.053
## Saechao, Suzanne 8 98 55545.250 68362.271 -12817.021 -1.157 -0.390 0.050
## Kralik, Laura 10 74 92681.190 75303.447 17377.743 1.535 0.287 0.026
## Anastasiou, Crystal 2 59 56508.320 49566.155 6942.165 0.636 0.270 0.025
## Langston, Matthew 5 94 49188.960 58681.106 -9492.146 -0.844 -0.268 0.024
## Afshari, Anbar 6 100 69441.930 61822.925 7619.005 0.689 0.264 0.024
## Cassinelli, Anastis 10 80 57562.360 75193.857 -17631.497 -1.554 -0.265 0.022
## Osterman, Pascal 5 69 49704.790 59137.730 -9432.940 -0.826 -0.216 0.016
## Bellingar, Samantha 10 67 66337.830 75431.301 -9093.471 -0.793 -0.198 0.013
## LaRoe, Maria 10 80 61961.290 75193.857 -13232.567 -1.148 -0.195 0.013
## Ritchie, Darnell 7 82 53788.260 65403.102 -11614.842 -1.006 -0.190 0.012
## Sheppard, Cory 14 66 95027.550 88455.199 6572.351 0.579 0.176 0.011
## Downs, Deborah 7 90 57139.900 65256.982 -8117.082 -0.706 -0.174 0.010
##
##
## PREDICTION ERROR
##
## Data, Predicted, Standard Error of Forecast,
## 95% Prediction Intervals
## [sorted by lower bound of prediction interval]
## [to see all intervals do pred_rows="all"]
## ----------------------------------------------
##
## Years Pre Salary pred sf pi.lwr pi.upr width
## Hamide, Bita 1 83 51036.850 45876.388 12290.483 20871.211 70881.564 50010.352
## Singh, Niral 2 59 61055.440 49566.155 12619.291 23892.014 75240.296 51348.281
## Anastasiou, Crystal 2 59 56508.320 49566.155 12619.291 23892.014 75240.296 51348.281
## ...
## Link, Thomas 10 83 66312.890 75139.062 11933.518 50860.137 99417.987 48557.849
## LaRoe, Maria 10 80 61961.290 75193.857 11918.048 50946.405 99441.308 48494.903
## Cassinelli, Anastis 10 80 57562.360 75193.857 11918.048 50946.405 99441.308 48494.903
## ...
## Correll, Trevon 21 97 134419.230 110648.843 12881.876 84440.470 136857.217 52416.747
## Capelle, Adam 24 83 108138.430 120658.778 12955.608 94300.394 147017.161 52716.767
##
##
## ----------------------------------
## Plot 1: Distribution of Residuals
## Plot 2: Residuals vs Fitted Values
## Plot 3: ScatterPlot Matrix
## ----------------------------------
Or, work with the components individually. Use the base R names()
function to identify all of the output components. Component names that begin with out_
are part of the standard output. Other components include just data and statistics designed to be input in additional procedures, including R markdown documents.
names(r)
## [1] "out_suggest" "call" "formula" "out_title_bck" "out_background" "out_title_basic"
## [7] "out_estimates" "out_fit" "out_anova" "out_title_kfold" "out_kfold" "out_title_rel"
## [13] "out_cor" "out_collinear" "out_subsets" "out_title_res" "out_residuals" "out_title_pred"
## [19] "out_predict" "out_ref" "out_Rmd" "out_Word" "out_pdf" "out_odt"
## [25] "out_rtf" "out_plots" "n.vars" "n.obs" "n.keep" "coefficients"
## [31] "sterrs" "tvalues" "pvalues" "cilb" "ciub" "anova_model"
## [37] "anova_residual" "anova_total" "se" "resid_range" "Rsq" "Rsqadj"
## [43] "PRESS" "RsqPRESS" "m_se" "m_MSE" "m_Rsq" "cor"
## [49] "tolerances" "vif" "resid.max" "pred_min_max" "residuals" "fitted"
## [55] "cooks.distance" "model" "terms"
Here, only display the estimates as part of the standard text output.
$out_estimates r
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 44140.971 13666.115 3.230 0.003 16337.052 71944.891
## Years 3251.408 347.529 9.356 0.000 2544.355 3958.462
## Pre -18.265 167.652 -0.109 0.914 -359.355 322.825
Here, display the numeric values of the coefficients.
$coefficients r
## (Intercept) Years Pre
## 44140.97140 3251.40825 -18.26496
Do a regression and request all prediction intervals. Then convert that output segment to a data frame named dp with base R read.table()
. As a data frame, do a standard search for an individual row for a specific prediction interval (see the Subset a Data Frame vignette).
This particular conversion to a data frame requires one more step. One more spaces delimit adjacent columns, but the names in this data set are formatted with a comma followed by a space. Use base R sub()
to remove the space after the comma before converting to a data frame.
<- reg(Salary ~ Years, pred_rows="all", graphics=FALSE)
r $out_predict = sub(", ", ",", r$out_predict, fixed=TRUE)
r<- read.table(text=r$out_predict)
dp row.names(dp) == "Pham,Scott"),] dp[.(
## Years Salary pred sf pi.lwr pi.upr width
## Pham,Scott 13 81871.05 84955.08 11805.96 60962.48 108947.7 47985.2
Because reg()
accomplishes its computations with base R function lm()
, lm()
parameters can be passed to reg()
, which then passes their values to lm()
. Here, first use base R function contr.sum()
to calculate an effect coding contrast matrix for a categorical variable with three levels, such as the variable Plan in the Employee data set.
<- contr.sum(n=3)
cnt cnt
## [,1] [,2]
## 1 1 0
## 2 0 1
## 3 -1 -1
Now use the lm()
parameter contrasts
to define the effect coding for Plan, passed to reg_brief()
. Contrasts only apply to factors, so convert Plan to an R factor before the regression analysis.
$Plan <- factor(d$Plan)
dreg_brief(Salary ~ Plan, contrasts=list(Plan=cnt))
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ Plan, contrasts=list(Plan=cnt), Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
## Predictor Variable: Plan
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 37
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 76737.724 3897.284 19.690 0.000 68817.491 84657.958
## Plan1 -4166.287 5113.762 -0.815 0.421 -14558.703 6226.128
## Plan2 -6866.355 4920.990 -1.395 0.172 -16867.009 3134.299
##
##
## Standard deviation of residuals: 21,456.776 for 34 degrees of freedom
## 95% range of residual variation: 87,210.831 = 2 * (2.032 * 21,456.776)
##
## R-squared: 0.085 Adjusted R-squared: 0.031 PRESS R-squared: -0.133
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 1.580 df: 2 and 34 p-value: 0.221
##
##
## df Sum Sq Mean Sq F-value p-value
## Model 2 1454537623.133 727268811.566 1.580 0.221
## Residuals 34 15653370109.356 460393238.510
## Salary 36 17107907732.489 475219659.236
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## RESIDUALS AND INFLUENCE
##
## PREDICTION ERROR
The \(R^2\) fit statistic compares the sum of the squared errors of the model with the X predictor variables to the sum of squared errors of the null model. The baseline of comparison, the null model, is a model with no X variables such that the fitted value for each set of X values is the mean of response variable \(y\). The corresponding slope intercept is the mean of \(y\), and the standard deviation of the residuals is the standard deviation of \(y\).
The following submits the null model for Salary, and plots the errors. Compare the standard deviation of the residuals to a regression model of Salary with one or more predictor variables.
reg_brief(Salary ~ 1, plot_errors=TRUE)
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(Salary ~ 1, plot_errors=TRUE, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: d
##
## Response Variable: Salary
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 37
##
##
## BASIC ANALYSIS
##
## Salary: Mean = 73,795.557 SD = 21,799.533
## Min = 46,124.970 Max = 134,419.230
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 73795.557 3583.821 20.591 0.000 66527.230 81063.883
##
##
## Standard deviation of residuals: 21,799.533 for 36 degrees of freedom
## 95% range of residual variation: 88,423.006 = 2 * (2.028 * 21,799.533)
##
##
## df Sum Sq Mean Sq F-value p-value
## Residuals 36 17107907732.489 475219659.236
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## RESIDUALS AND INFLUENCE
##
## PREDICTION ERROR
Can also get the null model output from the lessR function Plot()
with the fit
parameter set to "null"
.
The scatterplot is displayed as a bubble plot when both variables consist of less than 10 unique integer values. With the bubble plot, there is no overprinting of the same point so that the number of values that represent a point is displayed.
<- Read("Mach4") dd
##
## >>> Suggestions
## Details about your data, Enter: details() for d, or details(name)
##
## Data Types
## ------------------------------------------------------------
## integer: Numeric data values, integers only
## ------------------------------------------------------------
##
## Variable Missing Unique
## Name Type Values Values Values First and last values
## ------------------------------------------------------------------------------------------
## 1 Gender integer 351 0 2 0 0 1 ... 0 0 1
## 2 m01 integer 351 0 6 0 0 2 ... 2 1 3
## 3 m02 integer 351 0 6 4 1 1 ... 3 4 3
## 4 m03 integer 351 0 6 1 4 0 ... 3 4 3
## 5 m04 integer 351 0 6 5 4 5 ... 3 4 4
## 6 m05 integer 351 0 6 0 0 4 ... 2 3 3
## 7 m06 integer 351 0 6 5 3 4 ... 4 4 2
## 8 m07 integer 351 0 6 4 3 0 ... 4 4 2
## 9 m08 integer 351 0 6 1 0 5 ... 3 2 3
## 10 m09 integer 351 0 6 5 4 3 ... 3 3 3
## 11 m10 integer 351 0 6 4 4 4 ... 3 4 3
## 12 m11 integer 351 0 6 0 0 1 ... 1 1 2
## 13 m12 integer 351 0 6 0 1 4 ... 2 1 3
## 14 m13 integer 351 0 6 0 1 0 ... 3 1 2
## 15 m14 integer 351 0 6 0 1 0 ... 2 2 2
## 16 m15 integer 351 0 6 4 2 2 ... 3 5 3
## 17 m16 integer 351 0 6 0 4 0 ... 0 2 5
## 18 m17 integer 351 0 6 1 4 2 ... 0 2 2
## 19 m18 integer 351 0 6 3 3 4 ... 4 4 3
## 20 m19 integer 351 0 6 2 1 0 ... 0 0 1
## 21 m20 integer 351 0 6 4 0 1 ... 1 0 3
## ------------------------------------------------------------------------------------------
reg_brief(m10 ~ m02, data=dd)
## >>> Suggestion
## # Create an R markdown file for interpretative output with Rmd = "file_name"
## reg(m10 ~ m02, data=dd, Rmd="eg")
##
##
## BACKGROUND
##
## Data Frame: dd
##
## Response Variable: m10
## Predictor Variable: m02
##
## Number of cases (rows) of data: 351
## Number of cases retained for analysis: 351
##
##
## BASIC ANALYSIS
##
## m10: Mean = 3.991 SD = 1.138
## Min = 0.000 Max = 5.000
##
##
## Estimate Std Err t-value p-value Lower 95% Upper 95%
## (Intercept) 4.285 0.092 46.642 0.000 4.104 4.466
## m02 -0.168 0.040 -4.184 0.000 -0.247 -0.089
##
##
## Standard deviation of residuals: 1.112 for 349 degrees of freedom
## 95% range of residual variation: 4.373 = 2 * (1.967 * 1.112)
##
## R-squared: 0.048 Adjusted R-squared: 0.045 PRESS R-squared: 0.037
##
## Null hypothesis of all 0 population slope coefficients:
## F-statistic: 17.502 df: 1 and 349 p-value: 0.000
##
##
## df Sum Sq Mean Sq F-value p-value
## Model 1 21.632 21.632 17.502 0.000
## Residuals 349 431.343 1.236
## m10 350 452.974 1.294
##
##
## K-FOLD CROSS-VALIDATION
##
## RELATIONS AMONG THE VARIABLES
##
## RESIDUALS AND INFLUENCE
##
## PREDICTION ERROR
The parameter Rmd
creates an R markdown file that is automatically generated and then the corresponding html document from knitting the various output components together with full interpretation. A new, much more complete form of computer output.
Not run here.
reg(Salary ~ Years + Pre, Rmd="eg")
Specify multiple logistic regression with the usual R formula syntax applied to the lessR function Logit()
. The output includes the confusion matrix and various classification fit indices.
Logit(Gender ~ Salary)
##
## Response Variable: Gender
## Predictor Variable 1: Salary
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 37
##
##
##
## BASIC ANALYSIS
##
## Model Coefficients
##
## Estimate Std Err z-value p-value Lower 95% Upper 95%
## (Intercept) -2.6191 1.3715 -1.910 0.056 -5.3073 0.0691
## Salary 0.0000 0.0000 1.904 0.057 -0.0000 0.0001
##
##
## Odds ratios and confidence intervals
##
## Odds Ratio Lower 95% Upper 95%
## (Intercept) 0.0729 0.0050 1.0715
## Salary 1.0000 1.0000 1.0001
##
##
## Model Fit
##
## Null deviance: 51.266 on 36 degrees of freedom
## Residual deviance: 46.918 on 35 degrees of freedom
##
## AIC: 50.91807
##
## Number of iterations to convergence: 4
##
##
##
##
## ANALYSIS OF RESIDUALS AND INFLUENCE
## Data, Fitted, Residual, Studentized Residual, Dffits, Cook's Distance
## [sorted by Cook's Distance]
## [res_rows = 20 out of 37 cases (rows) of data]
## --------------------------------------------------------------------
## Salary Gender fitted residual rstudent dffits cooks
## James, Leslie 122563 F 0.8424 -0.8424 -2.1213 -0.7143 0.46299
## Langston, Matthew 49189 M 0.2900 0.7100 1.6237 0.3646 0.08559
## Osterman, Pascal 49705 M 0.2938 0.7062 1.6139 0.3586 0.08225
## Kralik, Laura 92681 F 0.6522 -0.6522 -1.4942 -0.3313 0.06402
## Ritchie, Darnell 53788 M 0.3243 0.6757 1.5380 0.3136 0.05962
## Skrotzki, Sara 91352 F 0.6416 -0.6416 -1.4698 -0.3161 0.05736
## Cassinelli, Anastis 57562 M 0.3539 0.6461 1.4703 0.2761 0.04409
## Link, Thomas 66313 M 0.4267 0.5733 1.3223 0.2111 0.02335
## Anderson, David 69548 M 0.4547 0.5453 1.2706 0.1967 0.01962
## Stanley, Grayson 69625 M 0.4553 0.5447 1.2694 0.1965 0.01955
## Capelle, Adam 108138 M 0.7632 0.2368 0.7586 0.2236 0.01954
## Knox, Michael 99063 M 0.7011 0.2989 0.8637 0.2179 0.01935
## Hoang, Binh 111075 M 0.7813 0.2187 0.7265 0.2228 0.01919
## Sheppard, Cory 95028 M 0.6706 0.3294 0.9132 0.2119 0.01869
## Wu, James 94495 M 0.6665 0.3335 0.9199 0.2110 0.01859
## Campagna, Justin 72321 M 0.4788 0.5212 1.2275 0.1888 0.01759
## Fulton, Scott 87786 M 0.6124 0.3876 1.0066 0.1980 0.01706
## Adib, Hassan 83014 M 0.5720 0.4280 1.0715 0.1892 0.01613
## Pham, Scott 81871 M 0.5622 0.4378 1.0875 0.1875 0.01599
## Portlock, Ryan 77715 M 0.5261 0.4739 1.1469 0.1841 0.01593
##
##
## FORECASTS
##
## Probability threshold for predicting M: 0.5
##
## 0: F
## 1: M
##
## Data, Fitted Values, Standard Errors
## [sorted by fitted value]
## --------------------------------------------------------------------
## Salary Gender predict fitted std.err
## Stanley, Emma 46125 F 0 0.2684 0.1161
## Langston, Matthew 49189 M 0 0.2900 0.1126
## Osterman, Pascal 49705 M 0 0.2938 0.1119
## Gvakharia, Kimberly 49869 F 0 0.2949 0.1117
##
## ... for the rows of data where fitted is close to 0.5 ...
##
## Salary Gender predict fitted std.err
## Campagna, Justin 72321 M 0 0.4788 0.08710
## Korhalkar, Jessica 72502 F 0 0.4804 0.08713
## Billing, Susan 72675 F 0 0.4819 0.08718
## Portlock, Ryan 77715 M 1 0.5261 0.09079
## Pham, Scott 81871 M 1 0.5622 0.09670
##
## ... for the last 4 rows of sorted data ...
##
## Salary Gender predict fitted std.err
## Capelle, Adam 108138 M 1 0.7632 0.1355
## Hoang, Binh 111075 M 1 0.7813 0.1364
## James, Leslie 122563 F 1 0.8424 0.1318
## Correll, Trevon 134419 M 1 0.8901 0.1174
## --------------------------------------------------------------------
##
##
## ----------------------------
## Specified confusion matrices
## ----------------------------
##
## Probability threshold for predicting M: 0.5
##
## Baseline Predicted
## ---------------------------------------------------
## Total %Tot 0 1 %Correct
## ---------------------------------------------------
## 0 19 51.4 16 3 84.2
## Gender 1 18 48.6 8 10 55.6
## ---------------------------------------------------
## Total 37 70.3
##
## Accuracy: 70.27
## Recall: 55.56
## Precision: 76.92
Specify additional probability thresholds for classification beyond just the default 0.5 with the prob_cut
parameter.
Logit(Gender ~ Years + Salary, prob_cut=c(.3, .5, .7))
##
## Response Variable: Gender
## Predictor Variable 1: Years
## Predictor Variable 2: Salary
##
## Number of cases (rows) of data: 37
## Number of cases retained for analysis: 36
##
##
##
## BASIC ANALYSIS
##
## Model Coefficients
##
## Estimate Std Err z-value p-value Lower 95% Upper 95%
## (Intercept) -0.1922 1.5968 -0.120 0.904 -3.3218 2.9374
## Years 0.4041 0.1801 2.244 0.025 0.0511 0.7571
## Salary -0.0001 0.0000 -1.317 0.188 -0.0001 0.0000
##
##
## Odds ratios and confidence intervals
##
## Odds Ratio Lower 95% Upper 95%
## (Intercept) 0.8251 0.0361 18.8658
## Years 1.4979 1.0524 2.1320
## Salary 0.9999 0.9999 1.0000
##
##
## Model Fit
##
## Null deviance: 49.795 on 35 degrees of freedom
## Residual deviance: 38.772 on 33 degrees of freedom
##
## AIC: 44.77244
##
## Number of iterations to convergence: 4
##
##
## Collinearity
##
## Tolerance VIF
## Years 0.274 3.655
## Salary 0.274 3.655
##
##
##
## ANALYSIS OF RESIDUALS AND INFLUENCE
## Data, Fitted, Residual, Studentized Residual, Dffits, Cook's Distance
## [sorted by Cook's Distance]
## [res_rows = 20 out of 36 cases (rows) of data]
## --------------------------------------------------------------------
## Years Salary Gender fitted residual rstudent dffits cooks
## Skrotzki, Sara 18 91352 F 0.9174 -0.9174 -2.4427 -0.6774 0.35497
## James, Leslie 18 122563 F 0.6923 -0.6923 -1.7374 -0.8959 0.28507
## Hoang, Binh 15 111075 M 0.5465 0.4535 1.1867 0.5568 0.08266
## LaRoe, Maria 10 61961 F 0.6635 -0.6635 -1.5384 -0.4480 0.06884
## Langston, Matthew 5 49189 M 0.3344 0.6656 1.5340 0.4122 0.05852
## Osterman, Pascal 5 49705 M 0.3286 0.6714 1.5451 0.4083 0.05799
## Saechao, Suzanne 8 55545 F 0.5495 -0.5495 -1.3072 -0.3747 0.04143
## Bellingar, Samantha 10 66338 F 0.6118 -0.6118 -1.4120 -0.3359 0.03598
## Campagna, Justin 8 72321 M 0.3409 0.6591 1.5015 0.3235 0.03574
## Ritchie, Darnell 7 53788 M 0.4712 0.5288 1.2641 0.3408 0.03351
## Correll, Trevon 21 134419 M 0.8048 0.1952 0.7135 0.3834 0.03266
## Kralik, Laura 10 92681 F 0.2905 -0.2905 -0.8663 -0.3233 0.02467
## Cassinelli, Anastis 10 57562 M 0.7117 0.2883 0.8576 0.2983 0.02098
## Kimball, Claire 8 61357 F 0.4754 -0.4754 -1.1573 -0.2510 0.01725
## Downs, Deborah 7 57140 F 0.4288 -0.4288 -1.0800 -0.2502 0.01643
## Sheppard, Cory 14 95028 M 0.6464 0.3536 0.9581 0.2575 0.01638
## Stanley, Grayson 9 69625 M 0.4707 0.5293 1.2459 0.2329 0.01568
## Anderson, David 9 69548 M 0.4717 0.5283 1.2442 0.2327 0.01564
## Link, Thomas 10 66313 M 0.6121 0.3879 1.0112 0.2391 0.01450
## Fulton, Scott 13 87786 M 0.6387 0.3613 0.9649 0.2213 0.01217
##
##
## FORECASTS
##
## Probability threshold for predicting M: 0.5
##
## 0: F
## 1: M
##
## Data, Fitted Values, Standard Errors
## [sorted by fitted value]
## --------------------------------------------------------------------
## Years Salary Gender predict fitted std.err
## Korhalkar, Jessica 2 72502 F 0 0.04339 0.05839
## Singh, Niral 2 61055 F 0 0.07533 0.07374
## Hamide, Bita 1 51037 F 0 0.08324 0.07439
## Billing, Susan 4 72675 F 0 0.09163 0.08914
##
## ... for the rows of data where fitted is close to 0.5 ...
##
## Years Salary Gender predict fitted std.err
## Ritchie, Darnell 7 53788 M 0 0.4712 0.13798
## Anderson, David 9 69548 M 0 0.4717 0.09822
## Kimball, Claire 8 61357 F 0 0.4754 0.11321
## Hoang, Binh 15 111075 M 1 0.5465 0.21933
## Saechao, Suzanne 8 55545 F 1 0.5495 0.14527
##
## ... for the last 4 rows of sorted data ...
##
## Years Salary Gender predict fitted std.err
## Correll, Trevon 21 134419 M 1 0.8048 0.19249
## Knox, Michael 18 99063 M 1 0.8822 0.09262
## Skrotzki, Sara 18 91352 F 1 0.9174 0.07830
## Capelle, Adam 24 108138 M 1 0.9815 0.02894
## --------------------------------------------------------------------
##
##
## ----------------------------
## Specified confusion matrices
## ----------------------------
##
## Probability threshold for predicting M: 0.3
##
## Baseline Predicted
## ---------------------------------------------------
## Total %Tot 0 1 %Correct
## ---------------------------------------------------
## 0 19 52.8 14 5 73.7
## Gender 1 17 47.2 6 11 64.7
## ---------------------------------------------------
## Total 36 69.4
##
## Accuracy: 69.44
## Recall: 64.71
## Precision: 68.75
##
##
##
## Probability threshold for predicting M: 0.5
##
## Baseline Predicted
## ---------------------------------------------------
## Total %Tot 0 1 %Correct
## ---------------------------------------------------
## 0 19 52.8 14 5 73.7
## Gender 1 17 47.2 6 11 64.7
## ---------------------------------------------------
## Total 36 69.4
##
## Accuracy: 69.44
## Recall: 64.71
## Precision: 68.75
##
##
##
## Probability threshold for predicting M: 0.7
##
## Baseline Predicted
## ---------------------------------------------------
## Total %Tot 0 1 %Correct
## ---------------------------------------------------
## 0 19 52.8 14 5 73.7
## Gender 1 17 47.2 6 11 64.7
## ---------------------------------------------------
## Total 36 69.4
##
## Accuracy: 69.44
## Recall: 64.71
## Precision: 68.75
Use the base R help()
function to view the full manual for Regression()
. Simply enter a question mark followed by the name of the function, or its abbreviation.
?reg