The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Shiny App: Grading

Introduction

The grading app is an instructor-facing shiny web application that provides a comprehensive environment for evaluating student submissions. Unlike the similarity app, which focuses on comparing submissions to each other, the grading app centers on assessing correctness against a model solution. It turns a plain model solution into an autoharp grading template, grades every submission with autoharp’s render_one() function, supports manual grading of the plots students produce, and saves progress so that grading can be spread over several sessions. To run the grading app locally:

library(shiny)
runApp(system.file(file.path('shiny', 'grading_app', 'app.R'),
                   package = 'autoharp'))

The application follows a single workflow across three tabs: Template Generation for turning the model solution into a grading template, Autoharp Grading for running render_one() on all student scripts, and Plot Grading for manual assessment of visualizations. Saving and loading sessions is available at any time from the buttons in the top-right corner of the header.

Session Management

A key design principle of the grading app is workflow persistence. Grading large classes often spans multiple sessions, and losing progress due to application restarts or system interruptions can be frustrating. The Save Session and Load Session buttons in the header keep this functionality visible without interrupting the main grading flow. Sessions are stored as RDS files containing the complete application state: the model solution and generated template, the grading template, folder settings, grading results, plot grades with comments, and the current position in the grading workflow. The application auto-saves after grading completes and after each plot grade is saved, and the status line under the buttons shows when and where progress was last saved. Session files are written to the sessions folder inside the grading output directory, and instructors can save manually at any time.

grading app session

To resume a previous session, the instructor clicks “Load Session”, uploads the saved RDS file and confirms. The dialog also shows information about the current session (session ID, when and where it was last saved, and how many students have been graded). The application restores all state, including input field values and the current student being graded, allowing work to continue exactly where it left off.

Template Generation

The Template Generation tab creates an autoharp-compatible grading template without requiring instructors to write test code. The instructor uploads a model solution (an R Markdown or Quarto file containing correct implementations) and clicks “Generate Template”. If the model solution reads data files, the instructor also sets the folder it should run from, so that relative paths such as ../data/file.csv resolve correctly; a solution that reads no files can be run without one. The student scripts folder is not needed here: it is set in the Autoharp Grading tab, where the grading happens.

grading app template generation

The key innovation in this tab is the type-based test generation. When the model solution is executed, the application inspects each created object and determines its type. Based on this type information, it suggests appropriate tests automatically: for example, length, mean and standard deviation checks for numeric vectors, dimension and column checks for data frames, and argument checks for functions. The objects found are listed with checkboxes next to a table of the suggested tests for each selected object, and instructors can choose which objects to include. If the model solution produces errors (for example because a data file cannot be found), the setup panel lists them so that the problem can be fixed before grading. The full list of automated tests generated is as follows:

Test Description
Numeric vectors
length Compares vector lengths
mean/sd/min/max/sum Statistical comparisons with tolerance
NA count Counts missing values
identical values Element-by-element comparison
Data frames
nrow/ncol Dimension comparisons
column names Checks column name sets match
column types Verifies class of each column
numeric column statistics Mean and SD for numeric columns
Matrices
dimensions Row and column count comparison
row/column names Name attribute comparison
element-wise difference Maximum absolute difference below threshold
Functions
argument count Number of formal arguments
argument names Argument name set comparison
Factors
level count Number of factor levels
level names/order Level set and ordering comparison
level frequencies Frequency distribution comparison

The generated template adds test chunks to the model solution. Each chunk includes the autoharp.objs option (specifying which objects to extract from student code) and the autoharp.scalars option (specifying which test results to record). The generated code follows autoharp’s conventions, using dot-prefixed names for solution objects (e.g., .X for the solution’s X object). Instructors can preview the generated template before downloading it, allowing them to verify the test structure and make any necessary adjustments. The downloaded template can be used directly with populate_soln_env() and render_one(), or passed straight to the Autoharp Grading tab with the “Use for Autoharp Grading” button, which also carries over the folder the solution runs from and the test chunk prefix.

Autoharp Grading Integration

The Autoharp Grading tab provides a graphical interface for running autoharp’s core grading workflow. Instead of writing R scripts to call populate_soln_env() and render_one(), instructors use the template from the Template Generation tab (or upload their own) and specify the student scripts folder through the interface. As the folder path is typed, the app reports how many student files it found and shows the full path, so relative paths can be checked at a glance.

grading app grading

The tab offers several configuration options:

After grading completes, results appear in a table showing run status (SUCCESS or FAIL), execution time, memory usage, and all correctness check scalars defined in the template. If plot grades have been assigned in the Plot Grading tab, they appear alongside the automated results. When the student scripts produced plots, the tab reports how many were found and offers a shortcut to the Plot Grading tab. The tab includes an Excel export feature that creates a workbook with two sheets: one containing all grading results together with plot marks and comments, and another containing the plot grades. This format facilitates integration with course management systems and grade books.

Manual Plot Grading

The Plot Grading tab addresses a challenge that automated systems cannot fully solve: evaluating the quality of data visualizations. While code correctness can be verified programmatically, assessing whether a plot effectively communicates its message requires human judgment.

grading app plot grading

When autoharp grading runs, the application collects all plots generated by each student script, whether it is an R script or an R Markdown or Quarto file. These plots are stored in the grading output folder.

The Plot Grading interface allows instructors to navigate through students sequentially (using Previous/Next buttons or a dropdown selector) and view the current student’s plots one at a time, together with details such as the file name, image size and creation time. A plot can be opened in a full-size view, making it easier to examine details. Instructors assign a numeric plot mark on whatever scale they use, with optional comments; saving the grade moves on to the next student, and a mark that was edited but not saved is kept when moving between students. The tab also displays all automated test results for the current student, providing context when assessing plots. For example, if a student’s data processing produced incorrect values, this context helps explain why their visualization may look different from expectations. A consolidated table combines the automated results with the plot marks and comments for every student and can be exported to Excel. A progress bar tracks grading completion, and all grades are auto-saved to the session file after each entry. This combination of automated collection, easy navigation, and persistent storage streamlines what would otherwise be a tedious manual process.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.