New R/exams Version: exams2forms, Written NOPS Exams, and More

Overview

The new version 2.4-2 of the R/exams package has been released on the Comprehensive R Archive Network at https://CRAN.R-project.org/package=exams along with version 0.2-0 of the accompanying exams2forms package at https://CRAN.R-project.org/package=exams2forms. For an overview of all changes in both packages, see their respective NEWS files: exams NEWS and exams2forms NEWS.

The development of exams2forms was originally prompted by requests for R/exams infrastructure in online books via R/Markdown or Quarto. However, it turned out to be very useful for quite a few other applications including:

• Testing exercises interactively during development without the need for import in a learning managment system.
• Standalone (non-graded) quizzes as self-contained files, e.g., for download as open educational resources.
• Browsing exercise pools, e.g., for sharing previews within a team.

The other main area of development were extensions to the written multiple-choice exams (NOPS) that can be automatically scanned and evaluated. This was prompted by the decision of the Faculty of Economics and Statistics at Universität Innsbruck to conduct all of their large-scale exams during the first years of the bachelor program using this infrastructure. Below we give an overview of the R functionality in the exams package that support this endeavor. Those who are interested in how the administrative aspects are organized in Gitlab repositories with corresponding issues can check out the workflow documentation and a demo project (both in German) on the university’s self-hosted Gitlab system.

Interactive web exercises via exams2forms

Version 0.1-0 of the package had been introduced in an exams2forms tutorial in a previous blog post. Prompted by our own needs as well as a lot of positive and constructive feedback, version 0.2-0 now contains many useful improvements and new features. Also, all exercise template pages on the R/exams web page now feature an interactive preview generated with exams2forms using three random variations of each exercise. See boxplots or lm2 for two examples.

Obfuscation

The function exams2forms() as well as all underlying forms_*() functions gained an argument obfuscate = TRUE which provides some basic obfuscation of the correct answers for all forms in the underlying HTML code. Thus, all correct answers are still stored in the HTML but, by default, they are not easily readable anymore.

Auto-display of filled-in exercises

The function exams2forms() gained additional arguments auto, show_filename, and show_tolerance, which are useful for inspecting exercises during their development. To see the results with all forms pre-filled, check enabled, and display of solution, tolerances, and the file name, try:

exams2webquiz("lm2.Rmd", auto = TRUE)

Regular expression in string solutions

The package now ships with some example exercises that illustrate a feature of exams2forms that is not (or not easily) available in other exams2xyz interfaces: regular expressions for the correct answers of string exercises. To generate a demo quiz with these you can use:

exams2webquiz(c("geography2.Rmd", "email.Rmd"), regex = TRUE, n = 3,
  edir = system.file(package = "exams2forms"))

• The exercise geography2.Rmd simply lists several solutions that are accepted using “or” regular expressions: ^(answer1|answer2|answer3)$
• The exercise email.Rmd has a more complex regular expression for checking the validity of e-mail addresses. geography2.Rmd exercise.

Multiple-choice NOPS exams

R/exams long provides infrastructure for large-scale multiple-choice exams that can be automatically scanned and evaluated, see the exams2nops tutorial for an introduction. Previously, the workflow just comprised exams2nops() for generating the PDF exams, nops_scan() for reading the filled-out and scanned exam sheets, and nops_eval() for evaluating all results, grading the exams, and generating reports for the participants.

However, addressing errors that occurred, in particularl during scanning, used to be rather inconvenient and necessitated editing some of the intermediate files manually. Therefore, version 2.4-1 of the package introduced a new function nops_fix() that greatly facilitates interactive quality control of exam sheets directly in R (and/or the browser). A dedicated tutorial for this functionality is under preparation.

Below we summarize the most important additions and improvements from version 2.4-1 (which had not yet been included in previous blog posts) and 2.4-2.

nops_fix

The new function nops_fix() can be applied to the ZIP file resulting from nops_scan(). By default it goes through all rows of the scanned data and interactively prompts for updates to fields from the scanned exam sheets that need updating. It can also be applied repeatedly to iteratively check and resolve different potential problems.

• Optionally, the user can specify the rows of the scanned data and/or the fields that should be updated.
• Different display options are available with the two most important being: (1) Displaying only a section of the scanned sheet in the R plot window with interactive prompts on the R console. (2) Displaying the entire sheet in the browser with a Javascript form to enter all data which can be passed to R through the clipboard.
• Display 1 is most convenient for iterating through smaller potential problems in sheets that have been scanned mostly correctly.
• Display 2 is most convenient for entering scanned data for sheets that could not be read at all (e.g., due to rotation or damaged sheets etc.).
• The default display tries to make an educated guess for the more convenient option.
• Dedicated check = "schoice" or check = "missing" arguments are provided to go through exercises with more than one answer or missing answers, respectively.

nops_scan

By default, nops_scan() now relies on the R packages qpdf and magick for merging/rotating/splitting PDF files and converting them to PNG, respectively. This greatly facilitates installation of the scanning infrastructure, especially for novice R/exams users.

Improvements have been made in nops_scan() for reading scans from different paper formats (e.g., letter paper instead of A4) and for reading the registration id more reliably. Also, scanner markings are found more reliably, especially if the top-left marking is missing completely (e.g., because it was cut erroneously).

exams2nops and language support

Additional NOPS language support has been added: Bulgarian (bg, contributed by Nikolay Rachev), Catalan (ca, contributed by Paco Rivière), and Polish (pl, contributed by Paweł Kleka). Corrections in French (fr, contributed by Jean-Philippe Georget), Russian (ru, contributed by Nikolay Rachev), and Spanish (es, contributed by Flavio Lozano Isla).

Moreover, there are various improvements and new arguments in exams2nops():

• New argument helvet = TRUE which allows to suppress the default Helvetica font in order to facilitate using other fonts.
• New argument newpage = FALSE to facilitate adding page breaks after every exercise.
• The logo can now also be a relative path (relative to the working directory). Also logo = "uibk" is supported as a convenience option for including the logo of Universität Innsbruck.
• The exam type now codes the number of exercises directly (rather than rounding the number to multiples of 5). This facilitates fixing the correct number of questions in nops_fix().
• Answer lists are now labeled “a.”, “b.”, etc. everywhere rather than “(a)”, “(b)”, etc.

More features and improvements

New exercises

There are two new demo exercises: flags illustrating the handling of Unicode characters for flags in a flexible way. sumdiff is a minimal example for using random numbers in arithmetic exercises.

Evaluation strategies in learning managment systems

The default evaluation rule for all learning management systems (including exams2moodle, exams2canvas, exams2openolat, etc.) is now consistently negative = FALSE and partial = TRUE with rule = "false2". Thus, there should not be negative points for any exercise type and partial credits should be used for mchoice exercises (and mchoice interactions within cloze exercises).

Stress-testing exercises

New arguments have been added in stresstest_exercise() to facilitate stress-testing exercises, especially for large collections of exercises:

• stop_on_error = length(as.character(unlist(file))) < 2: Should stresstesting stop or continue after encountering an error in one of the exercises?
• timeout = NULL: Set a time limit for running each exercise, e.g., to avoid running into infinite loops etc.
• maxit = getOption("num_to_schoice_maxit", -10000L): Default number of iterations in case num_to_schoice() is used (see below).

Standalone PDF files

Customizing the LaTeX template in exams2pdf() has been facilitated by adding the usepackage argument and streamlining processing of the header argument.

Additionally, the {Sweave} LaTeX package is no longer loaded in the LaTeX templates (like plain.tex, exams.tex, solution.tex, etc.). Instead the LaTeX environments for displaying R code and the accompanying LaTeX dependencies are loaded directly.

Misc

• Include better table formatting by default in exams2moodle() via styles table = "table_shade" (default), "table_rule", or "table_grid".
• New arguments in num_to_schoice() to facilitate converting numeric to single-choice exercises:
  • format = TRUE: Should the question list be formatted to a character vector with LaTeX math markup (default)? Alternatively, the question list can be numeric (format = FALSE or "numeric") or a formatted character vector without LaTeX math markup (format = "character").
  • order = getOption("num_to_schoice_order", FALSE): Should the question list be ordered numerically? If FALSE (default) the question list is shuffled randomly.
  • maxit = getOption("num_to_schoice_maxit", Inf): Maximum number of iterations to try to find a feasible set of wrong solutions for the question list.