R/exams

penguins: Cloze Quiz for Exploratory Analysis of Penguins Data

2026-01-07T00:00:00+01:00

Cloze exercise exploring sex differences in body mass for a randomly selected species of penguins, based on the eponymous data in base R.

Name:

penguins

Type:

cloze

Related:

lm2, boxhist2

Preview:

The penguins data in base R provides various measurements of adult penguins from three different species. See ?penguins for more details. Originally, the data was used to study sex dimorphism separately for the three species.

The first three rows of the data can be inspected as follows. Employ summary() to obtain a first overview.

data("penguins", package = "datasets") head(penguins, 3)

## species island bill_len bill_dep flipper_len body_mass sex year ## 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007 ## 2 Adelie Torgersen 39.5 17.4 186 3800 female 2007 ## 3 Adelie Torgersen 40.3 18.0 195 3250 female 2007

Explore the sex differences with respect to body mass (weight, in grams) of the penguins. Create parallel boxplots of weight by sex, such as the one below, separately for the three species.

Which species does this plot pertain to?

To complement the plot complete the corresponding table of groupwise statistics:

	median	mean	std. deviation
female
male

The average weight difference of is thus slightly than the median weight difference of .

Compute the full summary() of weight by sex for this species and select the correct statements in the following list.

Less than half of the male penguins weigh more than 3731.25 grams.None of the penguins weighs less than 2700 grams.The standard deviation of weight is lower for males compared to females.

One way to obtain the exploratory boxplots separately for the three species is:

par(mfrow = c(1, 3)) for(i in levels(penguins$species)) plot(body_mass ~ sex, data = penguins, subset = species == i, main = i, ylim = range(penguins$body_mass, na.rm = TRUE))

The question shows the parallel boxplots for the Chinstrap species.

Groupwise statistics of body mass by sex and species (including mean, median, and standard deviation) can be obtained by aggregating the data with the combined summary() and sd() functions.

aggregate(body_mass ~ sex + species, data = penguins, FUN = function(x) c(summary(x), `Std. dev.` = sd(x)))

## sex species body_mass.Min. body_mass.1st Qu. body_mass.Median ## 1 female Adelie 2850.0000 3175.0000 3400.0000 ## 2 male Adelie 3325.0000 3800.0000 4000.0000 ## 3 female Chinstrap 2700.0000 3362.5000 3550.0000 ## 4 male Chinstrap 3250.0000 3731.2500 3950.0000 ## 5 female Gentoo 3950.0000 4462.5000 4700.0000 ## 6 male Gentoo 4750.0000 5300.0000 5500.0000 ## body_mass.Mean body_mass.3rd Qu. body_mass.Max. body_mass.Std. dev. ## 1 3368.8356 3550.0000 3900.0000 269.3801 ## 2 4043.4932 4300.0000 4775.0000 346.8116 ## 3 3527.2059 3693.7500 4150.0000 285.3339 ## 4 3938.9706 4100.0000 4800.0000 362.1376 ## 5 4679.7414 4875.0000 5200.0000 281.5783 ## 6 5484.8361 5700.0000 6300.0000 313.1586

Based on this the remaining elements of the question can be answered.

The first three rows of the data can be inspected as follows. Employ summary() to obtain a first overview.

data("penguins", package = "datasets") head(penguins, 3)

## species island bill_len bill_dep flipper_len body_mass sex year ## 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007 ## 2 Adelie Torgersen 39.5 17.4 186 3800 female 2007 ## 3 Adelie Torgersen 40.3 18.0 195 3250 female 2007

Explore the sex differences with respect to body mass (weight, in grams) of the penguins. Create parallel boxplots of weight by sex, such as the one below, separately for the three species.

Which species does this plot pertain to?

To complement the plot complete the corresponding table of groupwise statistics:

	median	mean	std. deviation
female
male

The average weight difference of is thus slightly than the median weight difference of .

Compute the full summary() of weight by sex for this species and select the correct statements in the following list.

The standard deviation of weight is lower for males compared to females.Less than half of the female penguins weigh more than 3175 grams.None of the penguins weighs less than 2850 grams.

One way to obtain the exploratory boxplots separately for the three species is:

par(mfrow = c(1, 3)) for(i in levels(penguins$species)) plot(body_mass ~ sex, data = penguins, subset = species == i, main = i, ylim = range(penguins$body_mass, na.rm = TRUE))

The question shows the parallel boxplots for the Adelie species.

Groupwise statistics of body mass by sex and species (including mean, median, and standard deviation) can be obtained by aggregating the data with the combined summary() and sd() functions.

aggregate(body_mass ~ sex + species, data = penguins, FUN = function(x) c(summary(x), `Std. dev.` = sd(x)))

## sex species body_mass.Min. body_mass.1st Qu. body_mass.Median ## 1 female Adelie 2850.0000 3175.0000 3400.0000 ## 2 male Adelie 3325.0000 3800.0000 4000.0000 ## 3 female Chinstrap 2700.0000 3362.5000 3550.0000 ## 4 male Chinstrap 3250.0000 3731.2500 3950.0000 ## 5 female Gentoo 3950.0000 4462.5000 4700.0000 ## 6 male Gentoo 4750.0000 5300.0000 5500.0000 ## body_mass.Mean body_mass.3rd Qu. body_mass.Max. body_mass.Std. dev. ## 1 3368.8356 3550.0000 3900.0000 269.3801 ## 2 4043.4932 4300.0000 4775.0000 346.8116 ## 3 3527.2059 3693.7500 4150.0000 285.3339 ## 4 3938.9706 4100.0000 4800.0000 362.1376 ## 5 4679.7414 4875.0000 5200.0000 281.5783 ## 6 5484.8361 5700.0000 6300.0000 313.1586

Based on this the remaining elements of the question can be answered.

The first three rows of the data can be inspected as follows. Employ summary() to obtain a first overview.

data("penguins", package = "datasets") head(penguins, 3)

## species island bill_len bill_dep flipper_len body_mass sex year ## 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007 ## 2 Adelie Torgersen 39.5 17.4 186 3800 female 2007 ## 3 Adelie Torgersen 40.3 18.0 195 3250 female 2007

Explore the sex differences with respect to body mass (weight, in grams) of the penguins. Create parallel boxplots of weight by sex, such as the one below, separately for the three species.

Which species does this plot pertain to?

To complement the plot complete the corresponding table of groupwise statistics:

	median	mean	std. deviation
female
male

The average weight difference of is thus slightly than the median weight difference of .

Compute the full summary() of weight by sex for this species and select the correct statements in the following list.

The standard deviation of weight is lower for males compared to females.Less than half of the female penguins weigh more than 3362.5 grams.None of the penguins weighs less than 2700 grams.

One way to obtain the exploratory boxplots separately for the three species is:

par(mfrow = c(1, 3)) for(i in levels(penguins$species)) plot(body_mass ~ sex, data = penguins, subset = species == i, main = i, ylim = range(penguins$body_mass, na.rm = TRUE))

The question shows the parallel boxplots for the Chinstrap species.

Groupwise statistics of body mass by sex and species (including mean, median, and standard deviation) can be obtained by aggregating the data with the combined summary() and sd() functions.

aggregate(body_mass ~ sex + species, data = penguins, FUN = function(x) c(summary(x), `Std. dev.` = sd(x)))

## sex species body_mass.Min. body_mass.1st Qu. body_mass.Median ## 1 female Adelie 2850.0000 3175.0000 3400.0000 ## 2 male Adelie 3325.0000 3800.0000 4000.0000 ## 3 female Chinstrap 2700.0000 3362.5000 3550.0000 ## 4 male Chinstrap 3250.0000 3731.2500 3950.0000 ## 5 female Gentoo 3950.0000 4462.5000 4700.0000 ## 6 male Gentoo 4750.0000 5300.0000 5500.0000 ## body_mass.Mean body_mass.3rd Qu. body_mass.Max. body_mass.Std. dev. ## 1 3368.8356 3550.0000 3900.0000 269.3801 ## 2 4043.4932 4300.0000 4775.0000 346.8116 ## 3 3527.2059 3693.7500 4150.0000 285.3339 ## 4 3938.9706 4100.0000 4800.0000 362.1376 ## 5 4679.7414 4875.0000 5200.0000 281.5783 ## 6 5484.8361 5700.0000 6300.0000 313.1586

Based on this the remaining elements of the question can be answered.

Description:

This cloze exercise randomly selects one (out of three) species of penguins from the eponymous data set in base R and then asks for a number of exploratory analyses investigating sex differences in body weight for the selected penguin species. The exercise template illustrates how such a data-based cloze including all types of items (numeric, string, single-choice, multiple-choice) can be created relatively easily using the add_cloze() function.

Solution feedback:

Yes

Randomization:

Shuffling (1 out of 3 species)

Mathematical notation:

Verbatim R input/output:

Yes

Images:

Yes

Other supplements:

Template:

penguins.Rmd

penguins.Rnw

Raw: (1 random version)

penguins.md

penguins.tex

PDF:

HTML:

Demo code:

library("exams") set.seed(403) exams2html("penguins.Rmd") set.seed(403) exams2pdf("penguins.Rmd") set.seed(403) exams2html("penguins.Rnw") set.seed(403) exams2pdf("penguins.Rnw")

Quality Control for Scanned Multiple-Choice Exams

2025-09-15T00:00:00+02:00

Step-by-step guide to using nops_fix() for fixing problems that occurred during scanning written NOPS exams in R/exams.

Overview

The R/exams package provides a workflow for generating (exams2nops()), scanning (nops_scan()), and automatically evaluating (nops_eval()) large-scale pen-and-paper multiple-choice exams, see the corresponding NOPS tutorial for more details.

Recently, another function nops_fix() was added to the package that helps to find and correct problems that occurred when reading the scanned exams and that might impede the automatic evaluation process. Typical problems include scans that are rotated or with corrupted scanner markers, answer boxes that are not correctly checked or filled, or missing checks in the boxes for the registration IDs, among others. Therefore, nops_fix() helps the examiner to resolve such problems after nops_scan() and before nops_eval() to ensure accurate grading.

In this tutorial, we highlight the key features of nops_fix():

Automatic display of problematic sheets
Prompting for manual corrections
Specialized checks for implausible answer patterns
Manual review of specific exam sheets

These are illustrated by fixing the PDF scans from two demo exams: one with single-choice exercises (and only smaller problems) and one with multiple-choice exercises (and more severe problems). Some more detailed comments about all the features are provided at the very end of this tutorial.

Demo 1: Single-choice exam with typical problems

Create exam

First, we generate five PDFs (in a new subdirectory "nops") from a demo exam with five single-choice exercises (all provided in the package: swisscapital, Rlogo, deriv2, fruit2, hessian).

library("exams") sc <- c("swisscapital.Rmd", "Rlogo.Rmd", "deriv2.Rmd", "fruit2.Rmd", "hessian.Rmd") set.seed(403) exams2nops(sc, n = 5, dir = "nops", date = "2024-10-07")

Conduct exam and scan results

Suppose we conducted the exam with three participants and scanned their exam sheets (first page), rotated upside down (typically facilitating scanning because the top-left corner is damaged from the staples).

Here, you can download the PDF file scans-schoice.pdf:

Then we can read the information from the scanned sheets via nops_scan(), resulting in a file nops_scan_*.zip. The * placeholder will correspond to the date/time where you run the code. The file contains the three scanned exam sheets converted to PNG along with a text file Daten.txt containing the information that R has extracted from the scanned sheets.

nops_scan("scans-schoice.pdf", rotate = TRUE)

Fix scan results and final evaluation

Subsequently, calling the function nops_fix() without any arguments automatically searches for files named nops_scan_*.zip and checks for any obvious errors, such as incorrectly entered registration IDs in the provided demo sheets.

nops_fix()

When you run the code, R displays a section of the exam sheet and prompts you to manually enter the correct registration number. In this case, the program cannot read the number nine because the cross in the penultimate position has erroneously been entered in the last column.

After entering the registration number, the corrected data is saved in the updated zip file.

## Correct registration number (for 0000000, S0000002.PNG): 0157193 ## adding: Daten.txt (deflated 89%) ## adding: S0000001.PNG (deflated 8%) ## adding: S0000002.PNG (deflated 6%) ## adding: S0000003.PNG (deflated 6%)

Additionally, we can check whether in this schoice exam there is exactly one checked box per question. Implausible answer patterns without checked boxes or with more than one checked box might be actual answers from the examinees but might also be due to scanning problems where checked or filled boxes are not detected correctly. If the scanned answer needs correction, you can either enter the correct answer as a letter (a, b, …), as a number (1, 2, …), or in binary coding (e.g., 00010). To accept the scanned result, simply press <Enter>.

First, we check and (if necessary) correct cases with more than one checked box:

nops_fix(check = "schoice")

## Correct answer 3 (for 01010, S0000003.PNG): d ## Correct answer 4 (for 01001, S0000003.PNG): ## adding: Daten.txt (deflated 89%) ## adding: S0000001.PNG (deflated 8%) ## adding: S0000002.PNG (deflated 6%) ## adding: S0000003.PNG (deflated 6%)

Second, we check for cases with missing answers:

nops_fix(check = "missing")

## Correct answer 2 (for 00000, S0000003.PNG): ## adding: Daten.txt (deflated 89%) ## adding: S0000001.PNG (deflated 8%) ## adding: S0000002.PNG (deflated 6%) ## adding: S0000003.PNG (deflated 6%)

Finally, after resolving the problems above in the nops_scan_*.zip file, the single-choice exam can be evaluated. This additionally requires the correct solutions from the exams2nops() metainformation (produced above) and a list with the registered participants (containing the registration number, name, and user ID). The latter can be downloaded as participants-schoice.csv.

ev <- nops_eval( register = "participants-schoice.csv", solutions = "nops/metainfo.rds", scans = Sys.glob("nops_scan_*.zip"))

This produces the usual nops_eval.* output which are described in more detail in the NOPS tutorial.

To clean up before next demo you can use the following code:

unlink("nops", recursive = TRUE) file.remove(Sys.glob("nops_eval.*")) file.remove(Sys.glob("nops_scan_*.zip"))

Demo 2: Multiple-choice exam with severe problems

Create exam

Similar to the first demo, we first generate five PDFs in the subdirectory "nops" for a demo exam with six multiple-choice exercises provided in the package (capitals, switzerland, gaussmarkov, boxplots, scatterplot, ttest).

library("exams") mc <- c("capitals.Rmd", "switzerland.Rmd", "gaussmarkov.Rmd", "boxplots.Rmd", "scatterplot.Rmd", "ttest.Rmd") set.seed(404) exams2nops(mc, n = 5, dir = "nops", date = "2024-10-08")

Conduct exam and scan results

Again, suppose we conducted the exam with three participants and scanned their exam sheets (first page), rotated upside down.

Here, you can download the PDF file scans-mchoice.pdf:

Then we can read the information from the scanned sheets via nops_scan(), resulting in a file nops_scan_*.zip.

nops_scan("scans-mchoice.pdf", rotate = TRUE)

In this case, this leads to a warning message that there are errors in the scanned data and that running nops_fix() is necessary prior to nops_eval().

Fix scan results and final evaluation

As above, calling the function nops_fix() without any arguments automatically searches for files named nops_scan_*.zip and checks for any obvious errors. Here, the third page is very heavily rotated and the scanner markers are all corrupted so that the PDF could not be read at all.

Due to the bad rotation and corrupted markers, it is is not possible for nops_fix() to extract the individual fields reliably enough. Hence, it displays the entire scanned exam sheet in the browser, next to an interactive web form where all fields can be entered manually. As not even the “Type” field can be read, nops_fix() does not know that there are only six exercises and hence it displays 45 exercises, the maximum possible.

nops_fix()

After entering all fields, press the “OK” button in the browser (bottom right). This copies all the data entered into the clipboard (in a JSON string) and R will either attempt to read it from there or you can paste it into the R prompt. On Windows, reading from the clipboard should always work, while on other platforms the R package clipr would need to be installed to read from the clipboard reliably. In any case, you should make sure that the clipboard is not modified before reading or pasting the clipboard again in R.

Instead of using the interactive form in the browser, you can also use nops_fix(display = "browser") which only display the scanned sheet in the browser and then all fields can be entered individually in the R prompt (as in the previous examples).

Subsequently, we can run the additional precautionary checks of answer fields where all or none of the boxes were checked, respectively. In these three exams, we have none of these problems, though.

nops_fix(check = "mchoice") nops_fix(check = "missing")

To demonstrate another nifty feature of nops_fix(), let us assume that we know that answer fields on the second scan should be inspected more thoroughly.

nops_fix(exam = 2, field = "answer")

This iterates through all six answer fields, in this case turning up a number of ambiguities which need to be resolved by the examiner.

## Correct answer 1 (for 00001, S0000002.PNG): ## Correct answer 2 (for 10000, S0000002.PNG): ## Correct answer 3 (for 01100, S0000002.PNG): ## Correct answer 4 (for 01110, S0000002.PNG): ## Correct answer 5 (for 10000, S0000002.PNG): 1 ## Correct answer 6 (for 11010, S0000002.PNG): bd ## adding: Daten.txt (deflated 88%) ## adding: S0000001.PNG (deflated 8%) ## adding: S0000002.PNG (deflated 5%) ## adding: S0000003.PNG (deflated 6%)

Then, the nops_scan_*.zip file is ready for the final evaluation and you just need to download the participant list participants-mchoice.csv before running nops_eval().

ev <- nops_eval( register = "participants-mchoice.csv", solutions = "nops/metainfo.rds", scans = Sys.glob("nops_scan_*.zip"))

To clean up the working folder after the demos, use:

unlink("nops", recursive = TRUE) file.remove(Sys.glob("nops_eval.*")) file.remove(Sys.glob("nops_scan_*.zip"))

Details

Automatic display of problematic sheets
- If certain sheets are poorly scanned (e.g., misaligned or badly scaled) or if the boxes for the registration ID are not correctly checked (exactly one checked box per column), nops_scan() flags these sheets as erroneous and nops_fix() presents them automatically for manual inspection.
- The function displays either the entire exam sheet in the browser or the section in need of attention in an R plot and prompts the user for input on the console, allowing for step-by-step inspection and correction.
- The most suitable display is chosen automatically or can be set to "plot" (display scanned excerpt in R plot), "browser" (display full scanned image in browser), or "interactive" (display full scanned image along with interactive HTML form in browser). The default is "plot" unless errors occurred during scanning where "interactive" is used instead.
Prompting for manual corrections
- For each field in the exam sheet (either "type", "id", "registration", or "answers"), nops_fix() displays the detected content and prompts the user for potential updates.
- Just pressing Enter accepts the detected content without changes.
- When checking a certain answer field, the detected answer is represented by a five-digit binary code (even if the question had fewer than five answer alternatives) such as 01001 (when the second and the fifth box was checked). To correct this, say to the second box only, the following formats are supported: binary (01000), letters (b), or numeric (2).
Specialized checks for implausible answer patterns
- To support examiners in finding potential problems in the exam sheets before examinees complain, nops_fix() offers checks for various implausible answer patterns.
- Often such implausible patterns are errors on behalf of the examinees (and then just have to be preserved) but when the errors are due to incorrectly detecting the checked boxes in the scanned sheet, the answers can be corrected.
- nops_fix(check = "missing") checks answers without any checked boxes which are implausible unless there are negative points for incorrect answers.
- nops_fix(check = "schoice") checks answers with more than one checked box which are often caused by not fully filling an erroneously checked box.
- nops_fix(check = "mchoice") checks answers with all boxes checked which are implausible because such patterns are typically graded with zero points.
Manual review of specific exam sheets
- Sometimes specific exam sheets require more attention, e.g., when special conditions applied in the exam or when the sheet was not filled out correctly.
- Say if this applies to the first scanned sheet, then nops_fix(exam = 1) can be employed to check each field of the sheet.
- Such review ensures that every answer is scrutinized, which is especially helpful in high-stakes or verification-required settings.

The full details for all arguments are documented on the ?nops_fix manual page which also includes more typical application examples.

Creating and Managing Canvas Quizzes with R/exams and vvcanvas

2025-09-08T00:00:00+02:00

Canvas quizzes can be created with exams2canvas in R/exams and then imported and managed in Canvas using the R package vvcanvas.

Overview

This tutorial illustrates how to use R/exams in combination with vvcanvas for creating and managing quizzes for Canvas. exams2canvas() has long been available in R/exams and allows to create QTI zip files that can be imported into the learning management system Canvas (owned by US-based educational technology company Instructure). Until now the import of quizzes had to be done manually using the Import Course Content link in a Canvas course’s Settings menu. Also, quiz options, such as due dates and time restrictions, needed to be set manually in the quiz’s settings, which is rather time-consuming, especially if the number of quizzes is large. The R package vvcanvas provides a convenient interface to interact with the Canvas API. It allows users to authenticate, retrieve course information, fetch specific details, and perform various operations within the Canvas LMS, such as uploading QTI files and setting quiz options.

Here, two things are illustrated:

Steps required using R/exams to create a quiz in Canvas format.
Steps required using the vvcanvas package to (a) authenticate with the Canvas LMS API, (b) import the quiz into the Canvas system and (c) set quiz options.

Steps using R/exams

The code below first loads the exams package (see the Installing R/exams tutorial for instructions how to set it up for the first time).

Then, we create an exam called NewCanvasQuiz which is a list of exercises, in this case all provided as part of the R/exams package: tstat, tstat2, relfreq, function, vowels. It consists of, respectively, a num, schoice, mchoice, string, and cloze exercise. note that there is only limited support for cloze exercises, for now sets of schoice items are allowed, resulting in multiple dropdown menus; the possibility of sets of num items is currently considered.

library("exams") NewCanvasQuiz <- list( "tstat.Rmd", "tstat2.Rmd", "relfreq.Rmd", "function.Rmd", "vowels.Rmd" )

A quick preview in the web browser can be created using exams2html() or exams2webquiz() from exams2forms:

exams2html(NewCanvasQuiz) library("exams2forms") exams2webquiz(NewCanvasQuiz)

To actually create an exam for use in Canvas the exams2canvas() function can be used which generates a zip file (here: NewCanvasQuiz.zip), containing a collection of XML files with the exercises in Canvas format.

exams2canvas(NewCanvasQuiz, name = "NewCanvasQuiz")

If multiple quizzes are uploaded within a course it is advised to _not_ reuse file names as the vvcanvas functions use file names as part of the identification of imported quizzes.

Steps in Canvas

In order to communicate with Canvas from R a ‘token’, a kind of password, is required. To obtain a token, the instructions on Instructure’s community portal may be followed. For completeness, the (currently required) steps are provided below:

Go to your account (icon at the top of the left menu).
Go to Profile > Settings.
Find “Approved integrations” halfway down the page.
Click on “+ New access token” (red button).
Copy or save it.

Steps using vvcanvas

Load the package and authenticate

First, the vvcanvas package may be installed from CRAN using the following command:

install.packages("vvcanvas")

The most up-to-date description of authentication steps can be found on the vvcanvas Github page. Here, after loading the package, we set up variables for the Canvas token and the Canvas URL, using ‘MY_API_KEY’ and the Canvas page of the University of Amsterdam as placeholders. Please replace these with your actual token and URL. Then, the authentication can be carried out using canvas_authenticate().

library("vvcanvas") my_token <- "MY_API_KEY" my_url <- "https://canvas.uva.nl" my_canvas <- canvas_authenticate(api_key = my_token, base_url = my_url)

Specify the course and look into the available quizzes

Set the ID of the course in which the upload must take place and retrieve the list of quizzes that were already available before uploading the new QTI file:

my_course <- 7520 get_course_quizzes(canvas = my_canvas, course_id = my_course)

Upload the quiz and obtain its ID

The time that Canvas requires to process the uploaded QTI file is hard to predict. It depends on the traffic on the Canvas server, and possibly the number of migrations the user has recently performed. Therefore, in circumstances with much waiting time a different approach is advised than when waiting time is short. In the latter case the ID of the quiz is easily obtained, and in the former case the ID needs to be determined using a more extensive approach requiring two additional steps.

If processing takes little time (e.g., 5 to 10 seconds), one can obtain the ID of the quiz directly using the upload_qti_file_with_migration function setting wait = TRUE, by which the function waits 30 seconds after uploading the QTI file to check if it has already been converted into a new quiz. If a new quiz has been created it returns the ID of the quiz; if not it asks the user if further waiting is required.

my_quiz <- upload_qti_file_with_migration(canvas = my_canvas, course_id = my_course, qti_name = "NewCanvasQuiz", wait = TRUE)

If waiting takes too much time, or when waiting is undesirable all the same, users are advised to use the get_course_quizzes function to see if a new quiz has been created. In that case, experience shows it may be wise to wait a couple of hours before running the remaining code (because the Canvas API may have assigned low priority to the processing of the QTI file).

Update quiz parameters

Once the ID of the quiz has been established (saved in object ‘my_quiz’ in the example), the update_quiz function can be used to modify the parameters of the newly created quiz:

my_params <- list( title = "Updated Quiz Title", description = "Updated description of the quiz.", due_at = "2025-09-21T23:59:00+01:00", time_limit = 120, published = TRUE ) updated_quiz <- update_quiz(canvas = my_canvas, course_id = my_course, quiz_id = my_quiz, quiz_params = my_params)

Five parameters were set, the title of the quiz, its description, its due date, its time limit (120 minutes), and its status (‘published’). Note that Canvas demands dates in ISO 8601 format. For completeness, the structure of the updated quiz object is presented below:

str(updated_quiz)

## List of 56 ## $ id : int 119836 ## $ title : chr "Updated Quiz Title" ## $ html_url : chr "https://canvas.uva.nl/courses/7520/quizzes/119836" ## $ mobile_url : chr "https://canvas.uva.nl/courses/7520/quizzes/119836?force_user=1&persist_headless=1" ## $ description : chr "Updated description of the quiz." ## $ quiz_type : chr "assignment" ## $ time_limit : int 120 ## $ timer_autosubmit_disabled : logi FALSE ## $ shuffle_answers : logi TRUE ## $ show_correct_answers : logi TRUE ## $ scoring_policy : chr "keep_highest" ## $ allowed_attempts : int 1 ## $ one_question_at_a_time : logi FALSE ## $ question_count : int 5 ## $ points_possible : num 5 ## $ cant_go_back : logi FALSE ## $ access_code : NULL ## $ ip_filter : NULL ## $ due_at : chr "2025-09-21T22:59:00Z" ## $ lock_at : NULL ## $ unlock_at : NULL ## $ published : logi TRUE ## $ unpublishable : logi TRUE ## $ locked_for_user : logi TRUE ## $ lock_info :List of 2 ## ..$ missing_permission: chr "participate_as_student" ## ..$ asset_string : chr "quizzes:quiz_119836" ## $ lock_explanation : chr "This quiz is currently locked." ## $ hide_results : NULL ## $ show_correct_answers_at : NULL ## $ hide_correct_answers_at : NULL ## $ all_dates :List of 1 ## ..$ :List of 4 ## .. ..$ due_at : chr "2025-09-21T06:59:00Z" ## .. ..$ unlock_at: NULL ## .. ..$ lock_at : NULL ## .. ..$ base : logi TRUE ## $ can_unpublish : logi TRUE ## $ can_update : logi TRUE ## $ require_lockdown_browser : logi FALSE ## $ require_lockdown_browser_for_results: logi FALSE ## $ require_lockdown_browser_monitor : logi FALSE ## $ lockdown_browser_monitor_data : NULL ## $ speed_grader_url : chr "https://canvas.uva.nl/courses/7520/gradebook/speed_grader?assignment_id=618723" ## $ permissions :List of 12 ## ..$ read : logi TRUE ## ..$ create : logi TRUE ## ..$ manage : logi TRUE ## ..$ update : logi TRUE ## ..$ submit : logi TRUE ## ..$ preview : logi TRUE ## ..$ delete : logi TRUE ## ..$ read_statistics : logi TRUE ## ..$ grade : logi TRUE ## ..$ review_grades : logi TRUE ## ..$ view_answer_audits: logi FALSE ## ..$ manage_assign_to : logi TRUE ## $ quiz_reports_url : chr "https://canvas.uva.nl/api/v1/courses/7520/quizzes/119836/reports" ## $ quiz_statistics_url : chr "https://canvas.uva.nl/api/v1/courses/7520/quizzes/119836/statistics" ## $ message_students_url : chr "https://canvas.uva.nl/api/v1/courses/7520/quizzes/119836/submission_users/message" ## $ section_count : int 1 ## $ important_dates : logi FALSE ## $ quiz_submission_versions_html_url : chr "https://canvas.uva.nl/courses/7520/quizzes/119836/submission_versions" ## $ assignment_id : int 618723 ## $ one_time_results : logi FALSE ## $ only_visible_to_overrides : logi FALSE ## $ visible_to_everyone : logi TRUE ## $ assignment_group_id : int 11855 ## $ show_correct_answers_last_attempt : logi FALSE ## $ version_number : int 4 ## $ has_access_code : logi FALSE ## $ post_to_sis : logi FALSE ## $ migration_id : chr "NewCanvasQuiz_136488631" ## $ in_paced_course : logi FALSE ## $ question_types :List of 5 ## ..$ : chr "numerical_question" ## ..$ : chr "multiple_choice_question" ## ..$ : chr "multiple_answers_question" ## ..$ : chr "short_answer_question" ## ..$ : chr "multiple_dropdowns_question"

R/exams Presents: Fun with Flags

2025-09-02T00:00:00+02:00

A knowledge quiz about the flags of countries and their neighbors illustrates how R/exams and Quarto can be leveraged for standalone self-learning materials (and pays tribute to Sheldon Cooper).

Overview

The recently introduced exams2forms package extends R/exams so that its dynamic exercises can be embedded in R/Markdown or Quarto documents, web pages, online books, etc. One potential application scenario for this is the creation of standalone HTML documents including self-learning materials, e.g., for providing open educational resources (OER).

Here, this is illustrated by a knowledge quiz that pays tribute to the web series Sheldon Cooper presents: Fun with Flags, a spin-off from the popular sitcom “The Big Bang Theory”.

Knowledge quiz

The quiz is provided in the standalone file flags.html and it contains two tasks:

Guess the flag: The correct flag for a country has to be selected from a list of flags. The underlying exercise is provided in the exams package as flags.
Find the neighbors: The flags of neighboring countries (for a country only identified by its flag) have to be selected from a multiple-choice list. The underlying exercise is provided in the exams2forms package as geography.Rmd.

Click on the screenshot below to try the quiz yourself. (Note that you can also download the HTML file and play the quiz offline.)

Resources

To recreate the quiz file, the following resources are necessary. First, the exercise files for the two tasks are needed but, as pointed out above, these are shipped with the exams and exams2forms package, respectively. Second, the following files are needed to create a standalone HTML file containing 100 random variations from each of the exercises.

flags.qmd: Quarto source file embedding the exams2forms() calls, see below.
webex.js: Copy of the current Javascript code from the exams2forms package (version 0.2-0).
webex.css: Copy of the current style sheets from the exams2forms package (version 0.2-0) with the highlight color changed to the R/exams primary color.
cosmo.scss: Style sheets for customizing the cosmo theme to the R/exams primary color and button-style tabsets.
fun_with_flags.png: Title image taken from Michael Heap’s Fun with Flags presentation.

If these files are placed in the same folder, the Quarto source file flags.qmd can be rendered and then produces flags.html. When doing so in RStudio you can simply open the flags.qmd file and click on the rendering button. Alternatively, on the command line you can use quarto render flags.qmd.

Quarto source file

The complete Quarto source file flags.qmd is also shown below. It contains the following parts:

The YAML header (separated by ---) sets the title using the logo image, loads all style files and scripts, and specifies that all resources should be embedded in order to make the resulting HTML file self-contained.
The first R code chunk loads the exams2forms package (which also loads the exams package) and sets a random seed for reproducibility.
Subsequently, a panel tabset with two sections is defined, each containing an R code calling exams2forms().
For the first task 100 random variations can be set up easily because flags.Rmd is found in the exams package.
For the second task we need to set up the path to the geography.Rmd file in the exams2forms package, then its the display variable is fixed to "flag", and finally 100 random variations are created.

--- title: "![Fun with Flags](fun_with_flags.png){width=330px}" format: html: theme: - cosmo - cosmo.scss toc: false css: webex.css include-after-body: webex.js embed-resources: true --- ```{r setup, include = FALSE} library("exams2forms") set.seed(0) ``` ::: {.panel-tabset} ## Guess the flag ```{r flags, echo = FALSE, message = FALSE, results = "asis"} exams2forms("flags.Rmd", n = 100) ``` ## Find the neighbors ```{r geography, echo = FALSE, message = FALSE, results = "asis"} geography <- system.file("exercises", "geography.Rmd", package = "exams2forms") |> expar(display = "flag") exams2forms(geography, n = 100) ``` :::

New R/exams Version: exams2forms, Written NOPS Exams, and More

2025-05-05T00:00:00+02:00

New CRAN releases of the R/exams package and the accompanying exams2forms package with many new features and enhancements, especially for the written multiple-choice exams (NOPS).

Overview

The new version 2.4-2 of the R/exams package has been released on the Comprehensive R Archive Network at https://CRAN.R-project.org/package=exams along with version 0.2-0 of the accompanying exams2forms package at https://CRAN.R-project.org/package=exams2forms. For an overview of all changes in both packages, see their respective NEWS files: exams NEWS and exams2forms NEWS.

The development of exams2forms was originally prompted by requests for R/exams infrastructure in online books via R/Markdown or Quarto. However, it turned out to be very useful for quite a few other applications including:

Testing exercises interactively during development without the need for import in a learning managment system.
Standalone (non-graded) quizzes as self-contained files, e.g., for download as open educational resources.
Browsing exercise pools, e.g., for sharing previews within a team.

The other main area of development were extensions to the written multiple-choice exams (NOPS) that can be automatically scanned and evaluated. This was prompted by the decision of the Faculty of Economics and Statistics at Universität Innsbruck to conduct all of their large-scale exams during the first years of the bachelor program using this infrastructure. Below we give an overview of the R functionality in the exams package that support this endeavor. Those who are interested in how the administrative aspects are organized in Gitlab repositories with corresponding issues can check out the workflow documentation and a demo project (both in German) on the university’s self-hosted Gitlab system.

Interactive web exercises via exams2forms

Version 0.1-0 of the package had been introduced in an exams2forms tutorial in a previous blog post. Prompted by our own needs as well as a lot of positive and constructive feedback, version 0.2-0 now contains many useful improvements and new features. Also, all exercise template pages on the R/exams web page now feature an interactive preview generated with exams2forms using three random variations of each exercise. See boxplots or lm2 for two examples.

Obfuscation

The function exams2forms() as well as all underlying forms_*() functions gained an argument obfuscate = TRUE which provides some basic obfuscation of the correct answers for all forms in the underlying HTML code. Thus, all correct answers are still stored in the HTML but, by default, they are not easily readable anymore.

Auto-display of filled-in exercises

The function exams2forms() gained additional arguments auto, show_filename, and show_tolerance, which are useful for inspecting exercises during their development. To see the results with all forms pre-filled, check enabled, and display of solution, tolerances, and the file name, try:

exams2webquiz("lm2.Rmd", auto = TRUE)

Regular expression in string solutions

The package now ships with some example exercises that illustrate a feature of exams2forms that is not (or not easily) available in other exams2xyz interfaces: regular expressions for the correct answers of string exercises. To generate a demo quiz with these you can use:

exams2webquiz(c("geography2.Rmd", "email.Rmd"), regex = TRUE, n = 3, edir = system.file(package = "exams2forms"))

The exercise geography2.Rmd simply lists several solutions that are accepted using “or” regular expressions: ^(answer1|answer2|answer3)$
The exercise email.Rmd has a more complex regular expression for checking the validity of e-mail addresses. geography2.Rmd exercise.

Multiple-choice NOPS exams

R/exams long provides infrastructure for large-scale multiple-choice exams that can be automatically scanned and evaluated, see the exams2nops tutorial for an introduction. Previously, the workflow just comprised exams2nops() for generating the PDF exams, nops_scan() for reading the filled-out and scanned exam sheets, and nops_eval() for evaluating all results, grading the exams, and generating reports for the participants.

However, addressing errors that occurred, in particularl during scanning, used to be rather inconvenient and necessitated editing some of the intermediate files manually. Therefore, version 2.4-1 of the package introduced a new function nops_fix() that greatly facilitates interactive quality control of exam sheets directly in R (and/or the browser). A dedicated tutorial for this functionality is under preparation.

Below we summarize the most important additions and improvements from version 2.4-1 (which had not yet been included in previous blog posts) and 2.4-2.

nops_fix

The new function nops_fix() can be applied to the ZIP file resulting from nops_scan(). By default it goes through all rows of the scanned data and interactively prompts for updates to fields from the scanned exam sheets that need updating. It can also be applied repeatedly to iteratively check and resolve different potential problems.

Optionally, the user can specify the rows of the scanned data and/or the fields that should be updated.
Different display options are available with the two most important being: (1) Displaying only a section of the scanned sheet in the R plot window with interactive prompts on the R console. (2) Displaying the entire sheet in the browser with a Javascript form to enter all data which can be passed to R through the clipboard.
Display 1 is most convenient for iterating through smaller potential problems in sheets that have been scanned mostly correctly.
Display 2 is most convenient for entering scanned data for sheets that could not be read at all (e.g., due to rotation or damaged sheets etc.).
The default display tries to make an educated guess for the more convenient option.
Dedicated check = "schoice" or check = "missing" arguments are provided to go through exercises with more than one answer or missing answers, respectively.

nops_scan

By default, nops_scan() now relies on the R packages qpdf and magick for merging/rotating/splitting PDF files and converting them to PNG, respectively. This greatly facilitates installation of the scanning infrastructure, especially for novice R/exams users.

Improvements have been made in nops_scan() for reading scans from different paper formats (e.g., letter paper instead of A4) and for reading the registration id more reliably. Also, scanner markings are found more reliably, especially if the top-left marking is missing completely (e.g., because it was cut erroneously).

exams2nops and language support

Additional NOPS language support has been added: Bulgarian (bg, contributed by Nikolay Rachev), Catalan (ca, contributed by Paco Rivière), and Polish (pl, contributed by Paweł Kleka). Corrections in French (fr, contributed by Jean-Philippe Georget), Russian (ru, contributed by Nikolay Rachev), and Spanish (es, contributed by Flavio Lozano Isla).

Moreover, there are various improvements and new arguments in exams2nops():

New argument helvet = TRUE which allows to suppress the default Helvetica font in order to facilitate using other fonts.
New argument newpage = FALSE to facilitate adding page breaks after every exercise.
The logo can now also be a relative path (relative to the working directory). Also logo = "uibk" is supported as a convenience option for including the logo of Universität Innsbruck.
The exam type now codes the number of exercises directly (rather than rounding the number to multiples of 5). This facilitates fixing the correct number of questions in nops_fix().
Answer lists are now labeled “a.”, “b.”, etc. everywhere rather than “(a)”, “(b)”, etc.

More features and improvements

New exercises

There are two new demo exercises: flags illustrating the handling of Unicode characters for flags in a flexible way. sumdiff is a minimal example for using random numbers in arithmetic exercises.

Evaluation strategies in learning managment systems

The default evaluation rule for all learning management systems (including exams2moodle, exams2canvas, exams2openolat, etc.) is now consistently negative = FALSE and partial = TRUE with rule = "false2". Thus, there should not be negative points for any exercise type and partial credits should be used for mchoice exercises (and mchoice interactions within cloze exercises).

Stress-testing exercises

New arguments have been added in stresstest_exercise() to facilitate stress-testing exercises, especially for large collections of exercises:

stop_on_error = length(as.character(unlist(file))) < 2: Should stresstesting stop or continue after encountering an error in one of the exercises?
timeout = NULL: Set a time limit for running each exercise, e.g., to avoid running into infinite loops etc.
maxit = getOption("num_to_schoice_maxit", -10000L): Default number of iterations in case num_to_schoice() is used (see below).

Standalone PDF files

Customizing the LaTeX template in exams2pdf() has been facilitated by adding the usepackage argument and streamlining processing of the header argument.

Additionally, the {Sweave} LaTeX package is no longer loaded in the LaTeX templates (like plain.tex, exams.tex, solution.tex, etc.). Instead the LaTeX environments for displaying R code and the accompanying LaTeX dependencies are loaded directly.

Misc

Include better table formatting by default in exams2moodle() via styles table = "table_shade" (default), "table_rule", or "table_grid".
New arguments in num_to_schoice() to facilitate converting numeric to single-choice exercises:
- format = TRUE: Should the question list be formatted to a character vector with LaTeX math markup (default)? Alternatively, the question list can be numeric (format = FALSE or "numeric") or a formatted character vector without LaTeX math markup (format = "character").
- order = getOption("num_to_schoice_order", FALSE): Should the question list be ordered numerically? If FALSE (default) the question list is shuffled randomly.
- maxit = getOption("num_to_schoice_maxit", Inf): Maximum number of iterations to try to find a feasible set of wrong solutions for the question list.

vowels: Phonetics Cloze Quiz about Cardinal Vowels

2025-04-30T00:00:00+02:00

Cloze exercise in which two randomly-selected Cardinal vowels have to be described according to the three parameters vowel height, frontness-backness, and lip rounding.

Name:

vowels

Type:

cloze

Preview:

Describe the following Cardinal vowels according to the three parameters vowel height, frontness-backness, and lip rounding:

[e] is the vowel.
[ɶ] is the vowel.

For an overview of the 18 Cardinal vowels see https://en.wikipedia.org/wiki/Cardinal_vowels.

[e] is the close-mid front unrounded vowel.
[ɶ] is the open front rounded vowel.

Describe the following Cardinal vowels according to the three parameters vowel height, frontness-backness, and lip rounding:

[ɶ] is the vowel.
[o] is the vowel.

For an overview of the 18 Cardinal vowels see https://en.wikipedia.org/wiki/Cardinal_vowels.

[ɶ] is the open front rounded vowel.
[o] is the close-mid back rounded vowel.

Describe the following Cardinal vowels according to the three parameters vowel height, frontness-backness, and lip rounding:

[ʉ] is the vowel.
[i] is the vowel.

For an overview of the 18 Cardinal vowels see https://en.wikipedia.org/wiki/Cardinal_vowels.

[ʉ] is the close central rounded vowel.
[i] is the close front unrounded vowel.

Description:

This cloze exercise randomly selects two out of 18 Cardinal vowels and then uses three single-choice elements each to describe their parameters. It is particularly well-suited for drop-down menus and also illustrates how Unicode symbols can be used for the phonetic transcriptions in the International Phonetic Alphabet (which requires the fontspec package in XeLaTeX for PDF output).

Solution feedback:

Yes

Randomization:

Shuffling (2 out of 18 Cardinal vowels)

Mathematical notation:

Verbatim R input/output:

Images:

Other supplements:

Template:

vowels.Rmd

vowels.Rnw

Raw: (1 random version)

vowels.md

vowels.tex

PDF:

HTML:

Demo code:

library("exams") set.seed(403) exams2html("vowels.Rmd") set.seed(403) exams2pdf("vowels.Rmd", texengine = "xelatex", usepackage = "fontspec", header = "\\setmainfont{Doulos SIL}") set.seed(403) exams2html("vowels.Rnw") set.seed(403) exams2pdf("vowels.Rnw", texengine = "xelatex", usepackage = "fontspec", header = "\\setmainfont{Doulos SIL}")

flags: Fun with Flags

2025-04-30T00:00:00+02:00

Single-choice knowledge quiz question in which the correct flag for a country has to be selected from a list of flags.

Name:

flags

Type:

schoice

Related:

capitals, countrycodes

Description:

The exercise contains a large list of countries from all continents around the world along with their direct neighbors (sharing a border) and their secondary neighbors (neighbors of neighbors). Flags are generated from the ISO 3166-1 alpha-2 codes, either via their Unicode symbols (especially in HTML-based output) or using the {worldflags} LaTeX package (in PDF output).

Solution feedback:

Yes

Randomization:

Shuffling (1 out of 136 countries)

Mathematical notation:

Verbatim R input/output:

Images:

Other supplements:

Template:

flags.Rmd

flags.Rnw

Raw: (1 random version)

flags.md

flags.tex

PDF:

HTML:

Demo code:

library("exams") set.seed(403) exams2html("flags.Rmd") set.seed(403) exams2pdf("flags.Rmd", usepackage = "worldflags") set.seed(403) exams2html("flags.Rnw") set.seed(403) exams2pdf("flags.Rnw", usepackage = "worldflags")

sumdiff: Sum Minus Difference

2025-04-29T00:00:00+02:00

Exercise template for a very basic arithmetic task, namely computing the sum of two random numbers minus their difference.

Name:

sumdiff

Type:

num

Related:

deriv

Preview:

By how much is the sum of the numbers 52 and 30 greater than the difference between these numbers?

The result of the calculation is (52 + 30) - (52 - 30) = 82 - 22 = 60.

By how much is the sum of the numbers 32 and 22 greater than the difference between these numbers?

The result of the calculation is (32 + 22) - (32 - 22) = 54 - 10 = 44.

By how much is the sum of the numbers 42 and 31 greater than the difference between these numbers?

The result of the calculation is (42 + 31) - (42 - 31) = 73 - 11 = 62.

Description:

Minimal arithmetic task (sum of two numbers minus their difference) with randomly-drawn numbers but only short text description and no mathematical notation in LaTeX.

Solution feedback:

Yes

Randomization:

Random numbers

Mathematical notation:

Verbatim R input/output:

Images:

Other supplements:

Template:

sumdiff.Rmd

sumdiff.Rnw

Raw: (1 random version)

sumdiff.md

sumdiff.tex

PDF:

HTML:

Demo code:

library("exams") set.seed(403) exams2html("sumdiff.Rmd") set.seed(403) exams2pdf("sumdiff.Rmd") set.seed(403) exams2html("sumdiff.Rnw") set.seed(403) exams2pdf("sumdiff.Rnw")

Embedding R/exams Exercises as Forms in R/Markdown or Quarto Documents

2024-11-07T00:00:00+01:00

Introduction to the new exams2forms package for including quizzes or individual questions from dynamic exercise templates into rmarkdown or quarto documents, e.g., for self-paced learning and self-assessment.

Overview

The new package exams2forms, co-authored with Reto Stauffer, provides several building blocks for embedding exercises written with the R package exams (also known as R/exams) in interactive documents or quizzes written with rmarkdown or quarto.

The idea is that the dynamic exercises in R/exams’ Rmd (R/Markdown) or Rnw (R/LaTeX) format can also be reused in HTML documents, web pages, or online books. This facilitates their use for self-paced learning and self-assessment without the need for a learning management system etc. (By default the correct answers are obfuscated in the documents so that they are not obvious when inspecting the HTML source code.) And for (summative) assessments the same dynamic exercises could then be exported to different learning management systems or employed in written exams.

All R/exams exercise types are supported:

Single-choice (schoice).
Multiple-choice (mchoice).
Numeric (num).
Text (string).
Cloze combining all of the previous elements (cloze).

Many of the ideas as well as the code in the package have been adapted from the webexercises package, authored by Dale Barr and Lisa DeBruine.

First examples

As quick demonstration for R/exams exercises embedded into an HTML document, the two examples from the First Steps tutorial are included below: The single-choice exercise swisscapital and the numeric exercise deriv, both in three random variations.

What is the derivative of $f(x) = x^{2} e^{3.2 x}$, evaluated at $x = 0.59$?

Using the product rule for $f(x) = g(x) \cdot h(x)$, where $g(x) := x^{2}$ and $h(x) := e^{3.2 x}$, we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 2 x^{2 - 1} \cdot e^{3.2 x} + x^{2} \cdot e^{3.2 x} \cdot 3.2 \\ &= e^{3.2 x} \cdot(2 x^1 + 3.2 x^{2}) \\ &= e^{3.2 x} \cdot x^1 \cdot (2 + 3.2 x). \end{aligned} \] Evaluated at $x = 0.59$, the answer is \[ e^{3.2 \cdot 0.59} \cdot 0.59^1 \cdot (2 + 3.2 \cdot 0.59) = 15.153964. \] Thus, rounded to two digits we have $f'(0.59) = 15.15$.

What is the derivative of $f(x) = x^{4} e^{3.5 x}$, evaluated at $x = 0.65$?

Using the product rule for $f(x) = g(x) \cdot h(x)$, where $g(x) := x^{4}$ and $h(x) := e^{3.5 x}$, we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 4 x^{4 - 1} \cdot e^{3.5 x} + x^{4} \cdot e^{3.5 x} \cdot 3.5 \\ &= e^{3.5 x} \cdot(4 x^3 + 3.5 x^{4}) \\ &= e^{3.5 x} \cdot x^3 \cdot (4 + 3.5 x). \end{aligned} \] Evaluated at $x = 0.65$, the answer is \[ e^{3.5 \cdot 0.65} \cdot 0.65^3 \cdot (4 + 3.5 \cdot 0.65) = 16.763849. \] Thus, rounded to two digits we have $f'(0.65) = 16.76$.

What is the derivative of $f(x) = x^{7} e^{2.8 x}$, evaluated at $x = 0.72$?

Using the product rule for $f(x) = g(x) \cdot h(x)$, where $g(x) := x^{7}$ and $h(x) := e^{2.8 x}$, we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 7 x^{7 - 1} \cdot e^{2.8 x} + x^{7} \cdot e^{2.8 x} \cdot 2.8 \\ &= e^{2.8 x} \cdot(7 x^6 + 2.8 x^{7}) \\ &= e^{2.8 x} \cdot x^6 \cdot (7 + 2.8 x). \end{aligned} \] Evaluated at $x = 0.72$, the answer is \[ e^{2.8 \cdot 0.72} \cdot 0.72^6 \cdot (7 + 2.8 \cdot 0.72) = 9.430757. \] Thus, rounded to two digits we have $f'(0.72) = 9.43$.

In addition to the question and the interaction element, there are three buttons providing the following functionality.

Button	Function
✓	Check the answer and display whether it is correct or not. When clicked, the symbol is toggled and ↰ is displayed, which can be clicked to hide the solution again.
?	Display the full correct solution explanation.
↺	Switch to the next question.

Inclusion of the solution explanation is optional and the next question button is only displayed if there is more than one random variation of a question. The icons and mouseover text can also be adapted (see below).

To set up a similar standalone file with these two exercises, the exams2webquiz() interface from the exams2forms package can be used:

library("exams2forms") exams2webquiz(c("swisscapital.Rmd", "deriv.Rmd"), n = 3)

More elaborate examples

To showcase some more exercise types, the following examples from the R/exams package are used: capitals (multiple-choice), function (string/text), fruit (numeric with table and images), lm2 (cloze containing string, multiple-choice, numeric, and single-choice elements as well as an embedded data file).

Which of the following cities are the capital of the corresponding country?

True. Tokyo is the capital of Japan.
True. Warsaw is the capital of Poland.
False. The capital of New Zealand is Wellington.
True. Astana is the capital of Kazakhstan.
False. The de facto capital of Switzerland is Bern.

What is the name of the R function for negative binomial regression?

glm.nb is the R function for negative binomial regression. See ?glm.nb for the corresponding manual page.

Given the following information:

$+$	$+$	=	$143$
$+$	$+$	=	$437$
$+$	$+$	=	$433$

Compute:

$+$

$\text{?}$

The information provided can be interpreted as the price for three fruit baskets with different combinations of the three fruits. This corresponds to a system of linear equations where the price of the three fruits is the vector of unknowns $x$:

$x_1 =$

$x_2 =$

$x_3 =$

The system of linear equations is then: \[ \begin{aligned} \left( \begin{array}{rrr} 1 & 2 & 0 \\ 2 & 0 & 1 \\ 0 & 2 & 1 \end{array} \right) \cdot \left( \begin{array}{r} x_1 \\ x_2 \\ x_3 \end{array} \right) & = & \left( \begin{array}{r} 143 \\ 437 \\ 433 \end{array} \right) \end{aligned} \] This can be solved using any solution algorithm, e.g., elimination: \[ x_1 = 49, \, x_2 = 47, \, x_3 = 339. \] Based on the three prices for the different fruits it is straightforward to compute the total price of the fourth fruit basket via:

	$+$		$+$		=
$x_1$	$+$	$x_2$	$+$	$x_3$	=
$49$	$+$	$47$	$+$	$339$	=	$435$

Theory: Consider a linear regression of y on x. It is usually estimated with which estimation technique (three-letter abbreviation)?

This estimator yields the best linear unbiased estimator (BLUE) under the assumptions of the Gauss-Markov theorem. Which of the following properties are required for the errors of the linear regression model under these assumptions?

Application: Using the data provided in linreg.csv estimate a linear regression of y on x. What are the estimated parameters?

Intercept:

Slope:

In terms of significance at 5% level:

Theory: Linear regression models are typically estimated by ordinary least squares (OLS). The Gauss-Markov theorem establishes certain optimality properties: Namely, if the errors have expectation zero, constant variance (homoscedastic), no autocorrelation and the regressors are exogenous and not linearly dependent, the OLS estimator is the best linear unbiased estimator (BLUE).

Application: The estimated coefficients along with their significances are reported in the summary of the fitted regression model, showing that x and y are not significantly correlated (at 5% level).

 Call: lm(formula = y ~ x, data = d) Residuals: Min 1Q Median 3Q Max -0.59846 -0.17142 -0.01458 0.13740 0.68566 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.03055 0.02507 1.219 0.226 x -0.03080 0.04702 -0.655 0.514 Residual standard error: 0.2504 on 98 degrees of freedom Multiple R-squared: 0.00436, Adjusted R-squared: -0.005799 F-statistic: 0.4292 on 1 and 98 DF, p-value: 0.5139

Code: The analysis can be replicated in R using the following code.

## data d <- read.csv("linreg.csv") ## regression m <- lm(y ~ x, data = d) summary(m) ## visualization plot(y ~ x, data = d) abline(m)

Again, the exams2webquiz() function can be used to set up a standalone file based on the same exercises:

exams2webquiz(c("capitals.Rmd", "function.Rmd", "fruit.Rmd", "lm2.Rmd"))

Building blocks

To accomplish the functionality demonstrated above, the package provides the following contents:

exams2forms(): Main workhorse function from the package. Like other exams2xyz() interfaces this takes a vector or list of exercise files and returns Markdown text, including HTML snippets, than can be included in rmarkdown or quarto documents. This includes questions, suitable interactions for the different types of answers (with correct solutions also embedded in the HTML), and optionally full solution explanations.
webex.css, webex.js: CSS (Cascading Style Sheets) and Javascript files shipped within the package and providing the code necessary for the exercise and quiz display and processing the different types of interactions.
webquiz(): Small wrapper function that creates a knitr::html_document() but includes the CSS and Javascript files above.
exams2webquiz(): Convenience interface that combines all of the above elements. It sets up a webquiz() document in which exams2forms() is used to embed the specified exercises, calls rmarkdown::render() to process it, and by default displays it in the browser. This is most useful for quickly trying out how R/exams exercises can work in HTML documents.
forms_num(), forms_string(), forms_schoice(), forms_mchoice(): Helper functions for just embedding the user interactions for the different types of exercises. Typically not called directly by the user.

Demo files

While exams2webquiz() is convenient for quickly setting up an HTML document containing certain exercises, further customizations are typicallly needed for more elaborate documents. To demonstrate how this works the exams2forms package provides two demo rmarkdown files which can also be downloaded here:

quiz.Rmd.
questions.Rmd.

The first file quiz.Rmd renders a number of different exercises into a quiz using a single exams2forms() call, indicating that the results should be included "asis":

 --- title: "R/exams quiz" output: exams2forms::webquiz --- ```{r setup, include = FALSE} ## package and list of various exercises library("exams2forms") exm <- list( c("swisscapital.Rmd", "capitals.Rmd"), "deriv.Rnw", "deriv2.Rnw", "fruit.Rmd", "boxplots.Rmd", "ttest.Rmd", "function.Rmd", "lm2.Rnw", "fourfold2.Rmd" ) ``` ```{r quiz, echo = FALSE, message = FALSE, results = "asis"} exams2forms(exm, n = 3) ```

The questions.Rmd file looks similar but contains different sections, each with a single question set up via exams2forms().

Both files can be rendered to HTML via rmarkdown::render() or by clicking the knit button after opening the files in RStudio etc.

Setup and customization

When setting up a more elaborate document or even a full webpage or online book with rmarkdown or quarto, then the simple webquiz() HTML document provided by exams2forms is probably not sufficient. In this case, it is best to take the CSS and Javascript files from the package or download them here:

webex.css.
webex.js.

The files can then be placed in the same folder as the rmarkdown or quarto project. They can also be adapted relatively easily by changing the definitions of colors, icons, text, etc. in the first few lines of each file.

To include the CSS and Javascript in an rmarkown project, the YAML header should include:

output: html_document: css: webex.css includes: after_body: webex.js

Similarly, in a quarto project the YAML header or the _quarto.yml file should include:

format: html: css: webex.css include-after-body: webex.js

Standalone interaction forms

While it is not the primary focus of the exams2forms package, it is also possible to directly include interaction forms in documents like in the webexercises package, i.e., without setting up full R/exams exercises. A few simple examples are inlcuded below for numeric, text, single-choice, and multiple-choice interactions (both using drop-down interactions here), respectively.

What is the answer to the ultimate question of life, the universe, and everything?

Which superhero is the secret identity of Bruce Wayne?

Which of the following villains is not an adversary of Batman?

Which of the following characters are romantic interests of Spider-Man?

Mary Jane Watson
Pepper Potts
Selina Kyle
Gwen Stacy

The corresponding code snippets included in the inline code chunks are:

forms_num(42, width = 10)
forms_string("Batman", width = 20, usecase = FALSE)
forms_schoice(c("Bane", "Riddler", "Thanos", "Poison Ivy"), c(FALSE, FALSE, TRUE, FALSE), display = "dropdown")
forms_mchoice(c("Mary Jane Watson", "Pepper Potts", "Selina Kyle", "Gwen Stacy"), c(TRUE, FALSE, FALSE, TRUE), display = "dropdown")

Mind the Gap: Using R/exams to Ease the Transition into STEM Studies

2024-06-17T00:00:00+02:00

Ideas and experiences from an award-winning bridging course in mathematics at Universität Innsbruck, whose teaching and examination culture is guided by learning outcomes and extensively uses R/exams.

Guest post by Pia Tscholl & Lisa Schlosser (Universität Innsbruck).

Motivation

Universities in German-speaking countries face above-average dropout rates in STEM (Science, Technology, Engineering, Mathematics) subjects. These high dropout rates are worrying, as the demand for qualified workers in these fields exceeds the available supply. Especially deficient mathematical knowledge plays a major role during the dropout process in STEM fields. For this reason, the majority of the German-speaking universities offer mathematical remedial or bridging courses at the beginning of STEM degree programs. However, researchers criticize that many remedial courses do not offer audience-appropriate assessments to efficiently diagnose mathematical deficits. Echoing this critique, a team at the University of Innsbruck has developed a mathematical self-assessment, which is implemented via R/exams and made available to participants of the mathematical remedial course via the university’s learning management system OpenOlat. The implementation with R/exams offers the advantage of large test sets with (potentially) randomized tasks which can be easily modified and evaluated in an automated manner. Due to the automated evaluation, the self-assessment can be used in courses with a large number of participants. Moreover, the created assessment can be easily exported to other formats (pdf, moodle, …).

Creating task lists by content area

The self-assessment covers the following content areas (with abbreviations based on the German titles):

AK: General mathematical competencies.
ZF: Numbers and functions.
GU: Equations and inequalities.
KV: Coordinates and vector geometry.
GT: Geometry and trigonometry.
DR: Differential calculus.
IR: Integral calculus.
SW: Statistics and probability.

A separate test set with a certain number of R/exams tasks is created for each content area. All tasks are saved in the same folder and named as follows: “ContentArea_AscendingNumber.Rmd”, e.g. the first task of the content area ZF is named ZF_001.Rmd, the second ZF_002.Rmd and so on. Consistent naming makes it possible to quickly create task lists by content area:

testset_ZF <- list.files(path = "wd/task", pattern = "^ZF_[0-9]+\\.Rmd$")

where wd is your current working directory or the path where your task folder is located and task is the name of the folder where the tasks are stored. The argument pattern is a regular expression starting with ZF (or another content area) followed by _, some digits and ending with .Rmd. Subsequently, testset_ZF contains the ZF task list to be exported to OpenOlat.

Example: Defining a task

As an example, the code for a specific randomized task is presented below. It asks the participants to compare two randomly selected fractions to check which one is greater or whether they are equal. In addition to the code being shown below (in R/Markdown format) it can also be downloaded in both R/Markdown and R/LaTeX format as ZF_071.Rmd or ZF_071.Rnw, respectively.

 ```{r, include = FALSE} ## Generate two random fractions based on numbers from 1 to 10 f1 <- sample(1:10, 2, replace = FALSE) f2 <- sample(1:10, 2, replace = FALSE) a <- f1[1] b <- f1[2] c <- f2[1] d <- f2[2] ## Save random fractions a/b and c/d as character ## (don't forget escaping \\ within the math mode) fr1 <- paste0("$\\frac{", a, "}{", b, "}$") fr2 <- paste0("$\\frac{", c, "}{", d, "}$") ## Possible answers answers <- c( paste(fr1, "is greater than", fr2), paste(fr2, "is greater than", fr1), "Both fractions are equal" ) ## Correct solution sol <- c(0, 0, 0) if(a/b > c/d) { sol[1] <- 1 } else if(c/d > a/b) { sol[2] <- 1 } else { sol[3] <- 1 } ## Explanation k <- answers[as.logical(sol)] eq <- c(" > ", " < ", " = ")[as.logical(sol)] explanation <- paste0(k, " since $", a, " \\cdot ", d, eq, b, " \\cdot ", c, "$.") ``` Question ======== Which of these numbers is greater: `r fr1` or `r fr2`? ```{r, echo = FALSE, results = "asis"} answerlist(answers, markup = "markdown") ``` Solution ======== `r explanation` Meta-information ================ exname: Comparing fractions extype: schoice exsolution: `r paste(sol, collapse = "")`

One random version of the exercise is shown below as rendered in an OpenOlat test after entering an incorrect answer:

Further meta-information could be added to the exercise via the exextra tags, if needed. For num questions, the tag extol defines the tolerance range for numerical solutions. Exercise templates for different task types (num, mchoice, schoice, cloze, …) are provided on the R/exams web page at https://www.R-exams.org/templates/.

The elegant aspect of randomized tasks is that students can perform the same test multiple times for the purpose of practice or improved self-assessment. If implemented foresighted like in our example, solutions and explanations are automatically adapted - so there is no additional work for teachers despite many task variations.

The next section will explain how to export such randomized tasks from R/exams to OpenOlat.

Randomized export to OpenOlat

As explained in the previous section, consistent file naming makes it easy to create a list of test files, hereinafter referred to as testset_ZF. The following code exemplifies how such a test set, consisting of several randomized tasks, can be exported from R/exams to OpenOlat.

library("exams") # Load R/exams package seed <- 6020 # Select seed, so randomization can be repeated set.seed(seed) # Set seed export_ZF <- exams2openolat( # Function for exporting tasks to OpenOlat edir = "wd/tasks", # Directory where task files are stored file = testset_ZF, # File names of the tasks n = 20, # Number of randomized versions of the test set name = paste0("ZF_Testset_seed", seed), # Remember seed also in file name stitle = "ZF", # Section title ititle = "Aufgaben", # Item title solutionswitch = TRUE # Display explanation right after saving the answer )

Further arguments, such as cutvalue (threshold for passing), navigation (disabling switching between tasks), and duration (maximum processing time) can individualize the test implementation regarding the needs of the course (see this blog post for some practical illustration).

The exams2openolat function creates a zip folder with the chosen name in the current working directory.

Now, you can insert a new Test element in your OpenOlat course. In the editor mode, it is possible to import the created zip folder to the Test-Konfiguration element. At this point, the work is already done - yay! Each time the test is called up, one of the created randomized test sets is selected by OpenOlat. If desired, further settings can be done using the editor mode in OpenOlat, such as displaying the test results after the test has been completed (Testkonfiguration > Report), a winner’s podium for the three best (anonymous) participants (HighScore > Siegertreppchen), or an automated submission confirmation by E-mail (Email Bestätigung).

It is noteworthy - since this question occurs often among our students - that decimal input for num questions can be entered in the OpenOlat export with dot or comma.

Results and feedback

Once a class or group has finished the self-assessment, various tools provide information on the results. For example, an anonymous ranking showing the achieved number of points of the three best performing participants and a histogram over all point results can provide a first overview on the results.

Moreover, each participant is informed about his/her performance after finishing the assessment for one of the content areas. In particular, the total number of achieved points, the time needed to finish and the number of answered questions is listed. Additionally, each question, the given answer, and the expected answer can be accessed by the participant.

Experience and outlook

A problem that still arises with num tasks is the incorporation of fractions. While they can be properly in the tasks using mathematical notation, there is still no convenient way to insert fractions as solutions when completing the test set. As of now, fractions can only be transmitted as decimal numbers, which can be tedious and prone to errors. One possibility would be to implement the numerator and denominator as two num solution-elements of a cloze task in R/exams. However, this would have the disadvantage that reduced or expanded variants of the respective fractions are not recognized as solutions. In this case, the most reduced fraction would have to be requested as a solution, which in turn is again prone to errors. Furthermore, if only the numerator or only the denominator is inserted correctly (and the solution is therefore wrong overall), 50% of the points are still awarded.

Additionally, it would be desirable to provide students with the solutions, including implemented explanations for the tasks, after completing the test. It is currently only possible to display explanations immediately after saving the answer, but not as final feedback.

However, we received primarily positive feedback from students and colleagues regarding the implementation of our mathematics self-assessment using R/exams in OpenOlat.

The concept for this Bridging Course Mathematics even won the Austrian teaching award Ars Docendi State Prize 2023. The photo below shows the entire team Tobias Hell, Elisabeth Hell, Pia Tscholl, and Lisa Schlosser (left to right) being presented with the award by Secretary of Education Martin Polaschek (Photo: Martin Lusser/BMBWF).

One advantage of this digitalized self-assessment - in addition to those already mentioned - is the ability to collect data in larger quantities and without further effort. For example, a full PhD position has been realized to evaluate the data collected through the presented self-assessment.

\(+\)	\(+\)	=	\(143\)
\(+\)	\(+\)	=	\(437\)
\(+\)	\(+\)	=	\(433\)

	\(+\)		\(+\)		=
\(x_1\)	\(+\)	\(x_2\)	\(+\)	\(x_3\)	=
\(49\)	\(+\)	\(47\)	\(+\)	\(339\)	=	\(435\)