New R/exams Version: Improved E-Learning Tools During a Pandemic
The new version 2.4-0 of the R/exams package has been released on the Comprehensive R Archive Network at https://CRAN.R-project.org/package=exams. This is the first CRAN release after 2.5 years of very active development. Due to the challenges from distance learning during the pandemic a lot of new features were added and enhancements made, especially for the support of various e-learning tools and learning management systems.
Many of changes were prompted by the authors’ own needs (especially in the support of OpenOlat) but also by the many questions on StackOverflow, posts in the R-Forge forum, or tweets and e-mails from users all around the world. In the following the most important features from the update are highlighted. For the full list of changes including technical details and smaller bug fixes, see the NEWS of the package release. More blog posts are planned for introducing some of the new features in more detail, e.g., cloze exercises and evaluation strategies.
Facilitated support of UTF-8 and R/Markdown exercises
Initially, the package relied on Rnw (R/LaTeX) exercises only and optionally supported different encodings for non-ASCII characters, e.g., UTF-8 or latin1 (ISO-8859-1) etc. However, it was cumbersome to correctly set the encoding of input exercises and output file formats.
Given that the majority of users by now seem to rely on Rmd (R/Markdown) exercises and that most output file formats involve using the
pandoc document converter (which requires to use UTF-8) for some aspects, the entire
exams package and all its function now exclusively use UTF-8 as well. All templates etc. have been modified to support UTF-8 out-of-the-box. This greatly facilitates working with UTF-8 as there is no need anymore to set any
encoding argument anywhere.
Support for all other encodings like ISO-8859-* (latin1, latin9, etc.), which had previously been available for Rnw exercises in certain
exams2xyz() interfaces, has been disabled. This is a slight reduction of functionality but will be a great benefit for most users of the package.
Finally, to facilitate working with Rmd exercises and embedded graphics or data files, the packages
rmarkdown are now imported in the package (and not just suggested).
Open-ended text questions with text editor or file upload
The processing of
string exercises for learning management systems like Moodle, OpenOlat, or other QTI-based systems has been extended. By default,
string exercises are intended for closed-format short-text answers that have to be matched exactly by the participants (e.g., asking for the name of an R function or exact value of an output string etc.). Additionally, open-ended text answers can now be enabled by setting the
exstringtype meta-information to
file. The former requests a text editor for entering an answer while the latter requests a file upload dialogue. The essayreg exercise has been modified to leverage this new meta-information. Such exercises then have to be evaluated by the instructors/examiners, i.e., there is no automatic evaluation by the learning management system.
exclozetype meta-information now also accepts
file instead of
string for elements of a
cloze exercise. Currently, the combination of these types with
schoice elements etc. is only possible for QTI-based systems (i.e., OpenOlat in particular) but not for Moodle (whose cloze format does not support open-ended text answers). For illustration, see the new
"lm3" exercise templates.
Enhancements for OpenOlat
Several enhancements and extensions have been made in
exams2openolat() (or the underlying
exams2qti21() function, respectively). The most important new features are:
- Improved processing of the
cutvalue(for passing a test/exam/quiz). By default, this is
cutvalue = NULL(or equivalently
cutvalue = NA) which means that no pass/fail status is computed at all, i.e., only the sum of the points is reported. This is particularly useful if individual online tests during a semester contribute to an overall grade but cannot be passed/failed individually.
- New argument
navigation = "nonlinear". This can be switched to
"linear"enforcing that questions in the test must be answered sequentially while the default
"nonlinear"means that participants can switch back and forth between questions.
- New argument
selection = "pool"that controls how exercises and sections are sampled. By default, the function creates one section for each exercise from which one replication will be selected in the exam. If
selection = "exam"each section contains all questions and one section will be selected for the exam. The
"exam"variant has the advantage that
nsampcan be fully used now and that questions that build on each other can be used in the exam.
- New argument
shufflesections = FALSEcan be set to
TRUEin order to randomly shuffle the order of sections/exercises for each participant. For
selection = "pool"this corresponds to shuffling the sections that contain the pools of exercises. For
selection = "exam"it corresponds to shuffling the exercises within each exam section.
- New argument
cloze_schoice_display = "auto"that controls the display of
clozeexercises. By default, radio
"buttons"are used if the answer list appears in its own paragraph and a
"dropdown"menu is used if the answer list appears inline (and has no mathematical markup). Both options can also be enforced explicitly, independently from the answer list appearing in a separate paragraph or inline.
- The default
stitle(section title) and
ititle(item title) are now
NULLso that items are simply numbered consecutively (1, …, n) and section titles are omitted. Similarly, the default
sdescriptionis now empty omitting the section description as it is typically not necessary.
The are some additional new arguments that allow further fine control of a test’s configuration. The most important and flexible is the argument
config = TRUE. The logical specification
config = TRUE/
FALSE uses the default configuration or switches off the extra configuration entirely, respectively. Moreover, a list of options like
config = list(cancel = TRUE, scoreprogress = TRUE) can be provided for customizing how OpenOlat renders the QTI 2.1 content, see
?openolat_config for more details.
Finally. a new argument
envir has been added that is passed on to
xweave(). By setting this to a common environment, e.g.,
envir = .GlobalEnv (or some new dedicated environment), it is possible to re-use variables generated in one exercise in previous exercises, e.g., for creating a sequence of variables based on the same data set. Note that this needs to be combined with
selection = "exam" (see above).
Enhancements for Moodle
exams2moodle() interface has been improved, in particular for more flexible
clozequestions are now properly supported. By default they are shown as
MULTIRESPONSEcheckboxes and employ Moodle’s default evaluation strategy where each incorrect box eliminates one correct box. A different evaluation strategy can, in principle, be chosen but Moodle might not process all negative points correctly.
clozequestions are still rendered as
MULTICHOICEdrop-down menus by default unless they contain math markup. As this is not rendered by Moodle in drop-down menus, a
MULTICHOICE_Vcolumn of radio buttons is used in this case.
- To allow for customization of both
clozequestions, the are now both
cloze_schoice_displayarguments. This is not fully backward compatible because in previous versions
cloze_mchoice_displaywas also used to customize
schoiceelements. Now a warning is issued in this case.
- To fix the maximum width of fill-in-the-blank cells in
stringsub-items (e.g., when presented in a table), arguments
stringwidthhave been added to
make_question_moodle(). Alternatively, they can also be specified through
exextratags in each exercise. See the fourfold2 exercise for an example and
?make_question_moodlefor more details.
Moreover, there is a new experimental function
moodle2exams() that can take a Moodle XML quiz file with numeric, multichoice, shortanswer, and essay exercises and convert them to R/exams exercise files, either in Rmd or Rnw format. If the text formatting is more advanced (e.g., containing mathematical notation or tables etc.) the function might not lead to fully satisfactory results but still provide a useful starting point for subsequent manual editing.
New exams2xyz interfaces
Various new learning management systems or quiz engines can now be interfaced using R/exams. The work on
exams2grasple() was financially supported by the Dutch Ministry of Education, Culture and Science (Project code OL20-06), and the University of Amsterdam.
exams2ilias()for the open-source ILIAS learning management system. This is essentially a convenience wrapper to
exams2qti12(), tweaking a few defaults and employing a somewhat modified XML template. Not all question types are supported, though, mostly string questions with open-ended answers and multiple-choice and single-choice questions. Numeric and cloze questions are not supported, yet.
exams2testvision()for the Dutch testing platform TestVision. It is essentially a fork of
exams2qti21()that incorporates TestVision’s own strict implementation of QTI 2.1. See the online tutorial on how to upload the zip output from
exams2testvision()into the system by selecting it in the import menu and then moving the imported material to the appropriate directories.
Additionally, there is a new function
testvision2exams()to convert TestVision’s QTI 2.1 questions to R/exams exercise files, either in Rmd or Rnw format. The supported TestVision question types are ‘invul (numeriek)’, ‘een-uit-meer’, ‘meer-uit-meer’, and ‘open’ which are converted to
exams2grasple()for Grasple, a Dutch practice platform for mathematics and statistics education. It supports
schoicequestions which are exported to a zip file containing Grasple’s JSON format. Note that currently importing cannot be done by users themselves; it requires a request for manual import by a Grasple team member.
exams2particify()that can export exercises to a comma-separated values (CSV) format for import in the audience response system Particify, the successor to ARSnova. In particular, single-choice and multiple-choice exercises are fully supported while num and string question are converted to open-ended text questions.
exams2kahoot()that can export sufficiently simple single-choice and multiple-choice exercises to an Excel sheet via
openxlsx::write.xlsx()that can be imported by the game-based learning platform Kahoot! (suggested by Rushad Faridi). Exercises are converted to plain text and questions must not exceed 120 characters, answers must not exceed 75 characters.
Enhancements for written exams (NOPS format)
The exams2nops() function is the R/exams interface for generating exams of single-choice and multiple-choice questions, that can be scanned and evaluated automatically.
Various improvemens have been made in
nops_scan(), especially for scanning the boxes pertaining to the student registration ID. Rather than reading a very small area around each box and just shaving off its borders, a larger area is read now and then shaved iteratively. Hence, it is also easily possible to further increase the
size of the area which may sometimes lead to improved scanning results.
exams2nops() function itself, the handling of
reglength < 7 has been improved. Internally,
reglength = 7 is still enforced (and thus necessary in the registration CSV file) but the initial IDs are fixed to “0” in the exam sheet and corresponding boxes ticked already.
Finally, language support has been extended by two new languages: Czech (
cz, by Jindřich Marek) and Galician (
gl, by Marta Sestelo & Nora M. Villanueva).
Enhancements for Blackboard
exams2blackboard() interface the new argument
mathjax = NULL has been added that optionally embeds the MathJax
<script> in each of the exercises. This allows to render mathematical content by the MathJax plugin for all modern browsers. Without MathJax, the browser itself has to do the rendering directly (which is supported by Firefox and Safari but not by Chrome for example). The default is
converter = "pandoc-mathjax" is used. But also for the default converters (producing MathML output)
mathjax = TRUE can be used. (Suggested and tested by Sean Quallen and Gabriele Cantaluppi.)
Moreover, the following improvements may be useful for Blackboard users: Rendering verbatim code chunks can be fixed from
<code> tags if
fix_pre = TRUE (default) which is necessary in classical Blackboard systems. Points can be specified through
- When running exercises via
knitr::knit()errors in the R code will stop the evaluation now by default. This was always the default behavior for Rnw exercises (i.e., when processed with
engine = "sweave") but now is also the default for Rmd exercises and for Rnw exercises via
engine = "knitr". In exercises processed via
knitr::knit()it is possible to carry on with code evaluation after errors (the default in
knitr) by setting the chunk option
error = TRUE. Similarly, the default handling of warnings has been set to
warning = FALSEso that warnings are reported on the console rather than in the weaved/knitted exercises.
- Added new argument
texengine = "pdflatex"to
exams2pdf()which is passed on to
tinytex::latexmk(..., engine = texengine). Provided that
tinytexsupport is installed, this option can also be set to
texengine = "xelatex"or
exams2pandoc()gained an argument
exshuffle = NULLwhich can be modified to overrule the
exshufflesetting within an exercise (if any). For example, setting
exams2xyz(..., exshuffle = FALSE)retains the full answerlist without any permutation.
xexams()a new argument
rds = FALSEwas added that indicates whether the list returned by the function should also be saved as an RDS data file. If
rds = TRUEthe file name
metainfo.rdsis used but alternatively
rdscan also be specified as a character file name to be used. The new
rdsargument is leveraged in
exams2qti21(), as well as other interfaces built on top of these.