- EDIT
- 4. Questionnaires
- 4.1 Creating questionnaire exercises
Creating questionnaire exercises¶
One can easily create simple questionnaire exercises for an A+ course, such as one-line text answers and multiple choice exercises. This page shows many examples on these.
Technical remark. The most recent and complete documentation on this can be found at the source code of the A+ RST tools package. Technically A+ RST tools provide specific Sphinx directives for writing course HTML and configuration files, and then A+ and mooc-grader implement the actual functionality.
General description of the options¶
The questionnaire
directive implements a questionnaire exercise. Its
arguments define the exercise key (exercise identifier for A+) and max points
with the optional difficulty (A
, B
, C
, etc.). For example,
.. questionnaire:: easyexercise A50
sets key to easyexercise
,
maximum points to 50
and difficulty A
. The difficulty parameter
does not affect any scorekeeping, but only works as an indicator for the
student. (The A+ REST API can group earned points by the difficulty so that
the teacher may use the difficulties for computing final course grades.)
The questionnaire directive accepts the following options:
submissions
: max submissionspoints-to-pass
: points to passfeedback
: If set, assumes the defaults for a feedback questionnaireno-override
: If set, the conf.py override setting is ignoredpick_randomly
: integer. Set the pick_randomly setting for the quiz (select N questions randomly on each load)category
: exercise category
The contents of the questionnaire directive define the questions and possible instructions to students.
The question directives pick-one
, pick-any
, and freetext
take
the points of the question as the first argument. The sum of the question points
should be equal to the questionnaire max points. The question directives accept
the following options:
class
: CSS classrequired
: If set, the question is required and empty answers are rejectedkey
: a manually set key for the question. This affects the HTML input element and the key in the submission data. If no key is set, note that automatically added keys change when the order and amount of questions is modified.
The freetext
directive also accepts the following options in addition to
the common question options:
length
: (horizontal) length for the HTML text input in charactersheight
: vertical height of the text input in rows. If this is greater than 1, the textarea HTML element is used. Otherwise, a text input is used.- Also other options are defined in the questionnaire code of A+ RST tools, but they mainly affect the CSS classes and they were implemented for very narrow usecases.
The freetext
directive accepts a second positional argument after the points.
It defines the compare method for the model solution. A textual input can be
compared with the model solution as int
, float
, string
,
or unsortedchars
(unsorted character set). Another option is regexp
which takes the correct answer as a regular expression and tries to match the
submission with it using the Python re library.
Strings have comparison modifiers that are separated with a hyphen (-
).
For example, to create a freetext question which compares the answer to the
correct answer as string, and ignores whitespace characters and quotes, write
.. freetext:: 30 string-ignorews-ignorequotes
.
ignorews
: ignore whitespace (all space characters, applies to regexp too)ignorequotes
: iqnore quotes"
aroundrequirecase
: require identical lower and upper cases (only with the string type)ignorerepl
: ignore REPL prefixesignoreparenthesis
: ignore parenthesis( )
The question directives may define instructions. After the instructions, the contents of the directive define the choices, the correct solution, and possible hints. The hints are targeted to specific choices and they are shown after answering. See the example below.
Examples¶
The freetext
subdirective creates text boxes and grades based on their
input.
Testing the questionnaries¶
It is good practise to test your questionnaire, especially if there are several correct answers. Note that A+ will show the correct answers for the students who have submitted for the maximum number of times (but not for anyone else).
String, int, or float?
Use the int
answer type always when the answer is an integer. Of course
the answer could be compared to the right answer as a string. However, the
benefits of int
over string
are the following. First, extra space
characters are always ignored. Second, A+ shows a histogram of the numerical
answers to the teacher when they click on View all submissions on the
exercise box in A+, and then Summary. See Figure “the summary a
float-freetext questionnaire” below.
float
works the same way as int
. Currently it considers the answer
to be correct if the difference between student’s answer and the model
solution is at most 0.02.

Figure: the summary of a float-freetext questionnaire. In this case, 304 students (83 %) have answered the question and most of them (300) have received 20 points. There is only one question whose correct answer is between 4.8 and 5.0, with 300 students giving that answer. Four students have an incorrect answer having value between 1.4 and 1.6.

Figure: the summary of a string-freetext questionnaire. The answer is an SQL query, and A+ shows the unique answers. There are four identical answers of one type and three identical answers of another type.
Regular expressions are useful when there are multiple solutions, or when one wants to have some tolerance in numeric questions, like accept real numbers beginning with 0.014, 0.015, or 0.016.
Additional information¶
See the source code of the A+ RST tools questionnaire directive and the corresponding form implementation in mooc-grader.