This section is an introduction to creating programming exercises
We will use the “Hello Python” exercise on this course as an example.
The files related to this exercise are in directory exercises/hello_python.
The file programming_exercises/hello_world.rst is a sample page
which contains two programming exercises: the classic “Hello world!” program
for Python and Scala programming languages. Let’s look at the “Hello Python!”
exercise. It requires two lines:
.. submit:: python 10
The exercise is included with the submit Sphinx directive, which is from the
A+ RST tools package. That directive
places exercise submission forms. The number 10 is the maximum score that
the student will get from this exercise. The exercise has a configuration
file exercises/hello_python/config.yaml which is defined with the
:config: option. As was mentioned in the Docker chapter, the definition
of this directive is in the file a-plus-rst-tools/directives/submit.py,
but you don’t need to understand the contents of that file.
This file has the exercise configuration for Mooc-grader. The following is
a copy of exercises/hello_python/config.yaml.
title: Hello Python!
description: A simple grading example with Python code
In this exercise you must implement a function <var>hello</var>
that returns a string "<samp>Hello Python!</samp>".
- field: file1
Explanation of the settings:
The title of the exercise. The RST file having a reference to the
exercise may override this when using the “aplus-submit” directive.
This is not actually shown anywhere.
The HTML code for the instructions for the student.
value “access.types.stdasync.acceptFiles” defines that the
student must submit one or more files to complete the exercise.
This defines each file that the student can submit. Each file might have
different name at student’s computer, but they are renamed by the “name”
This specifies the Docker container which is used for grading.
image is the container image. The value apluslms/grade-python:3.6-2.7 means
that the container is grade-python made by organisation apluslms. The container
has Python version 3.6 installed and it is based on version 2.7 of the
“grading-base” container. For full documentation, see the repositories for
mount is the relative path of the directory which will be mounted to the directory
/exercise inside the container (read only). This directory should contain
the files required to run the grader program.
(The student’s submission files will be mounted separately by the platform
to the path /submission/user.)
If Python graderutils are used (covered later in this page), the mount directory
would contain, for example, the files config.yaml, run.sh,
test_config.yaml and various Python files (model solution, unit tests).
cmd describes what command is run inside the container. run.sh is the
actual grading script. The command may include parameters and it is not
required to be a shell script named run.sh.
The documentation of grading-base is a good
starting point for understanding the grading system.
This is the shell script which is run inside the grading container.
# The uploaded user files are always in /submission/user
# and named identically to config.yaml regardless of the uploaded file names.
# The directory /submission/user is also the default working directory
# in the container.
# The mount directory from config.yaml is in /exercise.
# Append the required support files to test user solution.
# Add the working directory to the PYTHONPATH so that the grader
# can import the student's submission. The grader program is started
# under the path /exercise since there is no need to copy it to
# the working directory.
# "capture" etc description in https://github.com/apluslms/grading-base
capture python3 /exercise/tests.py
Note! run.sh must have executing rights. That is, if you create the file from
scratch, you must do the following:
chmod a+x run.sh
Python-grader-utils (just “Graderutils”) is a Python library for test suite
management, file validation and test feedback formatting. It is used with
Python programming exercises. The source code and
documentation is here: https://github.com/apluslms/python-grader-utils
By default, Graderutils uses the configuration file test_config.yaml in the
exercise directory. A simple test_config.yaml looks like this:
- module: tests
display_name: Local tests
- module: grader_tests
display_name: Grader tests
- type: python_import
- type: python_syntax
test_groups defines which Python unit test files are executed.
module is the name of the Python file (without .py)
display_name is the title for the test group.
Typically there is file tests.py which is given to the student. It has some
very basic unit tests. Typically some points are given for passing these
tests. Another file is typically grader_tests.py which has the secret, more
complex and thorough unit tests. Most of the exercise points are obtained
by passing these grader tests.
validation instructs Graderutils to make a syntax analysis tests of the
submitted files before the unit tests are executed.
In the example above, Graderutils checks two items according to the validation settings:
With Graderutils, it is possible to forbid some Python syntax or libraries in
some particular exercise, for example, deny using the default sort function of Python
in an exercise where the student must implement their own sorting method.
If one needs to find out why a grader for some particular Python exercise
does not work, here are general tips.
exec >> /feedback/err
If all these fail, one can run a shell inside the grading container.
Probable cause: the run.sh file of this exercise does not have execution
If that does not help, debug the exercise grader inside the grading container.