Introduction to the Jutut service

The Jutut service is an add-on for A+ that enables teachers to collect feedback continuously during a course, to prompt students to comment on specific parts of course content, and to encourage messaging between students and course staff. Jutut is originally designed for use in large-class courses but it may find uses in smaller-scale teaching as well.

Note: In the context of Jutut, the term feedback has a very broad meaning and refers to any input from students to course staff, which could be a question about specific content, or a reflection about their own learning, or a comment about the course (i.e., “feedback” in the narrower sense), or an answer to a teacher-given multiple-choice item, or some other thought or anecdote the student wishes to share. It’s up to the teacher to decide which type(s) of feedback they seek with Jutut.

Here is a partial list of features:

• A course in A+ may be configured to use Jutut. The teacher may then embed, in rST source, a Jutut-based feedback form in any chapters of their course materials that they like.

  • The feedback form will be displayed to students as part of the chapter (could be a separate page too, if the teacher so decides). This is how Jutut is visible to the students: not as a separate system but as an integrated part of a course in A+.

  • The form is fully customizable and may include free text answers, radio buttons, and check boxes.

• Free text answers become messages that are visible to course staff in Jutut’s teacher interface.

  • Teachers are free to decide how or if they wish to act on these messages (and Jutut is not meant as a helpdesk).

  • In some cases, a teacher will want to respond to a message, in which case the student gets a notification about the response in A+. The student may respond back, resulting in a thread of conversation that is anchored in the chapter context from which the original message was sent.

  • Other things that a teacher may wish to do with messages include: compute statistics from answers, find common themes for discussion in class, make notes for course development, etc. (See below for an example of potential benefits of integrating Jutut in a course.)

• To assist teachers in dealing with vast numbers of messages in large-class courses:

  • The teacher may tag messages with message tags (much like students can be tagged with student tags in A+). The tags are customizable; they are only visible to staff. The tags may be convenient for distributing tasks among staff and for other things. (See the example below for more on this.)

  • The teacher may search for messages in various ways. For example, messages where the student has written a particular word; or unanswered messages from a specific chapter; or all messages with a “todo” tag.

• A specially supported question format is time usage: the student can be prompted to estimate how long they spent on particular course content.

  • Jutut’s teacher interface can generate statistics about these answers.

  • (Answers to non-time-usage questions may also be exported in order to compute statistics externally.)

• To encourage answering, teachers may make Jutut a part of how the course grade is formed:

  • Some, all, or none of the questions can be marked as mandatory so that they must be answered to submit the form.

  • Submitting a Jutut form may be made worth some number of points in the course’s grading scheme, or it may be made worth zero points but “mandatory” in the sense that the points for the main activities in a particular chapter/module aren’t official before the student has also submitted the corresponding Jutut form.

• A+’s student tags can be imported into Jutut so that the teacher sees them (e.g., when deciding whether/how to respond to a message).

• Student tags can be automatically generated by A+ based on students’ answers to multiple-choice questions in Jutut forms; i.e., everyone who picked option X automatically gets a particular student tag in A+. (Jutut forms are technically assignments in A+, so this works for Jutut just the same it works for any assignment.)

A course may use Jutut more or less extensively. An example of less extensive use is a course that simply accumulates feedback messages collected at various points during a semester (to complement the summary feedback collected from completers at the end of the course), with the teacher only looking at Jutut once the course is over, to improve future designs. For a more extensive use case, see below.

Example: A Course That Uses Jutut Extensively

Jutut in course materials:

• This is a large-class course that is based on an online ebook with embedded (programming) assignments. Over the course of a semester, the students work through the ebook, several chapters per week. There is a Jutut form at the end of every chapter.

• Each Jutut form asks the student to estimate how much time they spent on the chapter and provides a free-text box for open-ended comments, questions, reflections, anything. It is stressed to students from Day 1 that what they write in these boxes is read by staff and does make a difference.

  • The forms in some chapters have additional questions but the invariant components are time usage and the free-text box.

• Submitting the forms is mandatory in the sense that each student’s points (for the programming assignments) in the chapter only become official once they’ve answered the end-of-chapter form.

  • Within the form, however, only the time usage question is mandatory.

  • Not all students take up the option to send textual comments, but many do, especially after they realize that the comments are actually read and have an impact on what happens during the course (see below).

Teacher activities:

• While the course is running, the teacher keeps track of incoming Jutut messages via the teacher interface, stores questions and other messages that are of interest (to the teacher themself or to other students), and makes note of themes that come up often.

• Each week, the teacher publishes a Weekly Bulletin online. This bulletin covers various topical themes. In particular, it covers questions and themes that came up in the past week’s Jutut messages.

  • Students remain anonymous to each other, but usage of their messages in the bulletin is very explicit: many messages are quoted directly (but anonymously) and embedded into the bulletin.

  • These embedded student messages include, among others: questions (accompanied by an answer from the teacher); mood pieces (students telling stories of their struggles and accomplishments); comments on course content or organization (accompanied with responses from the teacher); weird/funny stuff shared by students; etc.

• Although no guarantees are given to students about receiving personal responses to the messages they send via the Jutut forms, the teacher does answer quite a few of the messages directly, time allowing.

  • A response may be something as simple as “Nice to hear that!” or “Don’t worry, you’ll get to practice that in the upcoming chapters.” or “Good question! I’ll put something about that in the next weekly bulletin.” or “Thanks! I went and rewrote that paragraph in the ebook; I hope it’s clearer now.” Or it may be something more complex.

  • Questions of the type “I read the chapter but I’m still wondering about X” are often answered, but questions like “I’m stuck on programming assignment X, please help!” are not: students who try to use Jutut as a helpdesk for debugging programming problems are directed to other channels.

• The main teacher uses Jutut’s message tags for several things:

  • to sort the teacher’s own work (marking “todo” messages, etc.)

  • to label messages by theme while searching for topics to discuss in upcoming bulletins and saving messages to include in relation to each topic

  • to delegate certain messages to other staff members (in order for them to respond and/or take other action)

• To keep track of workload, the teacher explores Jutut-generated time usage statistics for the various chapters.

  • Crucially, Jutut imports student tags from A+, and students are auto-tagged in A+ based on an enrollment (background) questionnaire. This means that it is possible to explore time usage by demographic—e.g., separately for students who are beginners in programming and those who aren’t, or separately for different degree programmes.

• Last but far from least, the messages collected via Jutut are a rich source of ideas for improving the course and its ebook. Students send in a lot of things, which range from tiny fixes (like spotting typos) to major suggestions for improvement.

  • Minor fixes are often implemented right away, so they affect the course while it’s still running. Bigger ones are saved for consideration later.

  • Usually, it happens that one or two insightful comments/questions each year end up improving the materials in a very direct way: the teacher integrates them into the ebook as quotations (from an “anonymous past student”).

Some benefits (as perceived by the teacher):

• Via the weekly bulletins and, sometimes, direct replies, students get answers to questions that puzzle them.

  • There is, perhaps, a lower threshold for asking questions via Jutut than there is, say, in person after a lecture, and the threshold is almost certainly lower in Jutut than for asking out aloud at a mass lecture.

  • The teacher can take the time to prepare good answers (possibly reusing old materials in the bulletins).

  • Students appreciate the responses from the staff (direct or via bulletins).

• Via immersion to many messages sent by students in situ, right after learning activities, the teacher gains a deeper understanding of student learning in their course and any challenges therein.

  • The teacher may also feel more connected to the students’ reality in the course.

• The messages provide a lot of ideas (of all sizes) for improving materials and the course design.

  • The anchoring of continuous feedback into the chapters means that there are many more concrete and detailed suggestions than what you can hope to get from a generic end-of-course feedback form.

• The Jutut-driven weekly bulletins appear to help foster a sense of belonging among students.

  • Not everyone reads the bulletins, but among those that do, comments like “These bulletins give a sense of community in a largely online course.”, “It was great to see that others had also struggled; I thought I was the only one.”, “Great that others are daring to ask 'stupid' but actually very good questions.”, “Fun to celebrate the eventual success together.”, etc., are not uncommon.

• The Jutut forms provide feedback also from those students who end up not completing the course.

  • In this respect, they are less biased than end-of-course surveys. (Still somewhat biased, since not everyone answers, but anyway.)

• The time usage statistics are (despite caveats) great for course planning.

• Reading and responding to the feedback messages feels personally rewarding (although responding can be a lot of work too).

• It is possible that the use of Jutut in an introductory course helps improve the feedback culture at the university, if students perceive giving feedback as valuable and impactful.

(For more information about this course and its use of Jutut, you may contact the teacher Juha Sorva at juha.sorva@aalto.fi.)

Example feedback questionnaire

Time spent:

Please estimate the total number of minutes you spent on this chapter (reading, assignments, etc.). You don’t have to be exact, but if you can produce an estimate to within 15 minutes or half an hour, that would be great.

“I feel that I have understood the most important things in this chapter.”

fully agree

somewhat agree

somewhat disagree

fully disagree

I’m unable to answer or don’t want to comment.

Give feedback on the chapter.

Posting submission...

×

Identical submission

This submission is identical to the previous submission . Are you sure you want to submit it?

Cancel Submit