Tips for adapting assessment for distance learning

Regardless of the emergency situation, it is important to understand how well the students have learnt and acquired the learning outcomes. As the content, volume and form of several courses has changed, it may also be necessary to also review the assessment of the course. If it is necessary to make changes in the assessment, the changes should be coordinated with students as soon as possible. The changes should also be recorded in the SIS. To change the form of assessment from graded to pass/fail assessment, you have to submit a proposal made in consultation with the programme director; the change will be entered in the SIS by Office of Academic Affairs.

GENERAL RECOMMENDATIONS

  1. Review the plans made at the beginning of the semester
    1. If the course in its content and form is, irrespective of the emergency situation, quite similar to what it was before (fully e-course or a course with extensive online support), it can be assessed as planned. 
    2. The assessment may take place in the planned form and no changes are needed (e.g. essay, project, electronic test).
    3. If these forms of assessment are not suitable, think whether formal rearrangements are enough (bringing a test paper into the online environment) or more substantive changes are needed (for example, changing the problem setting or type of questions).
    4. Check that the summative assessments and intermediate tasks are in balance. To support students’ learning, it may be reasonable to distribute assignments over time. Make a clear system how many points students can get from interim assignments, and negotiate it with students.
    5. Ensure that assessments are not accumulated to the end of semester. If you change the method and time of assessment, it is important that you consult with the programme director who will see that the schedule of assessment in different courses is reasonable.
  2. If changes are necessary...
    1. If major changes are needed, first of all think of the learning outcomes. What is the most important that the student has to obtain (knowledge, skills, attitudes)? Consider the activities and assignments that would show the acquisition of the learning outcomes. 
    2. If you have already reduced the planned volume of studies, consider which part of the course should definitely be tested and what assignment would be the best for testing.
    3. Instead of a classical test paper, other assessment methods can be used in distance learning: for example, case analysis and problem-solving tasks (incl. justification, references to learning materials); making a study aid or a short lecture (e.g. 3 min) on a specific topic; an essay or a short paper; a critical analysis of a scientific article or research report; students prepare questions about the materials; group work / group project (with report).
  3. How to plan the duration of an online test or exam?
    1. Taking an online exam may take the students more time (for example, adaptation to the technological environment, reading from the screen, and therefore, also reading the assignment online take more time).
    2. To better plan the exam duration, it is best to take the test yourself. The time you as a teacher spend on the test should be multiplied by the coefficient 1.5.
    3. When you set a time limit to an exam, consider that reproducing factual knowledge takes less time than discussions, analysis, case studies.
    4. It is possible to adjust the time for answering by specifying the recommended length of answers to open-ended questions (approximate maximum number of words).
    5. Before an important exam giving a test exam with similar technical requirements is recommended. The test exam may be short, consisting of just a few questions. It helps everyone to try out and ensure that the system works.
  4. In the case of distance learning, the teachers’ burden of providing feedback is likely to increase. How to give feedback efficiently?
    1. If you use the Moodle assignment or workshop activity for testing, you can use the marking guide or rubrics (see instructions, question 3). Create simple and clear assessment criteria with a scale. This way, instead of writing lengthy feedback, you can just tick the suitable boxes.
    2. Give audio feedback (with audio or video recording). For example, in Moodle forums, feedback to assignments, etc., where the Moodle text editor is used for giving feedback, you can record up to two-minute feedback in audio or video and present it to learners. It is also possible to use recording in Panopto to give feedback. Panopto allows recording of comments and assessments.
    3. Set a task for students to give feedback to a fellow student (for example, using the given marking guide). Ask students to justify their reasoning and assessment when giving feedback. This also makes it possible to assess the student who gives feedback.
    4. Specify the recommended length of answers to open-ended questions (approximate number of words), so you can avoid getting overly long answers. You can switch on the word count in Moodle forum, and the word limit can be set in the assignment.
    5. If the course is large and you give feedback to the whole course, there is a risk that students may not be able to associate the feedback with their individual work. When you give general feedback to students, also ask everyone to find comments about their own work in the feedback.
  5. How can you ensure that learners do not use external help while taking the test in distance learning?
    1. As teachers-lecturers we cannot ensure it, but we can trust the students and design the assignments in such a way that there is no point in using outside help. A well designed assessment task is a good opportunity to learn. Read about technical solutions and rules for identification of students below (No. 6).
    2. Instead of testing the knowledge of facts, you should test understanding and application of knowledge. Ask students to explain their answers and reasoning (you may ask them to (audio) record and present the answers in Moodle, for example in a forum or using the assignment resource or the test, as an answer to essay-like question).
    3. Use an open test for assessment (permitting the use of any materials) – prepare questions that require students to think more thoroughly (for example, compare, explain the reasons for this phenomenon).
    4. Set a time limit for answers.
    5. Prepare a larger test question bank or versions of the same assignment with different source data, and divide questions to students randomly.
    6. Before the exam, ask students to take a so-called oath of honesty (for example: I will answer the questions without using outside help/ I will answer the questions entirely on my own, using only the materials and resources permitted by the teacher).
  6. Use of technical solutions to evaluate student performance in tests and exams
    1. If it is important to ascertain the identity of a learner during the exam, various technical solutions can be used.
    2. One method of checking the exam is to video record the exam: both the screen and the student.
    3. In this case, you should inform the student (ask the student’s informed consent) and if the student refuses consent, offer a possibility to take a face-to-face exam.
    4. The student can record the exam, using, for example, the following computer programs:
      1. Screen-o-matic – enables to record the screen and the webcam video at the same time; record up to 15-minute videos with the free version
      2. Open Broadcaster Software (OBS) – enables to record the screen and the webcam video at the same time
      3. QuickTime Player (MacOS) – enables to record either the screen or the webcam video
      4. Panopto – enables to capture both the screen and the webcam video, the recording is saved in the UT server, in the Panopto folder of the course, subfolder Assignments. Only the learner and the teacher can access the recorded videos. See instructions.
    5. Students can save the files on their computer and later upload them to the UT server and make them available for the teacher. This can be done by means of OneDrive, which each student has the right to use. The student may delete the recording later. Alternatively, it is possible to use the ownCloud server that is offered by the UT and works similarly. Detailed instructions for using OneDrive and ownCloud are available on the Helpdesk page.
    6. The data recorded for identification, i.e. the videos, may be preserved no longer than until the end of the academic year. It is important to ensure technical readiness and a person responsible for deleting the data. If necessary, IT Helpdesk can help you.
    7. The Urkund plagiarism checking system can be used for plagiarism detection in exam assignments.
    8. See also Frequently asked questions on video recordings of exams and assignments.
  7. Organising of thesis defences during emergency situation
    1. Thesis defences may be held either with up to ten people present in the room, or online. For defences that are held in the physical room, the same measures apply as those applicable to face-to-face studies after 18 May.
    2. In the case of virtual defences, due to personal data protection requirements, only applications approved by Information Technology Office must be used. The permitted video transmission applications are BigBlueButton, Teams and Zoom. When the session is recorded it is important to ensure that the recordings are saved in storage locations under the control of the university. If you will record with Teams make sure that the recording is not available in Stream to all UT members.
    3. In order to record the defence meeting and to transmit or publish the recording, it is necessary to ask the student's written consent. The responsibility to prove the student’s consent lies with the person who further processes the video recording (views, transmits or discloses the recording).
    4. The recording is not stored longer than until the end of the academic year; the person who recorded the video is responsible for deleting the recording.
    5. Technical advice and support is provided by IT Helpdesk.

RECOMMENDATIONS BY ASSESSMENT METHODS

Planned form of exam Tips on how it could be done

WRITTEN 

EXAM 

(closed materials, classroom; multiple choice questions and open-ended questions)

In the emergency situation it is possible to have the written exam using online tools (for example, Moodle test, assignment). 

  1. Before the test, ask students to restart their computers – this helps to ensure that nothing goes wrong. 
  2. Set a time limit for both closed and open exam: remember that reproduction of factual knowledge takes less time than explorative, analysing problem tasks and questions. If possible, test how long it takes an average student to answer; if you try out the test yourself, multiply the time it takes you to answer with a coefficient 1.3–1.5.
  3. In Moodle assignment you may prepare the task so that the learner has to submit the answer in audio or video, recording it in the online assignment using the text editor.
  4. In Moodle assignment it is possible to make the exam assignment available for a fixed period only, so the learner can see the description of the assignment from the moment the assignment is open for submission.
  5. In Moodle assignment it is recommended that you require learners to use the button Add submission.
  6. If you use an open-end (essay-type) question, we recommend setting a word limit to the answer (specify the recommended maximum length of the answer). Prepare more questions to choose from – the system will randomly select a different set of questions for each student.
  7. Ask questions that require comparison, application, analysis.
  8. If you use the Moodle test, it is advisable to set the test so that after the test submission time has passed, the answers are revealed to students automatically, and if there are many questions in a test, they are not all on the same page. When the learner moves on to the next page, the previous answer is saved to ensure that the answer is not lost if internet connection is interrupted. If questions are placed on separate pages, it is more complicated for learners to share the answers with other learners.
  9. It is recommended you set in the test settings that after completing the test (until all learners have completed it) the learner is only displayed the score and the general feedback. This helps to ensure that the questions and answers are not share to other learners.
  10. If the time limit for the test is, for example, 30 minutes, definitely make the test available for more than 30 minutes (for example, 40 minutes), because not all students may be able to start exactly at the time the test is made available and as a result, cannot pass the test on time.
  11. If you use the test window with JavaScript security (full-screen pop-up, which covers other windows and has not navigation bars, the learner cannot paste an answer copied from another page into the answer box, and you cannot open the menu by right clicking), let the learners know in advance that they cannot paste the answer in the answer box. Otherwise it may happen that the learner types a long answer in Word and later discovers that it is not possible to paste the answer into the answer box and needs to start answering from the beginning. 
  12. For problem-solving tasks (e.g. mathematical tasks) that require showing the way to solution, ask students to write the way to solution on paper and send it to you as a legible image/photo (for example, in the Moodle course using Assignment or in the forum) – combine a written exam with brief oral discussions in the online environment.

When you plan an exam in an e-course, you may ask for help from the instructional designer of your faculty (which tool to use, technical solutions) or academic developers (methodological support). Instructions for Moodle test: https://sisu.ut.ee/juhendid/test

ORAL EXAM

(see example 2)

Taking an oral examination in the online environment differs from a similar exam taken in a classroom. This form of exam may be less formal (as you are at each other’s home) and therefore also the style of communication may be different. Before the exam, check what is visible in the background (if possible, blur the background) and ensure that no one or nothing interrupts you during the exam.

  • Inform the student in advance when and in which e-learning environment the exam takes place; the estimated duration of the exam and how it is conducted (describe the process).
  • As an introduction, ask the student how he/she has been doing, this helps both get adjusted to the e-meeting; once again, give the student an overview of how the exam will be conducted and give the student an opportunity to ask questions.
  • If necessary, ask the student to present an identity document.
  • Agree with the student on whether and how you use the web camera during the exam and whether the exam session is recorded.
  • Consider how to present the exam questions to the student. Before the e-meeting you may give a preliminary assignment (analysis, calculation, problem-solving task, case, e-test), on which basis you can start the conversation.
  • Give the student a fixed time for preparation before answering. Decide whether you continue online contact during the preparation or you make contact again after the preparation time ends.
  • While the student is answering, take notes about the answers. This will ensure that the information is recorded in case an unexpected technical failure occurs. 
  • After the exam questions have been answered, immediately give feedback to the student, or first ask the student to evaluate his or her performance.
PRESENTATION OF GROUP WORK OR PROJECT Students can present their group work using BigBlueButton. They can also provide feedback to each other. To assess each student’s contribution to group work, you may ask the students to add process analysis – the group will take records of each student’s ideas and contribution. Group members can also score each student’s contribution in Moodle.


Tips for online assessment (easy-to-read methods and principles): https://www.teachology.ca/knowledgebase/how-do-i-assess-student-learning-online