Instance Test Checklist#

Purpose: Things to check after making a deep change on a running instance, e.g. changing or upgrading the MySQL database.

Make sure user registration works#

  1. Register a new user (tip, in case your mail provider supports: use yourname+test1@example.com, yourname+test2@example.com, ... to not run out of email addresses).
  2. Make sure you received the activation email. If instance is set up to use an external SMTP server (SES, AuthSMTP, ...), verify that the email was actually sent via that external service and not directly from the EC2 instance.
  3. Make sure that the activation link works and uses the correct LMS domain.

Make sure course creation and import/export works#

  1. Log into the studio and create a new course.
  2. Add a new unit to the course.
  3. Make sure the "Preview" link works correctly.
  4. Import the Demo course to the newly created course.
  5. Export the course.
  6. Delete a unit.
  7. Reimport the exported course and make sure the units are back in place.

Test course settings in Studio#

  1. Click the Start Date of the course and modify the Start/End dates.
  2. Fill in or modify other course details. Refresh the page and make sure everything was saved correctly.
  3. Click "Invite your students" and check an email is composed with the course Name and URL.
  4. Click "Settings > Course Team" and "New Team Member". Add your email. Click "Add Admin" on your entry.
  5. Click "Settings > Course Groups > New content group". Add a content group.
  6. Click "Settings > Advanced Settings" and check everything displays. Change "Invitation Only" to "true", save and refresh, making sure the change was saved.
  7. Click "Tools > Checklist" and make sure it displays correctly.
  8. Click "Maintenance" in the top-right menu, then "Edit Announcements". Do not edit the current announcements if this is a live instance. Click "Create new", add a Dummy announcement, Edit and then Delete it.
  9. Click the Studio icon in the top-left to go back, find the Demo course. Use "View Live" to open the course in the LMS.

Test course in the LMS#

Test course content#

  1. Cycle through the options on "View this course as".
  2. Cycle through the units in the course, checking video, transcripts, inline discussions, polls and other XBlocks all work correctly.
  3. Make sure to submit answers to problems and check the Submission History feature works correctly.
  4. Click Bookmark this page on a few units.
  5. "View unit in Studio" link works correctly.
  6. Click the "Course" tab and make sure the outline displays correctly
  7. Click "Expand All" and check all sections and subsections are expanded.
  8. Check units have a checkmark for being already seen.
  9. Test the "Example handout" downloads correctly.
  10. Click "Bookmarks" in Course tools and make sure the previously bookmarked units are shown there.

Test other course content courseware features (some tabs might not be available)#

  1. Move to the "Course info" tab and verify both everything displays correctly and that the "View updates in studio" works.
  2. Move to the "Progress" tab, verify it displays correctly and "View grading in Studio" works correctly.
  3. Check the "FAQ" tab displays correctly.

Test discussions#

  1. Go to the "Discussion" tab and submit a new post, and add an image. Make sure the image is uploaded to S3/ObjectStorage successfully, and is displayed in the post correctly.
  2. Use the Edit, Follow and Report buttons.
  3. Make a search using the search box.

Test instructor tab features#

  1. Go to the "Instructor" tab and cycle through the tabs, making sure all content displays correctly.
  2. Go to "Membership" and add the created user to the course.
  3. Add this used as Staff, Admin and TA.
  4. Go to cohorts, add a manual cohort and add learners to this cohort. Select the content group created in Studio.
  5. On Discussions, set discussions as divided by Cohort.
  6. Query the user enrolled in the course using their email on "Student Admin".
  7. Select "Data Downloads" section. Click the "Download profile information as a CSV", "Generate Grade Report" and "Generate Problem Grade Report" buttons. Make sure the links to the reports works correctly.

Test LMS settings#

  1. Click the top-right menu and make sure the link to the "Dashboard" works.
  2. Select "Account" on the top-right menu. Modify profile information. Press the "Reset Your Password" button and wait for the email.
  3. Check the "Linked Accounts" display correctly, link Google or other account if available.
  4. If available, check the "Order History" displays correctly.

Other tests#

Check Django Admin#

  1. Make sure you can log into Django admin.
  2. Make a change to the test user created for the checklisting.

If using a custom theme#

  1. Make sure the theme looks ok in general.
  2. Pay special attention to discussion forums and the wiki (they are using different layouts that tend to break more often).

If using analytics#

  1. Make sure link from "Instructor" tab points to Insights correctly
  2. Make sure that OAuth between LMS and Insights works.
  3. Make sure tracking logs are being synced to S3. We want to sync tracking logs even if not currently using analytics, because the client might want to use them some day. Note that tracking logs don't rotate until they're larger than 1Mb, so if you want to force rotate them, run:
sudo logrotate -f /etc/logrotate.d/hourly/tracking.log

Cleanup#

  1. If the checklisting was done on a customer instance and a test course was created for it, log into the LMS as an admin user. Navigate to the 'Courses' tab in the 'Sysadmin dashboard' and delete the test course. Be careful to delete only the correct test course. Keep in mind there's a discussion to deprecate the sysadmin dashboard so this might not be possible after Ironwood.