Technology Supported Learning & Retention/Course Evaluation Checklist

Course Evaluation Checklist

In keeping with the theme of Seven Principles for Good Practice in Undergraduate Education, we have put together a list of questions that address each of the principles that may be included in the online portion of a course. In many cases, technology would be included to provide the support. A second list of items are provide to address the learning environment. Some sample technologies and their application are included as a guide.

Seven Principles for Good Practice in Undergraduate Education

These checklist items are guided by Chickering & Gamson's perspective that the goal of a proper undergraduate education should be active, cooperative, and challenging.

The specific items were adapted from Instruction and technology: Designs for everyday learning Mehlenbacher, © 2007 Research-based recommendations for practice in WBI across five dimensions http://www4.ncsu.edu/~brad_m/projects/litre_assess07.rtf

1. encourage student-faculty contact

Instruction

• Does the course include computer-based instruction, such as tutorials, simulations and drills?
• Are course management functions used to integrate the presentation and interaction for the course activities such as advising, delivery of course content, evaluation, collecting and returning assignments?
• Are online resources included to support didactic teaching, that is, supplying course content, posting assignments, or other information germane to course work?
• What technology options are used to personalize instructor "presence" - avatar, post signature, formatted text?

Individual learning support one-on-one

• Is mentoring, such as advising and guiding learners provided via email, web meetings, instant messaging?
• Can the learner experience project-based instruction, either within the classroom or in projects involving community, national, or international problem-solving?
• Is there Temporal independence (teachers and learners do not need to synchronize their meeting times and instructors have the ability to control their schedules)?

2. encourage cooperation among students

Student-student interaction

• Do learners engage in real dialog?
• Is there a public record of conferencing, such as discussion forum posting, threaded discussions, emails to lists of participants, archived chat sessions?
• Are learners encouraged to participate in personal networking and professional growth and such activities as finding persons with similar interests on scholarly discussion lists?
• Is interactive chat used to brainstorm with teachers or peers and to maintain social relationships?
• Are learners required to participate in peer review of writing, or projects involving peer learning, groups/peer tutorial sessions, and peer counseling?

Collaboration

• Are learner activities included for facilitating collaboration?
• Is technology used as a medium of collaboration, conversation, discussions, exchange, and communication of ideas?
• Are course activities provided to encourage collaboration and knowledge exchange between different groups of people (different professional groups, people from different domains, experts and novices, for example)?
• Are there examples of externalizing group processes and increasing awareness of them?

Knowledge building

• Are learner activities included for building of new knowledge on the basis of others' contributions, sharing cognitive load?
• Is computer mediation (for indexing, storing, searching, disseminating information) included?

3. encourage active/engaged learning

Personal experience

• Is there integration of theoretical knowledge with participants' practical experience?
• Is technology used as a medium for participating in simulated experiences, apprenticeships, and cognitive partnerships?
• Does the course include an international platform for the expression and contribution of artistic and cognitive understandings and meanings?

Participation

• Is there a simple, familiar interface so all learners can participate?
• Are there activities emphasizing the links between authentic work activities and e-learning material and virtual discussion?
• To what extent is there Geographic independence - learning without restriction to the physical buildings of the learning institution?
• What functionality is provided to increase learners' control over their own educational experience and the way that they are asked to learn particular materials?

4. give prompt feedback

Assessment / evaluation

• Are structured support and guidance for learning in all phases of the learning process included?
• What activities support conscious reasoning and self-assessment, setting one's own (ie., not set by the environment) learning goals (What do I know; What should I learn?)?
• Are assignments returned graded with notes and comments?
• How quickly are student questions answered?
• Are answers to students' questions accessible by all students in the class to ensure other students with the same questions do not need to ask the question as well?

5. emphasize time on task

Activities

• Are learners required to develop individual and group presentations?
• Is there integration of e-learning with face-to-face learning situations whenever possible?
• Are shared workspaces provided for collaborative activities?
• Are there tasks that force groups to collaborate and co-ordinate knowledge?

Resources

• Do learner activities include retrieval of information from online information archives, such as OPACs, ERIC, and commercial databases?

6. communicate high expectations

Framework

• Do learners have online access to instructor expectations, eg. course syllabus, assignment information, grading criteria?
• Is an evaluation rubric published showing grading critera and associated scoring for each major assignment?
• Do students provide input to the grading criteria?

Model behavior

• Is the instructor communicating frequently as a demonstration of appropriate communication?
• Are students talking more electronically (via e-mail or a chat program) than they do in a face-to-face situation?

7. respect diverse talents and ways of learning

Media

• Is there appropriate inclusion of hyperlinked textual material, incorporating pictures, graphics, and animation?
• Are learners offered different forms of representation and different forms of learning activities (reading, writing, discussing, using metaphors, audio, visual, etc.)?
• Are there videotaped elaboration of subject matter, including interviews, and panel discussions?

Alternative learning

• Are course materials provided in multiple formats for visual, auditory, read-write and kinesthetic learning?
• Are there guest lectures included, which promote interaction between learners and persons in the larger community?
• What tools are provided for the identification, evaluation, and integration of a variety of information?
• Is there Platform independence from any particular hardware or software type (eg. media players, browser-specific display requirements)?
• Are ADA compliant standards met or exceeded?

Learning Environment Technology Considerations

from Instruction and technology: Designs for everyday learning Mehlenbacher, © 2007 http://www4.ncsu.edu/~brad_m/projects/litre_assess07.rtf

Usability Heuristics for WBI ( Web-Based Instruction) Design and Evaluation

Learner Background and Knowledge

Accessibility

• Have the web-based materials been viewed on different platforms, browsers, and modem speeds?
• Is the site ADA compliant (e.g., red and yellow colors are problematic for visually-challenged users)?
• Have ISO-9000 standards been considered?

Customizability & maintainability

• Does printing of the screen(s) require special configuration to optimize presentation and, if so, is this indicated on the site?
• Are individual preferences/sections clearly distinguishable from one another?
• Is manipulation of the presentation possible and easy to achieve?

Error support & feedback

• Is a design solution possible that prevents a problem from occurring in the first place?
• When users select something does it differentiate itself from other unselected items?
• Do menu instructions, prompts, and error messages appear in the same place on each screen?
• Is the feedback of the appropriate type (textual, graphical, auditory) for the information being displayed?

Navigability & user movement

• Does the site clearly separate navigation from content?
• How many levels down can users traverse and, if more than three, is it clear that returning to their initial state is possible with a single selection?
• Can users see where they are in the overall site at all times?
• Do the locations of navigational elements and general layout of text and graphics remain consistent?
• Is the need to scroll minimized across screens and frames within screens?

User control, error tolerance, & flexibility

• Are learners allowed to undo or redo previous actions?
• Can users cancel an operation in progress without receiving an error message?
• Are multiple windows employed and, if so, can they be manipulated easily?
• Can learners annotate the instructional content themselves?

Social Dynamics

Mutual goals & outcomes

• Are learners rewarded for communicating?
• Are communication applications provided that allow synchronous and asynchronous interaction?
• Do communication applications allow information revision, organization, and management?
• Are interactions organized around instructional objectives and task deliverables?

Communication protocols

• Are instructions provided for engaging with other learners online?
• Are instructions and resources related to collaboration, teamwork, and group processes provided?
• Have methods and guidelines for copyright, fair use, and the management of group rights been made available?
• Are documents, resources, and task instructions shared across learners?

Shared resources

• Does environment support group presentation, analysis, problem solving, and artifact construction?
• Can learners control whether information and communication is private at the individual and group level?
• Are various media able to be exchanged with ease by learners?
• Can learners manipulate planning and scheduling resources individually and in groups?

Instructional Content

Completeness

• Are levels clear and explicit about the “end” or parameters of the site (thus avoiding unnecessary learning)?
• Are there different “levels” of use and, if so, are they clearly distinguishable?
• Are the beginnings and endings of tasks easy to identify?

Examples & case studies

• Are examples, demonstrations, or case studies of user experiences available to facilitate learning?
• Are examples divided into meaningful sections, e.g., overview, demonstration, explanation, and so on?

Readability & quality of writing

• Is the text in active voice and concisely written (> 4 < 15 words/sentence)?
• Are terms consistently plural, verb+object or noun+verb, etc., avoiding unnecessarily redundant words?
• Do field labels reside on the right of the fields they are closely related to?
• Does white space highlight a modular text design that separates information chunks from each other?
• Are bold and color texts used sparingly to identify important text (limiting use of all capitals and italics to improve readability)?
• Can users understand the content of the information presented easily?

Relationship with real-world tasks

• Is terminology and labeling meaningful, concrete, and familiar to the target audience?
• Do related and interdependent functions and materials appear on the same screen?
• Is sequencing used naturally, if sequences of common events or narratives are expected?
• Does the site allow users to easily complete their transactions or selections?

Interaction Display

Aesthetic appeal

• Does the screen design appear minimalist (uncluttered, readable, memorable)?
• Are graphics or colors employed aesthetically and functionally, accompanied and identified in text, meaningfully labeled, and reducing unnecessary information where possible?
• Are distractions minimized (e.g., movement, blinking, scrolling, animation, etc.)?
• Is the information design pleasant, engaging, attractive, fun, stimulating, or emotionally satisfying?

Consistency & layout

• Does every screen display begin with a title/subject heading that describes contents?
• Is there a consistent icon design and graphic display across screens?
• Is layout, font choice, terminology use, color, and positioning of items the same throughout the site (< 4 of any of the above is usually recommended)?

Typographic cues & structuring

• Does text employ meaningful discourse cues, modularization, chunking?
• Is information structured by meaningful labeling, bulleted lists, or iconic markers?
• Are legible fonts and colors employed?
• Is the principle of left-to-right placement linked to most-important to least-important information?

Visibility of features & self-description

• Are objects, actions, and options visible?
• Do users have to remember information from one part of a dialogue to another?
• Are prompts, cues, and messages placed where users will be looking on the screen?
• Do text areas have “breathing space” around them?
• Is white space used to create symmetry and to lead the eye in the appropriate direction?

Instructor Activities

Authority & authenticity

• Does the site establish a serious tone or presence?
• Are users reminded of the security and privacy of the site?
• Are humor or anthropomorphic expressions used minimally?
• Is direction given for further assistance if necessary?

Concurrency

• Are feedback and assessment made available for learner viewing?
• Are site features and materials germane, timely, and designed around learner needs and expectations?
• Is instructor-learner feedback thought-provoking (e.g., encouraging elaboration, clarification, questioning)?

Intimacy & presence

• Is an overall tone established that is present, active, timely, respectful, and engaging?
• Does the discourse model solidarity, acceptance, warmth, and trustworthiness?
• Does the site act as a learning environment for users, not simply as a warehouse of unrelated links?

Environment and Artifacts

Help & support documentation

• Does the site support task-oriented help, tutorials, and reference documentation?
• Is help easy to locate and access on the site?
• Is the help table of contents or menu organized functionally, according to user tasks?

Learn more...

• Seven Principles for Good Practice in Undergraduate Education

http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/7princip.htm

• Instructional Immediacy and the Seven Principles: Strategies for Facilitating Online Courses

http://www.westga.edu/~distance/ojdla/fall63/hutchins63.html

Online Journal of Distance Learning Administration, Volume VI, NumberIII, Fall2003 State University of West Georgia, Distance Education Center

• Development and Adaptation of the Seven Principles of Good Practice in Undergraduate Education

link to PDF doc

Arthur Chickering and Zelda Gamson. How the seven principles were developed and uses made of them in the late 1980s and early 1990s.

• Implementing the Seven Principles: Technology as Lever

http://icebreakerideas.com/implementing-the-seven-principles/

• Sloan-C Quality Matters - Rubric for evaluating online courses

http://www.esac.org/fdi/rubric/finalsurvey/demorubric.asp

an easy to use web form that includes a broad overview of components that should be considered as measures of quality in online courses. The rubric assumes that the course being evaluated is a complete, online course. This differs from the purpose of the checklists in that we want to encourage instructors to include new technologies to any course. There is no one standard measure of quality, as implied by the Quality Matters rubric.

• 21st Century Learning: 'We're Not Even Close'

http://www.thejournal.com/articles/21543

technology can be an aid in nine key areas of education:

* Building conceptual understanding of core content;

* Addressing misconceptions;

* Fostering inquiry and investigation;

* Applying knowledge and skills to interdisciplinary challenges;

* Creating and transforming knowledge for meaningful purposes;

* Collaborating with others;

* Apprenticing with experts;

* Engaging and motivating students; and

* Differentiating instruction to meet individual needs.