Quality Attributes and Metrics for a Unit

CSS 566: Sprint #1

Overview

In this document, we define quality attributes and metrics for a Unit. A Unit is defined as an element of a course we are creating in CSS 566 (the “Product”). Quality attributes are the benchmarks that describe a Unit’s intended behavior within the environment for which it was built. It also provides the means for measuring the fitness and suitability of the Unit within the Product. While defining a Unit’s quality attributes and metrics, we outline generic guidelines that may be applied to any single Unit of the Product. These guidelines are designed to enable developers and other related stakeholders that are responsible for developing a Unit to measure its quality.

The objective is divided into 3 sub-tasks as follows:

  1. Define usability qualities for a Unit.
  2. Define design qualities of the Unit’s structure.
  3. Define what metrics need to be measured for a Unit’s quality.

1. Define Usability Qualities For a Unit

Usability of a Unit is a quality attribute that assesses how easy user interfaces are to use. An alternative definition of usability of a Unit is, methods for improving ease-of-use during the design process.

a. Learnability - How easy is it for users to accomplish basic tasks the first time they encounter the design. [3]

  • Navigation-
    • Easy to find the content in the unit.
    • Easy to navigate the structure in a unit.
    • Easy to comprehend your place in a unit.
    • Easily comprehensible or self-explanatory navigation menu.
  • Intuitive interfaces [2] -
    • Easy to use interfaces through use of previously developed similar systems.
    • Use of contrast in the Unit’s content to catch user’s eye.
    • Use of abbreviated terms only after introducing their elaborated form.
    • The homepage shall clearly state what the product offers and what users can do on the site. [1]
    • Visual cues using different fonts, colors, and images.

b. Efficiency – Once users have learned about the Unit, how quickly can they perform tasks. [3]

  • Easy to learn structure of feedback system by observing. [2]
  • Easy to comprehend rating system that provides information on how each rating is defined as.
  • Use of text-boxes to accept open-ended suggestions to improve or add a unit.

c. Memorability – When users return to the Unit after a period of not using it, how easy can they reestablish proficiency. [3]

  • Modularity in the structure of a Unit.
  • Clear distinction between different hierarchical levels in a Unit.
  • Use of selective colors that make the website look uniquely attractive.

2. Define Design Qualities of the Unit’s Structure

Some of the design qualities for the Unit’s structure overlap with the qualities for its usability as outlined above, but there are fundamental differences. For instance, the Unit’s structure needs to be easily navigable and memorable as previously mentioned, but the actual direction for what goes into a Unit’s composition is governed by established practices around course and syllabus design. Thus, when defining how well a Unit’s structure is constructed, one needs to understand concepts relating to the content and how it influences the structure. This section defines design qualities for structure that are separate from Usability.

a. Logical Flow

  • Unit includes overview and description of its topics [4]
  • Unit topics are arranged in logical order of complexity and relevance [4]
  • Assignments, projects, and exams are woven in at appropriate intervals [4]
  • These are clearly broken down into how much impact on the student’s grade they will have.

b. Understandability

  • Students can tell what they need to learn by the Unit’s structure [5]
  • Students can tell what they need to do to learn the Unit’s content [5]
  • Students can tell when they need to learn and perform work [5]
  • Students can tell how much time they need to spend on a Unit [5]

c. Conformance to Institution’s Organizational Requirements

  • Length of Unit is appropriate to amount of time course has [5]
  • Amount of content in Unit is sufficient for curricular requirements [5]

d. Relevance to course level and difficulty [5]

  • Product is a Software Management (SM) course at a graduate (500) level, so each Unit must be structured towards a disciplined, research-oriented approach.
  • Structure should be strong or loose as appropriate to professor’s teaching style
    • Loose is more apt for a SM course that is structured as an exercise in agile software management

3. Define What Metrics Need to be Measured For a Unit’s Quality

Metrics for a Unit’s quality are related to sections 1 and 2; rather than defining what needs to be defined for quality, we are defining how the Unit needs to be measured. This is difficult since the main measure of a course’s quality (which is naturally composed of various Units) is “if the learner’s performance improved as a direct result of taking the course.” [6]

a. User Surveys

  • Students fill out Unit evaluation that rates each of Section 1 and 2’s quality attribute definitions [7]
  • Students fill out Unit evaluation on relevance, quality, and variety of content [6]
  • Professor’s colleagues can peer evaluate the Unit on quality of content
  • Surveys are carried out regularly (i.e. at the end of every Unit) [7]
  • Survey findings are used to continuously improve the Unit [7]

b. Aggregate student performance results

  • Record and observe students’ performance on a Unit’s assessments to measure how effective the Unit was at presenting concepts
    • Combine performance results with student surveys to separate out structure effectiveness vs. content effectiveness

c. Unit relevance

  • Unit is updated regularly and reflects current industry trends [7]
  • Unit is designed by a professor who has proper academic credentials [7]

d. Technical measurements

  • Interactive elements (i.e. audio and video) are high quality [6]
  • Look, feel, and overall visual design are consistent with other Units and the Product as a whole [6]
  • Unit content is presented in varied ways [6]

References

  1. http://www.softwarearchitectures.com/qa.html
  2. https://en.wikipedia.org/wiki/Usability
  3. https://www.nngroup.com/articles/usability-101-introduction-to-usability/
  4. http://www.washington.edu/teaching/teaching-resources/preparing-to-teach/designing-your-course-and-syllabus/
  5. http://www.tonybates.ca/2012/06/26/nine-steps-to-quality-online-learning-step-7-design-course-structure-and-learning-activities/
  6. https://community.articulate.com/discussions/building-better-courses/how-to-measure-quality-of-a-course
  7. http://publications.sreb.org/2006/06T05_Standards_quality_online_courses.pdf