Design Document for

Program Evaluation Course

By Lisa Berkland
Purpose of the Course The goal of this course is for County and University 4-H staff to fully utilize
the program evaluation process by focusing their evaluations, collecting
data, analyzing and interpreting data and using their findings.
County employed youth staff (approximately 100 staff).
Audience Description
ISU Extension and Outreach paid 4-H Youth Development Specialists
(approximately 30 staff). These specialists serve multiple counties around
the state. Some of them cover the entire state.

322053220.doc

6/23/2016

T.O. The learner will be able to determine the focus of program evaluation.
Major Course
Objectives (Terminal)
Enabling Objectives to support this above T.O.

The learner will be able to define what to evaluate.

The learner will be able to explain the purpose of the program
evaluation.
The learner will be able to identify potential users of the evaluation as
well as how those users might utilize the information.
The learner will be able to formulate a list questions to be answered by
the evaluation.
The learner will be able to select or develop indicators of
accomplishments, changes or progress that can be measured.
The learner will be able to select qualitative, quantitative, or both for
their evaluation design.
T.O. Create a plan for collecting data that identifies the sources, method and
schedule for collecting data. The plan will also articulate the pilot testing process.
Enabling Objectives to support this above T.O.
The learner will be able to identify existing data sources.
The learner will be able to compare different data collection methods.
The learner will be able to choose a method to collect data and explain
the rationale.
The learner will be able to explain the reasons for Human Subjects
Protection protocol.
The learner will be able to recognize components of a data collection
plan.
The learner will be able to develop a data collection plan.
The learner will be able to articulate the pilot testing process.
T.O. Interpret the data resulting in articulating conclusions, recommendations and
limitations.
Enabling Objectives to support this above T.O.
The learner will be able to match data analysis methods with data
collection methods.
The learner will be able to give examples of data analysis errors.
The learner will be able to select a data analysis method aligned with
chosen data collection plan.
The learner will be able to analyze data.
The learner will be able to interpret the meaning of the data.
T.O. Using the conclusions, recommendations and limitations identified previously,
staff will develop a plan to use the evaluation results.

322053220.doc

6/23/2016

RLO Enabling
Objective

The learner will be able to match data analysis methods with data collection
methods.

Learning Assessment
for Course

This course will feature formative assessments. The primary purpose of the
course is to improve staff skills in evaluation, not to award credit or
otherwise measure whether or not staff have achieved a desired level of
proficiency.

Learning Assessment
for RLO

Successful completion matching activity

Instructional Delivery This is intended to be an online course, not self-paced.

method for Course
(overall)
Instructional Strategy Absorb Presentation
for RLO
Do Scenarios
Connect Encourage participants to apply learning to their own project.
Media

The first part will use a PowerPoint style presentation and then the Scenario
will be branched using a slides and perhaps hyperlinks, depending on the
capacity of the multimedia tool.

508 Accommodations

The presentation will have a transcript available. The scenario will be

written on screen, but will also have an audio recording for each slide.
There arent any drag and drops used, so it should be accessible to those
with mobility challenges as well.

Course Structure
Description

This course is made up of 4 modules, matching up with each of the four

terminal objectives.

Seat Time of Course

8 hours

322053220.doc

6/23/2016

Seat Time of RLO

RLO Outline

322053220.doc

15 minutes
A. Overview of Data Collection and Analysis
B. Analysis methods for Qualitative Data
a. Get to know your data
b. Focus the analysis
c. Categorize information
d. Identify patterns and connections
e. Interpretation
C. Analysis methods for Quantitative Data
a. Numerical counts
b. Percentages
c. Measures of central tendency
i. Mean
ii. Mode
iii. Median
d. Measures of variability
i. Range
ii. Standard deviation
iii. Variance
e. Creating ranks
D. Alignment of collection methods with analysis techniques
E. Practice scenario w/knowledge check

6/23/2016

RLO Flowchart

322053220.doc

6/23/2016

Screens/Pages in RLO Estimated number of screens/pages: 22

Knowledge Checks or
Other Assessments or
Practices for RLO

____Dichotomous (T/F, Y/N, etc.)

__2_Multiple Choice
____Multiple Select
____Drag and Drop
____Custom describe; if appropriate, supply flowchart in an Appendix
and reference it here.
____Other describe

Rollovers/click events

____Rollovers
__4__Click Events

RLO Navigation

322053220.doc

Navigation will utilize previous/next words and arrows. Within the

branching scenario, once a selection has been made it will jump to the
corresponding next slide. If a learner selects an incorrect response, they will
be provided feedback and then there will be a button that will give them an
opportunity to try again. After a successful answer to the scenario, learners
will be provided feedback and then asked if they would like to try the
scenario again. If so, the learner will be taken back to the beginning of the
scenario. If not, they will be directed to apply the learning to their own
project and they will be reminded that they can come back at any time to
review.

6/23/2016

Screen Layouts for

RLO

Development Tools
for RLO

The RLO will be authored using Articulate Storyline. The storyboard will be
created in PowerPoint. Audio will be recorded within Storyline.

Ownership

Lisa Berkland will develop the initial course; however/and, Iowa State
University Extension and Outreach (ISUEO) 4-H Department will maintain
the course. The course is being developed for ISUEO.

Development Time of
entire course and
RLO

Estimated time for the entire course: 400 hours

322053220.doc

Estimated time for the RLO: 12.5 hours

6/23/2016

Support requirements LMS administrator

for RLO and course
Campus studio time or purchase of software for ID
Assigned staff time to monitor course/serve as facilitator
Project
Sign-off [optional]

322053220.doc

Please sign below indicating agreement with the proposed course plan and
approving start-up of the storyboard and development phases.

Instructional Designer

Date

Project Manager/Sponsor

Date

6/23/2016