Monitoring and Evaluation of E-Learning and Knowledge Network Program for Bosnia-Herzegovina

Goss Gilroy Inc. (Ottawa, Canada) and Productivity Solutions Inc. (Stillwater Lake, Nova Scotia) were awarded a five year contract by the Canadian International Development Agency (CIDA) to conduct a Project Review and Evaluation for the E-Learning and Knowledge Network Program for Bosnia-Herzegovina. This project involves capacity building in all aspects of learning design and delivery for the E-NET Centar at the University of Sarajevo, as well as capacity building and training for key sectors such as municipal management and judicial management.

This assignment involves the application of results based management to the evaluation of outputs, outcomes and impacts of the initiative (based in Sarajevo), including detailed needs assessments and baseline studies. Hal Richman’s special contribution has been his expertise in eLearning design, marketing and evaluation.

HealthCare Partners Medical Group (Los Angeles, California, USA)

Hal Richman and Nancy Valentine worked with HealthCare Partners in the San Gabriel Valley, CA to develop methods and approaches for dealing with the quality of care and cost issues in relation to end stage chronic disease and end of life. Through a combination of interviews, focus groups and document review, they conducted a baseline analysis to better understand:

• The current evaluation and measurement framework for outcomes and quality control measures
• Current practices regarding the administrative management of advance directives and utilization of these in acute clinical situations
• Current practices regarding the medical and psycho-social management of end stage chronic disease and end of life from the administrative, clinician and end user perspectives
• What administrators and clinicians are seeking with respect to learning, training, and performance support for end stage chronic disease and end of life

From the baseline analysis, they recommended a solution that combined usable information for patients and families, patient education via scenario based learning (both on-line and print) and decision support tools. They then developed an evaluation framework and measurable pilot objectives.

The Results Competency Assessment at the United Nations Development Programme (New York, NY, USA)

In 2002-2003, Hal and his team were requested to design and implement “blended learning” to support the world-wide roll-out of the Results Competency Assessment  (RCA) at the United Nations Development Programme. The RCA aligns individual performance planning and staff assessment with greater attention on individual results, knowledge sharing, feedback and learning.

The objective was to train 5,000 staff members in over 145 country offices in the principles and use of the new RCA. The hurdles that needed to be overcome were significant:

• The project needed to be completed within three months.
• The budget was tight.
• The country offices spanned every time zone.
• English was not the first, or even second language, for a number of staff members.
• Internet connectivity in some country offices, as well as phone connections, were poor.

The solution creative solution that Hal and his team developed included:

• CD-ROM training modules to provide “just-in-time” learning and performance support for the RCA. Hal’s team provided instructional design, editing, graphics design and production of the material using the UNDP standard, Click2Learn.
  
• PlaceWare sessions that allowed several offices at once to see and hear a Powerpoint presentation on the RCA, engage in question and answer dialogue with the presenter and take a tour of the Intranet for other learning and performance support resources. These sessions were recorded and made available for off-line viewing by other country offices and Headquarters units.
  
• A repository of questions and answers from face-to-face briefings and PlaceWare sessions were stored on an Internet-based FAQ application permitting staff to ask questions and give comments at any time.
  
  

This blended learning approach enabled UNDP to roll-out the RCA to a large group of people world-wide in a relatively short period of time for significantly less cost than traditional training.