John G. Richardson
Extension Program Delivery and Accountability Leader
R. David Mustian
Extension Program Evaluation Leader
Kevin J. Gamble
Director, Extension Technology Services
North Carolina State University
Abstract
In recognition of the increasing public demands for accountability, the North Carolina
Cooperative Extension Service embarked on a mission to develop a new
reporting/accountability system to meet current and anticipated future accountability
needs. These efforts involved a large number of persons in developing a conceptual
design for the new system. Focus was placed on being able to capture planned
program measures of progress and impacts as well as contacts, and program
successes. A computerized World Wide Web graphics based system was
developed for entry and accumulation of the reports. The system is now
implemented, and providing a vast resource for communicating program
accomplishments to both internal and external audiences. Uses have included
success stories specific to congressional districts to multiple information for a
Legislative mandated study of Cooperative Extension in North Carolina.
Introduction
The demands for increased levels of accountability seem to be ever-increasing for
practically all societal entities. With this expectation so prevalent, it is no surprise
that policy makers are increasingly focusing close attention on the relevance of
organizations and their value to their constituents, as well as to society as a whole.
Such emphasis on accountability led to the passage of the Government
Performance and Results Act (GPRA), which is now being implemented. Reports
from across the country indicate that states, counties, and essentially everyone else
appear to have similar ideas relative to making sure their tax dollars are being
expended as intended.
North Carolina has not been isolated in this age of accountability, and therefore, has been focusing considerable efforts to satisfy current and anticipated organizational and programmatic accountability needs and demands. One major component of this increased focus is the development of a completely new reporting system that essentially captures all of the accomplishment results emanating from programming efforts in all counties. These results include contacts, measures of program progress, impacts, volunteer time, cost-benefits, success stories, and delivery strategies for all planned programs. It also includes civil rights reporting as well as successes from any other program efforts not included in regularly planned programs, such as special educational efforts dealing with disasters.
Initial Process
In assessing the previous accomplishment reporting process, it became clear that
a system for reporting all required aspects of accountability functions needed to be
included in a single system. With this general concept in mind, state program
leadership appointed a special program Reporting and Accountability Task Force to
develop the criteria needed to address all future accountability needs of the
organization. This task force was charged with identification of all reporting needs;
developing goals and objectives for a new reporting system; identifying the
parameters of a new system; and ultimately developing a diagrammatic model that
could be studied, revised, and used as the design for the new system.
The Reporting and Accountability Task Force held meetings with Extension personnel, and conducted interviews with key County Extension Directors throughout the state. Those interviewed were then asked to canvas their associates by various means to secure as much input as possible as to what was and was not needed for local accountability, preferences of time-lines for required reporting, and special wishes, such as user friendliness of a new system. Throughout the entire process, the committee received direct input from agents in more than 60 of the 100 counties across the state. Altogether, more than one-half of all agents in the North Carolina Cooperative Extension Service (NCCES) had input into conceptualizing and designing the new reporting system. From the initial development of goals and objectives through the final roll-out of a World Wide Web based system, agents across the state were included in review and decision making on a continuous basis.
Development Stages
Building on the open dialogue that had been established throughout the state, the
task force developed a goal and a list of objectives for the new reporting system.
The goal was : To establish an effective and efficient reporting system that is user
friendly, easily accessible, and provides needed organizational accountability
requirements. With this goal as the guide, the objectives were developed which
included:
The next step was actual development of a diagrammatic model design. The design had to include all necessary components as well as those thought to be important for future reporting needs. Also, one objective was to design a system that was all-inclusive, i.e., only one reporting system rather than several different ones, all functioning slightly different. As the model was designed, all interested persons were given the opportunity to review and make suggestions. The State Major Plan Task Force spent hours during its monthly meetings intensely analyzing each component and recommending changes. Actual time-lines for required reporting probably created the greatest discussion, and ultimately, the Extension Administrative Council would have the final say in what was or was not included, and when reporting would occur. Similar long discussions were common among the administrative group as well. Often, seemingly tiny adjustments would precipitate lengthy discussion, analysis, and intensity. Ultimately, a final diagrammatic model was accepted as the blueprint for the system.
The next steps included development of input and output specifications to meet the needs of the Extension Technology Services programming group for their guidance in the design of the required computer programs. In tandem with this on-going process, parameters were being established by the State Major Plan chairs to identify specific program measures for each plan objective. These parameters indicating program progress would become the major component of the entire reporting system.
Components
With the key sections and components of each section identified, it was clear that
the major part of the system would be the measures of progress (MOPS) and
Impacts associated with each of the objectives within each of the twenty State Major
Plans (SMP). Since real program outcomes rather than inputs were now the focus,
the guidance and training requirements for actually developing realistic MOPS and
Impacts was a most daunting task. Altogether, seventy-six objectives were given
MOPS and Impacts in which reports would be completed. During this process, many
individual SMP task forces recognized the significance of their expected outcomes,
and many decided they had been perhaps too creative in developing a large number
of measures. Ultimately, due to the press of software development requirements,
final decisions had to be made with the MOPS and Impacts which would be used for
reporting 1996 accomplishments. An example of MOPs and Impacts is shown for
one objective in one of the twenty state major plans.
(NOTE: CELLS indicates the number of spaces allowed for entering numbers)
----------------------------------------------------------------------------------------------------------------
State Major Plan
AGING WITH GUSTO!
OBJECTIVE 1. Participants in aging issues programs will increase awareness, gain knowledge, change attitudes, develop skills, and adopt practices and behaviors to help make their later years more financially secure.
Measures of Progress:
*Increased awareness and knowledge of financial management techniques and
consumer issues.
NUMBER_______ (5 CELLS)
*Adoption of financial management and consumer practices.
NUMBER_______ (5 CELLS)
*Increased knowledge of estate planning.
NUMBER_______ (5 CELLS)
*Adoption of estate planning practices.
NUMBER_______ (5 CELLS)
*Increased awareness and knowledge of retirement planning and savings.
NUMBER_______ (5 CELLS)
*Adoption of retirement and savings practices.
NUMBER_______ (5 CELLS)
Impacts:
*Improved financial status through adoption of consumer and financial management
practices.
NUMBER ADOPTING__________(4 CELLS)
*Increased savings and/or increased retirement contributions for future financial
stability.
DOLLARS $ __________ (7 CELLS)
*Developed and implemented an estate plan.
NUMBER___________ (4 CELLS)
*Developed and implemented a plan for possible future incompetency and
dependency.
NUMBER ___________ (4 CELLS)
----------------------------------------------------------------------------------------------------------------
Other Components
Other components of the SMP reports include volunteers, volunteer hours and
calculated value, and cost-benefit analyses, plus a narrative description of the
program progress and results. While calculation of volunteer value is automatically
set at a rate of $10 per hour, the cost-benefit analyses may be most difficult to make
in some circumstances and relatively easy in others. Considerable discussion was
focused on whether to include a requirement for the cost-benefit information, and the
final decision was that this was a needed component, and therefore, was included.
A fact sheet was developed to assist agents in understanding cost-benefits. While
an assessment of this component may be difficult following initial use by agents, it
is clear that a new paradigm is emerging in which agents are making assessments
of the value of their time and its most worthy allocation for greatest impact.
Separate sections were included for success stories for planned SMP programs, and those of a non SMP focus. Often, valuable work is performed in a county of an unplanned nature, such as our experiences the past year with two major hurricanes, and the need to provide all types of disaster relief information and assistance on short notice. Successes in such program efforts can be reported in the new system. A training fact sheet has also been developed to guide agents in writing concise success stories of 150 words or less.
The civil rights section includes all components of previously needed information, and has only been adjusted to fit the new system, with time- lines being changed to twice a year rather than reports being required once a year.
The final section is one that allows those reporting to indicate their program delivery strategies. This section is optional, but will be accumulated at the state level so that use trends can be observed. N. C. State University is currently developing a university outreach reporting system, and reporting activities is expected to be a key part of that program. Therefore, while NCCES is more interested in MOPS and Impacts of programs, the types and numbers of program delivery activities remain important to a large number of people, and as a result, this optional component of the NCCES system may later become a required entity as well.
Computer System
Following the design and specifications phases of the system development,
emphasis was then placed on developing a computer system that could
accommodate all of the intricate components of the reporting system. Initial plans
rested with development of a text-based system linked to all county units.
Fortunately, an innovative idea for using a graphically-based user interface (GUI) in
conjunction with a client-server computing model began to emerge. This idea was
rendered possible by a major statewide investment in the NCCES information
technology infrastructure. These improvements provided for continuous Internet
connectivity to all of the NCCES county centers, and allowed for the development
of a modern software solution for the new reporting system. The skills required for
developing a graphics based system required development of new skills by members
of the Extension Technology Services group. This challenge was met, and a new
World Wide Web graphics based system was completed. All components of the
new system were developed, tested, and released within an eight month period
during 1996.
Testing of each computer program component involved initial release of an Alpha version of the program to six volunteer counties. Comments were received from the testing counties, and these were compiled into a punch list for the program development team to analyze and make adjustments in the system as indicated. Following this step, the second, Beta version was released to the same six counties for testing and review. Only then was the system released to all reporting units. This process is time consuming, but proved to be extremely valuable, in that the systems released were essentially bug-free.
Training for personnel in all 100 counties was conducted to introduce them to the new system, and to actually practice with Section A. Universal comments of user friendliness have been heard. Also, many agents are now finding the task of completing their reports so easy that they are entering their own information rather than giving it to secretaries for entering. Naturally, this is an evolving process, and some will continue to depend on others to make their entries. User friendliness of the system was further indicated when 97 of 102 reporting units met the accomplishment reporting deadline date of January 10, 1997, even though final release of all programs was on December 10, 1996.
Continuing Inputs and Uses
With the vast amount of accomplishment information provided, the State Major
Plan(SMP) teams accessed the system and developed statewide accomplishment
reports for their respective programs. Their deadline was February 10, 1997, which
they handily met. State Program Leaders then reviewed the accomplishment
information from the teams and developed program overviews. This information was
included with the SMP reports and developed into a printed program
accomplishment document to be used for both internal and external program results
communication. These accomplishment reports were also posted on the Web on
the NCCES Homepage that is accessible to the world.
Other uses of the system have included selecting specific success stories for individual congressional districts and developing a one page handout targeted to the 12 respective congressional districts in North Carolina. This information was hand delivered to the respective congressional offices in Washington, DC by representatives of professional Extension associations. Another example of a very quick use of the information contained in the Web ERS system was in response to a request from the Governor's office as to what Extension was accomplishing in the Welfare Reform movement. Through keyword searches of accomplishments and success stories, a multipage document was prepared for his office within hours of the request.
During 1997, field faculty and selected departments have continued to enter accomplishment data, as expected, with minimum intervals of reporting being every six months. Considerable accomplishment and success information was reported and a mid year summary was compiled by the SMP teams. More than 1,000 success stories were filed for 1996, and an even larger number have been entered in the system for 1997.
All of this information is being used continuously for accountability purposes, both internal to the university and externally. Altogether, as of November, 1997, twenty-four separate reports had been developed using the information contained in the ERS system. Fortunately or unfortunately, the North Carolina Legislature passed a bill in its 1997 session in which it is choosing to study Cooperative Extension. The ERS system is proving invaluable as a ready resource of information center for putting together specific information for that study.
Conclusion
The resources devoted to development of this new NCCES reporting system have
been enormous. Yet, in order to meet the organizational accountability needs that
are required and expected now and in the future, we believe the NCCES has
developed a system that will accomplish the objectives that were initially developed.
Thanks to rapidly emerging computer technologies, a system has been implemented
that could have only been a dream just a short time ago.
Based on experience and feedback from one year's experiences, NCCES have embarked on a new system that is providing the needed focus that should be placed on achieving actual program accomplishments. However, it is clear that the knowledge curve is quite broad as to what measures of progress really are, and what really entails program impacts. While obvious refinement will need to be made in the quantity and quality of indicated program results, the steps being taken so far have led far along the path of improved focus on programs and their accomplishments. Continuing training coupled with increased levels of knowledge and awareness of program impacts should provide a solid basis for helping the NCCES to meet all accountability needs both now and in the future. With the knowledge that their information is being used so extensively, field faculty and SMP teams are increasing their focus on obtaining and reporting "quality" program outcomes.
To Southern Association of Agricultural Scientists-Agricultural Communications Section Home Page