EFFECTIVENESS INDICATORS

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP83T00573R000600030027-7
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 12
• Document Creation Date: December 9, 2016
• Document Release Date: April 23, 2001
• Sequence Number: 27
• Publication Date: October 29, 1976
• Content Type: MF

^ SECRET
a4pprovcd of ~?eIeas? _ ggM3 IA-F~AI3.83T.
_
OD5z3
Q s 30
ROUTP,13 AND RECORD SW~Er
SUBJECT: (Optional)
POSITIVE INDICATORS PROGR:TODP
FROM.
EXTENSION
NO.
DDA/M&AS
- - - -
7C18, HQS.
5226
DATE
23 MARCH 1977
TO: (Officer designation, room number, and
building)
DATE
OFFICER'S
COMMENTS (Number each comment to show from whom
RECEIVED
FORWARDED
INITIALS
to whom. Draw a tine across column after each comment.)
1. DIRECTOR OF DATA
PROCESSING
Mr. May,
2D00
2. HQS.
Attached is a copy
ell
_ -
-
2
-
G
of the task force report
3
on effectiveness indica-
tors that forms the basis
C. ,Gi
q.5
of the OP Positive Indica-
4,
tors Program. This is
forwarded for your infor-
6114
oration in response to
5.
-
-
suggestion by a cou le of
the Office DirK+_,T Lat
the recent DDA Management
6.
Conferenc
a.
Distribution:
1 - Each DDA Office Dir,
with attachment
9.
except D/OP.
1 - D/OP wo/att.
12.
13.
---- -------- --------
14.
15.
,.Q~, /
~,~ved For Release 2001
.
?
05/23
:
IA-RDP
3T005734
000600030027-7
^ UNCLA.SSIFIED ^ INTERNAL 1 1 CO F OENT1AL
- - - -
$
Y
tf USE PRviOUS
FORM 6
3-62 k~ EDITIONS
SECRET ^ CJMFIDENTIAL
INTERNAL
USE ONLY
^ UNCLASSiFsZD
 Approved For Release 20Q0 /05/23 : CIA-RDP83T00573R000600Qp27-7
-il lli ill:l. SR 1l1-.. ? ?.. ... ~? .. ?...j F.r L!
.2 9 OCT 1976
MF IORANDUM FOR: DD/Pers-PP,C
DD/Pers-SP
DD/Pers-R&P
FROM : F. W. M. Janney
SUBJECT Effectiveness Indicators
REFERENCE : Report of OP Task Force on Effectiveness Indicators
June 1976
1. As the new Fiscal Year is underway, this is to remind you that
the system of effectiveness indicators is now operative. During final.
discussions of the task force, it was decided to eliminate Indicator
II-C, Alcohol Problems, which is now an On1S responsibility. It was also
decided to modify Indicator IV-C to measure the number of employees
taking optional insurance rather than the amount outstanding per employee.
2. I shall expect to receive reports on the following schedule!
INDICATOR I -
A.0 Recruiting - Semiannually
B.
EEO Recruiting - Annual
II -
A.
B.
C.
Reassignment - Annual
Retirement Benefits Counseling - Annual
Eliminate
III -
A. Position Grade Control - Annual
B.- Review of Positions - Annual
A. Employee Perceptions - Following survey
B. Level of Effort Indicators - As appropriate
C. Market Penetration (Insurance) Annual
A. Errors (Pers. Record;Keeping) - Monthly
B. Processing Time of Transactions - Monthly
C. Timely Statistical Reports - Monthly
Vt - Internal Management - As PDP and APP data become
available
3. Please send information copies of these reports, when made, to
Chief, Plans Staffs OP,
STATINTL
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
 Uses Only
Approved For Releas001/05/23 : CIA-RDP83T00573R000 0Q030027-7
RE OF OP TASK FORCE ON
EFFECTIVENESS INETCAMRS
OPfPlans Staff
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
qr ' ( ?t
,~l }
 adfinisI f : - is w O Q Only
Approved For Releasie~2001/05/23 : CIA-RDP83T00573R000 0Q030027-7
INTRODUCTION
On'instruction from the DDA, the Office of Personnel created a
task force in the fall of 1975 to "devise systems and procedures to
develop and establish a'series of positive indicators against which
program missions "functions) can be judged."
The task force was comprised of representatives from each of the
six program areas in the Office of Personnel. 1/ As a first step they
were asked to assess their programs, identify tlTeir clients, and list
potential indicators that would affect their clients' views of how
well they were being served. As the activities are quite diverse,
this produced a rather lengthy list of potential indicators which would,
however, be of value to program managers. From the viewpoint.of the
Office, it was desirable to develop a much shorter list which yet repre-
sented the range of programs. So the next step was to develop the
shorter list which is presented in this report.
I. CONSIDERATIONS
The Office of Personnel shares with other elements of Agency
management'the responsibilities for staffing, career development, and
supporting employee morale. In the personnel field the ultimate test
of management is the effectiveness of Agency personnel in the execution
of Agency programs. The Office of Personnel alone cannot assure this
effectiveness but poor personnel programs could do much to preclude it.
Given the inevitable pressure of resource constraints and the consequent
desire of management to assign the higher priority to the operating
programs, it is imperative that a staff function, such as personnel,
evaluates the effectiveness of its programs so that the available resources
are well-used and so that the resources required for essential personnel
functions may be requested and defended.
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
 Approved For Releane,2001/05/23 : CIA-RDP83T00573R000 QJ 30027-7
With this need in mind, it was possible to specify some criteria
for developing indicators:
a.- On a selective basis, represent a broad range of Office
activities.
b. Represent a significant portion (30-60%) of Office resources.
c. Indicate not how much but how well in aspects that pertain to
objectives (where they'exist) or ciien~t satisfaction.
d. Identify indicators that are based on client feedback or that
themselves would influence feedback.
e. Identify indicators that pertain to significant resources
(don't bother with activities that require few-resources).
f. Differentiate between factors controllable by OP and those
controllable outside of OP.
g. Provide D/Pers some indicators that pertain to the interaction
between OP and other elements of management (system indicators).
h. Reject indicators that involve great additional expense in
assembling data unless management perceives they are worthy.
i. Subject the indicators to revision after use if required.
These criteria have been applied, although not rigidly, in developing
the indicators presented in this report.
A complicating factor for the Office of Personnel is its unusually
broad clientele -- employees, components, Directorates, and the Agency
as a whole. Some personnel services -- such as position classification --
may produce conflicting perceptions of value among the employee concerned,"
his. Directorate, and the Agency as a whole. It is important, and not
necessarily easy, to identify the relevant client.
II. SYSTEM INDICATORS
The personnel side of Agency management already has an unusual
two part tool that provides significant indicators of Agency performance
and effectiveness in such key areas as placement, separations, promotions,
training, EEO, and executive development. The elements of this tool are
the Annual Personnel Plan and the Personnel Development Plan., The
monitoring of performance against expectation provides significant manage-
ment insights. The allocation of responsibilities for performance is
usually apparent.
2
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
 Adminiu rat v? 1r}cr 1 U B M M
Approved For Releas001/05/23 :CIA-RDP83T00573R000 Of~D30027-7
Additional system indicators might be applied. Given the high
cost of recruiting and the decentralized nature of hiring, the Director
of Personnel may wish to monitor the ratio between the nurci er of
applicants in process (over the course of time) to the number being hired.
A growth in the ratio might indicate. matters requiring management
attention. A decline in the ratio would be highly desirable and the
cause should be identified and encouraged. Along with this indicator,
it would be useful to set a unit-cost type of indicator, such as the
average cost per EOD (determined by dividing total "Personnel Input"
costs* by the number of EODs). Such an indicator could be expressed
in constant dollars to permit comparison of changes over time so that
trends of increasing or decreasing cost might be identified. Since costs
of recruiting different types of personnel (professionals, clericals,
blacks . .) tend to vary significantly, some disaggregation of the cost
data might be desirable.
Much of the cost of maintaining the personnel record system is
influenced by data errors in originating offices. Hence, it is useful
to monitor the number of errors found in personnel transactions as a
ratio to the total number and to take corrective action with major offenders.
Over time reduction of the proportionate number of errors would be a
positive indicator of a successful policy of turnback and training.
III. EFFECTIVENESS INDICATORS FOR OP
Much of the work of OP is susceptible to workload measurement and
therefore to unit-cost measurement.. Such measures are basically
quantitative. Clieht perception of the effectiveness of OP is affected
as much by the quality of service provided as the quantity, so measures
of effectiveness must reflect qualitative considerations as well. A
failure to do so would create incentive to increase quantity at the
expense of quality.
In the following discussion indicators of effectiveness are identified
for each of the program areas. A deliberate effort has been made to focus-
on a small number of indicators for the Office overall. Our success in
developing indicators has been uneven, as indicated by the appended
comments. - -
Program Area I. Personnel Input
Indicator A. Recruiting
Semiannually, by general occupational title, calculate the ratio of
applicants placed into process against manpower requirements,as stated in
the.Advance Staffing Plan.
Me total costs attributed to the resource area for "Personnel Input."
Approved For Release 2001/05/23 : CIA-RDP3T00573R000600030027-7
n; tie'e ~n
Vkn C1111
 11 it s
Approved For Relea e2001/05/23: CIA-RDP83T00573R000J030027-7
Background: As recruiting against requirements is a major
responsibility of the Office of Personnel, the provision of an adequate
flow of qualified applicants is a key objective. The Advanced Staffing
Plan has some inadequacies as an operational statement of recruiting
needs, but application of this indicator would.generate improvements in
the statement of requirements. Past experience suggests that requirements
normally can be filled if the proper number of applicants are put in
process. On an annual basis, the current norm for professional/technical
openings is 2.8 applicants in process per opening and for clerical, two
for one. If the statement of requirements is adequate, application of
the above indicator would help to identify for management attention
problem areas in the recruiting effort.
Critic use: The indicator may be affected by lags between the
estabTis nent of a requirement and recruiting against it. Thus the
.indicator may show no applicants in process even though the appropriate
recruiting efforts have been started and gray stimulate overcompensation.
This problem is the basis for using a se iannual indicator rather than
quarterly reporting.
A second problem is that of defining the occupational groups, which
should be relatively comprehensive.
A third problem is that applicants may be listed under occupational
titles that differ from those of the jobs in -khich they are actually
placed.
A fourth problem is that a successful. effort to reduce the' ratio of
applicants in process to EODs (a system effectiveness indicator) might
cause this indicator to signal an inadequate number of applicants in
process.
Indicator B. EEO-Related Recruiting
On an annual basis, measure the number of Black and Hispanic-American
professional/technical applicants placed into process against manpower
requirements stated in the Annual Personnel Plan.
Background: This indicator would signal success or lack of success
in EEO-related recruiting. The current norm is 2.8 applicants in process
.to 1 required.
Critique: The indicator suffers fraffn several of the weaknesses of
the preceding indicator.
Program Area II. 'Personnel Operations
Indicator A. Reassignment
This is a compound indicator consisting of four annual measures:
4
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
Ad min i0rr. P111V
 y a~~dr.c. ?i~
al Usad 0111Y
Approved For ReleaQe X001/05/23: CIA-RDP83T00573R000030027-7
~.r
1. The number of employees who have contacted SPD.concerning
reassignment.
2. The number subsequently reassigned..
3. The number of terminated employees (voluntary or involuntary)
whose reassignment cases have been closed by SPD within one year.
4. Number of employees who have contacted SPD concerning
reassignment and whose cases are still active.
Background: The objective is to develop an indicator of the
effectiveness the internal market. Since many of the cases that
contact SPD may fall into the problem category it is to be expected that
there will be many failures in the form of employees who eventually
terminate. Successful reassignments alone would not be a satisfactory
indicator, for successful counseling may induce some employees to remain
on the job.
Critique: The indicator is not clean in the sense that OP has
control. Obviously successful reassignment would involve factors
external to OP.
The choice of a one-year cutoff for previous contact with SPD con-
cerning reassignment is arbitrary. Experience may indicate that many
terminations occur between 12 and 24 months after contact.
? As a compound indicator, it may be difficult to interpret for
management purposes.
Indicator B. Retirement Benefits Counseling
Relate (1) the number of valid complaints concerning misinformation
or delays in the provision of benefits information to (2) the number of
retirees.
Background: The objective is to provide-a measure of the effective-
ness of retirement counseling. As valid complaints provide an indicator
of reduced effectiveness, the objective is to reduce the number of
complaints. (As some complaints may be without basis, it is necessary to
sort out those complaints that can be substantiated from those that can
not.) I
Crib ue: The indicator will be very sensitive to the determination
of whe er a particular complaint is valid or not.
It is implicit that there is a close relationship between the number
of errors or delays and the number of complaints. One can imagine
circumstances when complaints would be more likely to be registered.
5
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
 ~k[~~lili~ti w i+r ! : ::1 Use MY
Approved For Release 2001/05/23 : CIA-RDP83T00573R0006 0 30027-7
As there is a close correlation between the number of retirees
and the number receiving counseling related to retirement, it is.valid
to use the number of retirees in this indicator.
Indicator C. Problem Counseling -- Alcohol Problems
On an annual basis, compare the number of employees receiving OP
counseling related to problems of alcohol with the number terminated who
have received such counseling.
Background: The objective is to provide an indicator of the
effectiveness of problem-oriented counseling. If an employee with a
problem who has received counseling is unable to hack it, it represents
a failure in several respects but not exclusively the responsibility of
the counseling program.
Criti ue: There is a time lag problem so that it would be possible
in a year in which problem counseling is very effective to have many
terminations which really result from previous years in which counseling
was not so effective.
Successful outcome in problem counseling-would relate to many
factors that are beyond OP control. The same is true of failures.
Program Area III. Manpower Management
Indicator A. Position Grade Control
On an annual basis compare the'Agency's actual change in average
position grade with the average grade projection contained in the Agency's
budget.
Backgrouid: The objective is to control the Agency's position
average graces that it remains within the projection contained in the
budget -- which presumably would reflect programmatic and substantive
requirements.
Criti ue: The actual change in average position grade would be
determined by many factors outside of OP control., yet if that is under-
stood the measure has value.
In the event that the realized change does not equate to the budget
projection, it would be useful to disaggregate the data enough to identify
the major causes and-the components contributing.
Indicator B. Review of Positions
On an annual basis, compare the number of positions reviewed for
proper grade classification against the total number of Agency positions.
6
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
Ae~rrtini~A.>~e??fA 4'-1-- M -int
 A~l~ ~l alw : ; i ' Ij. Use Unly
Approved For Releaae001/05/23 : CIA-RDP83T00573R0006,p30027-7
Background: A lengthening time between reviews of components and
of their positions significantly reduces the validity of position
descriptions and classifications. It is necessary to monitor progress
in reclassification to preclude this.
Criti ue: This is a quantitative measure, not a qualitative one,
so it would be possible to improve the performance in the indicator by
degrading quality of position review.
Program Area IV. Benefits and Services
Indicator A. Employee-Perceptions
Employee surveys should include sections to elicit feedback concerning
levels of benefits and services provided, priority interests for new or
expanded benefits and services and the adequacy of the administration of
such programs.
Back ound: Management interests are served when employees are aware
of benefits and services available to them, are assured that management
will listen to their concerns for benefits and services commensurate with
their contribution to the Agency, and are reasonably satisfied with the
way in which programs are administered.
Critique: The linkages between benefits and services. and motivation
are not clearly established once certain.inimum requirements for economic
security and the working environment have been met. Surveys should be
careful not to establish an expectation that there is open season for new
or expanded services and benefits.
The level of adequacy of the administration of benefits and services
will be a function of the resources provided, which will be relatively
limited. Hence, some negative feedback is to be expected. The important
aspect would be a trend toward a perception of worsening administration
(longer times to process, more errors in processing). which management
might seek to avoid.
Indicator B. Level of Effort Indicators
Strictly speaking, these are not effectiveness indicators but might
provide early warning of trends that would cause adverse employee
perceptions. Examples would be measures of the average time to process a
claim, the number of errors reported, the number of employees briefed as to
the availability of benefits and services, the number of publications
distributed and the size of their distribution.
Background: Such indicators already exist and provide n}anagement
with some idea of the adequacy of administration of benefits and services
given the available level of resources. The additional need is for an
indicator that might alert management if its judgment of adequacy is at
odds with employees' perceptions.
Approved For Release 2001/05/23 : CIA-RDPB3T00573R000600030027-7
 Use OI1y
Approved For Release 2001/05/23 : CIA-RDP83T00573R00060000030027-7
Indicator C. Market Penetration
Where employee enrollment for benefits is voluntary, it is possible
to measure the extent to which employees take advantage of the opportunity.
Thus, one could measure the average amount of UBLIC insurance outstanding .
per employee.
Background: Such an indicator would measure progress in informing
and educating employees in the benefits available.
C__riitiqu~ee: Unlike the salesman on co .ssion, the objective is not
to sego inform.' The final decision for such voluntary benefits
should always be the employee's. What would a high value of UBLIC
insurance per employee really indicate?
Program Area V. Personnel Record Keeping
Indicator A. Errors
By month and by year, measure the ratio of the number of errors in
personnel transactions received for processing to the total number of
transactions received.
Background: A high error rate on personnel transactions received
from components introduces unnecessary costs in OP operations. Keeping
statistics on the error rate makes it possible to concentrate training
where it is most needed and most effective.
Critique: Though some cost is'involved in keeping the necessary
records, is indicator already has proven of-value.
Indicator B. Processing Time of Transactions
Record in and out dates of transactions received for processing and
report on a quarterly basis the average number of days required.
Background: Processing of certain personnel transactions cannot
be completed until the appropriate personnel folder is available.
Delays often result because folders are not in file or required update
material is not in the folder. Such delays create distortions in those
-personnel statistics that are based on processing date rather than
effective date. The indicator provides management a tool to see if the
problem is worsening or being kept under control.
Critique: Control of this indicator lies outside OP with those
offices that have personnel folders out on loan. Improvement of the
indicator would rest upon developing procedures to speed the circulation
of such folders.
8
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7
Adrnin"iMrnR'" n .. "',nn flr!t
 fly
_;jail au 11
Approved For Release 2001/05/23: CIA-RDP83T00573R0006600.0030027-7
Indicator C. Tinly Statistical Deports
Deport on a monthly basis the average time between a request for
data from SPB _ and the providing of such data.
B acRUound; The effectiveness of a central personnel statistical
service is related to its ability to provide timely information. Otherwise,
customers may seek .heir own data banks or decisions may be made with
inaccurate or incomplete information.
Criti ue; The delays in providing information may be driven by
a nee _ r prograrrrrlin and batch- rocessin in OJCS. The indicator would
a y be very sensitive to the type of request received and would not
be stable from month to month as the mix changes. Even so, the indicator
should be of management value, and its improvement would help to forestall
decentralization of the personnel data bank.
Program Area VI. Internal Management t` ~~S I
The degree of ful.fillTmnt of personnel goals for OP within the APP
and PDP provides an effectiveness indicator for internal management. The
Office may wish to establish supplementary goals for career management
and counseling.
Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030027-7