Document Type: 
Document Number (FOIA) /ESDN (CREST): 
Release Decision: 
Original Classification: 
Document Page Count: 
Document Creation Date: 
December 21, 2016
Document Release Date: 
June 16, 2008
Sequence Number: 
Case Number: 
Publication Date: 
September 8, 1983
Content Type: 
PDF icon CIA-RDP85B01152R000901240025-7.pdf461.84 KB
Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025 7 ROUTING AND RECORD SHEET SUBJECT: (Optional) Phase IV Long-Range Plan Action Items FROM: EXTENSION NO. ODP-83-1253 Chief, Management Staff, ODP DATE 8 S b t 1983 ep em er TO: (Officer designation, room number, and building) DATE OFFICER'S COMMENTS (Number each comment to show from whom RECEIVED FORWARDED INITIALS to whom. Draw a line across column after each comment.) 1. DDA Plans Officer 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. FORM 1 O USE PREVIOUS 1-79 EDITIONS Approved For Release 2008/06/23: CIA-RDP85BO1152R000901240025-7 uu/A Registry STAT SIAI DD/A Registry dJ-f//5`U /, ODP-83-1253 8 September 1983 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Chief, Management Staff, ODP SUBJECT: Phase IV Long-Range Plan Action Items REFERENCES: A. Memorandum for D/ODP, frm DDA, dtd 26 January 1983, Same Subject, (DD/A-83-0140/7) (ODP-83-155) B. Memorandum for D/ODP, frm DDS&T (via DDA), dtd 4 August 1983, Subj: Recommendations for Improving ADP Service to DDS&T, (DDS&T-640-83) (DD/A 83-1988) (ODP-83-1160) 1. Reference A tasks ODP "to conduct a general review and report on ... performance ... in response to customer requests." This Office has chosen to perform a survey of user satisfaction in response to the DDA's tasking. Due to time and resource limitations, an abbreviated survey was performed. We cannot, for certain, say that the results are fully representative of our user base. A more extensive formal survey would be required to ascertain user perceptions in a scientifically valid fashion. Based on preliminary results, we believe the approach is valuable and we plan such a large survey in the near future. In the meantime, we wish to share with you the user perceptions collected in our survey. Fortuitously, we are also in receipt of the results of a separate survey of DDS&T ADP Control Officers performed by the DDS&T staff. The second half of this memorandum discusses this DDS&T survey. Finally, in addition to the highlights of the ODP survey presented in the memorandum, the attachment provides a complete summary of the salient survey results. We think the results of both the ODP and DDS&T surveys are very interesting and address many of the concerns discussed in the DDA's memorandum. 2. Our general conclusion from the ODP survey is that user satisfaction is relatively high. We sent out 103 questionnaires and received 74 responses. The percent of survey respondents satisfied was in the low 90's for our key central services, VM and Batch. OCR services and GIMS scored Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 STAT Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 SUBJECT: Phase IV Long-Range Plan Action Items less well, with an 81 percent and 60 percent overall satisfaction level, respectively. Both, however, did relatively well with respect to two key components of overall satisfaction, response time and availability. OCR respondents were over 90 percent satisfied in both categories, and GIMS respondents' satisfaction level was in the high 80's. (The attachment provides more precise definitions of "satisfaction.") The follow-on survey will be used to learn more about the OCR and GIMS shortfall in overall satisfaction. 3. Satisfaction with major ODP software packages was also generally high. For example, overall satisfaction was in the mid-90's (percent of respondents) for AIM, RAMIS, and SCRIPT, which are, respectively, ODP's electronic mail, data base management, and mainframe text editing/document preparation system. Statistical packages, however, fared less well, with overall satisfaction in the 55-70 percent range. We did not obtain the detailed information in the preliminary survey that would explain these lower rankings, but can speculate that the inherent complexity of statistical packages is related to their relatively lower satisfaction level. 4. The user survey also addressed ODP support in the areas of consulting, documentation, trouble reporting, and training. Over 85 percent of respondents found ODP consulting support adequate or better. Similarly, 80 percent found ODP's main vehicle for user communications, Tech Notes, "Usually" or "Always" understandable. Overall, 90 percent of respondents were "Generally Satisfied" with information published or made available by ODP. 5. Almost 95 percent of survey respondents were "Generally Satisfied" with Trouble Desk performance. Finally, for Agency (not just ODP) ADP training, slightly over 75 percent of respondents were "Generally Satisfied." The latter figure is clearly not high enough and we need clarification on the sources of the user dissatisfaction. 6. In general, the Office is pleased with these preliminary results. On the whole, they indicate a generally positive user evaluation of our efforts, with only a few exceptions. We plan on acquiring more detailed information in future surveys to better understand the problem areas. 7. A second source of data providing a user evaluation of ODP services is Reference B. In Reference B, Mr. Hineman, the Deputy Director for Science and Technology, provided the results of a survey by his staff of DDS&T ADP Control ADMINISTRATIVE-INTERNAL USE ONLY Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 SUBJECT: Phase IV Long-Range Plan Action Items Officers. The DDS&T survey surfaced two major concerns: delay in applications development, and difficulty in "tracing requirements within ODP." (An additional concern was the DDS&T's desire to have ODP staff the ORD computer center, but that is a unique resource problem.) Overall, however, the DDS&T reported "frequent users of ODP services report excellent service." 8. We are, of course, aware of and concerned about delays in applications development for the DDS&T and other components. In fact, one of this Office's strategic objectives is to "Expedite Systems Development." We are looking for solutions to this problem, but, frankly, do not expect any quick or easy answers. New technology (e.g., the NOMAD2 database management system) will be helpful. Additional resources will also be required. With respect to the DDS&T's problem of requirements tracking, we plan to make improvements in this area by establishing a Requirements Management Organization in Applications at the start of FY 1984. 9. In summary, the preliminary feedback is encouraging. I foresee using surveying as a source of data for our continuing planning efforts. Our goal, of course, is to improve user satisfaction through reallocation of resources and management adjustments to mitigate any identified deficiencies or areas of weak performance. / Attachment: a/s Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 STAT Approved For Release 2008/06/23: CIA-RDP85BO1152R000901240025-7 Attachment Results of an ODP Survey on User Satisfaction Introduction 1. In late May of 1983, ODP initiated a preliminary survey of customer satisfaction. This inital effort was admittedly unsophisticated, and the sample was not scientifically chosen. The purpose of the effort was to get some feedback on the utility of the survey technique to judge customer views on office performance, and to get information useful in designing a follow-on survey. This planned follow-on will be a more sophisticated instrument, which will be sent to a larger and more representative sample of ODP users. 2. ODP sent out 103 questionnaires as part of its preliminary survey. Seventy-four responses were received. We found the results very interesting. We do not claim they are representative; they do, however, begin to give us a feel for areas which our performance is well-thought-of by the user community, and areas in which improvement is clearly needed. 3. This memorandum summarizes the most interesting results of our survey. Again, this is the only Agency-wide data we have, with all its limitations, on user perceptions. User Survey Results 4. As mentioned above, 103 surveys were distributed to a cross section of ODP systems users; 74 surveys were returned. The number of respondents answering a specific question varied. This variation is to be expected since, among other reasons, not all ODP customers use all available services. The respondents were from the following directorates/organizations: Directorate/Organization DCI DDI DDA DDS&T DDO ICS (& OTHER) No. of Respondents (%) 7 (10%) 37 (52%) 15 (21%) 7 (10%) 2 (3%) 3 (4%) 71* (100%) Approximate No. of Users ** * 71 out of 74 respondents identified their components. ** July 1983 data. STAT Approved For Release 2008/06/23: CIA-RDP85BO1152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 5. Survey respondents fell into the following self-identified job categories: Job Category No. of Respondents Professional 41 (58%) Technical 14 (20%) Clerical/Secretarial 12 (17%) Contractor 4 (6%) 71* (100%) * 71 out of 74 respondents identified their job category. Comparable data are unfortunately not available for ODP users as a group. 6. How do these ODP customers utilize ODP systems? The table below provides the number of respondents who use ODP systems, in the indicated manner, at least occasionally: Use No. of Respondents Data entry, correction or verification 64 (87%) Information query or retrieval 44 (60%) Computer programming 42 (57%) Production batch 35 (47%) Word processing (SCRIPT or ETECS) 30 (41%) Data computation or analysis 26 (35%) AIM electronic mail 21 (28%) Computer modeling or simulation 17 (23%) 7. The User Survey attempted to ascertain customer satisfaction with ODP's ADP and support services through the questionnaire approach. It is recognized that our sample was not ideal and that the results presented below are open to challenge. Notwithstanding this caveat, we believe the results provide a general indicator of how well this Office is living up to customer expectations. 8. The survey addressed the customer perceived quality of an ADP service, as well as general satisfaction. Most services were evaluated with respect to response time, availability, and overall customer satisfaction. Response time refers to the Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 customer's perception as to whether the service provides the required output in a timely fashion. Availability describes the customer's impression with respect to whether the system is generally functioning and able to provide useful output when required. The last factor, overall satisfaction, undoubtedly incorporates the two former factors, as well as concepts such as usefulness, ease-of-use and other undefined concrete and psychological criteria unique to individual users. From the survey, data were obtained for three major services, GIMS, OCR (Composite), and VM. These services are, respectively: the generalized database management system service; a group of Applications in support of the office of Central Reference, DDI (including AEGIS, COLTS, OLDE, OLDE3, OLTA and RECON); and the central timesharing service. 9. The following table summarizes survey results for the three services discussed above: Percent of "Satisfied" Survey Respondents* Avg. No. Respondents Answering Response Overall Question Time Availability Satisfaction GIMS 24 87% 88% 60% OCR (Composite) 35 91% 94% 81% VM 65 98% 98% 92% * A customer is defined as "Satisfied" vis-a-vis the various criteria when the following responses were provided: Response Time, either "Very Good" or "Adequate"; Availability, either "Always" or "Usually"; Overall Satisfaction, either "Like Very Much" or "Like." 10. For the Batch Service, the survey asked for customer evaluations in a slightly different format. Batch was evaluated with respect to turnaround time (i.e., response time), and the service and tools provided: No. of Respondents Answering Question Percent Satisfied Turnaround Time 47 92% Services/Tools Provided 44 95% ADMINISTRATIVE-INTERNAL USE ONLY Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 11. Due to limitations in the sample, usable data were not collected on CAMS, TADS, or DDO services. The SAFE service was also too new to collect user impressions at the time the survey was taken in May 1983. 12. In addition to services, survey data were also collected on several important software packages and tools provided to ODP customers. These tools primarily run on the VM timesharing service. Percent of "Satisfied" Survey Respondents* No. of Tool Brief Description Respondents Answering Question Response Time Overall Satisfaction AIM Electronic 20 81% Mail Database Management System Statistical 10 90% 70% Package SCRIPT Text Editing/ 40 100% 95% Document Preparation Software SPSS (SCSS) Statistical Package TELAGRAF Graphing/ 22 Charting System * A customer is defined as "Satisfied" vis-a-vis the two criteria indicated when the following responses were provided: Response Time, either "Very Good" or "Adequate"; Overall Satisfaction, either "Meets All My Needs" or "Meets Most of My Needs." Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 ADMINISTRATIVE-INTERNAL USE ONLY 13. Besides the ADP services themselves, ODP provides support to customers who utilize these services. This support runs the gamut from providing consulting and system administration services, documentation (vendor and in-house), and training, to staffing a Trouble Desk that acts as a central point for system status information and for reporting system or remote terminal malfunctions. The User Survey asked ODP customers their evaluation of office performance with respect to support activities. Specifically, the survey evaluated the performance of the Customer Services Staff, the ODP Processing component tasked with customer liaison and documentation support. No. of Respondents Answering Overall Customer Support Functions Questions Satisfaction* Consulting 42 86% VM System Administration 49 100% Batch System Administration 16 100% Administering Technical Library 45 and Providing Vendor Documentation * Percent respondents rating support "Very Satisfactory" or "Adequate." 14. ODP Tech Notes are published frequently to alert users to system changes, new capabilities, and new and better ways to use ODP systems and services. They are the main method of ODP communication with its customers. Respondents were asked if they found Tech Notes understandable, and if they were able to locate information in prior publications. Out of 58 respondents, 81% found Tech Notes, "Always" or "Usually" understandable. Sixty-three percent (63%) of 51 respondents indicate they "Always" or "Usually" could locate old items in Tech Notes. 15. Overall, 90% of 58 respondents are "Generally Satisfied" with information published or made available by ODP. 16. Finally, users were queried on the performance of the ODP Trouble Desk (staffed by Production Division, Processing), and their evaluation of Agency (not just ODP) computer training. Of the 66 respondents that evaluated the Trouble Desk, 94% were "Generally Satisfied." Agency ADP training was evaluated by 60 respondents. Of this group, 77% were, once again, "Generally Satisfied" with available training. ADMINISTRATIVE-INTERNAL USE ONLY Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7 Conclusion 17. ODP believes it would not be appropriate to draw specific conclusions from this preliminary survey. We are, however, quite encouraged by the results. Overall "Satisfaction" appears high among the ODP customers surveyed. Further in-depth surveying will be required to validate this thesis, and to identify specifically services that are not meeting customer needs and where they fall short. Approved For Release 2008/06/23: CIA-RDP85B01152R000901240025-7