Citation
Policy Analysis at OTA: A Staff Assessment, May 1993

Material Information

Title:
Policy Analysis at OTA: A Staff Assessment, May 1993
Creator:
Office of Technology Assessment
Publisher:
Office of Technology Assessment
Publication Date:
Language:
English
Physical Description:
132 pages.

Subjects

Subjects / Keywords:
Technology assessment ( LIV )
Genre:
federal government publication ( marcgt )
Spatial Coverage:
Washington, D.C.

Notes

General Note:
This document presents the findings of the Office of Technology Assessment (OTA) staff assessment of policy analysis that took place in 1992. It consisted of reviewing 18 sample OTA reports and soliciting opinions about OTA's policy analysis from outside observers. This report presents the findings of this assessment, along with options for OTA management and suggestions for OTA project directors.

Record Information

Source Institution:
University of North Texas
Holding Location:
University of North Texas
Rights Management:
This item is a work of the U.S. federal government and not subject to copyright pursuant to 17 U.S.C. §105.

Aggregation Information

IUF:
University of Florida
OTA:
Office of Technology Assessment

Downloads

This item is only available as the following downloads:


Full Text

PAGE 3

olicy analysis at OTA A STAFF ASSESSMENT -This is an internal report of an OTA working group to OTA management. The views expressed in this report are not necessarily those of the management, the study advisory panel, the Technology Assessment Board, the Technology Assessment Advisory Council, or individual members thereof-May 1993.

PAGE 5

FOREWORD This past year, the Office of Technology Assessment (OTA) celebrated its 20th anniversary_::providing an opportunity to reflect on how we11 the agency has been serving the U.S. Congress and on how it might improve. OTA was established in 1972 to provide Congress with "competent, unbiased information related to the physical, biological, economic, social, and political effects" of technology. In the fal] of 1992, former Director John Gibbons authorized a brief staff assessment of policy analysis in ful] OTA reports. The hope was that this internal exercise would yield insights that could improve OT A's ability to provide the Congress with information it needs to grapple with controversial policy issues involving science and technology. The five OTA staff who performed the assessment over the past several months were asked to take a critical look at the agency's past endeavors. In addition to reviewing a sample of 18 OTA reports, the project team solicited opinions about OTA's policy analysis from outside observers, including a bipartisan group of former and current House and Senate staffers who have used OTA's work (see study plan in app. A). The project team was assisted by a 14-member advisory panel composed primarily of senior OTA staff. This document presents the findings of the assessment, along with options for OT A management and suggestions for OTA project directors. While not confidential, the document is intended for internal use, and as such, was not reviewed or approved for release by the Technology Assessment Board. It is being distributed to OT A staff in the hopes that it will stimulate efforts to improve OT A's policy analysis by building on the successes and failures of the past. c/7&q(_O Roger C. Herdman Acting Director, OTA -------11---------

PAGE 6

------------titli-il;lMi-1~ -~--------Richard Rowberg, Chairman Division Chief, Science Policy Research Division Congressional Research Service Clyde Behney, Program Manager Health Program Peter Blair, Acting Assistant Director and Program Manager Energy and Materials Program Nancy Carson, Program Manager Science, Education, and Transportation Program Gerald Epstein, Senior Analyst International Security and Commerce Program Katherine GIiiman, Senior Associate Industry, Technology, and Employment Program Michael Gough, Program Manager Biological and Behavioral Sciences Program Emilia Govan, Senior Analyst Oceans and Environment Program Alison Hess, Senior Analyst Food and Renewable Resources Program James Jensen, Director of Congressional Affairs Congressional and Public Affairs Office William Keller, Senior Analyst Industry, Technology, and Employment Program Robyn Nishimi, Senior Associate Biological and Behavioral Sciences Program Linda Roberts, Senior Associate Science, Education, and Transportation Program Joan Winston, Senior Analyst Telecommunication and Computing Technologies Program ------lllf---------

PAGE 7

STAFF Project staff Robert Friedman, Project Director Kerry Kemp Anthony Fainberg Daryl Chubin JanLinsenmeyer Administrative staff Christopher Clary Kimberly Holmlund Publishing staff Mary Lou Higgs J. Christine Onrubia Chip Moore The project team appreciates the help and support of the advisory panel and of many of their OTA colleagues, including John Gibbons, former OTA Director; Roger Herdman, Acting Director; John Andelin, Assistant Direc tor; all nine Program Managers; several first-time project directors; and others who offered insights and assistance. The team also would like to extend special thanks to the cu"ent and former congressional staff, former OTA staff, and workshop participants who contributed significantly to the project. ------1111------

PAGE 9

Chapter1 Chapter2 Chapter3 CONTENTS FINDINGS AND OPTIONS, 1 Findings, 4 Needs of OT A's congressional clients, 5 Reader-friendliness of OTA reports, 5 Objectivity and recommendations in OT A analysis, 6 Timeliness of OT A reports, 8 Specific weaknesses or criticisms of OTA policy analysis, 9 Analysis of options, 9 Stakeholder analysis, 10 International context/comparisons, 10 Institutional analysis, 10 "Problem-driven" reports vs. "technology driven" reports, 10 Policy analysis methods and know-how throughout the agency, 10 Transfer of methods and know-how across programs and divisions, 11 Methods of analysis typically used by OTA, 11 Backgrounds ofOTA's analytical staff, 12 Options, 12 Options for OT A management, 13 Options to improve OTA's responsiveness to the needs expressed by congressional staff, 13 Options to address identified weaknesses in OT A policy analysis, 17 Options to improve the transfer of policy analysis methods and know-how across the agency, 19 Suggestions for OTA project directors, 23 ABOUT THIS STUDY, 27 Examination of a sample of OTA full assessments (Task #1), 28 Other views of OTA policy analysis (Task #2), 28 RESPONDING TO THE NEEDS OF CONGRESS, 33 Reader-friendliness, 35 Objectivity, 35 Timeliness, 37 "Context" or "options"?, 38 ------11------

PAGE 10

Chapter4 Chapter5 CONTENTS A PROFILE OF 18 OTA REPORTS, 39 Findings from the statistical analysis, 40 Scale of effort and disciplinary makeup of project teams, 40 Reader-friendliness of the 18 OT A reports, 40 Objectivity and recommendations in the 18 OT A reports, 42 Timeliness of the 18 OT A reports, 42 Policy analysis in the "context" vs. "options" part of the 18 OT A reports, 42 In search of differences by '"type" of OT A assessment, 44 Methods of assessment in the 18 OT A reports, 45 Implications of the analysis, 45 TELLING A GOOD STORY AND TELLING IT WELL, 47 Telling a good story, 47 The story told in Critical Connections, 48 The story told in Nuclear Power in an Age of Uncertainty, 51 The story told in Making Things Better, 53 Exception to the rule: Ownership of Human Tissues and Cells, 54 Characters and subplots, 56 Options as "empirically based policy prescriptions": Transportation of Hazordous Materials, 56 Analysis of options: Changing by Degrees and Mapping Our Genes, 57 Doing more with Jess: Exploring the Moon and Mars, 59 Stakeholder analysis: Power On!, 60 International context/comparisons: Enhancing the Quality of U.S. Grain for International Trade, 60 Institutional analysis: Indian Health Care, 61 Legal analysis: Finding a Balance, 62 Telling a story well: The Importance of reader-friendliness, 63 Table of contents: An outline of the story, 63 Summary chapter: A synopsis of the story 65 Main body of the report: The whole story 66 Index: The nitty gritty details of the story, 67 _____________ .__ __________

PAGE 11

Chapter6 Appendixes CONTENTS THE CULTURE OF OTA, 69 -Staff profile, 69 OTA alumni, 71 When outsiders look In, 72 First-time OTA project directors, 74 Implications for policy analysis, 75 A. OTA policy project: goals and study plan, 79 B. Form for examination of OTA reports, 89 C. Summary statistics for the 18 OTA reports, 93 D. Sourcebook information and related materials, 97 E. "Gems" of OTA policy analysis, 103 -------------111------------

PAGE 13

The Office of Technology Assessment(OTA)isasma11 analytical support agency of the U.S. Congress. Its pur pose is to provide thorough, objective information and analysis to help Members of Congress understand and plan for the shortand long term consequences of the applications of technology, broadly defined. The agency's authorization by the Technology As sessment Act of 1972 was something of an experiment. Never before had such an agency existed, in the United States or elsewhere. OT A's bipartisan governing board, the Technology Assessment Board (TAB), has 12 Members of Congress-six Senators and six Repre sentatives divided equa11y by party affiliation. By design, the in-house analytical staff of the agency is quite sma11, numbering only about 130 people at the end of 1992. To prepare reports for Congress, the agency relies not only on its multidiscip1inary analytic staff (about 70 percent of whom hold advanced degrees) but also on input from advisory panels, workshops, and outside contractors represent ing a broad range of interests and expertise. Each report undergoes CHAPTER ONE z. .~ Rilj~&gS .. ru:i~ ... tt several rounds of external review, involving a vast number of people with diverse backgrounds and per spectives. OT A was funded in 1973, so the agency has now been writing re ports and serving Congress in other ways for two decades. During this time, OT A has developed consid erable analytical and political cred ibility. In the past 20 years, con gressional committees have asked OT A to provide in-depth analyses of controversial, complex, and sometimes 1itt1e understood national policy issues involving science and technology. OTA has responded by issuing reports on topics that inc1ude-to name but a few--energy efficiency and conservation, global II climate change, national and international security needs, unconventional cancer treat ments, AIDS research, U.S. competitiveness in high technology industries, infra structure needs, the implications of revolutionary changes in telecommunica tions and computer technolo gies, the disposal of nuc1ear and chemical wastes, and the sustainabi1ity of natural resources. In many instances, OT A reports have helped frame congressional debate on a topic or provided options to help resolve the debate. In other instances, OT A reports have pro vided policy-relevant technical in formation that has helped illumi nate the debate. OT A reports are intended primarily for use by Con gress, but they often have a wider applicability and audience, includ ing the executive branch, state and local governments, industry, academia, and the public. Despite the agency's accom. p1ishments, this is no time for OT A staff to be complacent. Rather, the occasion of OTA's 20th anniver sary is an auspicious time to con sider how the agency might im prove the qua1ity of its work. The country has just entered a new

PAGE 14

phase-marked by events that include the election of a new President and a new Congress, profound changes in the global political and economic realm, and an everpresent need to grapple with diffi cult policy issues involving appli cations of science and technology. If we are to help our national lead ers meet the difficult challenges of the next decade and of the next century, we at OT A must continu ally strive to improve. In September of 1992, respond ing to widespread interest within OT A, then-Director John Gibbons 1 authorized a small internal assessment of OT A policy analysis-the OTA Policy Project, with a staff of five and an in-house advisory panel. This brief assessment was limited to an examination of policy analy sis in full OTA reports. 2The goal of the assessment was to produce a document that could help OT A staff-new staff and even "old hands"-improve the agency's policy analysis. This report is the culmination of that effort. The OT A policy project team decided at the outset of this study not to get bogged down in the effort to define policy analysis. Rather, we decided to use the same ap proach Percy Bridgman used in defining science as "the activity of scientists"-we simply defined policy analysis as the "activity of policy analysts." We started out by looking at OTA reports. The project team found that, despite their great diversity, OT A reports typically have two major components: analysis of the policy problem, including discussion of the policy context, findings, and issues; and identification and analysis of potential solutions, i.e., goals and options for congressional consideration (see box 1-A). Identifying these two components of OTA reports was the closest the team came to defining OT A policy analysis. Other key findings of the OT A policy project team regarding policy analysis in OT A reports are sum marized in this chapter. Also pre sented here are a set of options for OTA management (the director, assistant director, and program managers) and suggestions for project directors and analysts to consider as means of moving OT A toward a higher level of excellence. Chapter 2 of this report provides furtherdetailsonthepurpose,scope, and methods of this assessment (the study plan for this assessment is reproduced in app. A). In essence, the OTA policy project consisted of two major tasks. The core task was the review by the policy project team of a sample of 18 OT A reports (see box 2-A in ch. 2). The 18 reports, selected by program managers, were equally distributed among the nine OT A programs. A second task was to look beyond OT A reports to the following for insights: II 1. written evaluations of the policy analysis in a sample of 12 OT A reports by four former congres sional staff (using their own cri teria for good policy analysis), followed by a meeting with the project team that all four at tended (app. A-3 ); 2. telephone interviews with 13 cur rent congressional staff to learn their views regarding the strengths and weaknesses of OTA reports (app. A-4); 3. essays on the strengths and weak nesses of OTA policy analysis by five former OT A project di rectors (app. A-5); 4. a workshop with 10 outside experts in different fields, each of whom was familiar with some aspect of OT A's work (app. A-6); 5. interviews with a dozen recent first-time OT A project directors 1 OTA 's Director John Gibbons resigned in January 1993 to accept a position with the Ointon administration as Science Advisor to the President and Directorofthe White House Office of Science and Technology Policy. 2 Full OTA repons contain "major policy content" and are produced with the assis tance of an advisory panel. Although the focus of this assessment was on policy analy sis in full OT A repons, it is imponant to recognize that the publication of full repons is only one type of OT A activity. OT A also pubiishes background papers, briefs con gressional staff and answers their inquiries. provides testimony before congressional committees, etc. This assessment did not consider how to improve other OT A prod ucts or activities.

PAGE 15

Box 1-A, Components of policyanalysis in OTA reports (Identifying two components ~ically found in OTA re. < pons was the closestthe poucy project team came 1o .defining policy analysi$; OTA reports typically have two major components: 1) analysis of the policy problem, .. including discussion of the policy context/findings, and r issues; and 2) identification and analysis of .potential solutions, I.e.. goals and options for congressional roncontroversy or conflict), "problems to be solved, or '-"COngressional decisions" that must be made are among the approaches some reports use. for highlighting the most important concerns. Depending to a certain extent onthetopic,somereportshlghftghtverybroadphilosophical issues, and others focus on tar more narrow issues. fa po1iey~or~ue.or1~trie,pas~.Pfart .\ the goats are established by the congressional request, ~~~:=e.t:E!~ :g~g~t~~E'-&~i~ W~tlfc:iader~~ar~t(\?)r.. r ... }\~cc:>;) ...... >. . .. / /In additionto higtllighting 1fle~ aspecls.of. <'Ifs one might imagine,. options can be organized in . science and techn~. this part of .. the 1$poU may manyways. The challenge is to choose the approach that discussi~ ~f!'e people Whoai:e. niost seems to fit the assessment topic best. Organizing op. mvolved,.laws;mstitutions,..and economc and social > itionsby"problemtobefixed"or"goal"aretwoapproaches .. ~:f!l,e. pee>ple Jm,<*led include ~staJ<&. thatpflenwork. Organizing options by .technology fails to holders". (e.g.,. the AmtmCaO public and specffic gr(>ups / .. highlight~ specific matters of concern to OT A's con-~f flle":'~i:;: ::::~.t~t =:::=~~=:~~~=~~:~:::~~ entitles. ftelevant lawsandr~latic>ns arediscussedias >organizethemlnto strategles"or packages. Sometimes .. areithe lristitutions thatdmplement ~e lawsJitnd are {options can be organized by"values"orprior beliefs ( e.g., ........ affectedl>y~m. inV:()lveqlen~are r ... ,~ govemmentisbetter government"; "we do not give : .. ~l=~~=~~~fw,::;c ... :=~:~::::::::~~:,~ii:::;:: ~F~&a~='=~~a;E::: }ofonecongr~onalstaffer;areportpr0\fides"touch-. >near~termancklonger term decisions or by extent of stones" thatallovv staff to.go quickly from one l
PAGE 16

to learn what types of infonna tion they would have liked to have had available to help learn the craft of policy analysis; and 6. reviews of the literature on policy analysis and earlier examina tions of OT A (see ch. 2 and app. A for further details). Findings from interviews and evaluations of OT A reports by con gressional staff are summarized in chapter 3. Chapter 4 summarizes statistical data obtained by the policy project team's review of 18 reports. Chapter 5 offers narrative descriptions of especially good policy analysis in the sample of 18 OT A reports. Fina11y, chapter 6 turns to a discussion of the culture of OT A and the implications OTA' s culture has forthetransferof policy analysis methods and "know-how" (for lack of a better tenn) through out the agency. FINDINGS A central finding of the OT A policy project team is that policy analysis in OTA reports is often goodand frequently regarded as bet ter than that of other policy organizations-but there is consider able variation in the quality and methods of policy analysis from report to report, even within programs. Furthennore, there are some specific ways in which OT A re ports could be improved to better serve the needs of OT A's congres sional clients. Nearly all the individuals with whom the policy project team talked gave a very positive overall assess ment of OT A's work. The informa tion that follows should not obscure the near-universal agreement that OT A is either the best, or nearly the best, policy shop that deals with technological issues. OT A ranks extremely high in reliability, ob jectivity, and completeness. The high esteem in which the agency's work is held is reflected in the following rather effusive comments by outside observers who attended the project's policy workshop: OTA is, and has been for a de cade or more, the best policy analytic group in the U.S. Gov ernment. Its reports are the most balanced, even-handed, broadly based, and reliable. Over the past two decades, OTA has come to occupy a leadership role in a special and increas ingly important form of policy analysis. Countless state, na tional, and international policy organizations look to OTA for high-quality, unbiased, compre hensive analysis of difficult sci ence, technology, and policy is sues. This praise is echoed in a set of comments by minority and major ity congressional users of OT A re ports: OTA is the "most effective" [of a set of similarly tasked organi zations] ... OTA reports do the "best job of policy analysis." OTA is "by far the best" [of similarly tasked organizations]. OTA is "analytically more satis fying" than others. Apart from this praise, however, some criticisms emerged from dis cussions with current and fonner congressional staff, OTA alumni, and outside policy analysis experts (see ch. 3 and 4). Some of the criticisms were heard infrequently and perhaps should be taken with appropriate amounts of salt, but others were quite common. Con gressional staff noted problems with the readability of some reports and difficulty in finding conclusions and backup arguments. Many reports take too long to digest. Congres sional staff also mentioned the long time needed to produce full reports from OTA when pressures for leg islation were looming. And a few observers detected lapses of objec tivity in some of OTA 's work that concerned them. Some of the out side policy experts, perhaps view ing matters from an academic per spective, were of the view that OT A's institutional analysis, stake holder analysis, and inclusion of international aspects of issues were occasiona11y deficient. The OT A policy project team took an of these critiques into ac count and examined them in light of the 18 OT A reports and what team members know about OT A from personal experience. While disagreeing with some of the as sessments, the team found others to ---------------11--------------

PAGE 17

be justified. What follows is a dis cussion of problems that affect some OT A reports and suggestions for addressing those problems by means that include improving the transfer of policy analysis skills and know-how throughout the agency. Needs of OTA's congressional clients At OT A's request, four former congressional staff specified what they considered minimal and ideal criteria for good policy analysis in an OT A report (see box 3-A in ch. 3 ). The three criteria on which there was the greatest degree of unanim ity among the four were as follows: reader-friendliness, objectivity, and timeliness from a congressional perspective. 3 The consensus among current and former congressional staff was that OTA reports generally do well in terms of objectivitywith some lapses-and less well on accessibility/reader-friendli ness and timeliness. Reader-friendliness of OTA reports. Former and current congressional staff interviewed by the OTA policy project team underscored the im portance of reader-friendliness in OTA reports (see ch. 3). A number of staff said that if an OT A report is difficult to read, they may put it aside in favor of one of the many other documents that comes across q Copyright 1991 by Sidney Harris-"You Want Proof? 111 Give You Proof!", W.H. Freeman their desk instead. Or, if they use the OT A document, they may not be able to get the full benefit of what it has to say. Most of the current staff said they typically have time to read only the summary of an OT A report; others said they use the summary to point them in the direction of a specific chapter in which they are interested. Committee staff preparing legislation on a particular issue like to refer to the rest of an OTA document as needed to find more detailed information on topics or arguments found in the summary. For that reason, they said, it is extreme I y important to be able to track points raised in the II summary through the rest of the report. Many of the staff interviewed stressed the importance of an in dex. Former congressional staff re ported that one-third of the 12 OT A reports they were asked to evaluate had major problems in organiza tion and format that made the re ports extremely difficult to use ( see ch. 3). The OTA policy project's evaluation of a sample of 18 OT A 3 Other criteria included accuracy, key as sumptions identified, comprehensiveness/ thoroughness, apolitical, historical context given, strategies fro improving status quo well-analyzed, contains sound bites for mem bers (see box 3-A in ch. 3).

PAGE 18

reports confirms that some OTA reports are not reader-friendly for congressional staff pressed for time. The team judged just over half the reports (10) to be very good or excellent in terms ofreader-friend liness; they judged the other reports to be okay or worse (see ch. 4 and app. C). In some of the reports, the findings and options were so buried that it was almost impossible to find them. In many reports, it was difficult to find details on material presented in the summary. Some reports took hours to assimilate; others literally required days. Nearly half (8) of the 18 reports did not have an index. The OT A policy project team believes that broader use of tech niques such as those used in the most reader-friendly OTA reports (see ch. 5) would raise the average level of reader-friendliness in OTA reports, thus making the policy analysis more accessible to con gressional readers. A summary chapter with the major findings, issues, and options is essential. Clear labeling of the findings and policy options is essential. Parallel construction of at least part of the summary and the rest of the report can help make it easier to track points raised in the summary in the other chapters of the report. The most reader-friendly reports have a coherent chapter organization that outlines the story being told in the report. In many cases, the intelligi bility of a report can be enhanced through the use of techniques such as callouts or boldface type, boxes, or tables to summarize clearly im portant elements of the work, find ings, or options. Especially for long reports, but probably for all reports, an index is essential. Objectivity and recommendations in OTA analysis. A great majority of the former and current congres sional staff interviewed by the OT A policy project team noted that ob jectivity and independence are prime elements of OT A's credibil ity (see ch. 3). The importance of objectivity applies both to the analy sis of the policy problem and to the analysis of goals and options. Overall objectivity-In general, congressional staff and policy work shop participants credit OT A with a reputation for maintaining a high level of objectivity and balance, but they did find some lapses. The former congressional staff asked to evaluate a sample of 12 OT A re ports generally judged the level of objectivity in these reports to be quite high. In five of the 12 reports, however, these staffers identified major-or,moretypically,minorlapses of objectivity. Some reports had lapses in the "policy" context, findings, and issues" part of the report, some had them in the "goals and options" part, and some had them in both. The OT A policy project team found major or minor lapses of objectivity that it considered a problem in seven ( 40 percent) of the 18 reports it reviewed (see ch. 4 and app. C). What is meant by a "lapse of objectivity" in an OTA report? As one OTA program manager ex plains, there are ''several different possible meanings of objectivity in an OT A report, and ... the meaning has evolved over time and in differ ent programs and issue areas to the point where we now need to think through what we mean by the word. ... [O]bjectivity is important for OTA: it is our market niche." Absolute objectivity in an OT A report is virtually impossible,4 so the important question to be con sidered is: What constitutes an ac ceptable level of objectivity in an OT A report? Objectivity has sev eral aspects. Some types of lapses in objectivity are clearly unaccept able in an OTA report. It is gener ally understood, for example, that an OT A report should not be parti san in the sense of consistently ad vocating positions supported by Democrats or Republicans. The OT A policy project team found no criticism of OT A in this regard. A related but separate aspect of objectivity is ideological bias. One workshop participant criticized OT A for typically presenting op4As noted by one workshop panicipant, nor mative choices are made in every step of policy analysis," ... in making choices among analytical methods, among data. and more important, among alternative options or rec ommendations." -------------11-------------

PAGE 19

tions involving increased Federal intervention rather than market so lutions or greater delegation of re sponsibility to state and local gov ernments. The OTA policy project team did not examine the extent to which this criticism is justified; however, it did find that only a few of the 18 reports included options for reduced Federal intervention {see app. C). Given the agency's purpose and mandate, the nature of many issues it studies, and the fact that OTA' s primary audience is the U.S. Congress, a preponderance of options involving action by the Fed eral Government may not be unrea sonable. It is important for OTA staff to recognize, however, that the omission of non-Federal solu tions to problems as alternatives can lead to the perception of ideo logical bias. A third aspect of objectivity is in the selective use of evidence to point to a controversial conclusion or to a policy option. The OT A policy project team found one OTA report among the 18 it reviewed in which the arguments seemed so one-sided or so lacking in support that the report seemed open to the charge of outright advocacy {though, of course, it is not possible for the policy project team to as cribe intent or motive). In a few other reports, the objectivity ap peared questionable because the rationale for the report's controver sial basic premise was not explic itly stated {e.g., a report considered how to implement a policy that it apparently assumed was worthwhile but neglected to state any basis for that assumption). In some cases, there seemed to be "errors of omis sion," perhaps due to blind spots resulting from analysts' {and re viewers') values or prior beliefs. A few instances of concerns about objectivity stemmed from a report's making recommendations (see dis cussion below). Even though OT A is generally credited with high marks for objec tivity, the task of maintaining the agency's reputation requires eter nal vigilance. Congressional staff interviewed by the OT A policy project team noted that some other institutions that perform policy analysis have lost their credibility by repeatedly sacrificing their in dependence and objectivity. As one workshop participant noted, "It only takes a couple of 'bad apples' to cast doubt on OT A's reputation for reasonably bias-free analysis." To preserve OT A's reputation for ob jectivity, OT A staff need to become conscious of and explicit about the normative choices they make in doing policy analysis. Options and recommendations in OTA reports-A finding that was surprising to members of the policy project team was that seven {about 40 percent) of the 18 OTA reports the policy project team re viewed included what might be considered explicit or implicit rec ommendations or policy prescrip-II tions in the options (see ch 4 and app. C). From the policy project team's reading of reports, it is hard to discern a consistent pattern or ra tionale for when OT A reports do or do not make recommendations in options. Furthermore, there seems to be considerable confusion among OT A staff concerning the agency's policy about recommendations or policy prescriptions within options (see ch. 3). Contrary to in-house folklore, the Technology Assess ment Act does not prohibit OT A from making recommendations (see box 1-B). The policy project team was unable to locate any Technol ogy Assessment Board {TAB) or OTA director's policy explicitly prohibiting (or otherwise restrict ing) recommendations in OT A re ports. Several views on whether rec ommendations in options in OT A reports are appropriate were pro vided by congressional staff inter viewed for this assessment (see ch. 3). Most, but not all, said that OT A staff should recognize that they are treading on thin ice when making recommendations. On the other hand, many of the staff said that there will undoubtedly be a few cases in which available evidence is overwhelmingly in favor of a particular option to reach a speci fied goal; in these cases, they said, OTA might be obligated to present its "empirically based policy pre scriptions" accordingly.

PAGE 20

Box 1-B. Technology Assessment Act of 1972 An Act To establish an Office ofTechnology Assessment for the Congress as an aid in the identification and consideration of existing and probable impacts of technological applications ... Findings and Declaration of Purpose [l}t is necessary for the Congress to -(1) equip itself with new .and effective means for securing competent, unbiased information concerning the physical, biological, economic, social, and political effects of such applications; and (2) utilize this information, whenever appropriate, as one factor in the legislative assessment of matters pending before the Congress, par ticularly in those instances where the Federal Government may be called upon to consider support for, or management or regulation of, technological application. Establishment of the Office of Technology Assessment The basic function of the Office shall be to provide early indications of the probable beneficial and adverse impacts of the applications of technot-ogy and to develop .other coordinate information which may assist the Congress; In carrying out such function, 1he Office shall: (1) identify exi$ting or probable impacts .of technology or technological programs; (2) where possible, ascertain cause-and-effect relationships; (3} Identify alternative technological methods of implementing specific programs; (4) identify alternative programs for achieving requisite goals; (5) make estimates and comparisons of the impacts of alternative methods and programs; (6) present findings of completed analyses to the appropriate legislative authorities; and (7) identify areas where additional research or data collection is required to pr<>Vide adequate support for the assessment and estimates described in paragraph (1) 1hrough (5) of this subsection{~) The policy project team con cluded, after hearing from congres sional staff, former OT A staff, and others, that when implicitly or ex plicitly prescribing congressional action, OT A analysts must take special care in justifying their posi tion with very solid backup data and analysis, because it is precisely in such cases that OT A will find itself vulnerable to the charge of bias. Unless solid support for such policy prescriptions is provided, the policy project team believes that OTA runs the risk oflosing its hardwon-and vitally important-repu-tation for objectivity. The project team judged that "empirically based policy prescriptions" (sometimes labeled options) were present and well supported in three of the 18 reports in its sample (see ch. 5). Timeliness of OTA reports. Timeli ness is vital for the effective use by Congress of OTA's policy analy sis: From the congressional per spective, a report with solid analy sis that comes too late may not be useful.5 Congressional staff inter viewed for this project said that timeliness in the delivery of OT A reports was a problem they some times encountered in OT A's pro cess, if not its policy analysis (see ch. 3). An OT A report may be late for several reasons. Sometimes the agency commits to a schedule that it is unable to meet, either because the necessary staff are unavailable or because the time needed to com plete an assessment was underesti mated. In other cases, the responsi bility may lie with the requesting committee. The committee may not anticipate its needs early enough to allow OT A adequate time to com plete a full-scale assessment. The 'Sometimes, though, an OT A report issued too late to be of use by one Congress may be useful to a later Congress; there have been several instances in which legislation has been enacted five years or so after an OT A report and has borne many markings of the earlier OT A work. --------------------Bt.--------------------

PAGE 21

average 24-month time frame for an OT A report may be too long to satisfy some pressing legislative needs. It is under the latter circum stances that several staff expressed frustration that OT A could not un dertake a more modest assessment, complete with policy options. A few high-quality OT A reports have been produced quickly. Exploring the Moon and Mars and Improving Automobile Fuel Economy are examples from the 18-report sample. Both were di rected by long-time OT A project directors with at least some famil iarity with the topic. The existence of such reports supports the propo sition that OTA need not always invest 18 to 24 months to produce vital, high-value work.6 Specific weaknesses or criticisms of OTA policy analysis The analysis of options was identified as a weak point in some OT A reports by elements of all information sources used for this assessment, including the policy project team's review of 18 OTA reports, the one-day policy work shop, discussions with former and current congressional staff, and es says by former OTA project direc tors. There were more mixed sig nals on the adequacy of the fo]]ow ing aspects of OT A policy analysis: stakeholder analysis, international context/compari sons, institutional analysis, and "technology-driven" reports' policy analysis in comparison with "problem-driven" reports' policy analysis. Analysis of options. From its re view of I 8 OT A reports, the policy project team concluded that policy analysis in the "context, findings, and issues" (the policy problem) part of a report is typically better than that in the "goals and options" (potential solutions) part (see ch. 4 and app. C). Some of the reports did a fairly good job of analyzing the options. About eight of the 18 re ports in the sample, however, had little or no analysis of the options (e.g., estimates of effectiveness, cost, possible unintended conse quences, and a comparison among competing options) that were pre sented. Virtuany an of the project directors of the 18 reports said that they and their project staffs had devoted more time, effort, and re sources to analyzing "context, find ings, and issues" than to analyzing "goals and options." Opinion among current congres sional staff on which of the two major components of an OT A study-analysis of the policy prob lem or analysis of potential solu tions--shouldreceive more empha sis was closely divided (see ch. 3). Interviews with some staff sug gested they sometimes prefer that an OTA report emphasize analysis of the policy problem. Interviews with other staff suggested they considered the identification and analy sis of options the most critical com ponent of an OT A report. Some staff were of the opinion that there should be equal weight given to analysis of the policy problem and to analysis of potential solutions. On one point former and current congressional staff were virtually unanimous: In reports where it is important for OT A to present op tions, it is essential that the options be analyzed (see ch. 3). An infer ence may be that in OT A reports where the requesters really want OT A to present options, the analy sis of options should receive more attention than OT A project teams typically give it today. Interviews with OT A project di rectors of the 18 reports indicate the methods most commonly used for developing and analyzing options were "brainstorming by OT A staff' or "project director sitting and think ing" (see table 4-1 in ch. 4). The term "common sense" came up frequently. For the analysis of some While recognizing that OTA's role is to serve Congress, one policy workshop par ticipant strongly urged OT A's leadership to resist the tendency of Congress to ask for shon-tenn issue analysis, noting that "each assessment of an issue that has a relatively shon time line divens staff and consultant resources away from OT A's central organiz ing principal-to be the one place in the Federal Government that CAN sit and think about longer tenn issues, panicularly those where actions taken precipitously can lead to irreversible changes in society." This senti ment was strongly endorsed by one other panicipant. ---------------------11----------------------

PAGE 22

options, common sense may be suf ficient. In many cases, however, more rigorous analysis of the effec tiveness and effects of options is clearly warranted. Stakeholder analysis. To facilitate stakeholder analysis in OT A re ports, project teams rely on means such as the broad composition of advisory panels, the solicitation of a variety of perspectives through workshops, and an extensive exter nal review process for all full re ports. Some of the participants in OTA's one-day policy workshop, however, were of the view that OTA 's stakeholder analysis should be improved (see ch. 5). Congres sional staff interviewed for the project said they are typically very interested in what the effects of options on interested parties-including the general public-will be. Several congressional staffers interviewed for the policy project said they would like to see more stakeholder analysis in OT A work. The policy project team found that about one-third (5) of the 18 reports in the sample did a very good or excellent job of analyzing the positions of different stakehold ers in the analysis of"'context, find ings, and issues," but another third (6) of the 18 reports did a poor or fair job in this area (see ch. 4 and app. C). In the analysis of "goals and options," about half (8) of the reports included some discussion of the support for options by, and the effects of options on, the stake holders (including the American public). International context/comparisons. The project team had conflicting signals on the adequacy of interna tional analysis in OT A reports. A few of the policy workshop partici pants said OT A should try to do a better job of considering the inter national context of certain prob lems, especially problems (e.g., climate change) that cannot be ad dressed without international co operation. The policy project team found that inadequate attention to the in ternational dimensions of problems did not appear to be a major defi ciency in the 18 reports in its sample. Perhaps the workshop participants who cited this as a problem were reacting to reports that exhibited this deficiency more than the re ports in the sample. Institutional analysis. The project team also had mixed signals on the adequacy of institutional analysis in OT A reports. Most of the 18 reports in the sample did a good job on institutional analysis, which in cludes describing the roles of gov ernmental and other organizations-both as part of the problem and part of the solution. The team's fairly high ratings of the sample of 18 reports in this area conflicted with the perceptions of some par ticipants at the policy workshop. "Problem-driven" reports versus "technology-driven" reports. The OT A policy project team classified 13 of the reports in its 18-report sample as "problem-driven" (i.e., focused on how to fix a problem, usual1y specified in the congres sional request letter). The five re maining reports were classified as "technology-driven" (i.e., consid ering the implications of the devel opment, refinement, or use of a technology or class of technolo gies). The policy analysis received higher ratings in the problem-driven reports than in the technology driven reports (in both "context, findings, and issues" and in "goals and options") (see ch. 4). The sig nificance of this finding, especial1y given the small size of the sample, is not clear. Perhaps it is simply easier for analysts to get a handle on today's problems than to anticipate tomorrow's. Policy analysis methods and know-how throughout the agency The assumption by many OT A ana lytic staff and program managers that the way their program or they personal1y do something in an as sessment is the way it is done throughout the agency is typical1y not-correct. The culture of OT A is highly pluralistic (see ch. 6), and OTA reports reflect this. Differ ent OTA programs, and even dif ferent projects within a single pro---------------------111-------------------

PAGE 23

gram, have different styles and ways of approaching and conducting an assessment. The policy project team's re view of 18 OT A reports revealed striking variation in the methods and quality of policy analysis from report to report, even within OTA programs (see ch. 4). AU of the members of the policy team )earned agreatdeaJ from read ing and analyzing the sample of 18 OT A reports and talking to project directors from outside their pro grams and divisions. Team mem bers agreed that critically read ing OT A reports outside their normal purview and talking to the project directors of those re ports led them to new insights and ideas for policy analysis with ap plicability to their own future OTA work. There were .. policy analysis gems"-methods of analy sis,overaJJ conceptual frameworks, ways of making values explicit, ways of organizing and analyzing options, institutional analysis, stakeholder analysis, JegaJ analy sis, etc.-to be found in reports done by each of OTA's nine pro grams (see ch. 5 and app. E). Transferof methodsand know-how across programs and divisions. ThecoUective skiUsofOT A project staff and program managers in dif ferent parts of the agency represent a cornucopia of policy analysis skiUs and knowledge-but the culture of OT A is widely perceived as not facilitating the transfer of policy analysis methods and skilJs across programs and divisions (see ch. 6). Policy analysis appears to be seen as embedded in the .. technical" knowledge of subjects treated in program reports; thus, policy tools are viewed as irrelevant to others and not transferable across program, project, and problem boundaries. One former OT A project director commented: There is no incentive in OTA and strong disincentives for cross program: planning, project re view, report evaluation, collegi ality ... The organization is with out any internal program for cross-learning.As a result, there is no cross-over of policy gen eration, no cross-over of cre ativity ... The perception that OTA 's culture is not conducive to the transfer of policy knowledge was echoed in the following comment by one of the policy workshop participants: Internally, the agency manifests a fragmentation and lack of or ganizational learning that is a concern. This governs not only the culture of communication and learning about how to do policy analysis, but also proce dural issues, e.g., how could stakeholder analysis-who gains and who loses from the creation, application, or location of a tech nology-be improved? Some OT A staff who have felt a yearning for the opportunity to learn more about policy analysis from their coUeagues in other programs have turned to the creation of infor mal groups such as the Project Directors' Peer Group ( I 989) and the Research Assistants in Search of Empowerment (RAISE) ( 1991 ). Each met on an irregular basis to discuss issues of mutual interest. It is a maxim that any group will try to act to filJ needs not addressed by the formal organization. Infor mal efforts alone, though, may be neither efficient nor particularly effective when it comes to transfer ring policy analysis skiJls. Recently, informal efforts of a similar nature have been under taken by individuals from among another group at OT A-new project directors. Interviews with new project directors suggest that many of these individuals fee) somewhat adrift when they start a project and would like to have more guidance, especiaUy during the early phases of their studies (see ch. 6). Many new project directors expressed the desire for a mentoring program. Methods of analysis typically used by OTA. For developing the policy ..context, findings, and issues," the methods most often used in the 18report sample were the folJowing: workshops (used in half of the sample), literature reviews (half of the sample), ------mt------

PAGE 24

case studies ( about 40 percent of the sample), contractor reports (about30percent of the sample), legal analysis (about 20 percent of the sample), and quantitative analysis (about 20 percent of the sample) ( see table 4-1 in ch. 4). The overwhelmingly most often used methods for developing policy goals and options were "brainstorm ing by the staff' and the project director "sitting and thinking" (about half the sample used each). Other methods of developing and analyzing options that were used less frequently included quantita tive analysis, advisory panel com ments, scenarios to evaluate op tions, talking to lots of people, case studies, contracts, historical re views, literature reviews, and sur veys. Backgrounds of OTA's analytical staff. As of the end of 1992, OT A's analytical staff numbered 131. About 70 percent of the staff hold advanced degrees. 7 OT A's analyti cal staff come from a broad variety of academic disciplines (see ch. 6). In all, 45 percent ofOTA's analyti cal staff have, as their highest de gree earned, a natural science or engineering degree, 25 percent have a social science degree, and 8 percent a policy degree.8The other 22 percent have degrees in the hu manities (e.g., history, philosophy, English literature), business, and other miscellaneous subjects (e.g., communications, education, and social work). OTA's analytical staff typically work in teams of two to six people. About three-fourths of the 18 stud ies reviewed by the OTA policy project team had a mix of natural scientists and social scientists on the project team. Such diversity was viewed as a plus for good policy analysis by outside experts and former OT A project directors. Life experiences may be just as important as educational background in affecting policy analysis. Few OTA staff come to the agency with exposure to the Hill, so most of the staff have much to learn about writing reports that are responsive to congressional needs. The lack of exposure to the Hill was cited by some former OT A project directors as a flaw in the recruitment of staff to OT A; such experience, one former project director suggested, could help reduce the "academic" aspect of the OT A culture. New project directors who have typi cally gained most of their experi ence in policy analysis while at OT A nearly unanimously said they would like to have had more con tact with congressional staff and more guidance on how to deal with them when interviewed by the OT A policy project team. OTA's analytical staff includes few U.S. minorities (e.g., African Americans, Hispanics). The lack of racial and ethnic diversity on OT A's staff (which, among other things, may affect the composition of OT A advisory panels and selection of contractors) is viewed by some as a stumbling block to insightful policy analysis in certain areas. OPTIONS The "Findings" presented above suggest that, despite its many strengths, OT A policy analysis could be improved. To help move the agency's policy analysis toward a higher level of excellence, the policy project team offers a set of options for consideration and pos sible implementation by OT A man agement (see box 1-B). Each of the options is discussed and analyzed further below. Even without action by OT A management, there is some poten tial for improvement of OT A policy analysis at the grassroots level. If OT A project staff were more sensi tized to some of the agency's most 7 About 28 percent of the staff hold, as high est degree earned, a B.S. or B.A. degree; 25 percent hold an M.S. or M.A.; 37 percent have a Ph.D.; and 10 percent either an M.D. ( 3), a J.D. (9), or both (I). Natural science and engineering disciplines are most prevalent at all degree levels, ac counting for 55 percent of the Ph.D.s, 42 percent of the M.S.s, and 36 percent of the B.SJB.A.s. Social science expertise (includ ing psychology) is found among 20 percent of the Ph.D.s, 39 percent of the M.SJM.A.s, and 19 percent of the B.SJB.A.s. A "policy degree" (e.g., technology and policy) is held by 12 percent ofthe Ph.D.s and IS percent of the master's degrees. Staff with M.D.s and J.D.s are categorized by their next highest degree. -----------ml------------

PAGE 25

common deficiencies in policy analysis, they might make greater efforts to overcome them. With that notion in mind, the policy project team also offers here a set of"peer to-peersuggestions" to helpproject directors and other project staff improve their policy analysis. OTA' s program managers have an important role to play in improv ing OTA's policy analysis. Their styles permeate their programs and connect staff with a set of expecta tions and practices. Implicit in the discussion of options is the enor mous discretion that program man agers can exercise in advancing the cause of policy analysis as carried out in their respective programs. Furthermore, the suggestions pre sented at the end of this chapter can serve as guidance to program man agers about the types of weaknesses they should be looking for, and skills they may want to help strengthen, among their analytical staff. Options for OTA management One way for OTA management to improve the agency s policy analy sis would be to institute better qual ity assurance procedures by pro gram managers and assistant direc tors. OT A project staff look to OT A's director, assistant directors, and program managers for explicit clues about what is appropriate and needed. Even with improved quality as surance procedures, though, addi tional assistance to OTA 's analyti cal staff seems warranted. Several options that OT A management might consider to provide such as sistance are discussed below. The options fall, with some overlap, into three areas that correspond with the three general categories of "Findings" of the OTA policy analysis project: options to improve OTA's re sponsiveness to the needs ex pressed by congressional staff, options to address identified weaknesses in OT A policy analysis, and options to facilitate the transfer of policy analysis methods and know-how across the agency (see box 1-C). In keeping with OT A tradition, the options that grow out of this analysis are choices. Each choice is proposed as an experiment. What everexperiments are tried, of course, should be evaluated. Then the evalu ative information can be dissemi nated throughout the agency, so that new choices may follow. Options to Improve OT A's responsiveness to the needs expressed by congressional staff. Congres sional staff interviewed by the policy project team stressed the importance of reports that are reader-friendly, objective, and timely. These are, of course, criteria forusable and useful reports and thus stray well beyond the narrower topic of policy analysis, the subject that the project team was asked to address. However, given the im portance of each of these as ex pressed by our clients and the op portunities for improvement noted in the "Findings" section above, we begin with options to address each of these three areas. Option 1: Provide increased as sistance to.projects to improve reader-friendliness. If a report cannot be easily read by congressional staff, even the best policy analysis may go unno ticed and unused. The policy project team found-through its own read ing and interviews with congres sional staff-that there is consider able room for improvement in this area (see ch. 3). Four former con gressional staff who participated in the policy project identified major weaknesses in the reader-friendli ness of one-third of the 12 reports they read. Theprojectteam'sevalu ation was somewhat better; never theless, it judged 3 of the 18 reports in its sample as "fair" or "poor" in reader-friendliness, and another 5 were judged as just "okay." As discussed further below, OT A management could provide increased assistance to projects to improve reader-friendliness by 1) hiring a "managing editor" to read all OT A draft assessments and -----------~m~-----------

PAGE 26

. Box 1-C. Options for OTA management Options for OTA management-including the director, Options to address identified weaknesses assistanLdirectors. and program manag~e pre-'. ;..:.inOTA policy analysts. sented in three major cal$g<>ries that correspond to the categories in which the findings of the assessment were presented.The options are not mutually exclusive; one or a few could be chosen from each category. to good effect. Within each of the three categories, the options are presented in descending order of likely effectiveness, as judged by the project team. Options to Improve OTA's responsiveness to U. needs expressed by congressional staff. Option 5: Establish a bimonthly or quarterly tssues in Policy Analysis". lecture series on specific topics using people from outside OTA. Option 6: Contract for the development of-SOurcebooks" with key literature on specific topics in policy design and evaluation. Options to facilitate the transfer of policy analysts methods and know-how across the agency. Option 1: Provide increased assistance to projects to improve reader-friendliness. Optfon 1: Assign staff from outside the program in which tffrea"managingeditor;oramanagingeditorplusan ... an assessment is being done to help with that assess. ment through one or more of the following methods: .. .. additional senior writer/editor. to read all OTA draft .. n....~ assessments and advise project teams on how to __._... one or two senior staff people as "project kibitzers: or make the'.,eport more aci:essible to congressional Expand use of modified "Shirtsleeves policy ses->.'.~ =~~. ~.-.... = -shadow advisory panets composed of ---"" ,.,..,_, ......... .,. ..... :OTAstafffor.someassessments. freelance wi-it-.. editors. indexers who tiave worked forOTA,sothatsuchlnformation can be.used Option B: Give new project directors or all project direc. tofacllitatetheselectionof good freelancers by project. ... tors a few weeks to read reports from other programs. .. ... staff. :. : ... .:. . .... : ... .... Opti)n 9:.Establish a program to provide mentors for new Option 2: CJarlfy OT A's policy n,garding -recomme~ project directors tion$" and"pOlicy prescriptions" in options. < Option 10: Institute OTA staff-run seminars intended to Option 3: Appoint a standing panel of senior staff upon .. facilitate the transfer of policy analysis skills and knowlwhich the director can call when the objectivity of a report edge across the agency ;s caUed into question.. OTA staff-run "TJl)S for OTA Project Directors" semi > Option4; Encourage.experiments with shorter assess:. narsedes. .. .ments.and with policy-relevant Interim products and ser- Biweeklyormonthly seminars by OT A staff to present vices. the results and methods of OTA assessments. to advise the project teams on how to make the report more acces sible to congressional clients; and/ or 2) giving one individual the re sponsibility for collecting informa tion on, and advising project staff Option 11 :Reinstitute the OTA Congressional and Public Affairs Office'S lectures on how Congress works. about, freelance writers, editors, and indexers who have worked for OT A, so that such information can be used to facilitate the selection of good freelancers by project staff. A single managing editor, who also helps to coordinate the selec tion and use of freelance writer/ editors, is an approach used by sev eral other policy research organiza tions. -------------111---------------

PAGE 27

Hire a managing editor, or a managing editor plus an addi tional senior writer/editor, to read all OTA draft assessments. A managing editor, or a manag ing editor plus an additional senior writer/editor, could be hired to read all OT A draft assessments, begin ning with the earliest available fi nal draft. The managing editor would bean in-houseresource,con sistently available to project teams, to help the teams improve their reports. A managing editor (or editors) would read drafts of assessments for tone, organization, and clarity of presentation. He or she would serve in an advisory capacity to project teams and program manag ers, leaving final judgments with the project director or program manager. A managing editor could also develop and maintain a database or inventory of OTA writers, editors, and indexers and could advise project directors on the hir ing of writers and editors for spe cific tasks (e.g., composing a sum mary or copy editing). It is the policy project team's understanding that all three of OTA 's sister agencies have some form of centralized review, al) more rigid than envisioned in this option. Many other think tanks-e.g., Brookings, National Research Council, Institute of Medicine, World Resources Institute-have a managing editor or an equivalent position. Whether adding another level of centralized review at OT A is desirable is an open question. Because of concerns that such re view wiJl impose additional time burdens and be yet another bureau cratic hurdle, this option is likely to meet with at least some resistance from current OTA project direc tors, program managers, and other OT A staff. With the right skills and personality, and enough time to actually help rather than merely critique reports, a managing editor (or editors) might come to be re garded as a benefit rather than hin drance by OT A staff. Given the uncertainties about how this option would work in prac tice, the project team believes that OT A management might consider this option as an experiment to improve the reader-friendliness of OTA reports. It could hire a senior level writer/editor for a six-month to one-year trial period. The first month or so would probably be needed for the person to become familiar with OTA and the new position; ideally, then the person could assist several assessments from an early draft seen by the advisory panel through final publi cation. If the experiment is deemed a success and a managing editor is hired, each project could antici pate, on average, a week or two of the managing editor's time. This would be enough time to read most reports, allow hands-on involvement with a few, and provide assis tance lining up freelance editors for the rest. Adding a second senior writer/editor would allow the op portunity to offer significant assis tance to most full assessments in house. Again, the intent is to in volve the senior writer/editor as early as possible to help shape the final report. Each senior writer/edi tor is equal to about 1.5 percent of the staff resources at the level of senior analyst and senior associate. Give one individual the respon sibility for collecting informa tion on, and advising project staff about, freelance writers, editors, and indexers. Good writers, editors, and in dexers who might help improve the reader-friendliness of OT A reports are not always known to project staff or program managers.9Tohelp remedy this problem, OT A man agement could assign one person the task of keeping track of the performance of freelance writers, editors, and indexers so that other project directors could easily find out who might be available and their strengths and weaknesses. If a managing editor is hired, this could be one of his or her responsibilities. 9 OTA's experience with editors has been mixed. An April 1987 survey by the OTA Writing Task Force found that about half of the OTA projects that had hired an editor were happy with the editor's performance and half were unhappy. Interestingly, about half of the projects that did not hire an editor regretted not doing so. ------mi--------

PAGE 28

Otherwise, the responsibility might Jogically be given to the Publishing Office. The individual assigned this task cou]d compi1e a Jist of writers, edi tors, and indexers that OT A has actually used, with information that might inc1ude such things as l) a Jist of the projects on which they worked, 2) what they actuany did for each project, and 3) the project director's evaluation of what they did. The individua] might a]so peri odically so1icit new resumes (e.g., through an ad), go through o]d and new resumes and pun out the ones that Jook the most promising (e.g., those with experience most perti nentto OT A), get writing or editing samples, check references for the individua]s selected, and keep all the information on fi]e. Option 2: Clarify OTA' s policy regarding "recomnumdations" and "policy prescriptions" in options. Seven (about 40 percent) of the reports in the 18-report samp1e induded imp1icit or exp1icit recom mendations or directives (see ch. 4 and app. C). Some of the recom mendations or directives in the seven reports seemed empiricany based-i.e., supportable with data and ana]ysis in the report and not significantly dependent on the decisionmaker's values or other prior be1iefs. For lack of a better term, the project team decided to can these "empirically based po1icy prescriptions." The project team judged that "empiricany based po1icy prescriptions" were present and well supported in three of the 18 reports in its sample. '0 Some of the recommendations or directives in the seven reports did not appear to be empiricany based; instead, they seem to reflect preferences of OT A staff for reasons unstated. These the project team refers to as "recommendations." "Recommen dations" may pose some risk to OTA'shardwonandvital]yimpor tant reputation for objectivity (see ch. 3). There is considerable confusion among OT A staff concerning the agency's po1icy about the inc1usion of '"recommendations" or "empiri cany based po1icy prescriptions" as part of an OT A report's discussion of po1icy options. Some staff be Jieve an types of such directives are prohibited. Others appear to be Jieve that empiricany based pre scriptions, as defined above, are not on]y allowed but preferred. And individuals' judgments as to whether a particular directive is "empirically based" or a "prefer ence" seem to vary wide]y. The Technology Assessment Act is si1ent on this issue. Nor has the project team been able to locate either a Director's statement of po1icy or a TAB po1icy on OTA making recommendations orpo1icy prescriptions. The policy project team did not have the time to pursue this issue to the depth it would have 1iked, but believes consideration and clarifi cation of OTA's policy (or lack thereof) deserves further thought. Each of the congressional agencies has its own po1icy on recommenda tions, from the Congressional Re search Service's almost outright prohibition of recommendations to the Genera] Accounting Office's po1icy of anowing recommenda tions to be made routinely. Both approaches have advantages and disadvantages that are beyond the scope of this exercise to address. Option 3: Appoint a standing panel of senior staff upon which the Director can call when the objectivity of a report is called into quesnon. Congressional staff and others interviewed by the OT A po1icy project team stressed that it is es sential to preserve OT A's reputa tion for balance and objectivity ( see ch. 3). The OTA policy project team found ]apses or apparent lapses of objectivity-primarily minor ones-in about one-third of the samp]e of 18 reports. Three of the four former congressional staff who participated in the study also iden tified ]apses of objectivity in re ports that they read. 10 An example of such a report. Transporta tion of Hazardous Materials. is discussed in ch. 5. --------------1111-------------

PAGE 29

OTA's reputation for objectiv ity is one of the agency's indispens able assets-so vital that the Direc tor may want to appoint a standing panel of senior staff upon which he or she can can in those cases where concerns about objectivity have been raised. The panel would be used to carefully consider whether findings are wen substantiated, whether a directive is empiricany based, or other matters of objectiv ity in OT A reports. The panel would not routinely review all OT A re ports but instead would be used at the Director's discretion only in those cases where there was dis agreement within OTA. ,~:o,iio~ ~.: Enc~ge i~rfi r miilu :.;;,h shorter ~~ninelw. ia,.tlwii/,'i,o1iJ,,~ie~antim~~ p,,idui/Js.imd urvices. {r Congressional staff interviewed by the policy project team said that timeliness in the delivery of reports was a weakness in OTA's process. This option is intended to respond to the needs that congressional staff may sometimes have for informa tion that is provided to them in less time than the 18 to 24 months typi cally required to complete a fun OTA assessmenL It is certainly possible to pro duce good small assessments. Two ofthereportsinthe 18-reportsample reviewed by the OTA policy project team were completed with consid erably less staff time than the typi cal OT A report. About six personmonths of effort went into Explor ing the Moon and Mars, and about 15 person-months went into Improving Automobile Fuel Eco nomy. (Note that Fuel Economy actually took about 15 months be cause only one person worked on the assessment. The project direc tor estimates that the study could have been completed in less than a year if a second person had worked on it.) Moon and Mars focused on framing the key policy issues; Fuel Economy was a detailed examina tion of a controversial option. Both approaches seem quite reasonable for satisfying the needs for faster, more focused responses. Moon and Mars and Fuel Economy were prepared by project directors who: 1) had directed several assessments before, and 2) had some experience in related areas. Both factors con tributed to the project directors' abil ity to complete the assessments in less time. Several other experiments with shorter assessments have been tried over the last several years, but the OT A policy project team did not have the time to conect and evalu ate information on how wen they worked. The project team also did not consider what mix of shorter, faster assessments and more typi cal assessments might be desirable. For shorter, faster assessments, OTA might have to use smaller advisory panels and pay forreviewers' time to meet the accelerated schedule. Fun assessments are not the only products and services that OT A provides foroongressional commit tees. Other OT A products and ser vices include policy workshops, informal communications, and bill analyses. Such products were not often mentioned by the congres sional staff the project team inter viewed, possibly because we fo cused on full assessments and did not explicitly ask about these other products or perhaps because they did not consider them important. Further examination of the types of information needed by congres sional staff in a short time frame might shed light on what types of interim products and services might help satisfy the complaints the policy project team heard about timeliness of OTA assessments. It is important to note, however, that interim products do not fit well with current procedures for advi sory panel input and other forms of review. They also inevitably delay the final assessment and, if not care funy (and prudently) controlled, may involve substantial staff time. One assessment spent as much time on special responses (bill analyses) as on the full report. Options to address Identified weaknesses In OTA policy analysis. The policy project's review of 18 OT A reports and discussions with con gressional staff and others revealed some specific areas in which OT A policy analysis might be improved ------mf------

PAGE 30

(see "Findings" above). A weak ness that the policy project team believes merits special attention from OT A management is the analysis of the effects and effec tiveness of policy options presented in OT A reports. Congressional staff interviewed by the project team said that OT A might as well not provide options if options do not analyze their likely benefits and costs. Another weakness, at least in some OT A reports, was stakeholder analysis. Some OT A reports were very good in this area, but others neglected it entirely. Several con gressional staff mentioned that stakeholder analysis (which in cludes analysis of the public inter est) was often very helpful to them. The policy project team got mixed signals on the adequacy of atten tion in OT A reports to international dimensions and to analysis of insti tutions and laws. The two options presented be low might help address specific weaknesses of OT A policy analy sis. General options to improve the transfer of policy methods and know-how across the agency are described in the next section. Option$: Establish a bjmonthly or quarterly "Issues in Policy :Anmysis .. lecture series on spe-cific. topics using people from outside OTA. A bimonthly or quarterly lecture series focusing on topics such as analysis of policy options, stake holder analysis, international com parisons, and other topics appli cable to OT A policy analysis could open the agency to creative ideas developed by think tanks, academic researchers, and others inside and outside of government. Assuming the speakers are well chosen and the four to six one-hour lectures are well attended by OT A staff, this option might offer sig nificant returns. One person would have to devote a modest amount of staff time organizing the series, soliciting suggestions for speakers, and arranging the seminars. Travel and costs of small honoraria could be kept under $5,000 per year. If the speakers are not well chosen or the lectures are not well attended, the returns obviously would be minimal. Option 6: Contractfor the de velopment of "sourceboolcs" with key literature on specific topics inpolicydaignandevalu ation. The political science and public administration literature on policy design and evaluation is extensive, but the focus and orientation of the typical university researcher is of ten not quite right for OT A. Much of the literature is too theoretical, and that which is empirically based is often limited to one or a few case studies. OT A staff with little or no formal training in policy analysis often do not know where to begin. Properly focused source books with key literature on specific topics could be of great help if prepared by researchers who understand the more applied needs of OT A. The OT A policy project team searched the literature and prepared a sourcebook of articles on policy analysis that seemed particularly relevant to OT A's work (see app. D for the preface and table of con tents). The sourcebook includes information (with bibliographies) about I) definitions of policy analysis, 2) framing issues, 3) use ofOTA reports by and the impact on congressional and other users, 4) case studies and other study designs, and 5) specific topics in policy analysis, such as use of in centives and disincentives. Sourcebooks with information relevant to the types of options OT A often presents might help im prove analysis of the effects and effectiveness of these options. The sourcebooks might take the form of annotated bibliographies (with key literature reproduced) or more in terpretive surveys of the field. Ap pendix D includes material that points to the type of product that might be helpful. First is a contrac tor task statement prepared as part of this study (but never awarded owing to budget constraints) for a sourcebook to help us evaluate one of OTA's more frequent options, "More funds for R&D." The sourcebook would include empiri-------------m,-------------

PAGE 31

cal studies on how funding levels have been adjusted in the past, pri orities have been set, research agen cies have been organized, and simi lar questions. The second is the summary from a recent review pa per on "Reorganizing Public Orga nizations," prepared for the U.S. Department of Energy. It compares several structural alternatives for governmental organization (e.g., creating an independent agency, incorporating one into an executive branch department) and summa rizes the rather scanty empirical studies of past reorganizations. As a start, OT A management might want to try two contracting experiments on the order of $5,000 each to see whether in practice the idea has merit. Internal staff time to direct the effort would also be needed. At minimum, the results of these contracts would allow OT A analysts to make assertions, such as: "Remarkably few empirical studies have charted the intended and unintended effects of particu lar reorganizations. "11 Options to Improve the transfer of policy analysis methods and knowhow across the agency. Increased information transfer across the agency could help broaden the use of good policy analysis techniques in reports. Five options to encour age sharing of successful methods and know-how across the agency are presented below. Qption 7: Assign staff from out. side the program in which an ... assessment is being done to help with-tharassessment. :. To help share lessons and exper tise across the agency, manage ment could urge or require some type of formal cross-program par ticipation for most OT A assess ments. Such participation could take many forms-extensive involve ment of one or two people to more cursory involvement of up to eight to 10 people. The most appropriate type of involvement might vary by size and topic of the assessment and by the experience and personalities of the assessment staff. Participa tion is a two-way street: Both the project staff and staff outside the program are likely to learn and benefit from the interaction. Below are three variants of cross program participation; others are certainly possible. Again, the in tent of these options is to encourage collegial relationships across the agency, with a focus on improving policy analysis skills, not just shar ing substantive expertise. Designate one or two senior staff people as "project kibitzers." Each assessment could be as signed, or allowed to choose, one or two senior staff people who would follow the assessment from its inception, attending all major events, such as panel meetings and work shops, and read all assessment drafts. The kibitzers would be cho sen to offer advice on methods and fresh insights rather than for their knowledge of the topic of the as sessment. The kibitzers would be both offering advice to, and learn ing from, the assessment team. To be most effective for sharing les sons across the agency, the kibitz ers should be chosen from outside the program. The OTA policy project team estimates that this option might re quire 10 staff days per person per project. The costs of such a pro gram must be carefully considered, however. If kibitzers are restricted to project directors (current and former), and two are assigned to each assessment, each project di rectorwould be devoting about 4 to 5 percent of his or her time to the effort. If all senior research staffsenior associates plus senior analysts-were involved, each would devote about 3 percent of his or her time. All senior staff would have the opportunity to be kibitzers on one or two other assessments. The intuitive judgment of the policy project team is that such a program would be worthwhile but should be 11 See the summary of Craig Thomas' paper "Reorganizing Public Organizations: Alter natives, Objectives. and Evidence" in app. D. --------------------ml---------------------

PAGE 32

carefully monitored if implemented. Expand use of modified "shirt sleeves policy sessions" cur rently being tried in Division A. If the option above is deemed too intrusive or expensive, assess ments might try an approach simi lar to the "shirtsleeves policy ses sions" used in Division A to help improve their policy analysis and options. At the midpoint of an as sessment, a "shirtsleeves policy session" would be scheduled. The session would involve five or l 0 senior staff members, who would spend approximately one day each (including some background read ing) offering policy advice on the assessment. The meeting itself might last several hours. The key to the success of this approach is to make sure that the session occurs at the point in an assessment when the staff knows enough to be able to prepare a preliminary draft of is sues and options that seem worth pursuing, but not so far along that the direction cannot be modified. If five senior associates each spent one day participating on each of four other assessments, on aver age, each would be spending about 2 percent of his or her time. If the policy session involved 10 senior staff-senior associates plus senior analysts-each would spend on av erage about 1.5 percent ofhis or her time. The OT A policy project team's judgment is that the more time-intensive approach above would in most cases work better than this option. On the other hand, this option would offer the opportu nity for senior staff to be exposed to more projects. If all senior staff members participated about equally, each would be involved with threeorfourotherassessments per year. Most, but not necessarily all, of the participants would be chosen from outside the program. Establish "shadowadvisorypan els" composed of OTA staff for some assessments. Yet another variant for involv ing people from outside a program on each assessment is to establish a "shadow advisory panel" chosen from either senior staff or all project staff. Serving on a shadow panel might occupy five days per person, including time spent at panel meet ings and reading drafts. A shadow panel of 10 senior associates and senior analysts for each assessment would require about 7 or 8 percent of each staffer's time. A 10-person shadow panel drawn from all ana lysts, senior analysts, and senior associates would use about 5 percent of each staffer's time. This option might exceed the amount of timeonecanreasonablyexpectfrom senior staff involved with other as sessments. Of course, the amount of time could be reduced by limit ing the number of assessments that have shadow panels. Large shadow panels might be limited to that sub set of assessments that would most < benefit from substantive expertise from many programs around the agency. Even then, the project team has doubts about whether "shadow advisory panels" of this size are an efficient way to build sharing of policy skills into the OT A process. Option 8: Give new project di rectors or all project directors a few weeks to read reports from other programs. For learning how to design an assessment, there is no substitute for reading and analyzing OT A re ports and talking to the project di rectors of those reports to find out what methods they used and what lessons they learned. It is one of the methods the OT A policy project team used to prepare this report. Before beginning their first as sessment, new project directors could be given several weeks to read and dissect five or so reports outside their program. The reports should probably be chosen from a carefully selected list, for example, a list of reports nominated by the program managers and senior asso ciates. The reading and interview guides that the OT A policy project team constructed for this study would be a helpful adjunct ( see app. B). The key to success is for the project director to become con scious of the varied aspects of policy ------------fli-------------

PAGE 33

analysis that must be covereddescribing and analyzing the policy context, constructing and analyz ing options-so that they are incor porated from the beginning into the design of the new study. Allowing three weeks at the beginning of the study is equivalent to about 4 per cent of a typical study. If the read ing period is taken seriously. how ever, the improved feel for design ing a study might avoid an equal or greater amount of time's worth of potential mistakes. Rather than giving only first time project directors a few weeks to read OT A assessments, management could offer this opportunity to all OT A project directors. ThemembersoftheOTA policyprojectteam, though already quite experienced, all thought that they gained many insights from reading the 18 OTA reports selected for review in this assessment. It is remarkable how easy it is to identify mistakes that you made in your own reports by reading a report on a subject with which you are not familiar. 'fl::::,:~ri Many of the first-time project directors whom the OTA policy project team interviewed expressed great interest in having a mentor to help them with their first assess ment. Help on the substance of an assessment is often available from the program manager and senior staff within the program. The role of a mentor would be to provide additional guidance to the project director on the assessment process and methods of policy analysis. Mentors would have to be cho sen for both their teaching and policy skills. They probably would be cho sen from ranks of experienced ana lysts or project directors, although not all of these individuals are likely to be interested in serving as men tors. For the process to work, be coming a mentor must be volun tary. Mentors would probably spend on the order of a day a month or more helping the new project director-possibly as much as 10 percent of their time-so the commitment is substantial. The payoff for the new project director, however, is likely to be great. Probably not all first-time OTA project di rectors would need such intensive assistance, since many have been at the agency for several years and are already well prepared for the task. To help break down the barriers to transfer of policy analysis skills and knowledge throughout the agency. OTA management could initiate a series of staff-run semi nars. As discussed further below. the seminars might be of two types: 1. a series of one-or two-hour seminars in which senior OT A staff present "'Tips for OT A Project Directors"; and 2. periodic (e.g biweekly or monthly) shorter seminars to present the methods used by in dividual assessments, both suc cesses and failures. OTA staff-run "Tips for OTA Project Directors" seminar series. A "Tips for OT A Project Direc tors" seminar series could include panels of OT A presenters, indi viduals lecturing on specific topics, or some combination of the two. Topics that the seminar series might address include the following: 1. What OT A "Findings" Are All About and How To Find Them, 2. Options: Their Relationship to Findings, 3. FormsofDataCollection: Work shops Versus Contractor Reports and Other Forms of Data Collec tion, 4. The Organization and Presenta tion of an OT A Report, 5. Relations With Requesting Con gressional Committee Staff, 6. Planning an OT A Assessment, 7. Tips on How To Manage a Project Team. The seminar series could be de veloped primarily to serve OT A project directors and soon-to-be ------mf--------

PAGE 34

project directors or could be fash ioned with a broader OT A audience in mind. The series would have to be repeated every few years to ac commodate staff changes. One of the primary advantages of having a seminar series run by OT A staff is that the seminars would be directly relevant to OT A's work. Experienced OT A staff are more familiar than anyone else with the constraints, demands, and idiosyn crasies of the OT A process, and many project directors and analysts have developed insights or tech niques that could be useful to their peers. One disadvantage of this option, common to all seminars, is that it is relatively less "hands on" than the previous three options. In addition, organizing staff-run semi nars would be yet one more task for busy seniorstaff. Nevertheless, this option would require considerably less time than options 7 or 9. Biweekly or monthly seminars by OTA staff to present the re sults and methods of OTA as sessments. In addition to the seminar series just discussed, biweekly or monthly seminars by OTA staff to present and evaluate the results and methods used by individual assessments, might be instituted by OT A management. One former OT A staffer asserts that OT A provides for less cross-program review and "constructive feedback" than any other policy research organization of which he is aware. Presenting the findings of an assessment to OT A staff is rare. Presenting the-methods used to reach those conclusions or evaluate options-both the suc cesses and failures-is rarer still. Presentations on specific assess ments could occur a month or two after completion of a study. Semi nars could be quite brief: 30 min utes to briefly review the findings and highlight a few key methods with equal time for questions and discussion. Seminars could be held as lunchtime brownbags. The purpose of the presentations would be to share good ideas and to help others avoid mistakes. It is important to note, however, that some people may be reluctant to present failures; others may not even recognize failures. Unless OT A staff are open and honest, the series will not be worth the effort. Biweekly seminars, especially if other semi nars proposed in this report are adopted, may also be too frequent. Option II: Reinsdtutethe OTA Congressional 01lfl Public Alfairs Office's lectures on how Congress works Both current and former OT A staff cited lack ofHill experience as a hindrance to understanding con gressional processes and needs, with negative implications for the quality of OTA policy analysis. While it is difficult to determine exactly how the lack of Hill experience affects OT A's work, it is possible to edu cate staff through in-house lessons and lectures about how Congress works and how OT A can help. Though it may not be any substitute for the real thing, more information aboutthe Hill's operations will cer tainly help clarify for OT A's staff how their own work fits into the process and better equip staff to deliver reports that meet congres sional needs. OTA's Congressional and Pub lic Affairs Office has offered short courses on how Congress works. The courses, primarily aimed at new staff, instructed participants on the various functions of the Hill. Lectures included topics such as how a bill becomes law; roles played by various stakeholders, such as lobbyists and personal staff; and committee jurisdiction and activi ties. Guest speakers from the Hill provided an insider's view of con gressional activities by providing commentary on their own experi ence as Hill staff. Some mix of lectures and guest speakers in twoto three-hour "classes" overa period of four to six weeks was fairly effective and popu lar in the past. At minimum, lec tures given by Hill staff with a focus on their own use of OT A reports would be helpful in educating OT A staff. ------------ml--------------

PAGE 35

Suggestions for OTA project directors In this section, the OT A policy project team offers 1.0 "peer-to peer suggestions" to OT A project directors (and other project staff) to help them work toward the goal of producing high-quality policy analysis that is useful to, and usable by, OTA's congressional clients. These general suggestions are not intended to provide a template for OT A policy analysis-a template for all OT A studies would be overly constraining and rightly laughed out of the agency. Rather, the sug gestions are intended to help project directors avoid some of the most frequent pitfalls that the policy project team observed in its reading of OTA reports. The suggestions are offered not as firm rules but as collegial advice that may be helpful to many assessments. lbeycould all be boiled down to one sugges tion for project staff: Consider each of the weaknesses or criticisms of OT A reports with an eye to improv ing your own skills. Different project directors working on different topics will, of course, develop a variety of ap proaches to meet the minimal re quirements implied by these sug gestions. Program managers could encourage project directors to con sider these suggestions. S~ggestion 1: Devote~ttention. ... to policy ~lysis as ~ly as ik,ssible. \ .. Identify key policy issues, frame findings and options, and get initial thoughts in writing as early as pos sible. Distribute early versions of findings and options for review and discussion by all members of the project team. Early drafts can be revised as the project progresses, resulting in a basis for final drafts well in advance of the last panel meeting. Several project directors reported that late attention to policy issues and options often requires a crash effort in the last few weeks of a study. Avoid this mistake. Allow as much time as you can for analy sis of options, including costs, ef fects, and synergies of various pack ages of options. Meet with requesting congres sional committee staff to improve your understanding of the nature of the committee's request. An early meeting can be especially helpful in the identification of key policy issues. Invite congressional com mittee staff to workshops and panel meetings, and keep them informed of milestones and major changes in the assessment. Finally, consider with the program manager and as sistant director whether it would be beneficial or harmful to have fre quentmeetings ( e.g.,quarterly) with committee staff during the assessment. OT A program managers have expressed different views on the advisability of frequent meetings. Use early advisory panel meet ings to help identify key policy issues, and devote later meetings, in part, to help frame and analyze key findings and options. Work shops-probably the most common research method used at OT Acan also help. Suggestion 2: Give special at
PAGE 36

assessment. Consider including the congressional request letters in the report as an appendix. This added information will help the reader understand the ques tion you are answering. A section that outlines the over all organization of the report. Such a section will allow the reader to quickly understand what is included and where par ticular types of information can be found. At minimum, make your table of contents descrip tive. Suggestion 3: Do not bury your major findings and options by dispersing them randomly throughout the report in .the middle of paragraphs; Major findings and options should all be easy to find. Options can appear in the summary chapter, in a separate chapter clearly la beled "Policy Options," or in sev eral chapters, but the chapters should be clearly labeled as con taining options. If the options ap pear in more than one chapter, con sider collecting them in an easily referenced location (e.g., a box or table in the summary chapter) so that they may be read, assimilated, and used without demanding days of study by the reader. For an aca demic customer, ease of assimila tion may not be vital. For congres sional staff, it is. For help, consider asking some one from another program who is unfamiliar with the topic to read your draft. How long does it take them to find and understand the "bottom line" of the report? Seek out a good professional editor to help with the organization of the report early enough to be able to change it. Be very careful in the selection of an editor. Recognize that editors have different strengths (e.g., some can do substantial re writing and reorganizing; others can copy edit for punctuation, spelling, etc.). To avoid the perils of choos ing a bad editor, you might want to solicit recommendations from other OT A project directors who have used editors. Sometimes former OT A project directors, good ones, are good substantive and stylistic editors. Suggestion 4: Try to be self consciousaboul-am:l therefore more rigorous in analyzingthe normative choices you make in conducting.assessments. OTA's reputation for objectiv ity is central to the agency's useful ness to Congress. Be aware of the values and assumptions you bring when you select research methods and data, when you create narra tives about information, and when you choose your options. Be ex plicit about the basis for possibly controversial basic assumptions underlying your analysis. Suggestion 5: Keep your eye on the legislative schedule. Make sure that you know when the committee will be having hear ings or marking up legislation re lated to the topic of the assessment, so that you can be as responsive as possible to the committee's calendar. Suggestion 6: Devote increased effort to analyzing the effects and effectiveness of any options you propose. Congressional staff interviewed by the policy project team staff said that presenting options without analysis was not useful to them. Suggestion 7: Devote increased effort to stakeholder analysis. Congressional staff interviewed by the project team urged OT A to include discussions of stakehold ers' points of view and likely reac tion to options. Particularly con sider the public's interest, which is typically not addressed by lobby ists. Seek, through the literature or direct solicitation of authors' views, the greatest diversity of perspec tives you can. -------------m-------------

PAGE 37

Suggestio,iB: Considef the in-. <'ten,ati~i aspects of problems \as ail integral part of ihe analy sis of issues; where:applicable .. It is becoming increasingly dif ficult to treat many technology is sues as uniquely American prob lems. Moreover, different perspec tives may provide OTA with new insights in analyzing domestic is sues. For some topics (e.g., climate change), discussion of the interna tional context is clearly essential; for others, consideration of other nations' experiences and approaches may not be essential but may still be valuable for compara tive purposes. Sugg;,do; 9: Consfiier alt~natives to mo;e Federal intervention. OTA reports are replete with options for greater Federal inter vention. Rarely do the reports sug gest no action at all or less Federal intervention. In many of the studies that OTA undertakes, more Federal intervention may be the only fea sible or effective option for dealing with an identified problem. In other cases, however, there may be roles for state and local governments or for the private sector. Given OT A's role as an adviser to the U.S. Con gress, it may be appropriate to fo cus on options proposing greater Federal intervention. At a mini mum, however, note that there are roles for state and local govern ments or for market solutions and outline them (if you do not fully analyze them). Suggestion 10: Learn about policy analysis from your col leagues. Read reports from other pro grams to broaden your knowledge of successful or creative approaches to policy analysis. You might learn lessons that are applicable to your assessment. Consider the policy analysis experience of other OT A staff, both inside and outside of your program. Identify other staff whose work you find particularly thoughtful. Pick their brains for ideas and techniques for improving your own policy analysis. To those OTA staff who are approached by others: be collegial, your efforts will probably be recip rocated. ------------lfli-------------

PAGE 39

The Technology Assessment Act of 1972 (Public Law 92-484) created the Office of Technology As sessment (OTA)" ... to pro vide early indications of the probable beneficial and ad verse impacts of the appli cations of technology and to develop other coordinate in fonnation which may assist the Congress." Given little more guidance than that, the fledgling OT A embarked on an odyssey. confronting nu merous tasks and seemingly insur mountable obstacles. in the service of the legislative branch. Early OT A analysts spent many hours delving into the complexities of technological issues that they were charged to understand and skill fully analyze in order to inform their demanding client. Over the years, an OTA process evolved to equip the analysts with some "'tools" of the policy analysis trade; and overall, the agency has acquired an excellentreputation for competence and reliability in its work. The gen eral belief within the agency is that OTA does better policy analysis than it once did but that it can still improve. CHAPTER TWO Responding to widespread in terest within the agency, OTA's Director John Gibbons asked a small group of OTA staff to conduct an assessment of policy analysis in full OT A reports. Full OT A reports contain "major policy content" and are produced with the assistance of an advisory panel.1 The hope was that the assessment would provide information about OT A policy analysis that would enable OTA's staff to improve their work. The project began in September 1992 and was completed in the early part of 1993. It involved five staff members; in addition. $20,000 was al lotted for contracts and workshops. In essence. the goal was to discover how (and how well) OT A policy analysts transform the relevant facts regarding sci ence, engineering, econom ics, political science, law, etc., into a written report that provides useful information to congressional committees about the prob lems the country faces and potential solutions. Like other OT A assess ments, this project enlisted the help of an advisory panel, in this case composed of senior OT A staff and chaired by a formerOT A program manager, now a division director at the Congres sional Research Service. The advi sory panel members met twice over the course of the study, first to give advice on the project study plan, while the second meeting focused 1 Although the focus of this assessment was on policy analysis in full OTA reports, it is important to recognize that the publication of full reports is only one type of OT A activity. OTA also publishes background papers, briefs congressional staff and an swers their inquiries, provides testimony before congressional committees, etc. This assessment did not consider how to improve other OTA products or activities. ------f11--------

PAGE 40

Table 2-1. summary of information sources for the OTA policy project .Source Taskl1: OTA reports San1'>le of 18 OTA assessmen1s Project directors of the 180TAassessments Task 12: Other views Methods of collection Content analysis Interviews Former congressional staff Contract papers and interviews < .... Purpose Characterize and Judge various elementsof OTA policy analysis. Inform the reviewers about the policy relevant material in the report and methods used in the assessment process. ldentifY "kJear and "minimal" criteria for useful policy analysis; evaluate three OTArepo,ts on the basis of the selected criteria. ~~e,,slonalstaff Interviews Commenton.OTApolicy analysis, especially ~OTA~ .. .)~~ : < t ........INIP"fS .. :m..:::.._. ... polcy.....,. xO/~->.====~ . *st-ti~ ()~A: ; .. .. projectdirectors . . .-. ,. Poficy~literalure ... and early.examination of.OTA. assessmentprocess. Comment on their own experience with the OTAassessment process. tdentifyrelevant articles and reports from an existing body of literature on policy analysis and on OTA. and compile in a notebook for .::_,_:_.:_:.-:" :.:-= .... :~;~: ::-....... :::: .:. _:. .. OTA staff use ....... ~anddocumentrtwiew Help shape project work plan and report writing. on an initial draft of this report. Some advisory panel members also participated in the one-day work shop with a group of outside policy analysis experts. To appraise OTA policy analy sis in full reports, the project team developed a study plan with two major tasks. One was the examina tion and evaluation of a sample of OTA full assessments (Task #1). The second task was the solicita tionof views on OT A's policy analy sis from a broad range of individu als familiar with OTA's work, in--------------,111-------------

PAGE 41

eluding current and former con gressional staff, former OT A staff, and a special group of outside ob servers (Task #2). These two tasks are described in the next two sec tions of this chapter. Information sources for the two tasks are briefly summarized in table 2-1. The full study plan for the project is repro duced in appendix A. Examination of a sample of OTA full assessments (Task #1) Eighteen OT A reports identified by the OTA program managers as ex amples of "good policy analysis" were read and evaluated by the OT A policy project team (see box 2-A). In the selection of reports, program managers were asked to try to select reports completed since 1985, so that the project team would be able to interview the project director of each report. The project team decided not to consider the impact of a report ( e.g., legislation, testimony, media coverage follow ing a report's release) as a measure of the quality of its policy analysis, preferring instead to consider each assessment as a "stand-alone" docu ment. The process of evaluating the sample of 18 reports was extensive. At the beginning of the project, a questionnaire was developed as a guide to reading and characterizing the various elements of interest in the reports. The questionnaire, re produced in appendix B, included questions covering two major com ponents of policy analysis found in most OT A reports: 1) policy con text, findings, and issues; and 2) goals and options (see box r-A in ch. I). Each of the 18 reports was as signed to a pair of reviewers on the policy project team, a primary re viewer and a secondary reader. Using the first part of the question naire found in appendix B, the pri mary reviewer interviewed the project director to gain a quick ori entation to the material and help identify the policy-relevant con text and noteworthy features of the analysis. Following the interview, the report was read by both the primary and secondary reader aided by the second part of the project team's questionnaire. After read ing a report, the primary and sec ondary readers met to discuss their evaluations and, where possible, to reach some agreement on the as pects of the policy analysis identi fied during the evaluation. After the Jong questionnaire for each report was completed, the team wrote a one-page summary of find ings for each report. In addition, the team devised a shorter ques tionnaire to rate each report on some specific "dimensions" or "elements" identified in the first round by the team (see app. C for the form and frequency distributions for each rated dimension). This second round of scoring served as the basis for a statistical analysis (see ch. 4). The longer form helped in the elucidation of the variety of analytic ap proaches found in OTA reports (see ch. 5). The IS-report sample (equiva. lent to three-fourths of the full re ports released in a typical year) was large enough to allow the project team to recognize general tenden cies in policy analysis and report writing across the agency and iden tify a few patterns or themes that identify "problems" with OTA re ports. Many of these were ampli fied by other kinds of data collected through interviews and retrospec tive accounts by participants in the OT A assessment process. The sample was not large enough to support conclusions at the program or division level. Other views of OTA policy analysis (Task #2) The second major task for the policy project team was to gain a system atic sense of the perceptions and expectations of OT A's work from a broad range of individuals. This task had six major components. First, four former congressional staffers-who were from both Houses and parties of Congress and had handled a wide variety of issues while on the Hi11-were given con tracts to provide written evalua tions of the policy analysis in a sample of 12 OTA reports (using their own criteria for good policy analysis) (see app. A-3). In the first paper, the former staff were asked to identify criteria they would use --~---mt--------

PAGE 42

.!Box 2A. The sample of 18 OTA repor.ts.xamined by the project team .. lhefollowing 180TAreporhr-$elected byttle agency's program managers. using their own criteria, as examples of "goodpolicy,analysls"--=wereTead and evaluated byihe OTApolicyproject team.~&me program managers articulatecl the coteria on which they based their choices (e~g "sorted complex issues," *defined a number of .altemativescenarios, "Written in remar1cably .good English."~ one.criterion which the policy project team had urged not be used, ~eatest impact"); other program manegers. did not specify their criteria. Reports From Division A (Energy, Materlals,andlntemational Security): Energy and Matedals Program Nuclear Power in an Age of Uncertalrity,February 1984 Improving Automobile Fuel. Economy, October 1991 lntemational Secutity .,;.,.Commerce Ptogram Exploring the MoonandMars~ July 1991 Holding thffEdge: Maintaining the DefenseJecnnology April 1989 ............. .... .Jl.-,ortsf.--DJvlsloriB~~d.Ufe.Sciences) .. ~$8::,~1i:~ .. ,-. ; .:~-~-. ~-:~~$~:.._~,-< . ... HeaithP,~~> . <~ Health ServJces for Meclcare:Beneficiaries, February 1990 .. Indian Health C8reiAprU 1986 ~epo,IS From DivlsionC (Scien~~~and Natural Resources): < ~=~=~~~1991 ;,fs,;~=~~1988 :,~,;~~=~~ . 'NC>re~;~--~~~-.sioMlhmlle1111N~~.10rata1a1d27-..~A-2ft#aillOl111e~nal27reports).Time :cananlnlS1iwced111eplajlc:t-lD-111eG!Wnalamnpled%7nipons11>:18fllPOl18 ..... I.WOl'lll)ClllsparpnlglUI}. ThereducliOnOflheamnpleftOm .. 21111 .18111pG111 -doll&lllllllllMlltatbinllly, tlUl t.eNpC!itew'edrappedmbHf1M ablaMIIJclfttbattlley_.dilactedby a project diteCtor wtt~ atllllher~lbetam-*lbeavalualing. (Twoot the tine reports 11ominlllllCU1filve~managars-dNclad by Hingle projeCt dilector.J _________________ ,__ _______________

PAGE 43

to judge good" policy analysis. In the second paper. they were asked to apply these criteria to the evalu ation of three OT A reports, nearly all from the 18-report sample. 2 Af ter they submitted their papers, the four former Hill staff came to OTA for aroundtable discussion to elabo rate on their impressions of what was most and least valuable in the OT A reports they reviewed. In a second, related effort, I 3 current congressional staff were in dividually interviewed by telephone by members of the project team (see app. A-4). Each staff member was asked a series of questions designed to ascertain his or her familiarity with OT A reports and views regarding the strengths and weaknesses of OTA' spolicy analy sis. Third, five former OT A staff were given contracts to deliver two essays on the strengths and weak nesses of OTA policy analysis to theprojectteam(seeapp.A-5). The first paper identified the distinctive features of OT A policy analysis based upon a questionnaire pro vided by the project team. The sec ond, longer paper offered a per sonal retrospective of the OT A as sessment process and how it shapes both the content and presentation of policy analysis in reports. Fourth, the project team invited a diverse group of 10 outside ex perts who are familiar with one or several aspects of OT A's work to a one-day policy workshop to tap them for their perspectives on the strengths and weaknesses of OTA 's policy analysis (see app. A-6). These outside experts represented a wide range of science and policy backgrounds. Some of them had served on past OTA advisory pan els. Attheone-dayworkshop, which was attended by the project team and other OTA staff members, the 10 outside experts discussed vari ous aspects of OT A policy analysis and made a number of suggestions about how OT A might improve its work. After the workshop, partici pants sent a memorandum to the policy project team to convey their overall impressions of the work shop. Fifth, 12 recent first-time OTA project directors were interviewed by the project team. Questions fo cused on recounting how the new project director conducted his/her work and what kinds of guidance they found helpful while working through the assessment process for the first time. Finally, the policy project team conducted an in-house review of the policy analysis literature and previous analyses of OT A. A num ber of especially insightful articles and books were identified and or ganized in a separate loose-leaf notebook that will be made avail able to OTA staff (see app. D). 2 Two of the reports reviewed by former congressional staff were among the nine repons dropped from the original list of 27 reponsoriginally submitted by the program managers (see note in box 2-A) and were not reviewed by the OTA project team. ------------mt------------

PAGE 45

An important mea sure of OT A policy analysis is the degree to which the analysis in an OT A report is able to be assimilated and understood by Congress, usua11y through congressional staff. That measure, in tum, depends in part on how wen an OTA report responds to the needs ofboth the com mittee that requested it and the rest of Con gress. The project team decided that the most straightforward way of discovering OT A reports' re sponsiveness to the needs of Con gress would be to ask OTA' s con gressional clients. Thus, the project team turned for insights to a group of current and fonnercongressional staff who are fami1iar with OT A's work and also have worked for committees that have requested OT A studies. To help counter con cerns that some committee staff might be satisfied with OTA po1icy analysis for political reasons, the policy project team decided to so-CHAPTER THREE licit the opinions of a diverse, bi partisan group of 17 Hill staff. The team asked four fonner congres sional staff to provide written evalu ations of the policy analysis in a sample of 12 OT A reports (three reportseach),usingtheirowncrite ria for good policy analysis, and then to come to a fol1owup meeting with the project team (see app. A3 ). In addition, the team interviewed 13 current congressional staff by telephone to ascertain their views on the strengths and weaknesses of OTA reports (see app. A-4). The policy project team hoped to gain important in sights concerning the way OT A is regarded by its congressional clients, while taking advantage of the het erogeneous nature of the sample, which was intended to ensure that any perceptions of serious deficiencies would surface. To obtain a differ entperspecti ve on how we11 OT A reports meet congressional needs, the policy project team also consulted five former OT A project directors ("OT A alumni") (see app. A-5). The team hoped that these views would draw on OT A experience and perspective but also be tem pered by time and experience out side the agency .i 1 The five fonnerOTA project directors were also asked to comment on other aspects of OTA's culture and policy analysis. Addi tional findings on these topics are presented in ch. 5. -------------11Jr------------

PAGE 46

.Box 3-A Crlteria Identified by fonner congressional staff to Judge good"pollcyanalysis. Most Oftari cited: N > / .. Reac:fer.-friendlylusabJefaccessible: useful.summary/overview requesh:ontext ~en ... .purpose$1d objectives clearty stated .. coherentlwelt-organized conta1ns an index .Objective Tunety from.a congressional perspective Alaoctted: Reader-friendly/usable/aeoeasible: .. parallel structure between summary.tlnd report .. desaiptive table of contenta up front :i: .. :.i). .. -. ~ ... ;,a ......... .. ....,. ... : :=-~~c At OT A's request, four former congressional staff specified what they considered criteria for "good" policy analysis in an OT A report. The three criteria on which there was the greatest degree of unanim ity among the four were as follows: reader-friendliness, objectivity. and timeliness from a congressional perspective. Individual staffers also speci fied a number of other criteria (see box 3-A). but on these points, there was less unanimity. As discussed further below, the consensus among former and current congressional staff queried was that OTA reports generally do well in terms of objectivity-with some lapses-and less well on reader-friendliness and timeliness. READERFRIENDLINESS The former congressional staffers consulted for this assessment were in strong agreement that OT A should give considerable attention to structuring of OT A reports to make them more reader-friendly (i.e., quickly digestible) for con gressional staffers and other lay persons with a short amount of time to assimilate information. The over all document should be clearly or ganized to steer the reader to the important ideas and conclusions. The reader-friendliness of a report can make a huge difference in the usefulness of an OT A report to a congressional staffer. The views expressed by one former congressional staffer on these points are typical of those expressed by several others: In my experience, very few staff read a major report in its en tirety. If it is well done, it will be used as a reference document; if it is not easy to use, it will be used very little. if at all. Ease of use will depend, in the first instance, on how the overall topic to be addressed is defined and refined ... Ease of use will also depend on how well the docu ment is organized and visually presented to make finding par ticular information or answers to specific questions relatively easy. Creative use of tables, graphs, and other displays of data is highly desirable to pro vide quick access to complex material. Any temptation on OTA's part to slight attention given to the presentation of ma-------------111-------------

PAGE 47

terial in order to put more em phasis on content should, in my opinion, be resisted, The best analysis in the world will not be used if it is buried in pages of unbroken prose. The importance of good sum maries in OTA reports was stressed repeatedly by congressional staff ers. Most of the staffers interviewed for this project said that they typi cally read only the summary chap ter of an OT A report, so the sum mary should include all the most important items-i.e., clearly de fined policy issues, summary con clusions, action recommendations clearly separated in the text for easy location and identification. In addi tion, the staffers said, they like to be able to track the summary's find ings and conclusions in the body of the report. For that reason, they said, the summary of an OT A re port should summarize the impor~t parts of the analysis and pro vide the reader both a quick over view and a guide to digging deeper. In the words of one former staffer: Summaries that are as concise as possible without sacrificing clarity are the ideal, with backup material for any particular point easily referenced in the body of the report. One staffer mentioned the impor tance of a full table of contents at the front of the report, and several Slressed the need for an index (missing in nearly half of the 18 OT A reports in the policy project's sample). The former congressional staff asked to evaluate 12 OT A reports reported that one-third of the 12 reports they reviewed had major problems in organization and for mat that made them difficult to use. 2 Some reports were criticized because the major findings and options were buried. Some reports were criticized because the sum mary did not fully represent the contents of the report or did not facilitate the tracking of arguments in the rest of the report. These and other difficulties, congressional staffers said, detracted from the reports' usefulness and value to them because they need to be able to assimilate a report quickly and easily The absence of indexes in OTA reports was criticized on simi lar grounds. 3 One former congres sional staffer suggested that OT A make its work available on disk as well as in print. OTA recently be gan an experiment to make the full text of our reports available through the House Information Systems (HIS). OBJECTIVITY Many congressional staff inter viewed for this project stressed that one of OTA's chief assets is its reputation for objectivity, noting that if the agency were to lose that reputation-as it could do very quickly if a few biased OT A reports surfaced-OT A would be of little use to Congress. One former con gressional staffer put it this way: On the whole, I believe that OTA produces objective reports. Nev ertheless, continuing that tradi tion of objectivity is critical to the stature, and future, of OTA. In fact, it is probably the most important aspect of an OTA re port. Without a reputation for objectivity, the agency will lose the intellectual support of their f sic] colleagues, and the quality of the members on advisory pan els and the Technology Assess ment Advisory Council will suf fer.Moreover, without a reputa tion for objectivity, the agency will certainly lose support in Congress ... Because the agency is dependent on the support of TAB [the Technology Assessment Board] and the Legislative BranchAppropriations Subcom mittees, serious allegations of bias could undermine the agency. 2 Such _proble~s were also encountered by th e pohcy proJect team in the sample of 18 ?TA repons. The policy project team judged Just over half the repons ( 10) to be very good or excellent in terms of reader-friendliness they judged the other repons to be okay o; worse (see ch. 4 and app. C). lNearly half(8)ofthe 180TA repons in the sample read by the policy project team did not have an index. _______ __J ________

PAGE 48

What is meant by objectivity in an OT A report? Judgments about OT A's objectivity are themselves highly subjective. As one OT A pro gram manager explains: [There are] several different pos sible meanings of objectivity in an OTA report, and ... the meaning has evolved over time and in different programs and issue areas to the point where we now need to think through what we mean by the word. Although three of the four former congressional staffers asked to give criteriaforgoodOTApolicy analy sis explicitly cited objectivity as one criterion, it is not clear that they always meant the same thing by the term. One staffer elucidated what he meant as follows: Objective: Is the analysis unbi ased? Does it, intentionally or unintentionally.favor the posi tion of any particular advocate party involved in the issue? Another staffer who said that ob jectivity was "'essential" for an OT A report gave this interpretation of the term: Objective: Reliable valid data, scientifically based is critical. Current literature review coupledwithadvicefroma spec trum of qualified experts must be evident. In any event, the former congres sional staff asked to review and evaluate a sample of 12 OT A re ports generally judged the level of objectivity of these reports to be quite high. However, three of these staffers identified five reports with lapses of objectivity (lapses in the "con text, findings, and issues" part, orin the "goals and options" part, or in both).4 A few of the examples the former staff pointed to in these reports seemed to be major. One of the OT A reports in the sample of 12, for example, was severely criti cized by the staffer who criticized it for unsubstantiated advocacy. This criticism applied to both the "con text, findings, and issues" part of the report's policy analysis and to the "goals and options" part. Most of the complaints voiced by con gressional staffers about objectiv ity in the policy analysis in OT A reports were less serious. Some complaints stemmed from specific findings in the report that did not seem adequately supported. Other complaints about objec tivity by staffers stemmed from a report's making explicit or implicit recommendations rather than pre senting options.5 Some options were perceived as recommendations be cause the report presented them in a way that seemed to favor a par ticular course of action; some were perceived as recommendations because the report identified only one general course of action. Thus, for example, one staffer complained about one report: The policy options ... are not really options at all but recom mendations hidden under an other name. Another former staffer specifically noted the risk of making implicit recommendations by presenting only one general course of action rather than presenting a range of alternatives: It is through the use of clearly stated alternatives that the reader can be impressed that OTA hasn't just derived "an answer" and is now trying to figure out how to derive the right question to fit the answer. The policy project team and the fonner congressional staff read IO of the same OT A assessments. Both the project team and the fonner staffers found what they considered lapses of objectivity-either real or apparent-in five of the I Oreports. The two groups differed, however, in their judgments of which five reports had these lapses. Both groups agreed that there were lapses of ob jectivity in two specific reports. They also agreed that there was no problem with objec tivity in two other specific reports. For the remaining six reports, the two groups came to opposite conclusions about the reports' objectivity. Given the lack of agreement about the meaning of objectivity in an OT A report and the possibility of subjective interpreta tions, it is perhaps not surprising that people come to different judgments about the level of objectivity in specific OT A reports. 5 Seven ( 40 percent) of the 18 OT A reports the policy project team reviewed included what might be considered explicit or implicit recommendations or policy prescriptions in the options (see ch. 4). ---------------------111--------------------

PAGE 49

Anotherstaffer,inviewingareport's apparent Jack of objectivity in the presentation of an option, per ceived the problem primarily as an error of omission rather than of commission: I found the basis for suggesting such a change intuitively defen sible, but not intellectually so. Should OTA reports make recom mendations in options, either ex plicitly or implicitly (e.g., by pre senting one set of possible courses of action more strongly than oth ers)? Contrary to in-house folklore, the Technology Assessment Act does not prohibit OT A from mak ing recommendations. Furthermore, the policy project team was unable to locate any Technology Assessment Board (TAB) or OTA Director's policy explicitly discuss ing recommendations or other types of policy directives in OT A reports. Several views on whether rec ommendations in OTA report op tions are appropriate were provided by congressional staff interviewed for this assessment. Most, but not an, of the congressional staff said that OTA staff should recognize that they are treading on thin ice when making recommendations, either explicitly or implicitly. Sev eral congressional staffers were of the opinion that OTA should never recommend or prescribe a particu lar path, but should present options with supporting analysis comparing their effectiveness. One current staffer made this argument on the grounds that, unlike Congress, OT A is not "accountable" for its pro nouncements (except indirectly through TAB). On the other hand, many con gressional staff consulted for this assessment said that there wiH un doubtedly be a few cases in which available evidence is overwhelm ingly in favor of a particular option to reach a specified goal. In those few cases, they said, OTA owes the Congress an honest statement to that effect and may present its "em piricany based policy prescriptions" accordingly.lfareportdoespresent an "empirically based policy pre scription," however, there should be extraordinarily solid analysis and arguments in support of the direc tive. 6 Otherwise, OTA wm run the risk of losing its hard-won-and vitally important-reputation for objectivity. There seems to be considerable confusion among OTA staff con cerning the agency's policy about recommendations or policy pre scriptions in options. As an ex ample, the views on this topic among former OTA project directors were polarized. At one end of the spec trum was a project director who felt that OT A should never make rec ommendations. OTA had no reason to exist if it fen into the trap of advocacy: Congress has little need for ad vice from another advocacy or ganization .... On this reason ing, no OTA report can be too even-handed. At the other end of the spectrum was one who found fault with OT A for not being activist enough in advocating viewpoints. He asserted: The one stylistic tendency that should be avoided, or minimized, is an orientation and priority to be factually correct, balanced, fair, objective, comprehensive, and academically correct, with little explicit strategic design to be effective as an agent of change. The policy project team did not have the time to pursue this issue to the depth it would have liked but believes consideration and cJarifi cation of OTA's policy (or Jack thereot) is required. TIMELINESS On another issue, timeliness, many congressional staffers felt that OT A has often fanen short. Reports often take too Jong to produce and their usefulness may thus be diminished. Timeliness has two aspects. First, 6 The project team judged that "empirically based policy prescriptions" (sometimes labeled options) were present and well sup ported in about three of the 18 repons in its sample(seech.4).0neofthese, Transporta tion o/HaZJJrdous Materials, is discussed in ch.S. ------m------

PAGE 50

there is the overall length of time it ta1ces to produce an OT A study, which averaged close to two years for the studies read by the project team--coincidentally the lifetime of a Congress.7 This gives rushed staffers the impression that OT A cannot be relied upon to produce useful answers within the main timeline of interest. However, it should be noted that the policy project team found several cases where an OT A report provided important input to legislation passed up to five years after the report release date. Second, complaints were heard that OT A reports are often not available in time for im portant mark-ups. There is a miti gating argument here, too: that the report release date is partially de termined by the request date, which is the province of committee staff.8 Nevertheless, staff would gener ally be pleased if at least some OT A reports were produced more rap idly, knowing full well that their scope or depth would have to be reduced. In fact, the policy project team found that at least two reports of the 18 were produced relatively quickly: Exploring the Moon and Mars (six months) and Improving Automobile Fuel Economy (15 months). In both cases, the project director had substantial experience at OT A and had some knowledge of the topic in general, which made the quick turnaround possible. The two reports were considered good and useful reports, both by OT A personnel and congressional staff, leading to the conclusion that, in some cases, by calling on experi enced project directors who have worked on related topics, it is pos sible for OT A to do analytically sound and useful work for Con gress in considerably less than two years. The other reports read by the project team ranged up to over 36 months in length of time to com plete and averaged 26 months. "CONTEXT" OR "OPTIONS"? In the course of reviewing OT A reports,theOTApolicyprojectteam found that OT A reports typically have two components: 1. analysis of the policy problem, including discussion of the policy context, findings, and issues; and 2. identification and analysis of potential solutions, i.e., goals and options for congressional consideration (see box 5-A in ch. 5). These two components are referred to below in the shorthand "context" and "options." The policy project concluded from its evaluation of 18 reports that OT A reports typically devote more time, effort, and pages to analysis of the context than to pre sentation and analysis of options (see ch. 5). Thus, an important ques tion to be asked is whether, in the views of congressional requesters, OT A tends to allocate the correct fraction of energy to each. The opinions of current requesting staff vary, perhaps indicating that hard and fast generalities on this topic are impossible to ma1ce. Of 13 indi viduals queried, four felt that analy sis of options was more useful to staff (with some emphasizing tech nical analysis). Four felt that analy sis of context was more important. The other five wanted both, but three of them thought that options were more important. Although congressional staffers interviewed by the policy project team disagreed on the importance ofincluding options in reports, there was one point on which they were virtually unanimous: namely, that OTA should analyze the options that it presents. Some stated this sentiment even more strongly: Do not bother providing a list of op tions unless the report analyzes their effects and effectiveness. An infer ence may be that in OT A reports where the requesters really want OTA to present options, the analy sis of options should receive more attention than OT A project teams typically give it today. 9 7 Two years was the average length of time required to produce an OT A study. based on the periods reported as being required to complete the 18 reports read by the project team. This difficulty might be avoided by closer contacts between OTA and requesting com mittee staff, both preceding the issuance of the request letter and during the study. 9The policy project team judged the analysis of policy options to be deficient in about half of the reports in its 18-report sample (see ch. 4). ----------------111-------------

PAGE 51

CHAPTER FOUR A profile . .. The core of the OT A policy project, at least in terms of person-hours devoted to analysis, was a review of a sample of OT A reports. Eighteen of 18 .. ()TA tents of the report sample to a set of summary sta tistics. Such numerical representation misses detail and nuance alike. Nevertheless, it provides an overview that can ori ent the reader to dimen sions of OTA's policy analysis. In the discus sion that follows, these errors of omission and commission are discussed in the aggregate (i.e., without citing report OT A reports identified by the nine OT A program managers as examples of "good" policy analysis were read and evaluated by the OT A policy project team (see box 2-A in ch. 2). The process used to evaluate these 18 reports took shape only as the project got underway. The process involved the use of two forms developed by the project team (see box 4-A). A rather long questionnaire was used to interview project directors and to characterize aspects of policy analy sis in the reports (see app. B); this resulted in the elucidation of, among other things, the variety of analyti cal approaches found in OT A re ports (see ch. 5). A second, shorter form a1lowed the numerical rating of each report on some specific "dimensions" or "elements" of policy analysis. The ratings from this shorter form, which is repro duced in app. C, are the basis for the statistical analysis in this chapter. The policy project team's focus on 18 OTA reports a1lows for sev eral characterizations of how OT A reports present the results of an assessment. The diversity of what OT A does is indeed reflected in its reports: scope. style, length. and a host of other dimensions that the policy project team developed in ductively are presented below. The sources were many: suggestions by advisory panel members, criteria suggested by former committee staff. and debate among the policy project team about their own sup positions and experiences with the OT A process. In general, the policy project team seeks in this chapter to reduce patterns observed in the contitles). Emphasis on positive fea tures of OT A reports, with attribu tion. is given in the next chapter on ways OT A tells a story ( or parts of a story) well. It is important to note that the nonrandom. "purposive" sample of 18 OT A reports read and rated for this study makes any statistical analysis more heuristic than defini tive. The policy project team can not generalize to the program or division level. but can present a picture of agency tendencies in policy analysis and report writing. (A project team joke is that this is a sample of 18 statistical "outliers" on which we comment at our peril.) Thus, our findings focus on a few --------------------Im--------------------

PAGE 52

patterns or themes that identify "problems" with OTA reports. Many of these are amplified else where in this report by other kinds of data collected through interviews and retrospective accounts by par ticipants in the OT A assessment process. FINDINGS FROM THE STATISTICAL ANALYSIS Statistical data from the sample of 18 OT A reports indicate that the average report is 254 pages long ( with a range of 104 to 395 pages), organized in eight chapters with three appendices. Only about half (8) of the reports in the sample had an index. Two-thirds of the 18 OT A re ports in the sample were requested by a single congressional commit tee, and the rest were requested by multiple committees. The list of requesters for the 18 OT A reports reviewed by the policy project team encompasses 19 different congres sional committees. These commit tees represent a fairly good cross section of OTA' scongressional cli entele. Since 1985, 19 House and 16 Senate committees have asked for OT A assessments. Topping the agency list of House requesters of studies since 1985 are the House Energy and Commerce Committee and the House Science, Space and Technology Committee; these same committees also topped the list of requesters of the 18 reports in the sample (requesting five reports each). The Senate requesters of the 18 reports were more varied, origi nating from eight different com mittees. The Senate Labor and Human Resources Committee and the Senate Commerce, Science, and Transportation Committee are the agency's most frequent Senate re questers since 1985 but requested only one and two reports, respec tively, in the I 8-report sample. Two other Senate committees that re quested two reports each were the Senate Energy and Natural Re sources Committee and the Senate Armed Services Committee. The dimensions on which the policy project team gathered statis tics on the I 8 OT A reports in the sample form clusters of themes and issues of varying prominence (see app. C for the frequency distribu tion of reports or mean scores for each rated dimension). These are reviewed below. In most instances, average values for the entire report sample are reported. It is important to point out, however, that there was significant report-to-report variation. Scale of effort and disciplinary makeup of project teams Two indicators of project scale of effort are budget and average size of the project staff. Almost two thirds (11) of the 18 reports in the sample cost from $300,000 to $600,000; the remaining one-third cost less than this (4) or more (3). The 18 projects in the sample had an average staff size of four, including the project director. Half of the 18 assessments were directed by a natural (physical or biological) scientist or engineer. The othernine assessments were directed by an economist, by a social scientist or a person with a '"policy degree," or by a person from some other disci pline. Two-thirds of the projects included a social or policy scientist (by degree) on the staff; and two thirds included a natural scientist or engineer. One-third of the projects included a lawyer; and one-sixth of the projects included an economist. 2 Reader-friendliness of the 18 OTA reports The preceding chapter noted the importance to congressional staff of being able readily to grasp the central findings, issues, and op tions in an OT A report and being able to track the points made in the summary chapter through the rest of the report. Various features of a report enhance a report's accessi bility/reader-friendliness for con gressional readers-a good sum mary chapter(e.g., useful summary/ overview), request context given, purpose and objectives clearly stated,coherentand well organized, 1 One of the 18 reports was requested by six committees (two House com mittees and four Senate committees). 2 For information on the disciplinary composition of OTA 'sentire research staff, see ch. 6. -------------------ml---------------------

PAGE 53

.-: . .-. . eox~A. A note on methods for the statistl~I analysis of the 18 OTA reports. ii:::-~:=.=r,= :..:=-z:.o~ :~s:: .. overthecourseofthepolicyproject.Atthebeginningof > /by the team (see app. C):'This "statistical tabulation .theproject.aquestionnairewasdevelopedasaguideto (form" was drafted, pilot tested, and revised several reading and characterizing the. various elements of S8nJ)le of 18 OTA reports. The primary and secondary .... policy anatysis found inmost OTAreports: 1) policy readers used the form independently and then dis-< .. context, findirlgs. and issues; and 2) goats and options cussecltheirratingsandagreed upon a single combined .(seeibox 1--A inch. 1). Each of.the 18 reports was .(consensus) rating for each question. Many of the .assigned ~:a pair of reviewers on the policy project / questions required assignment of a numerical rating on team, a primary reviewer and a secondary reader.The .. a 1 to51lC81e~witM being "p0or;2being"fair:ao1cay, .. primary,:eviewerinterviewedtheprojecfdirectortogain; 4-Verygood." and 5 "excellent: ? . :::::=:a::=~= ;ieam'::~~n~a::::,: lanaiysi&ThentJoththeprirnaniand~reader .:. )or each ~eddimension (see app. C for the report \readthe report,ostiJgihe secdndpart.oi1hequestio&ffrequencydstribtltion~meanscores, or other measures '.na1re:;~nt1y;jthe tvwij\~ 10 diseti#. < '.:ot rateddimenston)~.:fhe team also generated a ==::.==:== ii;~=:~:=:':w= :i== .... :::lfiepduri.ffl1~~~r ... .. ....... .. ... .~inilning, type of assessment (e.g narrow vs / r Aft .. \COll'lp1etingJhe.~ qui;$ti0rinaire for each < .broad)/~ primaryreviewer.ontheprojectteam. There t .{reportittieprhnl,ya.,d~ndaiyijwiewerwroteaoije,;\ .were. -~:.~no systeniatlcvariations and very few :: tpe.gesummary Qffindiqgs/That,sunmary'.proved~ ;;: identifiable dustera(whlcti is why none lsreproduced / : ~ lass~than 1he lol)g.questioMBJr, ;/ : In Olher words~ what was cause and what was > : liowever.sothe,;policyiprojectieamdevised &;shortet'; :.,~ .. .. :.fi-:: ..::-::( ... .. .. contains an index, effective use of typography, graphics, etc. (see ch. 5). Onasca]eofl (poor)to5(excel-1ent), the policy project team gave the 18 reports in the sample an averageratingof3.4(between okay and very good) for reader-friendli ness (see app. C). The team gave just over half the reports (10) a score of 4 or above (very good or better) for overall reader-friendli ness; the other eight reports fe]] below this level. Thedetai1s are sobering. In some of the reports, the findings and op tions were so buried that it was almost impossib]e to find them. The ease of Jocating issues and findings was another dimension of reader-friendliness the team con sidered in each report; the mean rating was3.4(with less than half of the samp]e garnering a 4 or 5). In many reports, it was difficult to find where in the report details on mate rial presented in the summary could be found. Some reports took hours to assimi1ate; others literaUy re quired days. The team found close parallel ism between the organization of the summary and the rest of the report in only seven of the 18 reports; four reports had no paraUelism at all. The absence of paraUel construc tion of the summary and other chap ters often makes it difficult for a reader to use the summary as a guide to the rest of the report. Ten of the 18 reports included no index. -------------mf-------------

PAGE 54

Objectivity and recommendations in the 18 OTA reports OT A's reputation for objectivity is viewed by some as one of its chief assets (see ch. 3). If OTA were to lose this reputation-as it could do very quickly if a few reports with serious problems in objectivity surfaced-the agency would be in se rious trouble. The OT A policy project team found lapses of objectivity that it considered a problem (major or minor) in one-third of the 18 reports it reviewed (see app. C). Some re ports had lapses in the "policy con text, findings, and issues" part of the report, some had them in the "goals and options" part, and some had them in both. In one report among the 18 the team reviewed, the arguments seemed so one-sided or so lacking in support that the report seemed open to the charge of outright advocacy. In a couple of other reports, the objectivity ap peared questionable because the basis for a report's controversial basic premise was not explicitly stated (e.g., the report considered how to implement a policy that it apparently assumed was worthwhile but neglected to state any basis for the assumption that the policy was worthwhile). In some cases, there seemed to be "errors of omission," perhaps due to blind spots resulting from analysts' (and reviewers') values or prior beliefs. A few instances of lapses of objectivity stemmed fromareport's making recommendations or policy prescriptions rather than present ing options. Seven of the 18 reports included, either explicitly or im plicitly, at least some recommen dations or policy prescriptions. In three reports, these were judged by the project team to be "empirically based policy prescriptions" support able with data and analysis in the report and not significantly depen dent on the decisionmakers' values or prior beliefs, and thus not open to criticism as a breach of objectiv ity. 3The implicit or explicit recom mendations in the remaining four reports were questionable, some times because OTA's unsubstan tiated preference seemed apparent and sometimes because reasonable alternatives just were not presented. Timeliness of the 18 OTA reports The OT A policy project collected data on the time required to com plete each of the 18 reports. The team found that the average num ber of months was 26.4 Unfortu nately, though, information on elapsed time from approval of re quest by the Technology Assess ment Board (TAB) to report release is unreliable because of imprecise recordkeeping and other reasons (e.g., delays in project startup after requests have been received, and the production-in response to a single Jetter-of a series of reports over many years whose initiation and completion cannot be accu rately determined). Number of fis cal years spanned for each deliv ered OT A report would at best be a crude approximation. There were no alternative data sources, how ever, to cross-check on this dimen sion that was included in the statis tical tabulation form. Policy analysis in the "context" vs. "options" part of the 18 OTA reports The policy project team rated each of the 18 reports in the sample on their presentation of the two major components of policy analysis in an OTA report: I. analysis of the policy problem, including discussion of the policy context, findings, and issues; and 2. identification and analysis of potential solutions, i.e., goals and options for congressional consideration (see box 5-A in ch. 5). For the sake of brevity, these two components are referred to below in the shorthand "context" and "op tions." For funher discussion of "empirically based policy prescriptions" in an OT A report. see ch. 5. Congressional staff complained that two years was often too long to wait for an OTA assessment (see ch. 3). A report in the 18-report sample that was completed in six to seven months, Exp/orinl( the Moon and Mars. is discussed in ch. 5. -------------------ml---------------------

PAGE 55

To rate these two components, the policy project used a scale of 1 (poor) to 5 (excellent). For the de scription of "context" overall, the team gave the 18 reports a respect able overall score of 3.8 (almost very good) (see app. C). In addi tion, the team scored some specific aspects of "context." Three aspects-treatment of current Fed eral policy and activities, the legal and regulatory context, and the in stitutional context-were rated around 4. 5 It is important to note, however, that there was significant variation from report to report. A fourth aspect of "context"-analysis of stakeholders/affected parties-was rated decidedly lower, 3 (okay). The policy project team found that about one-third (5) of the 18 reports in the sample did a very good or excellent job of analyzing the positions of different stakehold ers. 6 Six of the reports did not do a particularly good job (i.e., were rated 1 or 2) in this area. In considering "context," the team also judged the quality of "explanation of the status quo" in the 18 reports. It considered one third of the reports to be "very good," another third "fair," and another third either not very good in treating the current state of af fairs or excluding such a discussion altogether. On a final dimension of context--consideration of interna tional context-the policy team found that in five of the 18 reports, the international context was criti cal to an understanding of the issue; these reports received an impres sive 4.8 score for the discussion of the international context.7 Six reports out of the 18 noted the inter national context but did few com parisons, and seven reports focused solely on the national context. The policy project team found that while the 18 reports on average did a fairly good job of analyzing the "context," many of them fal tered on the presentation and analy sis of"options." For the description ofoptions" overall, the team gave the 18 reports an overall score of only 2.8 (less than okay) (see app. C). Nearly two-thirds (11) of the reports provided from six to 20 options; and nearly one-third (5) provided more than 20 options. Nearly half (8) of the 18 reports included options creating a new governmental institution. Three reports had options involving a re duced Federal role (e.g., options relying on market or other mecha nisms). About half included an op tion that advanced the deployment of people/human factors as a policy solution. Since congressional staff inter viewed for this project said OTA analysts should not bother present ing options without analysis of the effects and effectiveness of the op tions (see ch. 3), the policy team specifically rated the 18 reports on whether the options presented were analyzed. More than half ( 10) of the 18 reports included analysis of op tions (e.g., in terms of their effec tiveness of achieving specified goals, costs, and impacts on and likely reactions by stakeholders).~ The bad news is that analysis of options was lacking in the other eightreports.Furthermore,onlynine of the 18 reports contained analysis that actually compared options and/ or strategies. The project team also estimated that the proportion of the report measured in pages devoted to op tions in the sample of 18 reports averaged about 15 percent of the report ( with a range ofless than 5 to 40 percent, calculated as a fraction of chapter pages only, not appendi ces). In terms of the dimensions of ease of finding the options and re lating the options to the rest of the report, the ratings were 3 .3 and 3 .1, respectively. More telling, only two of the 18 reports received scores of 4 or 5 on both of these dimensions. 5 A couple of OTA reports from the 18-report sample that did a good job of legal and institutional analysis are discussed in ch. 5. 6 One of the 18 reports that did a g1xx1 job of stakeholder analysis, Power On, is discussed inch.5. 7 One of these reports, Enhancing the Quality of U.S. Grain for International Trade, is discussed in ch. 5. A couple of OT A reports from the 18-report sample that did a g1xxl job of analyzing options are discussed in ch. 5. -------------11-------------

PAGE 56

In search of differences by "type" of OTA Table 4-1. Methods cited by project directors of 18 OTA reports The policy project team found that the policy analysis in the assessment The policy project team struggled todeviseataxonomy of OT A reports that might help us inter pret some of the results about the strengths and weaknesses of OT A policy analysis. The team considered and dis carded several taxono mies. It had expected glaring differences in the scope ( wide-ranging vs. well-circumscribed) of reports in the sample, but such differences were not readily appar ent. The team also considered a taxonomy based on whether the focal issues of a report were emerging issues or mature and whether they were closely linked to the immediate legisla tive agenda or more long term. Though plausible in the abstract, these dis10 Identify policy context, findings, and issues problem-driven reports was generally betterthan that in the "technology driven" reports, though Methods Workshops Frequency: 9 tinctions are impossible literature review case studies Contracts In-house legal analysis Quantitative analysis Executive branch discussions Site visits Historical reviews International comparisons Panel identifies issues Survey Talking to lots of people Session at conference To formulate goals and options 8 7 5 4 4 3 3 2 2 2 2 2 1 Methods Frequency Brainstorming by staff 9 Project director sitting and thinking 7 Quantitative analysis 5 .Advisory panel 3 Scenarios to evaluate options 2 Talking to lots of people 2 Casest~es 1 Contracts 1 Historical reviews 1 Literature review 1 Surv~ 1 to make for a whole report (i.e., they either are not mutually exclu sive or cannot be ascertained). on how to fix a problem, usually specified in the congressional request letter), and "technology driven reports" ( which consider the implications of the development, refinement, or use of a technology or class of technologies). After a great deal of effort, the best taxonomy of OT A reports the teamcouldcomeupwithwas "prob lem-driven reports" (which focus there were variations within these categories. For their treatment of "context," the problem driven reports got an average score from the projectteam of 4.1 (very good) vs. an average score in the technology driven reports of only 3.1 (okay). On the "op tions," the problemdriven reports got an average score of 3 (okay), while the technology-driven reports got an average score of only 2.3 (a little better than fair). The signifi cance of these findings, if any, is not clear. Perhaps it is easier for OT A analysts to get a handle on today's prob lems than to anticipate tomorrow's. Finally, the team took a second look at the two categories of re ports. The 13 problem-driven re ports were assigned to three subcat egories (though fourof the 13 could not be assigned in a mutually exclusive way): ----------,mi--------------

PAGE 57

1. problem-technical (the search for technical fixes, either the use of anew technology or the appli cation of an existing one to solve a problem, e.g., to improve fuel economy); 2. problem-organizational (how to organize for the management of a problem, e.g., the defense tech nology base); and 3. problem-legal (evaluation of a problem to ascertain, e.g., that questions of ownership of hu man tissues and cells cannot be handled by the current body of law). The five .. technology-driven" re ports typically asked about a tech nology (e.g., sequencing the hu man genome): What can it do for us or will it do to us? The project team's interpretation of these dif ferences is guarded at best: Some thing systematic may be going on, but the sample size, especiaJJy in the subcategories, is too small to warrant generalizations. Methods of assessment in the 18 OTA reports Finally, the policy project team was curious about the methods used by the staff responsible for the 18 as sessments in the sample. In the interviews with the project direc tors for the 18 reports, team mem bersaskedthem to identify the major methods they used, first in develop ing "context" and then specificaJJy for formulating "options." In addition, when team members were reading the reports, they noted specific methodological tools that were apparent. A listofassessment methods was also identified in the sample of 18 reports. A frequency distribution of methods is presented in table 4-1. The methods most commonly cited as being used for framing the "context"partof a report were work shops and literature reviews (half of the sample); case studies, legal analysis, and quantitative analysis each were cited for no more than one-third of the sample. Methods such as site visits, historical re views, issue identification at an advisory panel meeting, or con ducting a survey are reportedly much less common. For developing and analyzing "options," the most common meth ods by far were .. brainstorming by the staff' and the "project director sitting and thinking" (about half the sample used each). The term "com mon sense" came up frequently. Other methods (e.g., using the ad visory panel or scenarios to evalu ate options) were mentioned infre quentJy.9TheconclusionoftheOTA policy project team is that common sense and coJJective rumination prevail in the transformation of re search findings to policy options. The project directors, with and with out other team members, muddle through solo rather than depend on particular tools or other peopleinsideandoutsideofOTA-in drafting policy options. Missing from this inventory of methods is the extensive reviewing of contractor reports and draft chap ters. On such review (both external and in-house), project directors depend religiously. But chapter re view occurs near the end of the process, not in its formative stages. IMPLICATIONS OF THE ANALYSIS Quantitatively and qualitatively, the sample of 18 OT A reports exhibits some notable tendencies that in clude the following: a mix of natural scientists/engi neers and social scientists on most project teams; room for improvement in the reader-friendliness of a substan tial portion of OTA reports (e.g., improvements in the ease of lo cating the findings and options, inclusion of an index, more par allelism between the summary and the rest of the report); occasional )apses in objectivity (e.g., arguments so one-sided or so lacking in support that the report seemed open to the charge of outright advocacy) and other instances in which objectivity appeared questionable (e.g., be-9 If the policy project team had first presented to the project director the items in table 4-1 and asked which were pan of the assessment resulting in a particular repon. the frequency distribution of methods might have been different from that shown. ----------------,11-------------

PAGE 58

cause the basis for a report's controversial basic premise was not explicitly stated); less even-handedness in the pre sentation of options than OT A folklore would lead one to sus pect; a typically better job of policy analysis in the "context, find ings, and issues" part of the re port than in the "goals and op tions" part, but significant varia tion from report to report; little analysis of the effects and effectiveness of options in a sub stantial portion of OT A reports, but significant variation from report to report; a generally good job of treat ment of current Federal policy and activities, the legal and regu latory context, and institutional analysis in the context part of most OT A reports, but variation from report to report; a less than satisfactory job of stakeholder analysis in many OT A reports, but significant variation from report to report; little attention to market solu tions (as well as the role of nongovernmental organizations and international bodies) as an alter native to Federal intervention, but some reports in which mar ket solutions were considered; and the use of "brainstorming" and the "project director sitting and thinking" more than any other methods to develop policy op tions in OT A reports. Some of these tendencies depart from the "folklore" (probed in ch. 6) about what OTA reports d~ and do better than documents pro duced by other policy organiza tions. Are they causes for concern? That depends. If one believes that each OT A report can be judged only in the context of a specific request and the expectations that the client brings to the document (and its scale of effort), then the findings of this overview may not raise concerns. Perhaps the contents of each report can be highly variable and still re sponsive to congressional commit tee needs. On the other hand, if one believes that every OT A report should satisfy some minimal set of minimal criteria (e.g., those speci fied by congressional staff in ch. 3), then this statistical overview has uncovered some deficits. One sure implication of this over view is that OT A satisfies its con gressional clients in many ways and disappoints in others. OT A re ports do many things well, some things not so well, and a few things not at all. In the context of a specific request, some of the apparent defi ciencies and omissions in a report may not actually be problems. In many cases, however, the apparent deficiencies and omissions prob ably reflect problems that can be addressed through heightened sen sitivity and better time-management and allocation of time to different tasks in the assessment process. With these preliminary findings in mind, we need to illustrate the dimensions of good OT A storytelling as found in the 18-re port sample. That is the task of the chapter that follows. --------------m--------------

PAGE 59

What is policy analysis in an OTA report? The OT A policy project team decided at the outset of this study not to get bogged down in the effort to define policy analysis. Rather we de cided to use the same approach Percy Bridgman used in defining sci ence as "the activity of scientists"-we simply defined policy analysis as the "activity of policy analysts." We started out by looking at OT A re ports. In reading a sample of 18 OTA reports judged by program managers to have "good policy analysis" (see box 2-A in ch. 2), the policy project team found that each of the 18 reports tried, more or less successfully, to "tell a story." The stories were very dif ferent, but despite their diversity, all of the 18 reports typically had two major components: analysis of the policy problem, including discussion of the policy context, findings, and issues; and identificationandanalysisofpo tential solutions, i.e., goals and CHAPTER FIVE options for congressional con sideration (see box 5-A). Part of the art of OT A policy analy sis is making sure that the policy options flow from the information andargumentsadvancedelsewhere in the report and that the options are thoughtful and reflective of an un derstanding of what policy can achieve. If a report is well crafted, there is a clear and logical link between the two components. TELLING A GOOD STORY Only a few of the I 8 re ports in the sample reviewed by the OT A policy project team were judged to be consistently good in telling and link ing the two parts of the story. Three of these reports are summarized below: Critical Connections: Communication/or the Future; Nuclear Power in an Age of Uncertainty; Making Things Better: Competing in Manu facturing; All of these reports were fairly big budget (around $500,000) assess ments. Critical Connections took about three years; the other two took 18 months or so to complete. Some congressional staffers in terviewed by the policy project team said that the development of op tions was sometimes or often less important to them than having OT A provide scientific and other infor mation related to the policy issues at hand (see ch. 3 ). Thus, in some ---------------lml------------

PAGE 60

cases, an OT A report that concen trates on the problem-identifying part of its story may sometimes be quite acceptable to congressional staff. One such report, Ownership of Human Tissues and Cells, which is discussed below, is an exception to the rule that an OT A report should tell a good story from beginning to end. Most memorable stories do have a plot that unfolds from begin ning to end, but an author can occa sionally get away with leaving the ending up to the reader. The policy project team concluded that the same is true of certain OT A reports. When congressional requesters are primarily interested in learning about the context, findings, and issues, it may be desirable for project staff to devote most of its energies to this aspect of the report. The story told in Crlttca/Connecttons Critical Connections is an example of a very broad, forward-looking report that takes a strategic look at rapidly changing communications technologies and their implications for U.S. society, institutions, and policymakers. Critical Connections is a "technology-driven report" in the sense that it considers the impli cations of the development, refine ment, or use of a technology or class of technologies. It is a cre ative report that generates several potential goals and a large number of options to attain those goals. This report is extremely well written and superbly organized. The structure of Ch. 1: Summary closely parallels the structure of the rest of the report, making it easy for the reader to find further details on points made in the summary. The body of the report has three major parts. Part I: Changing Communica tions Infrastructure, Goals, and Policymaking (with three chap ters, including a conceptual framework for analyzing com munications issues); Part II: Opportunities and Con straints Provided by New Com munication Technologies (with four chapters on opportunities and constraints in four specific areas in which communication technologies are used); and Part III: Crosscutting Commu nication Issues and Alternative Policy Strategies for their Reso lution (with five chapters, in a uniform format, that deal with five major policy areas and present numerous options in each area). The problem-identifying part of the report's story is told in Parts I and II, which consider how recent advances in communication tech nologies are transforming the U.S. communications infrastructure. Part III, which is devoted to the discus sion of potential solutions, identi fies crosscutting issues engendered by these technologies and evalu ates alternative strategies and op tions to address these issues. Problem identifying-Critical Con nections begins its story in Part I by noting that revolutionary advances in information storage, processing, and transmission technologies are rapidly reconfiguring the U.S. com munications infrastructure. The lines that have historically divided domestic and international com munication systems and markets are gradually disappearing, the re port says, so decisions concerning communication systems and indus tries must increasingly reflect a glo bal perspective. The U.S. communication infra structure is defined broadly in the report as the underlying structure of technical facilities and institutional arrangements that support commu nication via telecommunication, broadcasting, film, audio and video recording, cable, print, and mail. Most of the U.S. communications infrastructure, the report notes, is currently held by private individu als and firms. Historically, Federal involvement in developing policy in this area has been minimal. Whether that historical pattern should continue is a central ques tion that remains to be answered. The report suggests that techno logical changes in communication technologies and their socioeco nomic impacts are "unraveling the --------------,m-------------

PAGE 61

foti,~~~i1r;:i1~~~ ... story: cc,mpon~ts~f:OTA pollriy a~alysis 1Ilrt~~:.====:~gh:h~ :/(certm.n extent on the topic,.some reports highlight very \broad philosophical issues, and others focus on far ;~o~al ~utlons: :goals and options .;Jhisl)al1 d an OTA report moves from the craft of .. pc)lacyanalysistohigh art, from analyzing the status quo .. :,;;;:~;}. ::=::cc~:a::: ;;~~c~!s~ S;lbrainstormiog~ SOiiciting outside views, etc., the project jjeam'k1entifiespotentiafgoals and concrete options for :=:~1~.6T2~~to propose and evaluate ... U.Jmpl~ons of put$uiqg alternative goals. In other ~~~~~a::~~::::! .o~"4~aryat~ Options for achieving a .J~inpartonthetopicofthe assessment. ------------mi.------------

PAGE 62

existing U.S. communication sys tem," creating new opportunities, players, and problems and raising new questions about the goals of the communication system, as well as about how, and by whom, future communication policy decisions should be made. Thus, Congress now has the opportunity to make a number of choices. If Congress fails to act decisively, the report warns, .. the opportunity to make deliber ate choices about new communica tions technologies-and about the nature of American society itselfwill be overtaken by rapid techno logical advances, the hardening of stakeholder positions and alliances, and the force of international de velopments and events." To determine the role that gov ernment might play in the realm of communication, the report notes, Congress will need to consider the opportunities that new communi cations technologies offer society, as well as the obstacles that prevent those opportunities from being re alized. Part II of Critical Connec tions has four chapters that identify "opportunities and constraints" pro vided by new communications tech nologies in four realms of life: 1) the business world, 2) the political arena/democratic process, 3) the shaping and development of cul ture, and 4) individual efforts to achieve personal autonomy and self realization. Potential solution&-The five chap ters in Part III of Critical Connec tions identify and discuss five "crosscutting communication is sues" engendered by changes in communication technologies: I. issues involving equitable ac cess to communications oppor tunities, 2. issues related to the security/ survivability of the communica tions infrastructure, 3. issues entailed in achieving interoperable communication systems, 4. issues related to modernization and technological development in the U.S. communication in frastructure, and 5. jurisdictional issues that are likely to arise in formulating and implementing a national com munication policy. A full chapter is devoted to explor ing factors contributing to prob lems in each of these areas and to identifying and evaluating alterna tive policy strategies and options to address the problems. This orga nizing approach works very well. One of the most innovative and distinctive features of Critical Con nections is that it deals with differ ing values in the selection of goals and options by making them ex plicit. Potential goals for policymakers are the elimination of specified problems in one or more of the five aforementioned areas. The report provides extensive background information for understanding the importance of each of these policy areas. Further more, it repeatedly emphasizes that policymakers will not be able to maximize goals in all five areas simultaneously; rather, they will have to make tradeoffs between the various goals (e.g., between maxi mizing security/survivability of communications systems and maxi mizing access to communications technologies). The choice of op tions, therefore, depends in part on which of the five areas policymakers think should be emphasized. Critical Connections identifies more than 50 options-roughly I 0 options in each of the five issue areas. The options presented really seem to be alternative approaches; there is no sense that they are rec ommendations. Most of the options presented are foresight options that were new to this assessment, per haps because of the cutting-edge character of this report. In each issue area, an effort is made to suggest strategies and options that range from minimal amounts of Federal intervention to greater amounts. To encourage modern ization of the communication in frastructure, for example, the re port identifies three general strate gies: l) direct government involve ment; 2) provision of tax credits or other indirect incentives to the pri vate sector; and 3) creating a regu latory environment that is more conducive to the modernization. -------------------111---------------------

PAGE 63

Several options are presented un der each strategy. Figures summa rizing the options in each of the five majorissueareasappearin the sum mary chapter and are duplicated in the five chapters of Part III; these figures greatly aid comprehension. Apart from presenting options that span a spectrum from minimal Federal intervention to greater in tervention, Critical Connections identifies which strategies/policy options in different issue areas are consistent with what the report cans "three alternative visions of com munication": 1) the market vision, i.e., communication as a market commodity; 2) the economic vi sion, i.e., communication as a springboard for economic growth and development; or 3) the social vision, i.e., communication as a basic societal infrastructure. The report notes that the choice of con gressional policy strategies and options will depend primarily on how Congress views communica tion in 21st century America. "If Congress can agree on a consistent vision of communication goals," the report notes, "many policy choices will fo11ow." Fina11y, this report admirably features extensive discussion of the many options it presents. Most of the discussion focuses on historical and legal precedents for the op tions. Also, the analysis of options focuses on how the options will be viewed by stakeholders and other affected parties (the industry, the Federal Communications Com mission, universities and individual users of communications technolo gies, etc.). The story told in Nuclear Power in an Age of Uncertainty Nuclear Power in an Age of U ncer tainty is another example of a re port that is equa11y strong in de scribing the policy context and ex ploring options available to Con gress. Moreover, it is another ex ample where the organization of the report leads one directly to the options presented at the end. More important, for those readers who start by reading the policy options and then need to know the analyti cal support for the option, this type of organization allows one to locate readily the desired detail. Nuclear Power in an Age of Uncertainty is organized using six "factors that affect nuc tear power's future"; four congressional goals; and 15 options to help realize the goals. The 15 options are also grouped into three "strategies," which correspond to level of gov ernmental intervention. In the sum mary, the six "factors" organize the report's findings (though the term "finding" is not stressed.) Problem Identifying-The policy problem is stated briefly in the very beginning of Nuclear Power in an Age of Uncertainty: Fora variety of reasons, the United States may want to preserve nuclear power as a vi able energy option. Nuclear power "has advantages that may prove crucial to this Nation's energy sys tem in the coming decades, but at present is an option that no electric utility would seriously consider." Without improvements in a num ber of areas, this situation will not change and the nuclear industry will not survive. Keeping the U.S. industry alive while it is decided whether nuclear power's benefits outweigh its costs becomes the premise, or overall goal, of the as sessment. Again, the report tells the story through a focus on six "factors that affect nuclear power's future": 1) financial and economic conditions, 2) prospects for new technologies, 3) management of construction and operation of nuclear power plants, 4) regulation, 5) viability of the vendors, and 6) public attitudes. In the summary, key findings are or ganized under these "factors," with good use of boldface type to help those readers who are just skim ming. The report includes a chapter devoted to each of these factors, which accomplishes two things. First, if one desires more detailed explanation related to a specific factor than that presented in the summary, it is very easy to find. Second, the organization instills confidence in the report; the reader ------mf-------

PAGE 64

knows that more detailed discus sion and substantiation exists, re gardless of whether he or she uses it. "Factors" is not an exciting word but is nonetheless an appropriate and helpful organizing concept. Note that these factors span a wide spectrum of policy-relevant con siderations: whether new technolo gies can help solve the problem, strengths and weaknesses of cur rent laws and regulations, govern ment performance, human factors, the private sector, public opinion, and soon. Even before the factors are pre sented, the report tells the story from the very different perspec tives of the key participants in the "nuclear debate": the Nuclear Regu latory Commission, state regula tors, utility investors, utilities them selves, the nuclear industry, critics of nuclear power, and the public. The discussion is brief, about two pages plus an excellent summary figure (reproduced as one of many OTA policy "gems" in app. E) but captures the differing needs and desires of each stakeholder group. It also identifies those objectives on which everyone agrees (e.g., no major accidents and a convincing waste disposal program). The de bate is a harbinger of the six factors that organize the rest of the report. Several research methods were used to gather the information pre sented. Three workshops were held to discuss several of the factors identified above. Two contracts were let on public acceptance. The staff also arranged a small work shop at an industry conference to understand better the views of in dustry executives. The report includes a few other noteworthy devices to improve readability: a one-page "overview and findings" at the beginning of the report; a clearly labelled statement of "the policy problem"; and an explicit statement of the purpose of the study, the methods used, and its organization. Potential solutions-The last chap ter of Nuclear Power in an Age of Uncertainty, "Policy Options," makes the grand leap from the sta tus quo to alternative, more desir able futures. It identifies four goals that should be viewed favorably by the stakeholders. The four goals are 1) reduce capital costs and uncer tainties, 2) improve reactor opera tions and economics, 3) reduce risks of accidents, and 4) alleviate public concerns and political risks. Again these are not alternative goals, but components necessary to achieve the overall goal of preserving the nuclear option. Fifteen options are discussed under the four goals in thepolicyoptionschapter. The sum mary discusses only the most im portant seven options. The sum mary also does not attempt to force the options under a single goal. Note that the organization of the policy options chapterin this report does not parallel the organization of the rest of the report. In this instance, the lack of parallelism is not a problem, because the options themselves correspond quite well to the factors (i.e., chapters) identi fied earlier. For example, technol ogy options (primarily covered in ch. 4) are found under goal I, re ducing capital costs and uncertain ties and goal 3, reducing risks of accidents. The options are also organized into three strategies that reflect how actively one might want to inter vene to keep the industry alive-a continuation of the status quo, re moving a few obstacles to more nuclear orders, and providing a moderate stimulus to more orders. These strategies are uninforma tively titled "base case," "strategy l ," and "strategy 2," but it is still quite easy for most congressional staffers to identify which strategy (i.e., package of options) his or her boss is likely to prefer, so the con cept is quite helpful. These are, of course, the same options that are discussed by goal, just organized in a different, equally policy-relevant, form. Nuclear Power in an Age of Uncertainty analyzes the effective ness of the strategies under four plausible future scenarios. (These are combinations of electricity growth rate and industry success in solving their own problems-both key unknowns.) Because the ef fectiveness of the options in some ------------11.------------

PAGE 65

cases depends on unknown future conditions, the report is able to examine the robustness of the alter native strategies. The attempt is admirable, but owing to its com plexity is not altogether successful. The analysis is mostly by appeal to common sense-very thoughtful, well reasoned, and thorough-but one is left with a desire for more data to support many of the asser tions. The story told in Making Things Better Making Things Better is the second in a series of three OT A reports on the health ofU .S. manufacturing in a world economy. It is a .. problem driven report" in the sense that it focuses on how to fix a problem (rather than on a specific technol ogy or cJass of technologies). Prob lem-driven reports are the most common types of studies inc1uded in the sample of 18 OT A reports. The report is extremely well written and organized. These fea tures, for the most part, overcome the report's almost total lack of other types of assistance to the reader (such as summary tables of find ingsoroptions). The foreword states the goal of the assessment: to iden tify "ways to promote the restora tion of American leadership in manufacturing technology." Tech nology is defined in its broad sense as "not only new products and ad vanced manufacturing equipment but also efficient organization of work and effective use of people." With only the foreword to serve as an explanation of the purpose and scope of the assessment, the report jumps right into a very forceful, clear summary which begins, ''American manufacturing has never been in more trouble than it is now. Its biggest challenge is from Japan ... While some American compa nies and institutions have redoubled efforts to improve manufacturing, the government is dozing at the switch." And even the first para graph makes it clear that the report will not shy from proposing alter natives: .. As a nation, we owe it to ourselves to help with [the] solu tion." Making ThingsBetterestablishes key themes early and uses these themes to organize the entire re port. This type of highly parallel structure is of great advantage to the congressional client (see fur ther discussion under "Telling a Story We1J" section below). Problem Identifying-Making Things Better begins its story with a brief review of trends in the U.S. manufacturing economy since 1960. Seven figures illustrate pro ductivity, trade balance, and other economic trends-plus compari sons to the country's major eco nomic competitors-in as many pages. These are the manifestations of the problem (and the subject of an earlier OT A special report that was part of the series). The report then launches into its major themes-the root causes of these undesirable trends and unfavorable comparisons. These include defi ciencies 1n 1) investments in tech nology, 2) investments in people, 3) cooperation among firms, and 4) technology transfer and diffusion. Chapters of the report (and sections of the summary) are devoted to explaining the importance of each of these factors and comparing the United States to its competitors. According to the project direc tor, the four themes that eventually emerged were by no means identi cal to the key issues the project team hypothesized at the beginning of the study. To understand "the story," the methods used by the team included a review of the ex tensive literature on the topic; in depthcomparisons with other coun tries, in particular Japan; several case studies (e.g., high-definition television); identification of key issues by their advisory panel; and open discussions with analysts in relevant executive branch agencies. Potential solutions-The second half of the summary and the second chapter of Making Things Better are devoted to four somewhat overlapping "strategic targets" for 1 For further discussion of "technologydriven" and "problem-driven" OTA studies, see ch. 4. -----------~-------------

PAGE 66

policymakers. Again, these flow almost directly from the policy con text. These four strategic targets are as follows: 1) improving the financial environment for U.S. manufacturing firms, 2) upgrading education and training of the work ers, managers, and engineers need ed in manufacturing, 3) diffusing technologies throughout the sector, and 4) supporting research and de velopment (R&D) for commer cially important technologies (i.e, "strategic technology policy"). About one-quarter of the text is devoted to explanations of these strategies and the options that might constitute them. The scope of the options is extremely broad, ranging from expansive goals (e.g., balanc ing the budget and encouraging savings) to modest and very spe cific changes to existing antitrust law. Close to 50 options are dis cussed under three of the four strat egies. (While education and train ing are flagged as one of the four strategic targets for improving manufacturing, the identification and analysis of specific options is left to another report in the series.) Unfortunately, finding the options is like going on an Easter egg hunt: Options are hidden in the middle of sections with no identifiers other than sentences beginning with the phrase "Congress could .... "There are no summary tables or section identifiers to aid the hurried reader. Most of the options presented in Making Things Better had already been proposed. Reviews of the lit erature, bills already introduced, programs in other industrialized countries, and discussions with ex ecutive branch staff and other knowledgeable people appear to be the source of most. The strength of the report comes from "knitting together a network of possible ac tions" in the four areas in greatest need of policy attention. A common flaw in many OTA reports is that options are often identified but not analyzed.2 Mak ing Things Better endeavors to ana lyze many of the options proposed, primarily using three methods. In the section on financial policy, for example, the report relies on exist ing literature for estimating the ef fectiveness of such options as in vestment and R&D tax credits. Analogies to similar programs in other areas are also used to great advantage. For example, design of an industrial extension service re lies heavily on the model of the Agricultural Extension Service and the concept of a civilian technology agency looks to the Defense Ad vanced Research Projects Agency (DARPA). Finally, the report con siders policies in other countries as a source of insight on the effective ness of several of the options pro posed. (A comparison of industrial and trade policies in other countries is the focus of the last report of the series.) Only some of the 50 options are analyzed, however, and little at tempt is made to compare the op tions. Given limited time and re sources, one must make a tradeoff between attempting a fairly com plete list of options and analyzing and comparing a select few. This report opts for the former course. Policy project team interviews with Hill staff suggest that most of OT A's congressional clients would prefer that analysts choose the latter course (see ch. 3). Exception to the rule: Ownership of Human Tissues and Cells The policy project team found that, in rare cases, an OT A report con centrates on the first part of the story-with very little discussion of options-and is quite acceptable to congressional staff. Ownership of Human Tissues and Cells, which was done with a budget of about $150,000, may be such a report. A minuscule portion of this report (only five of the 168 pages) is de voted to "Policy Issues and Options for Congressional Action." Ownership of Human Tissues and Cells analyzes the legal, economic, 2 For a discussion of congressional views on the imponance of analyzing options in OT A repons, see ch. 3. For a summary of how OT A reports in the I 8-repon sample fared in this regard, see see ch. 4. -------------mf--------------

PAGE 67

and ethical rights of human sources of tissues and cells and also those of the physicians or researchers who obtain and develop these biological materials. The study describes the potential of three rapidly moving technologies (tissue and cell cul ture, cell fusion to produce mono clonal antibodies, and recombinant DNA) for manipulating human tis sues and cells to yield commer cially valuable products. At the time of its release, in March 1987, this study was antici patory. When the study was re quested, a court case was pending over who owns a cell line-the human source of the original tis sues or the scientist who developed the cell line. Ownership of Human Tissues and Cells alerted Congress that this case was not an isolated incident but rather involved a ques tion that might arise again and again because of the uncertainty of how the courts would apply current law to the new and unforeseen products of biotechnology. The court ruled in 1990 and apparently in a manner that did not stimulate further con gressional attention. Problem Identifying-Ownership of Human Tissues and Cells tells the story in a completely straightforward-almost textbook-manner. After a clear and quite concise summary, the report Jays out a six-page introduction that describes some of the key issues by presenting four disputes over ownership of tissues and cells, including the one, stiU pending, that led to the study re quest. A l 5-page chapter describ ing the technologies fo1lows. The policy context is then ex plained in two chapters. The first of these "'The Interested Parties," is an 18-page discussion of the stake holders: the sources of human tis sues and cells, the research com munity, and the biotechnology in dustry. The chapter explains how an eventual commercial product results from the contributions of both the sources and researchers, and often, many sources and re searchers. Itis followed by the single longest chapter in the report, on "Legal Considerations. "The report reviews the dozen or so areas under both the common law and specific statutes that are relevant, conclud ing that existing law does not pro vide definitive answers about own ership of the products of these new technologies. The report states that it is the uncertainty of how the courts might rule that is hindering the development of products using these new technologies. The report then presents three chapters on the key issues: "In formed Consent and Disclosure," "Economic Considerations," and "Ethical Considerations." In the first, the report presents the argu ments in favor and against disclos ing to the donor the potential com mercial gain from the use of his or her tissues or cells. Full disclosure would respect the right of the indi vidual, but one might not want a subject to prefer any medical pro cedure because of what might be a slim probability of commercial gain. The report then devotes a chapter to the tensions between two important economic considerations. Concerns for equity argue in favor of paying human sources; added costs of pay ments, and how this might slow down the development of benefi cial technologies, argue against. The last chapter discusses ethical considerations, addressing topics often not included in OT Areportssuch as religious perspectives-but which in this case are highly rel evant for helping Congress under stand the multifaceted problem they face. The structure and organization of chapters for the report's discus sion of the policy context and issues verges on the mundane-the tech nologies, interested parties, legal considerations.economic concerns, ethical issues-but is extremely successful. Ownership tackles what was then an emerging and thorny topic, explains who cares about it, and clearly states some important, policy-relevant bottom lines. The identical structure is used to organize both the summary chapter and the body of the report. Such parallelism is very helpful to the typical staffer who reads the sum mary and uses the remainder of the report as a reference document. The summary does not have a separate ------11------

PAGE 68

section on key findings but rather highlights these through effective use of boldface. Potential solutionS-Ownershipof Human Tissues and Cells devotes only five pages to "Policy Issues and Options for Congressional Ac tion" (as the last sectio~ of the summary). It is not one of the report's strengths. According to the project director it was prepared quite hastily. Along with the early stage of development of the issue, it is difficult to second-guess the staff's decision to allocate almost all of the time and resources to the policy context. Nevertheless, earlier and greater effort to developing and analyzing options would certainly have improved this section. CHARACTERS AND SUBPLOTS In the course of the OT A policy project, outside workshop partici pants, former OT A staff, and con gressional staff identified some specific aspects of OT A policy analysis that they thought were im portant and, in some cases, might be improved: treatment of directives in policy options, analysis of options, timeliness (e.g., the preparation of reports that take less time than the typical OT A assessment), stakeholder analysis, international comparisons or as pects of problems, institutional analysis, and legal analysis (see ch. 3 and 4).3 The policy project team found that some of the 18 OT A reports did a particularly good job of dealing with one or another of these spe cific aspects of policy analysis. A few of the 18 reports that might be used as models for dealing with these types of problems are identi fied below. Options as "empirically based policy prescriptions": Transportation of Hazardous Materials The policy project team was sur prised to find that many of the 18 reports in its sample included some type of policy directive in the policy options for Congress. In fact, seven ( 40 percent) of the 18 reports in cluded implicit or explicit policy directives (see ch. 4). Some of the directives in the seven reports seemed to reflect pref erences of OT A staff for reasons unstated. These the project team decided to refer to as "recommen dations. "4 On the other hand, some of the directives in the seven re ports seemed empirically basedi.e., supportable withdataandanaly sis in the report and not signifi cantly dependent on the decision maker's values or other prior be liefs. For lack of a better term, the project team decided to call these "empirically based policy prescrip tions." The policy project team judged that "empirically based policy prescriptions" were present and well supported in three reports (or about 15 percent of the 18 re ports in the sample) (see ch. 4).5 An example of an assessment with "empirically based policy pre scriptions" that seem reasonable to the policy project team is Trans portation of Hazardous Materials. This report offers a number of such prescriptions, presented largely as findings or options. There are no alternative sets of options for Con gress and other bodies, so by impli cation, they appear to be prescrip tions that should be acted upon. However, the justification for these "options" is so straightforwardbased on data, analysis, and a lot of J Another aspect of OT A reports that was viewed as important and needing improve ment was reader-friendliness (e.g., elements of organization and format that make reports easy to use). Reader-friendliness is discussed in the next section of this chapter entitled "Telling a Story Well." 4 Concerns have been raised by congres sional staff and others that including "rec ommendations" in policy options for Con gress may pose some risk to OTA's hard won and vitally important reputation for ob jectivity (see ch. 3). 5 Most of the congressional staffers interviewed for this project indicated that in the small number of cases where the evidence is overwhelming in favorof a particular pattern of options, OTA would be justified in pre senting what the team has called "empiri cally based policy prescriptions" (see ch. 3). ---------------111------------

PAGE 69

common sense-that the options seemneithercontroversiaJ nor 1ike1y to get OT A in troubJe. It is difficuJt to argue against them. The options in Transportation ofHaztJrdous Materials are divided into a number of categories: train ing (for emergency response and enforcement), reguJatory consis tency, databases, containers, and coordinatingprograms. Undertrain ing, the report presents as a finding the fo11owing: OTA concludes that a national strategy to provide an appropri ate level of haztJrdous materials emergency response training, either basic or advanced, to local personnel is an urgent priority. This "empirica11y based poJicy pre scription" is based on data, incJud ing data suppJied OT A by the lnter nationa] Association of Fire Chiefs and the FederaJ Emergency Man agement Agency. Cost estimates are made, but there seems to be 1itt1e doubt that the training courses given by state, 1oca1, and FederaJ agencies were not avai1ab1e to Jarge numbers of local emergency response personnel, particularly firefighters. Another "empirically based poJicy prescription" is found later in the same section: OTA concludes that an annual F ederalfunding level of approxi mately $5 to$7 million, added to $10 to $15 million derived from other sources and monies now being spent, could provide ad equate assistance, if existing resources are reorganized and tightly managed. ThisisdefactotheOTA-prescribed expenditure for training, supported by a cost analysis. Under regulatory consistency, an option is found that "Congress couJd authorize the development of [a national truck driver's Jicense requiring special training] with spe cial certification requirements for aJI hazardous materials, including gaso1ine." Again, no alternative is presented. Nevertheless, the idea of providing at ]east nationa11y con sistent Jicensing for drivers of trucks carrying hazardous materiaJ, ap pears dictated by common sense. In fact, a few years after the issuance of the report, such a requirement was put into FederaJ law. As a final example, the section on containers includes the fo11owing "finding": OTA finds that adoption of the proposed changes calling for stringent and more specific manufacturing standards, an nual leak testing of all cargo tanks, and stronger manhole covers on gasoline tankers, will improve the performance of cargo tanks. There are similarJy positive "empirically based po1icy prescriptions" in the section on coordination of Federal programs. Again, all are grounded in convincing argument and appear quite reasonab]e, but the format is c1ear1y a collection of poJicy prescriptions rather than al ternative options for action ( or in action). Analysis of options: Changing by Degrees and Mapping Our Genes Congressional staffers interviewed by the policy project team disagreed on the importance of including op tions in reports, but there was one point on which they were virtually unanimous: namely, that OT A should analyze the options that it presents (see ch. 3). Some stated this sentiment even more strongly: Do not bother providing a list of options unless the report analyzes their effects and effectiveness. Analysis of policy options was a problem in about half of the 18 OT A reports in the poJicy project samp]e (see ch. 4). Little or no detail was presented on the effec tiveness of the options for achiev ing the intended goals, the costs of options, unintended consequences, or who might support or object to the option. A few reports offered sets of options but gave so little information about their relative strengths or weaknesses that the reader was left wondering how to choose among them. ------111--------

PAGE 70

Thus, continuing on how OT A tells the story, this section looks at two aspects of analyzing options that are important to OT A's clients. The first aspect is analysis of the broad benefits and costs of options. Included here are estimates of ef fectiveness (e.g., improvements in environmental quality, health, in dustrial productivity, and delivery of services), costs and other eco nomic consequences (e.g., job im pacts), and other measures of what society gains and loses if the option is chosen. (Which groups might support and oppose each option was mentioned by several congres sional staffers as another helpful facet to include.) The second as pect is analysis of the organiza tional and institutional facets of options. These include which agency or other organization might best implement the option, how the new option might affect other ac tivities of the organization, neces sary interactions with other organi zations, and so on. As discussed below, Changing by Degrees: StepsToReduceGreen house Gases is an example of an assessment that spent considerable effort analyzing the effectiveness of a long series of technical options. Mapping Our Genes: Genome Projects: How Big? How Fast? is an example of a report that focuses on the second aspect, the choice of agency to implement the program. Broad benefits and costs of op tions-six congressional commit tees asked OT A: "Can the United States reduce carbon dioxide emis sions in the near term?" Changing by Degrees answered by quantify ing potential emissions reductions in six key sectors of the economy: buildings, transportation, manufac turing, energy supply, forestry, and food. In each sector, the assessment identifies the technical measures (e.g., more energy efficient prod ucts or use of lower carbon-emit ting fuels), and estimates the car bon reductions achievable if Con gress were to require or success fully encourage the use of the lower emitting product or practice. Changing by Degrees evaluated and compared about 100 different technical options. These were grouped into two scenarios: All "cheap and easy" measures, whose increased capital costs were offset by fuel savings, were grouped into a "Moderate" scenario. A second scenario considered more aggres sive "Tough" measures, i.e., those products and processes that are available, or close to being avail able today, but are more expensive or technically challenging. To put these two scenarios into context, a "Base Case" scenario, forecasting emissions under current laws and regulations and anticipated fuel prices, was also developed. The study considered a 25-year time horizon and used a simple emissions model of the U.S. economy as a basis for consistently comparing the reductions achiev able from, for example, auto fuel efficiency standards of 39 mpg by 2010, to stricter efficiency stan dards for residential furnaces and air conditioners, to a moratorium on coal use in industrial and utility boilers. The modeling was done in house and consisted of several large, linked spreadsheets. Net costs of the more aggressive "Tough" sce nario were estimated, but rather crudely. Regulatory approaches are, of course, only one type of policy instrument that could be used to lower emissions. Other approaches, such as fuel and carbon taxes, financial incentives, information pro grams, and R&D were discussed in the report, but not evaluated. Re sults from literature reviews were presented for these other policy in struments, but the vast majority of the analytic effort went to evaluat ing technical measures that, with a few exceptions, could be imple mented through regulatory ap proaches. Nevertheless, the report presented a large array of options for lowering emissions, with quan titative estimates of the potential of each. By assigning each of the measures to the "Moderate" or "Tough" category, Changing by Degrees gave at least a crude mea sure of the cost-effectiveness and difficulty of implementing each option. --------------------lie---------------------

PAGE 71

Organizational and institutional facets of options-In Mapping Our Genes, the central issue addressed by one set of options was where in the Federal Government to locate the genome research effort and how to manage it. The following possi bilities were identified: I designate a lead agency to coordinate work (the practical choices were the U.S. Depart ment of Energy or the National Institutes of Health); 2. establish an interagency task force; 3. establish a national consortium, including private sector partici pation; and 4. rely on congressional oversight of interagency coordination. Mapping Our Genes discusses each of these options in terms of poten tial advantages and disadvantages to existing agencies. A full chapter is devoted to analysis of the first option, the issue of which agency should take lead responsibility: chapter 6, entitled .. Organization of Projects." In addition to describ ing the options in some detail, this chapter cites historical analogs and compares them with the proposed structure. It also presents possible scientific advisory structures. Fur ther, the chapter outlines the issue of big science vs. little science in relation to a human genome project, together with the effect on this ten sion of the organizational structure selected to manage the effort. It treats the pros and cons of whether the genome effort should be a large, centrally managed science project, or a loosely organized effort by many small research groups. Alto gether, several different options are compared and analyzed in some detail, allowing the policymaker to make a decision based on a coher ent set of arguments. Doing more with less: Exploring the Moon and Mars Oneofthecomplaintsthatthepolicy project team heard most often from congressional staff was that two years was often too long to wait for an assessment (see ch. 3). Some Hill staffers expressed a desire for at least some fraction of OT A's work to be devoted to shorter, more focused assessments. Such assess ments are possible, albeit with an inevitable sacrifice of breadth, depth, or both. Of the 18 reports reviewed, Exploring the Moon and Mars was completed most quickly (about six to seven months), for the smallest budget (about $50,000, but a re quested study, not one produced at the Director's discretion), and with a staff of one. Nevertheless, it pre sented what appears to be a pointed discussion of an ambitious proposal by President Bush for establishing a permanent lunar base and sending human crews to Mars. The report presents some alternatives to the original Bush proposal to land hu mans on Mars by 2019, at an esti mated cost of,$300 billion to $500 billion. The OT A project director was quite experienced before undertak ing the assessment, having worked on four related studies previously. He had also studied space issues for many years, though had no famil iarity with the key issue of the report: manned missions versus use of automation and robotics tech nologies. His prior experience run ning assessments and his contacts in the field helped him successfully complete the assessment on a short deadline, as specified by the requesting committee. Exploring the Moon and Mars is much shorter than the typical OT A assessment, about 100 pages. The project director relied on one work shop, literature review, and many interviews with experts in the field forinformation. The strength of the report lies in raising a wide range of policy issues, from the very broad societal benefits explicitly or im plicitly ascribed to space explora tion to the more focused question of humans versus robots. Its chief weakness (inherent in any months long, solo effort) is that though these issues can be raised, they cannot be analyzed in great depth. The report was able to sketch three alternatives to the Bush adminis tration proposal, which primarily involve stretching out the proposed -------------------111---------------------

PAGE 72

timetable and relying more heavily on robotics. The strengths and weak nesses of these options are ana lyzed in cursory fashion. Given the time and resources devoted to the assessment, the re port includes an impressive amount of policy analysis that would ap pear to be quite useful in the early stages of debate on the issue. Though OT A is unlikely to adopt this "small and quick" approach as the stan dard for its assessments, Exploring the Moon and Mars does iUustrate that such an approach is possibleif a significant knowledge base (probably embodied by a single OTA staffer or program) exists in house and when the congressional timetable demands expedience. Stakeholder analysis: Power On! Several congressional staffers and workshop participants interviewed forthepolicyprojectsaidtheywould like to see more stakeholder analy sis in OT A work (see ch. 3). The policy project team found that about one-third (5) of the 18 reports in its sample did a very good or excenent job of analyzing the positions of different stakeholders in the analy sis of "context, findings, and is sues," but another third did only a poor or fair job in this area. In the analysis of ''goals and options," about half (8) of the reports in cluded some discussion of the sup port for options by, and the effects of options on, the stakeholders (in cluding the American public) (see ch. 4). Power On! is a report in which extensive stakeholder analysisdifferent perspectives on the same issues and on the effects of possible policy actions-is presented. This report had three main areas of em phasis: teacher training, software, and R&D. In the area of teacher training, the stakeholders discussed are teachers, local boards, and Fed eral education officials, students in general, and students with special problems. In the second area, soft ware producers are added to the list. University researchers and Federal funding agencies are key actors in the R&D discussion. Power On! is particularly con cerned with teachers' problems in introducing computer-aided in struction; it analyzes the issue ex haustively. OT A surveyed an states to learn about state and local efforts in the field of teacher training for computer use in the classroom, and thus provided information on the issue from administrators' points of view. The leadership role of the Federal Government was also dis cussed. From the "demand" side, the role of teacher training in com puters to meet the needs of elemen tary school students and of special (]earning or otherwise disabled) stu dents was presented in boxes. Under software, a long discus sion of the marketplace, copyright laws, and types of software produced provides a context for under standing the needs and concerns of the software producers. Similarly, the uses to which various types of software are put in the classroom are outlined from the teachers' and students' points of view. Pricing is discussed as a factor in acquisition decisions by local school boards and states. Costs play a significant role. In R&D, stakeholders such as the academic research community and the Federal funding structure (U.S. Department of Defense, Na tional Science Foundation, U.S. Department of Education) are the principal actors. The issue of how wen, if at all, computer-aided in struction actually can improve edu cation is vital to determine; re search and development constitute the vehicle for assessing the effect and designing improvements. The roles and needs of the various stake holders are outlined, together with options for changing their interac tions: increasing resources, trans fer ofR&D results, new initiatives. International context/ comparisons: Enhancing the Quality of U.S. Grain for International Trade Several outside experts who par ticipated in the OTA policy analy sis workshop noted that for under standing certain problems (e.g., global climate change), it is essen tial to consider the international context of those problems. More ---------------------111-------------------

PAGE 73

and more problems, they suggested, will fall into this category. One participant thought that OT A's at tention to international issues in its reports too often amounted to an "afterthought." Enhancing the Quality of U.S. Grain for International Trade is an OT A report that dealt in great detail with the way other countries ap proached the issue. Although inter national comparisons in the first volume of the report took up just one chapter of 11, this chapter was essentially a summary of findings of a second volume devoted en tirely to international comparisons. The OT A project staff contracted with teams that spent a few weeks in each of four countries (Argen tina, Brazil, France, and Australia) and reviewed each country's sys tem for assuring quality. The Cana dian system was also reviewed. Technical and policy aspects (pric ing, regulation, storage) were ana lyzed for all cases. Finally, the for eign examples were compared with the United States in terms of ap plied technologies and structures of policy and institutions. The pur pose was not to recommend that one or anotherofforeign techniques should be adopted, but to present a variety of other approaches for deal ing with similar export problems. The report notes that not all other approaches would be practical in the U.S. political and economic context, but some might be. Institutional analysis: Indian Health Care Analysis of institutions was identi fied by some outside experts at the one-day policy workshop as a weak ness in OT A's policy analysis. The policy project team found in the sample of 18 OT A reports it re viewed, however, that the reports were generally strong in their institutional analysis. One report that was strong in this area, for example, was Indian Health Care. This report first ana lyzes the legal history of the rela tionship between the Federal Government and the many Native American nations, including the provision of medical services. The report performs a comprehensive analysis of the structure and effec tiveness of the Indian Health Ser vice (IHS), which provides Native Americans with medical care. These institutional analyses, including a legally oriented historical discus sion and then a detailed description and investigation of the institutional problems of IHS, constitute a sig nificant portion of the entire study. The chapter on the Federal-In dian relationship begins with a three-page review of the legal his tory. The emphasis is on health services, from the beginning of trea ties between the United States and the tribes to the Indian Health Care Improvement Act of 1976. After this comes a lengthy discussion of how the Federal Government de cides to recognize particular tribes and individuals as eligible for health services. The point is made that IHS is the residual health care pro vider for Native Americans-after care from other governmental and private providers for which an indi vidual is eligible is exhausted. The fifth chapter of the report is entitled "The Indian Health Ser vice." Only IHS is the subject of this institutional analysis; no other governmental organizations are dis cussed. The description is devoted to the "context"half of policy analy sis, with remarkable depth and de tail of discussion. The programs for both direct care and contract care are described, with emphasis on eligibility, funding, staffing, ser vice delivery, operations and major issues. In addition, urban health projects, apart from reservation projects, are outlined. Finally, there is a discussion of the IHS facilities construction program, even includ ing discussion of methodologies for assessing the need for new and re placement hospitals and clinics. Ample data on cost and types of services and demands, along with trends, are given. A large number of tables and graphs support the discussion, and an appendix is de voted to data on cost allocations by year, area, and category. In the summary chapter, overview data on IHS are given, including (to orient the reader) the agency's location in an organizational chart of the U.S. Department of Health and Human Services. ------ml-------

PAGE 74

Institutional analysis is also cen tral to options in Indian Health Care. Since the report's options all involve changes related to the be havior of IHS, the discussion of options forms a long series of insti tutional analyses. The first three options focus on eligibility require ments for access to IHS services and on a possible consequence: making the services .. more of an entitlement program" than a residual source of health care. The next three options discuss means of reallocating resources within IHS to provide more uniform services across different areas served by lllS. There are three fiscal options to help IHS deal better with problems of high-cost care in the contract care program of IHS, one suggest ing that services sometimes be pro vided non-Indians, and three op tions revolving around the occa sional assumption of lHS responsi bilities by tribes. The analysis of all these possibilities is the most ex tensive example of institutional analysis in reports reviewed by the policy project team. Legal analysis: Finding a Balance In most OTA reports, legal analysis is not a critical component. The policy project team's review of 18 reports found only a few that had extensive legal analysis. One of these was Finding a Balance: Com puter Software, Intellectual Property, and the Challenge ofTechno logical Change. It was part of a series of OT A reports exploring the intellectual property law challenges presented by new information tech nologies, including the move to electronic representation of infor mation and the proliferation of digi tal means of transmission, adapta tion, and copying. Finding a Balance identifies three major policy issues: 1) the appropriate scope of copyright pro tection for computer software; 2) patent protection for software-re lated inventions and algorithms; and 3) complications facing libraries as well as commercial and private pro ducers and users of digital informa tion (including computer-based mixed media products). Given the focus of the first two issues, it is not surprising that several chapters of this report include legal analysis. Chapter 2, entitled .. The Law," gives an overview (goals, case law, statutory provisions, etc.) of three types of law in the United States that are potentially applicable to computer software and digital in formation: 1. U.S. patent law, 2. U.S. copyright law, and 3. trade secret law in the United States. The chapter notes that most intel lectual property protection for soft ware has come through copyright and trade secret laws, and some through patent law. Software de velopers and users, the courts, and policymakers have been locked in a continuing effort to sort out what should and should not be protected ( from a social perspective) and what is and is not protected ( according to current law). The patent and copy right systems in the United States are both administered under Fed eral jurisdiction, while laws con cerning trade secrets vary from state to state. For comparative purposes, the chapter includes three boxes that summarize patent, copyright, and trade secret law and their appli cability to computer software in countries of the Pacific Rim, Westem Europe, and Latin America. The chapter concludes with a sec tion discussing relationships among patent/copyright/trade secret laws and their applicability to computer software. Chapter 3, entitled "The Inter national Arena," examines the na ture of the global software industry, the issue of software piracy, and multilateral and bilateral negotia tions and treaties entered into to provide protection for international property rights (e.g., the Berne Con vention, the General Agreement on Tariffs and Trade, and the Euro pean Community's directive on le gal protection for computer software). It also describes efforts to harmoni:ze international intellectual property law. -------------------m--------------------

PAGE 75

Chapter 4, .. Software Technol ogy and the Law," outlines the ap plication of existing intellectual property laws to each of four iden tified elements of a computer soft ware program (program function, external design, user interface de sign, and the program code). It also discusses policy issues associated with the current level of protection and various policy positions ad vanced for maintaining or chang ing the scope of protection. The final section of the chapter summa rizes legal arguments that have been used to support these policy posi tions and evaluates them in light of OTA's own analysis of software technology. Chapter 5, .. Digital Information and Copyright," reviews the grow ing field of digital information, points up differences between digi tal information and information in more traditional forms, and dis cusses copyright issues for digital information. Much of the legal discussion in Finding a Balance is highly de tailed and technical. Though not readily accessible to some educated lay readers, the legal discussion should be comprehensible to Mem bers of Congress and their staffs, a high percentage of whom are law yers. It is certainly more accessible to lay readers than law journal ar ticles, digests, and the actual court decisions themselves. TELLING A STORY WELL: THE IMPORTANCE OF READER-FRIENDLINESS Even a good story may not be per ceived as such if the story is riot well told. In discussions with con gressional staffers, a theme that consistently emerged was that OT A should work more on elements of presentation to make OT A reports more easily comprehensible (see ch. 3). Congressional staffers are inundated with information from a variety of sources, they said, and if an OTA report is too difficult to read, they may put it aside in favor of one of the other documents that comes across their desk. Or, if they use the OT A document, they may not be able to get the full benefit of what it has to offer. Several staffers said they typically read only the summary of an OT A report; others said they use the summary to point them in the direction of a specific chapter in which they are inter ested. When asked to evaluate a sample of 12 OTA reports (three reports each), four former congressional staffers were very critical of many of the reports' reader-friendliness, reporting that that one-third of the reports had major problems in or ganization and format that made them difficult to use. The OT A policy project team's reading of a sample of 18 reports similarly found much room for improvement in this realm (see ch. 4). The team judged just over half ( I 0) of the 18 reports to be very good or excellent in terms of overall reader-friendliness; the other reports were judged "okay" or worse (see ch. 4). In some of the 18 reports, the findings and options were so buried that it was almost impossible to find them. In many reports, it was difficultto find where in the report detail on material pre sented in the summary could be found. More thanhalf(I0)ofthe 18 reports did not have an index. Three OT A reports in the 18report sample that had numerous features contributing to reader friendliness were Critical Connec tions, Enhancing Agriculture in Africa: A Role for U.S. Develop ment Assistance, and Exploring the Moon and Mars. Examples of the types of elements that contributed to the reader friendliness of these reports and other are listed in box 5-B. Additional models are pro vided in appendix E. Table of contents: an outline of the story The foreword and the table of con tents are often the first things a prospective reader of an OT A re port consults. A good table of con tents with sufficient level of detail at the front of a report gives the reader a general idea of the story the report is going to tell; a bad one leaves a prospective reader won dering from the outset what the -------------11-------------

PAGE 76

Box 5-B. Telling a story well: elements of reader-friendliness .:ManyfeatuiesofanOTAreport<:<>ntrlbutetoreaderfriendliness: .Some outie .features that have worked welUn previous OTA reports are noted below. Table of contents at the beginning of the report 1. coherent chapter organization that tells the out lines of the report's story 2. sufficient levet of detail presented Summary chapter 2. organization of options into strategies or other pac::kages. especially When there are large num. bers of options 3. box; figure. or table of options 4. effective use of bold or other type to present .options 5. effective use of bold type or callouts to emphasize 1111)0rtant points In the text 6. effective use of photo captions to emphasize im portant points in the text Overall ... Overallreport 1. astand--alonedocumentwith a fairly highdegreeof 1. coherent chapter organization, with some degree parallelism.between the summary and the Jeslof or para1lelism between the summary and other the report chapters of the report (or other techniques to '2. clear and engaging prose (newspaperormaga.. .facllftatelhetrackingofpointSinthesummaryto zine-etyteproseissspeciallyimportantintheSl.lffl." .. .... therast.of.tflerepor:t) .... maty.which is intended for a broader audience. 2: a e:hapte.l'(ordlapters)with the word "Options" than lhetestofthereporl) :~inthe.titletoletreadersknowwhere 3~ .effective use of headings to tell a story ... .. the options.appear ... 4. effective use of type. callouts. or other me8f\S to 3. t8 airnniary'.at:the beginning (or end) of each .,erq,hasizekey points ... : .. .. .. -~ 5. effectiVe useofgraphics 4 ... anintroducliontoeachchapterthattetlswhatthe 5; good ~.layout chapter':is;going to do .. .. .. .. > .5~ :parallel construction of similar types of chapters ... Problem kientifying (context. findings. i$sut!isJ i& effedive use of headings to ten a story .. 1. clearstatementoflhestudy'spurposeandllCOpe ... .. 7 effectiv9useofboxes 2. clear statement of congressional context in which 8 effective use of graphies .a request for the study was made 9; clear.and engaging prose 3. asectiori(orflgure)thatdescribestheorganization >10. goodpagelayout .ofthereport >11~ amethodofthestudyappendix(thismayinclude 4. effective.useofcalloutsfortypetoemphasizekey the congressional lettefsof request, a list of points contractorpapers,alistof workshopsandpartici-. 5. effective use of bold type to emphasize findings .. ... pants; etc.) 6 .. clear identification of policy-relevant conclusions .12.. ,a glossary of terms and abbreviations asFincings" . ... 7. box.:tableto surrmarlze key findings .. Jndex ~:::::::='-.. ......-: . ratfterJhantmdertechnologies or congressional ~ (e.g., oversight). ----------------,llf---------------

PAGE 77

story is going to be about. Several of the OT A reports read by the policy project team had tables of contents that left an uninitiated reader frustrated or confused about the story the report was going to tell. A good table of contents de pends above all on coherent chap ter organization. If there is no co herent story, the table of contents will reflect this. An example of a report in which the table of contents reflects coherent chapter organiza tion is Critical Connections: Com munication/or the Future. Despite the report's length ( 400 pages, 12 chapters), a reader can quickly grasp the outlines of the story Critical Connections tells by looking at the contents. The table of contents uses "parts" to group similar chapters. Two other reports with coherent chapter organization reflected in the tables of contents are Changing by Degrees: Steps To Reduce Green house Gases, and Ownership of Human Tissues and Cells. The OT A Publishing Office has begun experimenting with changes in the format of the tables of con tents of OT A reports. Recently, for example, some OT A reports have been published with a table of con tents at the front of the report that indudes not only the chapter titles but first level headings (see app. E for an example). In this new format, the individual tables of contents for each chapter are omitted. Summary chapter: a synopsis of the story The apparently heavy reliance on the summaries of OT A reports by congressional staffers (see ch. 3) underscores the importance of hav ing a summary chapter that effec tively communicates the key find ings, issues, and options of an as sessment. The summary also should give the context of the request for the assessment and dearly state the purpose, objectives, and scope of the assessment (see examples in app. E). According to congressional staffers, the dear labeling of policy relevant findings and options in the summary (and elsewhere in the re port) is a must(seeexamples inapp. E). A section or figure that de scribes the organization of the re port may be helpful in orienting readers to the report's contents (see examples in app. E). The OT A policy project team concurs with the 1987 OT A Writ ing Task Force that each summary chapter should be a document that can stand on its own as a substitute for the fu)I report. 6 It also agrees that each summary should contain the fo11owing: 1. enough background or introduc tory material to enable the reader to understand the subject matter and point of the report (usua11y Jess than in the full report); 2. the findings, plus enough of the logic to show those findings are supported, and 3. the options, plus sufficient ratio nale to demonstrate their cred ibility. This material should be written in a style that is clear and engagingmore in the style of newspaper or magazine prose than in the style of technical or academic journals. To some extent, the OT A policy project team differs with the OT A Writing Task Force on the structure ofthesummary. The OTA Writing Task recommended "a double structure-a cross-cutting discussion of the issues in addition to a straight summary of the report." The task force noted that a cross-cutting dis cussion often is "very difficult for the staff to produce, having worked within the structure of the fu11 re port for a least a year." The OT A policy project team believes that a double structure is acceptable but believes that-from the standpoint of making the summary reader friendly for congressional readerseffort is better spent trying to struc ture a11, or at least a substantial portion, of the summary chapter to parallel the rest of the report. Con gressional staffers interviewed for this project emphasized that it is extremely important for them to be able easily and quickly to track The "Report of the OTA Writing Task Force," headed by Audrey Buym, was issued in June 1987. -------------m-------------

PAGE 78

ideas and information in the sum mary in the rest of the report (see ch. 3). Parallel construction of the summary and the rest of the report is one of the best ways to facilitate such tracking. A "cross-cutting dis cussion" may be better policy analy sis, but to the extent that it intro duces a whole new conceptual framework and pulls things together in a way that the full report does not, it may very well detract from a report's reader-friendliness. When considering the benefits of writing a "cross-cutting discussion" for the summary, this is at least a point that should be kept in mind. The technique of parallel con struction of the summary and the rest of the report was used ( to a greater or lesser extent) by all of the most reader-friendly OT A reports in the sample of 18 reports reviewed by the policy project team. One report that uses this technique is Making Things Better. This report establishes key themes early and uses these themes to organize the entire report. The parallel structure in the report first introduces the reader to the major themes in the summary and then explores them in greater depth and richness, first in the policy options chapter, and then again in the back chapters. The staff er who is primarily interested in one of the policy themes (e.g., technology transfer) can easily find the material he or she seeks. Because the options are organized in a way that corresponds to the rest of the report, the skeptical staffer who wants to know why his or her boss should bother with some of the options presented can directly find the answer. Not all reports, of course, can be forced into such a structure, but when it works, it seems to be worth the extra time, effort, and discipline that it takes. Perhaps the discipline of organizing the problem and possible solutions into a common format may even iden tify flaws or holes in the analysis! Main body of the report: the whole story For the main body of a report, one of the things that aids reader-friend liness is coherent chapter organiza tion. Enhancing Agriculture in Af rica: A Role for U.S. Development Assistance, for example, grouped similar chapters in two main parts:. Part I: Low Resource Agriculture and Development Assistance (with four chapters) and Part Il: Promis ing Technologies (with six chap ters). A report that took a rather innovative approach to organizing the chapters was Critical Connec tions. A "conceptual framework for analyzing policy issues engendered by new communication technolo gies" is depicted in a figure in chap ter 2 (fig. 2-3 on p. 32) and is used to provide the rationale for the scope and structure of the report. The organization and the subjects of the chapters reflect th~ flow and logic of this model. For congressional readers pressed for time, it is extremely important that a report's findings and options not be buried or hard to find. To help congressional readers find a report's policy options, one approach is to use "Options" or "Policy Options" in the title of any chapter or chapters that contain them. In most OT A reports, the options appear only in the first chap ter. Examples of names for chapter 1, when the options are presented in that chapter, include the following: Summary and Options Summary, Issues, and Options Summary and Policy Options Findings and Policy Options In reports in which policy options are presented and analyzed in detail in a chapter other than chapter 1, again it is helpful if the labeling of the chapter reflects this. Examples of names for such chapters include the following: Policy Issues and Options Policy Options for Enhancing Grain Quality Policy Initiatives To Improve Cleanup Prospects ------------lie--------------

PAGE 79

Index: the nitty gritty details of the story Many of the congressional staffers interviewed by the OT A policy project team indicated that an in dex was essential or extremely use ful to them. Some OTA staff swear by them. Of the 18 reports reviewed in the policy project, fewer than half (8) had an index. Serious Re duction of Hazardous Waste and Enhancing Agriculture in Africa were among the reports that had an index. One advantage of an index, if it accurately reflects the content of the report, is that the reader need not depend solely on the table of contents. Quirks in the organiza tion of material can be overcome by locating key words and phrases in the index. In this sense, an index is another map to the report's con tents. The reader with a specific need can probably locate details most quickly by consulting the in dex. ___________ _Jm------------

PAGE 81

This chapter summarizes what the OT A policy project team has learned from a va riety of sources about how OT A's culture-especially its formal and informal or ganization-shapes OT A re ports.1 The concern is for or ganizational clues that may help explain the strengths and weaknesses of the policy analysis found in individual OTA reports. Previous chapters have il lustrated the diversity of styles that characterize OT A re ports. The culture of OTA is highly pluralistic, and OT A reports reflect this. 2 Notably, however, many OT A reports share certain features. Un derlying all OT A policy analysis is something known as "the OT A as sessment process." The OTA as sessmentprocess, which is familiar to participants in OT A's work and recognizable by those outside, tran scends individual congressional committee requests for studies, pro grams, project staff, and reports. The process involves two general types of activities: 1) staff activities (e.g., planning a study, choosing contractors and an advisory panel, synthesizing various. types of infor mation, writing and revising, brain-CHAPTER SIX --.T-. .. .. ..h' .... .. ? .::: :: _. stormingaboutoptions),and2)pub lic activities (e.g., advisory panel meetings, workshops, external re view). Observations related to OT A's culture from a variety of sourcesfive former OT A project directors (see app. A-5), IO outside observ ers who have participated in the assessment process as mem bers of advisory panels and workshops, reviewers, and contractors (see app. A-6), and a dozen first-time OT A project directors-are sum marized below. First, how ever, we offer a profile of OTA's analytical staff. STAFF PROFILE A snapshot of OTA's per manent analytical staff, at least in terms of disciplinary makeup, may provide some insights into OTA's capability for doing policy analysis. OTA's per manent and temporary research staff, including program managers, numbered 131 as of December 1992 (for a summary, see table 6-1).3 The staff can be character ized first by degree and then by broad disciplinary field: 1 The "culture" of an organization consists of its fonnal structure (e.g., the divisions and programs in which the organization does its work) and its climate. The "climate" of a culture refers to the mood, the in-house tensions, and the rumors of which folklore is made. Climate also captures the ways individual staff experience, negotiate, and accommodate the demands made by the organization. 2 Many units of analysis-division, program, project-could help explain some of the agency characteristics discussed below. The scope and time frame for this study, however, precluded the policy project team from learning much about the various subcultures that thrive at OTA. 3 This number excludes in-house contractors and detailees. The core research staff may differ significantly from other categories of staff, but it is also the segment with the longest tenure and experience with the OT A assessment process. --------------Im------------

PAGE 82

About 28 percent of the staff hold, as highest degree earned, a B.S. or B.A. degree; 25 percent hold an M.S. or M.A.; 37 per cent have a Ph.D.; and IO per cent have either an M.D. (n 3), a J.D. (n -9), or both (n -1 ). Natural science and engineering disciplines are most prevalent at all degree levels, accounting for 55 percent of the Ph.D.s at OT A, 42 percent of the M.SJM.A.s, and36percentoftheB.S./B.A.s. (Fiveofthe 13OTAstaffwithan M.D. or J.D. have their next highest degree in one of the natu ral sciences or engineering.) Socialscience expertise (includ ing psychology) is found among 20percentof the Ph.D.s at OT A, 39 percent of the M.SJM.A.s, and 19percentoftheB.S.IB.A.s. Economics, long rumored as un dervalued at OT A, is formally represented among 9 percent of OT A's staff in the following dis tribution: one Ph.D. economist, three other Ph.D.s with an economics baccalaureate or master's degree, and eight addi tional staff with a B.S. and/or M.S. in economics. A newer category of credential, which varies in the amount of natural science and engineering versus social science content, is the "policy degree." Policy de grees are almost always hybrids ( e.g., technology and policy, health policy, and energy man agement and policy) and are typi cally not offered at the B.A. level. Twelve percent of the Ph.D.s and 15 percent of the M.S./M.A.s represented on the OT A staff are policy degrees. In all, then, 45 percent of OT A's research staff have, as the highest degree earned, a natural science or engineering degree, 25 percent have a social science degree, and 8 per cent a policy degree. The remain ing 22 percent have degrees in the humanities (e.g., history, philoso phy, English literature), business, and a myriad of other fields (e.g., communications, education, and social work).4 The lesson to be drawn from these figures is that many courses of study lead to doing policy analysis at OTA. No doubt some of the dimensions identified as "excellent" in the sample of 18 OT A reports evaluated in this as sessment can be traced to different ways of looking at the world. OT A staff usually work on project teams of two to six people. Thus, it is the combination of back grounds, talents, andexperienceswhat might be called "complementary strengths"-that is likely to be reflected in OT A reports. The project directors interviewed for this assessment, neophyte and ex perienced alike, often commented on the roles that individual staff play in an assessment. Individuals' roles depend on far more than de grees and disciplines. Yet degrees and disciplines are a proxy for some of the dimensions of the assessment process and report style that form the core of OT A analysis. About three-fourths of the 18 studies re viewed by the OT A policy project team had some mix of natural sci entists and social scientists on the project team (see ch. 4). This mix was viewed as a plus for good policy analysis by outside experts and former OT A project directors. The policy analysis in a given OT A report is shaped not only by the team members' educational backgrounds but also by things such as their life experiences, personali ties, and roles on a project team. In view of the importance oflife expe riences, it may be important to note that few OT A staff come to the agency with exposure to the Hill. A lack of exposure to the Hill was cited by some former OT A project directors as a flaw in the recruit ment of staff to OTA; such experi ence, one former project director suggested, could help reduce the "academic" aspect of the OT A culture. Some observers have also noted that OT A's research staff includes few U.S. minorities (e.g., African Americans and Hispanics). To the extent that racial and ethnic diversity extends networks, it may affect the composition of OT A ad visory panels and selection of con tractors. Furthermore, the lack of diversity is viewed by some as con stricting novel policy insights in certain areas. Staff with M.D.s and J.D.s are categorized by their next highest degree. --------------------~-------------------

PAGE 83

Individual creativity, project group dynamics, and on-the-job ex perience mingle, through the OT A assessment process, to make each OT A report a unique document. A considerable amount of interper sonal transfer of skills and knowl edge goes on in every project. Dif ferent members of a project team with expertise in certain aspects of the assessment (e.g., scientific or technical) learn from other mem bers of the team, as well as other sources, about aspects of the topic with which they are less familiar (e.g., legal, ethical, or economic). OT A staff without training or prior experience in policy analysis may learn about doing policy analysis from colleagues on the project team. Staff without exposure to Capitol Hill may pick up tips from their colleagues about writing reports that are responsive to congressional needs. Most new project directors at OTA learn how to conduct an OT A policy analysis, and report the results in an option format, infor mally as members of OTA project teams. The contribution made by an individual depends on things such as whether the individual is the project director or a research assistant, how creative the person is, how good the person's interpersonal skills are, and what the person's life experiences have been.' OTA has a core cadre of experi enced research staff. The collective skills of OT A project staff, program managers, and assistant di rectors in the agency represent a cornucopia of policy analysis skills and knowledge. As discussed fur ther below, however, some former OT A project directors, policy work shop participants, and new OTA project directors have suggested that the culture of OTA does not ad equately facilitate the transfer of policy analysis methods and skills across programs and divisions. The policy project team's review of 18 OT A reports seems to lend cre dence to this observation (see ch. 4 and 5). OTA ALUMNI Former OT A project directors are "alumni" of the assessment pro cess. Upon their departure, they typically join other policy organi zations and are therefore well-posi tioned to view the comparative strengths and weaknesses of such organizations. As long-time par ticipants in the OT A process, these alumni may be particularly percep tive of the agency's culture. Some of the five former OTA project directors consulted for this project (see app. A-5) suggested that the climate of OTA is not effective in transmitting policy know-how to those deep in the trenches of project work.6 One former OTA project director put it this way: There is no incentive in OTA, but strong disincentives, for cross program planning, project review, report evaluation, collegi ality. There is no significant in centive ... to present briefings on completed or finished work, to relish or enjoy constructive feedback from one's colleagues. The organization is without any internal program for cross learning. As a result, there is no cross-over of policy generation, no cross-over of creativity, and no significant pressures to ori ent the future into the work. In the words of another former OT A projectdirector, "OTA policy analysis is often too 'safe' because OTA staff are not risk takers." A third OTA alumnus sees this as a tradeoff: "Look at hiring policies. Are there too many inexperienced people being brought in at the high 'One current OTA staff person asks point edly: "Why doesn't OTA, like most other research and policy organizations, bring in experienced people, in their forties and fifties, who have worked at other centers? Why don't we try to recruit people who have retired after a lifetime of service in govern ment, academe, or industry? I think in large part because they would not buy into the "culture" of how wonderful and smart we are and would raise questions that are embar rassing for the established order .... The common reason given fornotrecruiting many senior people to OT A that the agency can't afford to pay their salaries. 6 As discussed at the 1992 Senior Manage ment Retreat, "people issues" pose challenges throughout the agency to develop staff competencies and facilitate intell)TOgramcooperation. See the Rockwood Retreat Rapporteur Reports, especially by Phyllis Windle, Nov. 16-17, 1992, pp. 3-5. ------------mt------------

PAGE 84

cost of internal training and partial education?" Risk-taking or risk aversion, one could argue, is a pos ture learned and reinforced by the environment in which it is prac ticed. Decisions to encourage or discourage risk-taking are doubt less project-specific but also be come associated with the agency as a whole. Decisions made both at the program level and at the top of the agency will produce a climate that values certain intellectual charac teristics and devalues others. Tangible evidence of a desire of current OT A staff for greater shar ing of policy analysis methods and skills is found in the creation of two voluntary grassroots organizations-the Project Directors' Peer Group (1989) and Research Assis tants in Search of Empowerment or RAISE (1991). Each met on an irregular basis to discuss issues of mutual interest, i.e., to enlarge the community of staff whose respon sibilities are unique. RAISE even issued a handbook for all incoming research assistants. It is a maxim that any group will try to act to fill needs not addressed by the formal organization. However, relying on this mechanism may be neither ef ficient nor effective when it comes to improving policy analysis. Several former OTA project di rectors said there was a need for improving the transfer of policy analysis skills across the agency. Mentoring was mentioned as one possible approach. One former OT A project director commented: [T]here is need for more --mentori.ngofOTAanalysts.One aspect of the OTA culture which is troubling is that the older, [ or] wiser people do not seem to train and influence the younger staff effectively. For example, senior managers and the most experienced and successful policy analysts (possibly from different programs and divisions) should balance their concerns about liabilities which can sink the ship [with] more positive influences on staff doing policy analysis and writing reports. One current OT A staffer noted that one reason more mentoring does not occur is that OT A's existing organizational incentives do not encourage it. The five former OT A project directors also made comments re lated to how the culture of OT A shapes the policy analysis in OT A reports. One former OT A project director commented: The academic backgrounds of so many OTA staff and their lack of practical policy experience has shaped OTA's culture in general and especially its policy analysis. The lack of exposure to the Hill is seen by some former OT A project directors as a fundamental flaw in the recruitment of staff to OT A-with serious implications for the orientation of the agency's work: ... too many OTA staff have no .real congressional experience or insights into the lives and needs of members and senior staffers. There should be oppor tunities to work on member and committee staffs. Such experi ence could greatly reduce the "academic" aspect of the OTA culture. Although the causal link may be challenged here, the concern that the academic mode of discourse differs significantly from the con gressional mode is legitimate. It is the difference between writing for a scholarly (and typically, disci plinary) audience and writing for decisionmakers (and especially their staff) who want to act on what has been found through analysis. WHEN OUTSIDERS LOOK IN Knowledgeable outsiders provide a different mirror in which we can view the culture of the agency. In November 1992, IO outside experts in different fields, each of whom was familiar with some aspect of OT A's work, participated in a one day policy analysis workshop (see app. A-6). Much of the commen tary from the workshop participants (some provided in memoranda af ter the event) centered on the cur rent and future organizational struc------ml--------

PAGE 85

\ >rable.&,;1. Distribution cf OTA'stesettrch staff'~ by degree and broad field ture of OTA rather than on the particulars of policy analysis. The perceptions of these outside ob servers, who are most likely to f ocus on the outcomes of the assess ment process, are certainly worth considering. By and large, the workshop par ticipants were of the opinion that what OTA does well (e.g., compe tent and objective analysis of the science or technology that may have economic,regulatory,ormoralcon sequences), it does very well-and probably better than other policy organizations. Despite the many accolades for OTA, however, there were criticisms. For example, one participant observed that .. the whole of OTA is less than the sum of its parts ., Several workshop particin.:. :13 44 27 \i::is9 ,~~: /~i~nce .. in \1:I/.: ;10 ;3 \):33:: .. 1:25~::. pants noted that OTA's structure may promote or inhibit the experi ences and habits of mind that con tribute to policy analysis: Internally, the agency manifests a fragmentation and lack of or ganizational learning that is a concern. This governs not only the culture of communication and learning about how to do policy analysis, but also proce dural issues, e.g., how could stakeholder analysis-who gains and who loses from the creation, application, or location of a tech nology--be improved? ... In house seminars will not suffice in addressing these needs and 'reinventing itself in its third decade of service to the Con gress. Policy .. -.science Other n n 0 ... 5. 16 1 6 5 6 0 11 28 22% .\ 8%. .. A few workshop participants were concerned that there was insuffi cient attention to racial and ethnic diversity. That view is reflected in the following comment: I am concerned that there is not an appreciation of the intellec tual reasons to make sure that staff and advisory committee [panel] diversity receive greater attention within the agency. Who the staff and AC [panel] mem bers are make a big difference in the way arguments are framed, whatnetworksaretapped,what research is accessed. I think OTA should be concerned as much with the present makeup of staff, for example, as with the lack of a process to affect that makeup. ------11------

PAGE 86

Most of the policy workshop par ticipants, as well as the former OT A project directors consulted in this project, recommended broader thinking about: 1. what constitute the tools of policy analysis at OTA, and 2. how the agency's structure can introduce these tools formally to the staff. Nobody doubts that policy tools are a valuable input to the assessment process. The concern is how the agency culture facilitates the learn ing that would benefit the staff, especially new analysts for whom both the process and OT A itself are still relatively abstract notions. One policy workshop participant high lighted the importance of finding a balance between opportunities for staff development and service to congressional committees: ... OTA should devote more resources to staff development. Staff feel that they are being "mined" and need time that need not be "billed" directly to a project so that they can catch up on progress in their fields. Among other things, this would allow staff more time to follow up on reports that have already been completed. Finally, workshop participants of fered additional pieces of advice: OTA should try to get new ideas/ options into its reports. The op tions should reflect more ereativity. This may involve, inter alia, more effective cross-disci plinary work. Staff should become more self conscious about-and therefore more rigorous in analyzingthe normative choices they make when they select research meth ods and data and when they cre ate narratives about informa tion. Staff would be more receptive to using the rigorous methods of the historical social sciences to assess the institu tional, organizational, and po litical dimensions of the issues they are studying. OTA simply must direct more creative attention to the means and attitudes for taking advan tage of its collective experience in a)attending to its responsibil ity to younger staff (who are thrown pretty much into the breach at present with little overarching OTA guidance), and b) its increasing obligation to show intellectual, as well as policy analytic, leadership in this field .... Activities taking smallish steps in that direction are those that strive to integrate association across groups, in crease the willingness of junior staff to learn from (and ques tion) each other and more expe rie~cedanalysts, take the chance of appearing vulnerable, and testing half-formed, tentative analyses and conceptions well before they become crystallized (and ego bound). FIRST-TIME OTA PROJECT DIRECTORS How well--0r poorly-does OT A help staff become productive project directors? Asking some first time project directors reveals some special sensitivity to the culture of the agency. The policy project team's interviews with a dozen re cent first-time OT A project direc tors supports the impression by outside observers that OT A is not always adept at transferring ana lytic skills across its divisions, pro grams, and project teams. New project directors become team leaders for various reasons. Most new project directors, how ever, have worked as project staff on at least one or two OT A studies before being given the opportunity to direct an OT A project them selves. Thus, becoming an OT A project director is usually a "pro motion" within the organizationa promotion that can be based as much on need (there's no one else to do it) and a program manager's comfort (e.g., with the staffer's personality and sty le) as on the new project director's analytical or ad ministrative acumen. That first-time OTA project di rectors do as well as they do may be more a tribute to their bootstrapping capabilities and perseverance than ---------~-~----------

PAGE 87

to an organizational culture that strongly encourages the sharing of policy analysis skills. After having worked on a project or two as team members and just learning by "os mosis," some new OT A project di rectors are left to fend for them selves in directing a project. It is no surprise if, during this ordeal of learning-by-doing-under-duress, some new project directors do not acquire the fine points of style. Many newprojectdirectorsarevery resourceful in seeking and getting advice and support from staff within and outside their program. Even so, a number of new project directors interviewed by the policy project team said that they would like additional help and support. Directing an OTA project that produces good policy analysis is a challenge even for many old hands. For new project directors, the task is sometimes more daunting. It re quires, among other things, that they figure out unwritten program norms,7 overcome communication barriers, and shed inhibitions about appearing unprepared or ignorant of certain procedures and customs in the OT A process. Most new project directors observed that the teaching and learning of policy les sons is informal, haphazard, and ultimately dependent on personali ties and a willingness on the part of experienced OT A staff to mentor staff with less experience. Some new project directors, typically those who are outgoing, reported that they had successfully sought out individuals within the agency (sometimes outside their own pro gram) who had been extremely help ful to them, offering support and/or advice on techniques and methods for policy analysis. Other new project directors, however, reported feeling extremely isolated and un supported. In the absence of other help, some project directors resort to ''trusting the [OTA] process," hoping, for example, that the inten sive OT A review process will keep them from getting too far off the track. The new project directors interviewed by the OT A policy project team typically gained most of their experience in policy analysis while at OT A. With near unanimity they reported that they would have liked more contact with congressional staff and more guidance on how to deal with them. Congressional re lations remain a bit of a mystery for some new OTA project directors; others complain that senior OT A staff"hoard" their connections with congressional staff. Some program managers restrict interactions with congressional staff to a few senior people in the program. A number of new project direc tors interviewed by the policy project team said they felt some what adrift, especially during the early phases of their assessments. Many new project directors spe cifically expressed the desire for a more formal mentoring program. IMPLICATIONS FOR POLICY ANALYSIS OTAhasa20-yearhistory and folk lore about its staff and its process. The truism that one "hires the best people and then stays out of their way" is subscribed to by many at OT A. Indeed, the policy project team found that the in-house work environment is simultaneously col legial and insular. In general, OT A staff are not very reflective about the structure and climate of the agency. Most of the analysts move laterally from project to project, carried by the intellectual challenge. Thus,asmuch as OT A studies probe how technology works, the staff tends to be indifferent to asking how OT A works. Stepping back from the workaday process of"tech nology assessment" can help to clarify where the folklore differs from actual practice. Furthermore, it can aid intra-agency learning. Collegiality at OT A is practiced in various ways. Intellectual interes~nergy, environment, educa tion and training-seem to define functional areas and also relation ships that cross program and divi sion boundaries. The power of per sonality and interactive style con-7 OTA programs differ in their unwritten program norms. An example of an unwritten norm in one program is that one should never use the word "should" preceded by the word "Congress" in a repon; other programs observe an unwritten norm that prohibits making recommendations in any fonn. ------11------

PAGE 88

nects staff informally, wherever they may be located in the agency. But the structure inhibits formal collaborations and even routine transfer of policy knowledgemethods, experiences, tools, etc. -across OT A's programs and divi sions. Insularity dominates collegial ity when it comes to policy analysis at OTA. Staff capabilities are not easily tapped from the inside. Ironi cally, such capabilities are solicited-especially through advisory panels and workshops-from the outside. But OTA staff seldom get the benefit of hearing from experts beyond those participating in their own program's assessments. In other words, there are missed op portunities for otganizational leam ing. As one former OT A project director stressed: [T]here should be some cross fertilization of projects. More discussion of each study being undertaken by a program needs to take place within the program andwithotherprograms.l know these efforts have been tried at various times, but they still have merit, if the time constraints and lack of interest in doing such extra work can be overcome. My overall point is that the more discussion and review takes place, the better will be the policy analysis. Every organization seeks a man ageable level of creative tension that propels its work forward. This level may vary by program and certainly among members of a project team, but it is healthy and, if properly marshalled, can have synergistic effects on many others in the organization's environment. Does OT A create such synergy? In pockets of programs, yes; during some stretches of time, yes. Can we do better? Probably. Certainly the climate can alter staff attitudes to ward "doing a better job" in the assessment process.8The advice of outsiders familiar with the process (and OTA's history) reinforces many of the perceptions and expe riences of the OT A culture by cur rent and former staff conveyed to the policy project team. This assessment of OTA culture leads the policy projectteam to ask: How willing is the agency to draw what is done informally into its formal organization to enhance the practice of policy analysis within and across programs and divisions? Although current OTA staff may disagree on the particulars, few would deny the following words of one of the agency's long-time out side observers: "Many things about OTA make cross-program work much harder than it ought to be .... Indeed, many current and former OT A staff see the agency as suffering from: a staff constrained by an organi zational culture steeped in its folklore and uncommitted to sharing-teaching and learning-the lessons of plying the policy analysis craft, work structured into niches that honor an historical division of program labor, and a climate that seldom supports mentoring of staff and experi menting with the formation of project teams (as was done in this project). If these judgments are correct, then perhaps somebody should enter tain alternatives to the way OT A's staff is organized. In OT A, as in any organization, the staff must look up, not down, for that "somebody." Perhaps the exhortations of the in siders and the outsiders should be interpreted as a need to "experi ment." As OT A enters its third decade, the job of policy analysis is not 8 One person at the 1992 Senior Management Retreat (Peter Johnson) provided seven op tions, including term limits for OTA pro gram managers, required rotation of program managers and senior associates, and designating one program to do cross-pro gram work (Rockwood Retreat Rapporteur Reports, p. 4). Another approach is outlined in a recent proposal to establish an "OT A Institute" that would bring in, for short peri ods (weeks to months), experts of interna tional stature in technology assessment and policy analysis. These experts would work, on a rotating basis, with senior OT A staff, providing stimuli in the form of seminars, collaborative analysis, and opportunities to prepare policy studies for joint publication. The cross-fertilization would be a two-way street-OT A influencing the audience of academic policy research practitioners, and OTA staff approaching the assessment pro cess with additional perspectives and ana lytical tools. See Todd M. LaPorte, "OT A Institute" Memorandum, Dec. 12, 1992. --------------------mi---------------------

PAGE 89

getting any easier. The expecta tions of congressional committees that request OT A studies keep ris ing. OTA staff are regularly heard to remark that they are expected to do more, better, faster-without compromising the integrity of the assessment process. They refer to it as a problem related to their current project, but it is more accurately an agency-wide issue. It will ultimately take the agency's collective will to act on it and, above all, some con structive management. If the above criticisms are accu rate, then OT A would continue to serve Congress in reliable ways, but get no better. Change, however, would require taking periodic critical looks at ourselves, our routines, and the people who perform policy analysis. It means being more re flective and receptive to OT A staff needs while effectively drawing on their skills. To do this assumes a culture that fdsters staff satisfac tion and team morale. That culture must resist inertia and self-satisfac tion with what is done, how it is done, and who is doing it. Some of this energy can be sustained at the program level, but much of it origi nates at the top as a central value to be cherished and nurtured. ---------,m------

PAGE 91

APPENDIX A OTA polic_y project: goals and study plan PROJECT GOALS The OT A policy project's goal is straightforward yet ambitious: We hope to produce a document that can be used by OT A staff to improve their policy analysis skills. We hope to learn useful lessons from some of the better OT A assessments, as wen as to gather insights from current and former OT A staff, congressional committee staff, and outside observers of OTA. We will not attempt to produce an agency "report card" on how well we are currently doing. Regardless of whether one believes we currently do a good job or not, clearly we can do better. We do not aspire to define what policy analysis is. (Fortunately, we are not alone. Aaron Wildavsky. in the introduction of his text on policy analysis, Speaking Truth to Power, raises the question, '"How can you write a book about a subject if you can't say what it is?" He does anyway, and so will we.) We wi11 proceed pragmatically. Our focus is on the issues that we raise for congressional attention and the options proposed to address them. We will not be focusing on those parts of our reports that examine whether a gizmo will fly, make sick people well, harm the environment, help us compete in world markets, and so on. These parts of our reports are often extremely policy relevant and vital to our work but are not the focus of this inquiry. OTA reports inform Congress about what they can do to change something and in many cases, what and why they cannot. We will focus on how we transmute an that science, engineering, economics, political sci ence, law, etc. into an assessment that tells Congress about the problems they face and potential solutions. We will not consider whether our reports have impact; nor will we examine how to communicate our findings in ways other than reports. Obviously such questions are important, but they are beyond our charge. We want to know how we figure out the story and the ways we tell it in our assessment reports. Of special interest are the chapters and sections with "policy" in the title. Although we will not attempt to define what policy analysis is, if coerced, we might draw from Percy Bridgeman's definition of science as "the activity of scientists" and say policy analysis is the activity of policy analysts-those who work at OT A and else where. By looking inward for a brief, three-month period, we hope to find out what we actually do and share what we learn with the rest of the agency. STUDY PLAN The OT A policy project will run for three months and will be staffed by three full-time and two part-time OTA staff members. The assessment has a 14-member advisory panel composed of OTA program managers and senior research staff representing all nine OT A programs and the Congressional and Public Affairs Office (see app. A-1 ). The advisory panel is chaired by a former OT A program manager who is now with the Congressional Research Service. -------------11..---------------

PAGE 92

The study will be divided into two complementary efforts explained in further detail below. The first effort (Task #1) will be an examinationof27 (or possibly 18) of OT A reports by the in-house project staff and interviews with the project directors of those reports. The second effort (Task #2) will be the solicitation of other views of OTA policy analysis through means such as short essays by former congressional and OT A staff, interviews with current congressional staff, a workshop with a carefully chosen group of outside experts, and reviews of the literature on policy analysis and earlier examinations of OT A. Slightly more than half of the staff time will be devoted to the first effort (see app. A-2). Task #1: Examination of OTA reports (see app. A-2) To get a list of 27 good OT A reports, the OT A policy project team has asked each program manager to nominate three of his or her program's reports that, in the program manager's opinion, were examples of "good" policy analysis. In selecting reports, the pro gram managers were allowed to use their own concep tion of "good policy analysis" with the following stipulations. First, we asked them not to let their choices be influenced by legislative use of the report. Second, we said we preferred reports where we could easily interview the project director. Finally, to remove a possible source of bias in the review, we said that the reports named should not be ones for which the mem bers of the OTA policy project team had had major responsibility. Each of the OT A reports that is reviewed will be assigned to two members of the OT A policy project team: one person will have primary responsibility for the review; the other will act as backup reviewer and independent check of the conclusions of the primary reviewer. Tony Fainberg, Bob Friedman, and Kerry Kemp will be the primary reviewers for nine reports each, all of which will be outside their areas of exper tise. Daryl Chubin, Tony Fainberg, Bob Friedman, and Kerry Kemp will be backup reviewers, with as much diversity in pairing as is possible. Backup reviewers will be chosen for their familiarity witl:! the assessment subject matter where possible. Each report will be reviewed using a common set of questions as a guide. Given the limited time available for this study, primary reviewers will spend two days reading the report; backup reviewers, one day. In the event that we decide thatthis amount of time is inadequate to the task, one assessment per program will be dropped. The project director for every OT A reports that is reviewed will be interviewed prior to the review of the reports. If necessary, the project directors may also be interviewed after the review of their reports. Task #2: Other views of OTA policy analysis (see app. A-3 through A-6) We are soliciting other views through a series of contract and in-house research efforts: Task 2-A: Evaluations of OTA policy analysis by former congressional staff We have identified four former congressional staff ers who were both 1) quite familiar with OTA, and 2) considered thoughtful about the policy analysis needs of the Congress (see app. A-3). The list includes critics as well as supporters, with staffers from both Houses and parties. The former staffers will be given several OT A reports (probably three from the list recom mended by the program managers). We will ask them 1) to specify the criteria that they would use to judge "good" policy analysis, and 2) proceed to evaluate the reports. We expect short papers and will pay them a small honorarium for their services. Each former con gressional staffer will be interviewed by two members of the project team after his or her short paper is submitted. Task 2-B: Interviews with current congressional staff Insights gained from task 2-A will be tested and explored further through interviews with current con--------------------11--------------------

PAGE 93

gressionaJ staff. We wi11 interview at ]east 10 current staffers, chosen from committees that are frequent requesters and TAB staffs, with a good mix from both parties and Houses (see app. A-4). Task 2-C: Retrospectives on policy analysis by OTA alumni Former senior OT A staff now with other organiza tions that do similar types of analyses present a unique opportunity for this study. These individuals under stand OT A yet have the benefit of some time away from the agency and experience with how analysis is done in another organization. For this part of the project we have chosen five former OT A staff members who directed OT A studies and are reputed to be among the most accomplished policy analysts who have worked at OTA(seeapp. A-5). We wi11ask them for their thoughts on how our policy analysis can be improved. Task 2-D: Workshop on OTA analysis with outside experts About 10 outside experts familiar with OT A assess ments (see app. A-6) will meet with the policy project team and invited members of the advisory pane] for a one-day workshop. Discussions at the workshop will provide a comparative basis for describing the diversity of OTA "styles" of policy analysis, as well as assessing the agency's analytical strengths and weakness as seen by policy scholars and practitioners. Task 2-E: Interviews with recent first-time OTA project directors We anticipate that first-time project directors wil1 be among the primary users of the report prepared by our group. We will interview several recent first-time project directors to ]earn what types of information they would have liked to have had available to help learn the craft of policy analysis. We wi11 also find out how they did learn their craft (e.g., read OTA reports, worked cJosely with another project director, or sought help from the program manager). Task 2-F: Review of the policy analysis literature and previous analyses of OTA Though agreement on what constitutes policy analysis-good or otherwise-is Jacking, the scholarly lit erature continues to grow. This task is devoted to providing a window on the Jiterature most relevant to OTA. Does our policy analysis reflect the "state of the art" (if there is one), lead it, or lag behind, and in what ways? Through a literature review, we have identified about 35 promising articJes and books ( out of about 500 items) that incJude discussions of how to think about, carry out, and interpret policy analysis. This task, which will be carried out entirely in house, aims to distil] what is known, and sometimes pre scribed, as important dimensions of policy analysis. A preliminary set of dimensions to relate to styles of analysis is being developed from this literature. There is dearly both a richness and a diversity in the literature that will be of some help. In addition, there have been a few doctoral disserta tions and case studies about OT A, though little has been targeted to our poJicy analysis per se. These "outsider" analyses of the agency will also be reviewed and culled for insights. -------------mf--------------

PAGE 94

A P P E N D I X A-1 OTA POLICY PROJECT ADVISORY PANEL Members of the advisory panel for the OT A policy project, all of whom are current OT A program manag ers or senior staff, are listed below. The panel will be chaired by a former OT A program manager, Richard Rowberg, who is now Chief of the Science Policy Research Division of OT A's sister agency, the Con gressional Research Service. Division A-Energy, Materials, and International Security Peter Blair, Program Manager Energy and Materials Program Gerald Epstein, Senior Analyst International Security and Commerce Program Katherine Gillman, Senior Associate Industry, Technology, and Employment Program William Keller, Senior Analyst Industry, Technology, and Employment Program Division B-Health and LHe Sciences Clyde Behney, Program Manager Health Program Michael Gough, Program Manager Biological and Behavioral Sciences Program Alison Hess, Senior Analyst Food and Renewable Resources Program Robyn Nishimi, Senior Associate Biological and Behavioral Sciences Program Division C-Science, Information, and Natural Resources Nancy Carson, Program Manager Science, Education, and Transportation Program Emilia Govan, Senior Analyst Oceans and Environment Program Linda Roberts, Senior Associate Science, Education, and Transportation Program Joan Winston, Senior Analyst Telecommunication and Computing Technologies Program Congressional and Public Affairs Office James Jensen, Congressional Affairs Director ------------------ml-------------------

PAGE 95

A P P E N D I X A-2 TASK #1: EXAMINATION OF OTA REPORTS The following OTA reports, selected by current pro gram managers, will be reviewed by the OTA policy project team: Reports from Division A: Energy and Materials Program: Electric Power Wheeling and Dealing: Technological Considerations for Increasing Competition Nuclear Power in an Age of Uncertainty Improving Automobile Fuel Economy International Security and Commerce Program: Round Trip to Orbit: Alternatives for Human Space Flight Exploring the Moon and Mars Holding the Edge: Maintaining the Defense Technology Base Industry, Technology, and Employment Program: Serious Reduction ofHai.ardous Waste Making Things Better: Competing in Manufacturing Worker Training: Competing in the New International Economy Reports from Division B: Biological and Behavioral Sciences Program: Losing a Million Minds: Confronting the Tragedy of Alzheimer's Disease and Other Dementias Mapping Our Genes: Genome Projects-How Big? How Fast? Ownership of Human Tissues and Cells Food and Renewable Resources Program: Beneath the Bottom Line: Agricultural Approaches To Reduce Agrichemical Contamination of Groundwater Enhancing the Quality of U.S. Grain for International Trade Enhancing Agriculture in Africa: A Role for U.S. Development Assistance Health Program: Preventive Health Services for Medicare Beneficiaries: Policy and Research Issues Health Care in Rural America Indian Health Care Reports From Division C: Oceans and Environment Program: Changing by Degrees: Steps To Reduce Greenhouse Gases Polar Prospects: A Minerals Treaty for Antarctica Complex Cleanup: The Environmental Legacy of Nuclear Weapons Production Science, Education, and Transportation Program: Transportation of Hazardous Materials Power On! New Tools for Teaching and Learning Delivering the Goods: Public Works Technologies, Management, and Financing Telecommunication and Computing Technologies Program: Global Standards: Building Blocks for the Future FindingaBalance:ComputerSoftware,lntellectual Property, and the Challenge of Technological Change Critical Connections: Communication for the Future Note: This list includes 27 OTA reports. Time constraints may necessitate reducing the number of reports reviewed to 18 by dropping one report per program. ------------------11-------------------

PAGE 96

A P P E N D I X A-3 TASK #2: OTHER VIEWS OF OTA POLICY ANALYSIS Task 2-A: Evaluations of OTA policy analysis by former congressional staff Fonner congressional staff Katherine Y. Cudlipp, Former Counsel, Senate Environment and Public Works Committee, Republican; Currently Consultant. Anne Scott, Former Legislative Director and TAB Staffer to Rep. Morris K. Udall, Democrat; currently Legislative Representative, City of New York. William H. Smith, Former Staff for the Senate Armed Services Committee, Democrat; Currently Director of Federal Relations, Georgia Institute of Technology. David N. Sundwall, former Director, Health and Human Resources Staff (Majority), Senate Labor and Human Resources Committee, Republican; Currently Vice President and Medical Director, AmHS Institute. Work statement In support of the in-house assessment on policy analysis, OT A seeks the retrospective views of former congressional staff on key committees requesting OT A work. They will be asked to discuss the strengths and weaknesses of OT A policy analysis, as reflected in agency reports. While the concern is not the ultimate impact of a particular OT A report, staff perspectives provided in both written and oral forms on what consti tutes useful policy analysis could yield important les sons. Each former staff person, within his/her area of committee experience, would be asked: 1. to nominate and briefly explain an "ideal" and a "minimal" set of criteria that a useful policy analysis should include; 2. to apply the criteria through written evaluations of the policy sections of three to four OT A reports; 3. to describe, in an in-person interview with project team members, how the sample of reports satisfy the criteria used in the written evaluations, and to elabo rate on impressions of what was most and least valuable in the policy sections of the OT A assess ment reports. Tasks-The contractor would be responsible for com pleting three tasks: 1. A brief (twoto four-page) memorandum describing the ideal and minimal criteria that any policy analy sis should meet to satisfy the needs of a congres sional committee. These criteria would be based on the ex-staffer's expectations of what serves as sup port for committees (i.e., legislation and oversight in various forms), as well as his/her experiences with OT A reports (and other policy documents, for that matter). 2. A report (ca. 15-page) that contains written evalua tions of the policy sections of a sample of OT A reports. (The number of reports will vary with the substantive area and experience of the contractor, and the extent and form of policy treatment in the sampled reports.) 3. An in-person interview and discussion of the criteria for identifying a useful policy analysis, with empha sis on the features of the reports reviewed in task 2 that are deemed most and least valuable. (Questions to help structure the discussion may be circulated in advance of the interview, which would involve at least two members of the project team.) -----------llt-------------

PAGE 97

A P P E N D I X A-4 Task 2-B: Interviews with current congressional staff Current congressional staff Chris Aldridge, Majority staff, House Anned Services Ben Cooper, Majority staff, Senate Energy and Natural Resources Rick Counihan, Majority staff, House Energy and Commerce, Energy and Power Subcommittee Gary Ensworth, Minority staff, Senate Energy and Natural Resources Dan Finn, Minority staff, House Foreign Affairs Dave Finnegan, Majority staff, House Energy and Commerce Jim Greene, Minority staff, House Science, Space, and Technology Judy Greenwald, Majority staff, House Energy and Commerce, Energy and Power Subcommittee Eric Hamburg, Majority staff, House Foreign Affairs Jimmie Powell, Minority staff, Senate Environment and Public Works Jack Riggs, Majority staff, House Energy and Commerce, Energy and Power Subcommittee Skip Stiles, Majority staff, House Science, Space, and Technology Len Weiss, Majority staff, Senate Governmental Affairs Background The policy project team will conduct individual telephone interviews with the 13 current congressional staff members will be listed above. Each staff member asked the following questions designed to ascertain their familiarity with OT A reports and their views regarding the strengths and weaknesses of OT A's policy analysis. The staff members were chosen from commit tees that are frequent requesters of OT A reports and are representative of both parties and Houses of Congress. Questions for current congressional staff I. How many (and which) OT A assessments did you play some role in requesting? How many other reports (and which) have you used in some way? 2. What do you typically read (summary only, sum mary plus options chapter, technical chapters, etc.)? Is this similar to what others on your committee typicaJly read? 3. What do you find of greatest value in our reports? OT A policy analysis typically includes two compo nents: 1) analysis of the "status quo" (policy context, findings, and issues), and 2) discussion of options for change (alternative congressional goals, options to meet those goals, and an analysis of the effective ness and effects of the options). Where should we put the bulk of our time and effort? 4. What are the strengths and weaknesses of our reports in comparison to other organizations (CBO, GAO, National Academies, etc.)? 5. How should OTA staff interact with committee staff? Do you believe it is helpful to meet during the course of an assessment? If so, how often? Once an assessment is completed, what type of interaction would be most valuable? 6. How should OTA improve its reports to be of greater service to you/the committee? (Consider both the content and presentation of our assessments.) 7. Do you know of staffers who do not find our work to be useful that we might talk to? ----------Iii-----------

PAGE 98

A P P E N D I X A-5 Task 2-C: Retrospectives on policy analysis by OTA alumni OTA alumni Joe Coates, Fonner Senior Researcher at OT A Currently President, Coates & Jarratt, Inc. Ronnie Goldberg, Fonner OT A Project Director, International Security and Commerce Program; Currently Senior Vice President for Policy and Program, U.S. Council for International Business Joel S. Hirschhorn, Fonner OTA Project Director, Industry, Technology, and Employment Program; Currently Consultant, Hirschhorn & Associates, Inc. Larry Miike, Fonner OTA Project Director, Health and Biological Applications Programs; Currently Professor of Medical Policy, University of Hawaii at Manao and Executive Director, Papa Ola Lokahi, Honolulu, HI Edith Page, Fonner OT A Project Director, Science, Education, and Transportation Program; Currently Manager of Federal Programs, Bechtel Group, Inc. Work statement In support of the in-house assessment on policy analysis, OT A seeks the retrospective views of fonner agency senior staff on the strengths and weaknesses of OT A policy analysis. The alumni would be asked to comment on the OTA style of policy analysis, as reflected in assessment reports, compared to reports on similar topics that are issued by other governmental (e.g., CRS) and non-governmental organizations (e.g., Brookings). Each OTA alumnus/alumna would be asked to respond in writing to a brief set of questions concerning his or her chief impressions of how OT A approaches policy analysis. This will prime the alumni for drafting a short (ca. IO-page) paper describing the distinctive ness of the OT A assessment process and how it shapes the policy analysis sections of reports. The paper would be prescriptive as well as descriptive, and develop ideas introduced in the earlier written responses. For ex ample: 1. What could OT A do better in preparing and present ing its policy analysis discussions? Should OT A rely more or less on certain literatures, analytical tools, or proposed options? 2. Is there something identifiable as an OTA style of analysis? Are there certain characteristics of OT A policy analysis that should be emulated or avoided? 3. Within the culture of the agency, how might one introduce change to the approaches taken or fonnats used in doing/presenting policy analysis? Tasks-Each OTA alumnus/alumna would complete the following: 1. A brief written response to a set of questions that outline his/her impressions of the strengths and weakness of OT A policy analysis. 2. A 10-page paper comparing OT A's style of policy analysis with that of other organizations with which the alumnus is familiar. The paper would probe alumnus prescriptions for improving how OT A does and presents policy analysis. Questions to OTA alumni-Based on your experience at OT A, we would appreciate your responses to the following questions. A memorandum containing your responses may serve as a preliminary outline for your paper, in which you may develop particular themes. 1. Is there something identifiable as an OT A "sty le" of analysis? What are the major characteristics of OT A policy analysis? Please describe one characteristic or stylistic tendency that more of the agency should (a) emulate and (b) avoid. -------------------mt---------------------

PAGE 99

2. In genera] terms, what is the most sa1ient thing OT A could do better in preparing and presenting its po1icy analysis discussions? For example, should OT A reports rely more or Jess on certain literatures or analytical tools, spend more time developing a few key options, present a broader range of options, devote more time to analyzing the impacts of op tions? 3. There are many elements of a po1icy analysis, e.g., research design/methodology, data col1ection, data analysis, interpretation of findings, and form of presentation of policy options. In your area of exper tise, how does OTA policy analysis (not the impact of reports or their coverage in the media) compare with that of other prominent producers of policy reports? Please list the elements of a policy analysis that contribute, in your mind, to its quality and usefulness (especially to the congressional client). 4. Taking the elements of a policy analysis as Jisted in three above, which, if any, do you consider a particu lar strength or a particular weakness in OTA assess ments? What is missing from the list above that OTA could do more or a better job of? In your view, is the oft-heard criticism that OT A reports are too even handed warranted? Why or why not? 5. We have many congressional clients for our assess ments: the specific requester(s), all committees of jurisdiction (both Houses and parties), the Congress as a whole, as well as future Congresses. How should we ascertain and address their diverse needs in our analyses? 6. Is there any other issue or theme absent above that you associate with OT A policy analysis, or the assessment process? 7. Since you know the OT A culture, do you have any top-of-the-head ideas on how change could be intro duced in the approaches taken or formats used to conduct and/or present po1icy analysis in our re ports? --------------------imt---------------------

PAGE 100

A P P E N D I X A-6 Task 2-D: Workshop on OTA policy analysis with outside experts Outside experts Philip H. Abelson, Consultant (former Deputy Editor for Engineering and Applied Science, Science Magazine) Claude Barfield, Director of Science and Technology Policy Studies, American Enterprise Institute Daniel M. Fox, President, Milbank Memorial Fund Edward M. Gramlich, Director, Institute of Public Policy Studies, University of Michigan (former Acting Director, Congressional Budget Office) Don E. Kash, Hazel Chair of Public Policy, Institute of Public Policy, George Mason University Todd R. LaPorte, Professor of Political Science, University of California, Berkeley Shirley Malcom, Head of the Directorate for Education and Human Resources Programs, American Association for the Advancement of Science (AAAS) Rodney W. Nichols, Chief Executive Officer, New York Academy of Sciences Norine Noonan, Vice President for Research, Florida Institute of Technology Charles Weiss, Global Technology Management, Inc. Background In support of the in-house assessment on policy analy sis, OT A seeks the advice of a few experienced outside experts who have participated in the OT A assessment process. These senior policy experts will be familiar with OT A through their sustained contributions as members of advisory panels, chairmen of workshops, project contractors, and reviewers of draft report mate rials. They will serve as sounding boards for the project team, augmenting the suggestions of the advisory panel and offering commentary on assessment activities and findings. Tasks The tasks of each external expert will include: I. critique, in the form of a brief memorandum, of the OT A project work plan; 2. attendance at a one-day workshop convened to solicit the views of external experts on various aspects of policy analysis; and 3. written reactions to the workshop, either in memo randum form or in response to specific questions posed by the project team. -----------~m-------------

PAGE 101

APPENDIX B Form for examination of OTA reports -------------------111-----------------

PAGE 102

6. Course of the assessment: Atwhat.point in the assessment were the major policy issues framed? At what point in the assessment did you know what 1he findings would be? At what point in the assessment were the majorpolicyoptions framed? Did the assessment ever take a major unexpected tum? What exactly happened and how did you deal with it? 7. What methods and data cfld the staff in this assessment use to ascertain the policy context, identify policy relevant findings, and identify policy issues? Which methods did it find to be most helpful and why? ( e.g., brainstorming, historical reviews, case studies, scenario building, modeling, policy workshops, synthesis of research) 8. How were the policy options developed? What methods were used to.develop policy options in this assessment? Who had primary l'e$p00Sibilityfor developing them? The project director, certain members of the project team? What role did the advisory panel play? The program manager? The AD? What role did the advisory panel play? How important a role did outside contractors play? What about outside reviewers? Workshop participants or others? How much time and effort were spent developing policy options as opposed to writing the rest of the report? .. .. .... .... .. .. To what extent was politic:aUeasibility a consideration in.developingletiminating options? For whom were ttl4:9 pptions intended? (e.g., Congress, executive branch.agencies, other parties) Were theymore optionsorrecomrnendations? .. <> .. How mi:Jny Of the options we~ already"on the table'tartd how many were new to this assessment? 9. i. H~ much time~ ~spentevaluating.O: ~artd effectiveness of the options and their unintendedconsequences? Whatme1hodswereused? tO: What do you conslder~~strengthsandweaknessesouhepoticy.anaJysisandoptionsln this report? .. ... 11. .. If yeti~ to do the report over again. what flf anythi~).woukJ y~ do differently? 12. How was the report usedby Congress? (e.g.,.to structure debate on the issue; options were translated into Jaw;used as the basis for oversight hearings) How was the report used by other parties that you know of? .. ..:: ... 13. Whafother comments or suggestions do you have? .. .. : : \ .... .... .. > BI'Guide forstaff revlewofassessments ........ .. 1; .... i~~~~~ioucharactenzethe~? .. (e.g.,narrowvs;broadinscope;cJosely linkedtoanlmrnedlatetegi$1ativeis9uelagendaormorelong,,term;problemorientedvs. technology oriented; tetatedto lll,lernergiOg issLle ys>a mature one;short-term vs.4o~term) 1:~~~~~~assess~nt ciear1y stated? ------------------11~-----------------

PAGE 103

~i Metoodsfor ~yam: What :~s~ .(e.g braJnst~ing, hmtoricaJrev~ case srudies, scenario )l;buUdtng, modeling/policy.workshops, synthesis of research) does the report use 1) to ascertain the policy >conie>. CAre pot~relevant conclusions distinctly identified.as "findings? c'.)iWhai are some interctStingor unique features of the analysis.that deserve comment? 5.{ Pdkr ~{;, :. .. .. .. .. .. .... ... .. .... : a !Doesthe.reportclearly identify majorpolicyJssties? Are they logically linked to the technical analysis? \\b. 'Does the report address the request Ofthe requesting(iolT1rTlittee(s)? To what extent does it go beyond ,:~&~xdi'.C~~~:.O-:!.C:01 i::!jl~~. i(~1t'.~~-~~t~:!tl!;:~i~~~li~~~j;~ad;~y.Hnkedto legislative ~l[f~~~~-WM-~-~a :};~ii..:=1~~~~, l~JEme~~-=~=~!.-?@;{fail~}th~t.: .' ., .. ., .:.. ,. ; !;. .. i~ ~&~~=r=-==~ i;.: ,t;~~..,k~~~~ .... --... ,. e:~t~;f~!. \: ........ :: Jt ,,, ::::~. .. .. __ mEIDYof~e~ require lawslq,aiorchanges .tn existing laws or.major new .. ~: '.~;1~~~t:J#;l~:~~~J:~1 .. ,', .. ,. '.r-1iit~T.::~~~:.~~~-:::s:::; (incJuding .. .-= --w ... .. :.::=:.:.-._ .. =:.-::":.!~.-.. =:~ .. -.' .. :-.-.:.:=. .. ...... :Ji{_ .. ::\1:-.: :-:-::.-... ::':i:=. :j{~t, .. -------------m-------------

PAGE 104

h. Howmuchdetailisincludedinthepolicyoptions(i.e.,couldtheoptionsbeeasilytranslateclintolegislative language?) 8. Analysis of policy options: a Is there a clear statement of evaluation criteria? Are consistent criteria used to evaluate the options? b. Are the institutional changes needed to implement the options identified and discussed? c. If monetary incentives (or new govemmental programs) are proposed, are funding and other require ments estimated? d. Does the report evaluate the effectiveness of the option for achieving the goal(s)? e. Are the various options compared to one another? f. Does the report analyze any of the following types of common issues jn a particularly creative way? If so please identify: 1) R&D funding (e.g., how does one select a mechanism; how does one decide how much to spend?); 2) Level of government (e.g., how does one decide where action is appropriate, Federal, state, or local government level}; 3) Federal institutions (who and why-neworganlzatk)n.choice among existing agencies, methods for lnteragency coordination. etc.)?>; .. ... ... 4) Level of specificity of congressional directives. (e~detailed. highly specific congressional mandates to the executive branchvs. broad~ mandates with delegation of specific actions to.the executive branch). ... t i \ )i. 5) {Identify other common issues as reviews.proceed] < . g. How are values dealt with in the options1 (e;g.idifferent opfiorlepresented to address different sets of values; "if one believes, then;___;_" constructionsareused;otfler means-specify) Does the analysis of options seem even-handed? .. Does the~ seem forced? Do the options seem to have .any particutar bias (e.g., liberal Grconaervatlve; more Federal lnteivention or less)? Please .elaborate. Mechanical aspects of presentation 9. Mechanical aspects of report: a. How long is the report (how many chapters and appendices/page)? How much of the report is devoted .. to policy analysis (including options) asopposedtodescnption?. (e.g two of five chapters, policy options at the end of each of 11 chapters, number of.pages for policy analysis vs. description) b .. How is the report organized? ... < .. 1 .... c. Are there any specific techniques used to enhance1he presentation of the policy analysis and options in this report that might serve as a model for other reports? What are they? (For example, is there a box, table, orfigurethatsummarizespolicyoptlons? Areheadingsusedinan especially effective way? What about graphics or tables?) Please attach a. copy of anything that you think might serve as a model for other reports. .. .. . d. Comment on writing style (clarity, tone, journalistic VS,;~micstyle,etc}. Is the slyle in the summary a alldifferent from that in the rest otthe report? Other comments. ------------------,mi-------------------

PAGE 105

APPENDIX C Summary statistics for the 18 OTA reports Average 254 8 209 3 .. 3.4 26 ... --------------11--------------

PAGE 106

5 .. .Staff (total number_): A. Project director's.background .....:.;.::Natural 8Cientist/engineer _Social scientist/policy degree ___;Economist _lawyer .~ Other (specify) B. Disciplines on project team Natural scientist/engineer _.:,_Social scientist/policy degree _Economist _lawyer .;.._ Other (specify) e; 9ercent of report devoted to policy anafysis: 9 3 3 0 4 14 13 3 6 ~-8 ... ....:_ "'on.policycontextlfindings/issues. 38% __:_:9'1 on setting/analyzklg.goals. options 15% ~. N1~ :~izldwn of~ ~1e1s:,etSt
PAGE 107

y~~ > .J1~~. Opt~~at~ies are an~i~~~t ;i ==~~:ls t.':E~orOl'bliiw~ .. :.::}~%1i;:: 2~8 3.3 3.1 ---------------llr-----------------

PAGE 109

APPENDIX D Source book information and related materials The policy project team conducted a review of the policy analysis literature and previous analyses of OTA to prepare a sourcebook with especially insightful articles and reports describing important dimensions of policy analysis most relevant to the work conducted at OTA. The "Sourcebook on Policy Analysis" now exists as a separate, loose-leaf notebook, available to OTA staff who wish to consult the literature on some aspect of policy analysis. The preface and table of contents for the "Sourcebook on Policy Analysis" are included in this appendix. In addition to preparing the general policy analysis sourcebook, the policy project team had initially planned to develop a series of sourcebooks on specific policy issues that commonly appear in OT A reports. These ubiquitous or perennial issues, such as R&D funding and federalism, each have scholarly and policy litera tures that provide both a history and an evaluation of "what works." As envisioned by the project team, the sourcebooks would represent a compendium of basic source material from which any project team could draw as it approached the formulation of actions based on the policy analysis featured in an assessment. Unfor tunately, time did not permit the development of these sourcebooks by the project team. The team did prepare a contractor work statement for "The R&D Funding Option" sourcebook, and that work statement is in cluded in this appendix. The agency might consider pursuing the development of this or similar sourcebooks in the future. Also included is an excerpt from an academic paper that examined another ubiquitous policy issue titled, "Reorganizing Public Organizations: Al ternatives, Objectives, and Evidence." SOURCEBOOK ON POLICY ANALYSIS Preface: The OT A policy project team has assembled, in a separate loose-leaf notebook, a user-friendly starter kit for those who wish to consult the literature on some aspect of policy analysis. It contains bibliographic references and excerpts from a selection of the litera tures on policy analysis dating to 1976. Over 500 items were retrieved through a keyword search done by the OT A Information Center (IC) and reviewed for inclu sion in the sourcebook. In addition, the OTA document archive (also maintained in the IC) was scanned for itemsrelevanttoOTAasaperformerofpolicyanalysis. Although there is no cookbook or a tried-and-true formula for the conduct of policy analysis, there have been several attempts to define it. The diversity of definitions attests to the "arts and crafts" status of the enterprise. Certainly one of the distinctions between "policy research" and "policy analysis" is that the latterusually is performed for a client and is problem-centered. The former is generated from a disciplinary knowledge base and is limited by the interests of the researcher. Policy analysis has a literature and vocabulary, but the audiences for it are so diverse that assuming shared meanings is perilous. For example, after evading the ------------------m,-------------------

PAGE 110

issue of how to define "technology assessment," the OT A policy project team heard this simple, yet elegant definition: Technology assessment is policy analysis where technology is a variable. Contents: The "Sourcebook on Policy Analysis" is organized as follows: I. Select Bibliography (1980-Present) on Policy Analysis II. Pre-1980 "Obscure Classics" in Policy Analysis III. Exemplary Sources on OT A as Policy Analysis Performer IV. Reprints: A. Defining Policy Analysis B. Orientations/ Approaches/Frame works/Frames of Reference C. Use by/Impact on Client and Other Consumers D. Case Studies and Other Study Designs E. Specific Topics in Policy Analysis F. Other Bibliographies and a Glossary WORK STATEMENT FOR SOURCEBOOK ON R&D FUNDING In support of the in-house assessment on policy analy sis, OTA seeks to develop a series of "sourcebooks" on key policy issues that commonly appear in OTA re ports. These ubiquitous or perennial issues each have scholarly and policy literatures that provide both a history and an evaluation of "what works." The contractor would summarize what is known about the issue, append a bibliography of classic and recent sources, and discuss this "policy knowledge" relative to the kinds of policy considerations advanced in OTA reports. The project team would identify the candidate issues for sourcebook treatment. Each sourcebook would be a brief guide to the state of knowledge about and use of a particular policy option or tool. Taken together, these sourcebooks would rep resent a compendium of basic source material on which any project team could draw as it approaches the formulation of actions based on the policy analysis featured in an assessment. Congressional attention to "R&D funding" is one issue addressed in the vast majority of OT A reports. For example, a. "Congress could increase Federal research on the impact of work schedules on workers. This could include directing the pertinent Federal agencies to expand existing programs and to develop new pro grams, as well as increasing appropriations of funds to support these efforts. From Biological Rhythms: Implications for the Worker, OTA-BA-463, Sep tember 1991, p. 23. b. "Continue funding and support for the NRC to evaluate the state of reliability of the U.S. commu nication infrastructure for purposes of national secu rity and emergency preparedness .... Provide fund ing and support for studies of the security of commu nication systems." From Critical Connections: Communication for the Future, OTA-CIT-407, January 1990, p. 15. c. "Congress could maintain the current emphasis of increased funds for competitive grants and level or decrease funding of formula and intramural funds. Implicitly, this would indicate that Congress places greater emphasis on basic research than on adaptive research, extension, and teaching activities. Congress could award certain competitive grants to basic research that clearly shows ties to adaptive research. This would be a clear signal that Congress considers the original mission of land-grant univer sities to be appropriate today." From A New Tech nological Era for American Agriculture, OT A-F474, August 1992, p. 30. d. "Some technologies of great potential benefit to society do not get adequate private backing because the payoff for individual firms is too small, uncer tain, and far in the future. The U.S. Government has sometimes given special support to R&D for com mercially important technologies, but in an ad hoc -------------------illi--------------------

PAGE 111

rather than proactive way. A coherent, strategic policy requires having an agency in charge that can set goals and choose technologies to support that fit the goals .... Any Civilian Technology Agency (CTA) ... would certainly start small, and might remain so ... After a few years' experience, a CT A might take over some technology projects of the National Science Foundation .... From Making Things Better: Competing in Manufacturing, OT A ITE-443, Feb. 1990,pp.21, 76. Scope of the contract The R&D funding option, in other words, can take a variety of forms. But what do we know about each form? How are they treated in OTA assessment reports, and more important for this contract, in the policy and scholarly literatures? What analysis has been done of the use and effectiveness of variants on this option? What, in short, is the empirical wisdom on the following: Adjust funding levels: What are the rules of thumb on how much more (or less) R&D makes a differ ence? Is this determined by the magnitude of the problem, e.g., dollar increase as a percentage of the cost of Federal programs devoted to the problem? Set priorities: Since there is never enough money to do everything, how are agencies directed to emphasize certain types of research, development, demon stration, test, or experimentation? How specific or general are such options presented? Is a time frame specified? Accountability requirements suggested? Reorganize the R&D effort: Problems fall within the oversight of agencies. Sometimes this is seen as a strength, other times a weakness. How often does the funding option feature not dollar amounts, but rather changes in the management of the effort? Is cooperation between existing agencies or other organizations prescribed? The shifting of responsi bilities from one agency to another? A change in an existing agency program? Creation of a new agency? Focus on the performer: The R&D funding option can also stress the virtues of the performer. Which institutions-universities, national labs, intramural labs, private companies-are noted for what advan tages? Does analysis indicate that these virtues are manifested in expected ways? Are some performers seen as more expedient, efficient, or creative? Implicit in the funding option are assumptions about how R&D represents a "solution" to a technology "problem." Actually, there are many solutions to a multidimensional problem that go beyond possible changes in dollar amounts, cognizant agencies, and funding mechanisms. How have the variations de scribed above been discussed in the scholarly and policy literatures? What, if anything, do they add to OTA'sknowledge base,confirm, or contradict OT A's tendencies in formulating the R&D funding policy option? Tasks The contractor shall, in consultation with the project team: 1. draft a memorandum that outlines in three to four pages generic issues to be addressed on the R&D funding policy option; 2. submit a final outline for the sourcebook, as de scribed above (including its scope and like! y sources); 3. deliver a draft of the sourcebook, including a bibli ography (OTA will provide some source materials and discuss a format); 4. submit a revision of the sourcebook that takes into account comments by the project team. ------------------111--------------------

PAGE 112

EXCERPT FROM ACADEMIC PAPER Reorganizing Public Organizations: Alternatives, Objectives, and Evidence Craig Thomas if Institute of Governmental Studies ti University of California, Berkeley g .----=-----------'----------------------, Conclusion Prepared for The Secretary of Energy Ad, U.S. Department ofE1 This paper emphasized the incompatibility of various values and the necessity of choosing bet~een them when deciding whether and how to reorganize a public organization. Since these decisions are pro foundly political and should be made by the political actors involved, it would be inappropriate at this point to suggest that certain values, and thus particular alternatives, are necessarily "better" than others. More over, even if the actors involved in a specific reorganization were to agree on value proprieties, no evidence exists to suggest that specific organizational designs would apply in all circumstances. As Herbert Kaufman (1977:402) noted: August 1992 .,., .-. --~-~ .. ~Obviously, no reorganiuition is inherently right or wrong. No given administrative pattern will invariable increase efficiency, effectiveness, or responsiveness. In particular circumstances, identical organizational arrangements may produce diametrically opposite effects while radically different arrangements may pro duce identical effects. Despite the large volume of rhetoric surrounding reorganizations in general, remarkably few empirical studies have charted the intended and unintended effects of particular reorganizations. Table I presents a "quick and dirty" summary of this evidence and some of the logic behind organizational design theories.1 Given the remarkable gaps in our knowledge or the effects of reorganization, any future attempt to reorganize which is driven by instrumental rather than symbolic purposes should seriously consider whether the benefits desired ex ceed both the political effort required as well as the unknown and unintended effects which will surely follow. Though actors often latch on to reorganization as a panacea for governmental ills, notall ailments require major surgery. In this vein, efforts short of grandiose reorga nizations may be more likely to achieve certain desired outcomes. Table 1 does not summarize all of the issues, evidence. and speculations presented 1n this paper. It is intended solely as an aid to the reader, and is not intended to be conclusive. --------------------1111---------------------

PAGE 113

!obi I '"'11','.W.il!&."'ll!c",,\!!.?"..5'.C''"' -lfflel....,. IIINllVIMH Polltlc1I Nolle ,,., ... .. ,.,, ... lqulty Pollllc1I ....... (Cllt coc111 .... Ace- Portie! ,,.,., lllblllty lfllcocy '""' ond ....... or, ..... ,,.,., """' .... ,.11 .. 111 Aut-Cw,fldfflc ..... C..trel .. 1,,-llokllll ,,. ..... Cllllnlht ,....,, ,_, -,u.,,1,11, ,.,~1,11, Po111111,11, -,.,,., ~Ith Unit,-, lrerchlc1I di 1crtdlttd1 dhcrtdlttd; hcro11td for ""~!"~ """""' Pr11ldtnt: ltlHIINlllpo tvlcltllco 111t 1Yldtnct lllt Prffldlnt rt1llltd by llllhlll lht ,_11 .. ,_11 .. ether actors f11eullve lrMlch lncorpor,11 ., "'*-.,,,._ -ht111tl111, Pot111tl1tly Po1111t1111, -"'*-lllllltd by ..,.,_, llllltptndtnt l11ert111dlor ""'"""' dlCroalod -ttt1 Ortlllhat ,.,. l11to Pruldlnl ond olltcttd 111 INtcutfve lfltlrt:lt lrOIICh D-rtNnt trM-c, .. ... -Unt,-, -,0111111,11, Polt11tl11ly hl011t1111, ..,.._ -lt1lllodby -,....,,...,...,._, decrt11td for lnerH1ed lncrH1td ,, .. 111en1 or c-1111111 ,, .. 111en1 Cr11tt I ""'-ltllllvtly ,.11 .. ,, ..,._ "'*-Po1t11tl1lly -.,,,.,_ Ct11tnlly Unit ...... Gav1r111tnt urone "'"" tho"""",. lncr111ed _, .. Corporu IOII or tvldtllCo 111 .,,,_, 111 l11l1l1tl111 apllOII l11t1rprl1t llte CIH ef ti ..... ..... ,,. ... Ilk llkl" _,.ti_ -,111111 Prlvella1 V1rl1111 1YldtllC1 Theory ond ,...,, ... ._, ... --N-,......,.,. Unt,-....... 11, -Operttllllll ., .... ldtnc1 tvldtnco l11eru11d ......... ... _,., c..,......11 thNIUlh '""" llul Hrlllf llut -1tor1,. _,,,.,1,. aptllll c .. ... ,, ... requlr11 .... ... ._11,1. eopoclty of -''''"' e-tltlM ,..11 ,..11. ......... Ind ..... or1111hu1.,. ..... .. ,, .... 1pecllltd lpoClflld eontraetl ,_,tracH Purouo ltrotqlH ""'-Unit---ti--ti--Generally Unit,_, Not lpoclflcolly 1lfcft1 l111 ""' ,. r11l1tenc:1 or1111l111I0111I than Structure 1truc:tvr1\ ttr1tttlet

PAGE 115

APPENDIX E "Gems" of OTA policy analysis Gems of OT A policy ana1ysis, inc1uding effective or creative methods of presentation, have appeared in numerous OT A reports over the years. Some of the gems of policy ana1ysis in the 18 reports reviewed by the OT A policy project team were discussed in chapter 5. This appendix includes a representative subset of several types of"gems" solicited from the entire OTA staff by the OTA po1icy project team. The examples selected for inclusion in this appendix typica1ly illus trate or present information on one page in a report. These gems are by no means exhaustive of OTA 's best work, but the project team believes they might serve as helpful models or sources of creative inspiration to OTA analysts. The gems in this appendix are organized under five major categories: Category I: methods for summarizing key findings. Category II: tables and figures summarizing options. Category Ill: information that helps orient a reader to a report (e.g., text that describes the purpose, objectives, or context, etc., of the report). Category IV: methods for summarizing information about the types of questions OTA confronts in many of its assessments, (eg., stakeholder analysis, inter national analysis, and legal analysis). Category V: information about the methods used to conduct an OT A assessment. category I: Examples of methods for summarizing key findings a. Pharmaceutical R&D: Costs, Risks and Rewards, Health Program, p. 1, February 1993. b. Nuclear Power inanAge of Uncertainty, Energy and Materials Program, p. ix, February 1984. c. Complex Cleanup: The Environmental Legacy of Nuclear Weapons Production, Oceans and Environment Program, p. 7, February 1991. category II: Examples of tables and figures summarizing options a. Adolescent Health, Volume I: Summary and Policy Options, Health Program, p. 1-47, April 1991. b. Critical Connections: Communication for the Future, Telecommunication and Computing Tech nologies Program, p. 13, January 1990. c. Indian Health Care, Health Program, p. 37, April 1986. d. Access to Space: The Future of U.S. Space Transportation Systems, International Security and Commerce Program, p. ix, April 1990. e. Round Trip to Orbit: Human Space Flight Alterna tives-Special .Report, International Security and Commerce Program, p. xi, August 1989. -----------llr-----------

PAGE 116

category Ill: Examples of information that helps orient a reader to a report (e.g., text that describes the purpose, objectives, or context, etc., of the report) a. Nuclear Power in anAge of Uncertainty, Energy and Materials Program, p. 8, February 1984. b. Changing by Degrees: Steps To Reduce Greenhouse Gases, Oceans and Environment Program, p. 4, February 1991. c. Technology and Handicapped People, Health Pro gram, p. 6, May 1982. d. Enhancing Agriculture in Africa: A Role for U.S. Development Assistance, Food and Renewable Re sources Program, p. 45, September 1988. category IV: Examples of methods for summarizing Information about the types of questions OTA confronts In many of Its assessments (e.g., stakeholder analysis, International analysis, and legal analysis) a. Nuclear Power in anAge ofUncertainty,Energy and Materials Program, p. 7, February 1984. b. Rural America at the Crossroads: Networking for the Future, Telecommunication and Computing Technologies Program, p. 22, April 1991. c. Changing by Degrees: Steps To Reduce Greenhouse Gases, Oceans and Environment Program, p. 285, February 199 I. d. Transportation of Hazardous Materials, Science, Education, and Transportation Program, p. 8, July 1986. e. Enhancing the Quality of U.S. Grain for Interna tional Trade, Food and Renewable Resources Pro gram, p. 6, February 1989. f. Facing America's Trash: What Next for Municipal Solid Waste?, Oceans and Environment Program, p. 8, October 1989. g. Federally Funded Research: Decisions for a De cade, Science, Education, and Transportation Pro gram, p. 261, May 1991. category V: Examples of Information about the methods used to conduct an OTA assessment a. Indian Health Care, Health Program, p. 352, April 1986. b. Grassroots Development: The African Development Foundation, Food and Renewable Resources Pro gram, pp. 23, 26, June 1988. ------------------11-------------------

PAGE 117

CATEGORY I category I: Examples of methods for summarizing key findings I a IIP __ ,, Ille Otlice of'lec:lmology AllelPN!U amiiaed lbe costs of pharmaceurical aaean:11 aad (R&D), rbc ecaaamic n:wanls from dW iaveamcnt, aad lbe impact of public po1ic:ics OD boch com aad Rllllas. Below is a lrief syaopsis oflbe l&lldy's major coaclvsioal SUMMARY OF FINDINGS Pbar-*al R.&D is a costly aad Ii.sty basiaal, liar in ncmt :,ars lbe fimmcialmvuds from R.&D baff mme dum cdflet its COIII aad rim. Tbe aaage llf'tatalt R.&D cash oaday far eada-drag lbat readied die 1111ml ia lbe 1980s wa &boat S6S milliaa (m 1990 dollm). Tbe R&D pocea liDOlt 12 ,-aoa-=nge. Tbe fullaftlllrmcmtoflllllleoadayl, Clllllllll!H.ciod fiO dim 'ftlae OD lbe day of ambt appamI, -lUllgbly $194 miJlioa (1990 dollm). TbeCOlt ofllliagillga-drag IO ambtis Yf -*ive fD dlaages ia 11:ieace 11111 iecbaology, lhifll ia lbe kinds of drags 'Dllllcr~ 11111 dlaages ill lbe n,gala-1111)' eavirw All of lllllle dlaages -oc:amiDg fut.~-it is impossible topedictlbe Cllltof llriDgiDg a -drag IO ambt IIOday from alilmled costs for drags wbole developlmDl bepD-dum a decadeago. Eachaewdragilllmduced IOlbe U.S.llllmt._ 1981 ad 1983.nlblmlld,-of1Des,atblt$36millioallD Ira iD..raa dum WU aeoded IO -,.Y off lbe R.&0 'Ibis mrpllll nlllm amaalS ID &boat 43 pm:eatofebe plil:e of each drag cm1r ila Jllodaet lifc. 1 Summary 1 Source: Pharmaceutical R&D: Costs, Risks and Rewards, Health Program, p. 1, February 1993. ------Iii----------

PAGE 118

CATEGORY I OVERVIEW AND FINDINGS Without signifiant ch.1nges in the technology, m.1n.1gement, and level of public acceptance, nuclear i,o-r in the United States is unlikely to be expanded in this century beyond the reactors already under construction. Currently, nuclear powerplants present too many financial risks as a result of uncertainties in electric demand growth, very high apital costs, operating problems, increasing regulatory requirements, and growing public opposition. If all these risks were inherent to nuclear power, there would be little concern over its demise. Ho-r, enough utilities have built nuclear reactors within acceptable cost limits, and operated them safely and reliably to dernonstr.1te that the difficulties with this technology are not insurmountable. Furthermore, there are national policy reasons why it could be highly desirable to have a nuclear option in the future if present problems can be overcome. Demand for electricity could grow to a level that would mandate the construction of many new powerplants. Uncertainties over the long-term en vironmental acceptability of coal and the adequacy of economical alternative energy sources are also great and underscore the potential importance of nuclear power. Some of the problems that have plagued the present generation of reactors are due to the immaturity of the technology, and an underestimation by some utilities and their contractors of the difficulty of managing it. A major commitment was made to build large reactors before any had been completed. Many of these problems should not reoccur if new reactors are ordered. The changes that have been applied retroactively to existing reactors at great cost would be incorporated easily in new designs. Safety and reliability should be better. It is also likely that only those utilities that have adequately managed their nuclear projects would consider a new plant. While important and essential. these improvements by themselves are probably not adequate to break the present impasse. Problems such as large cost overruns and subsequent rat
PAGE 119

CATEGORY I Summary 7 Bo:r A-l
PAGE 120

g, C 0 !1? ),. 8-ai" i i Ill c;: C: I [ d' '-<: -e> 5 _iii :I: I :T "ti ii> _3 -p )> 1 (0 (0 ..... Tabla 5-StrateglH for Malor Option 1: Congre11 Could Take Steps To Improve Adolescents' Access to Approprt111 HHlth and Related Services Access 1...,. _,eond Timo tor Strategy PhyalCIII Legal Knowledge Aftordablllty App,oachablllty expected Impact StrataoY M: Congro11 could 1upport tha do val opmont of cenlera that provide comprahan11v1 and ICCHllbll halllh and ralllad MrYICH 1paclflc1lly lor adolelconl9 In achooll and/or cornmunltlea. 1 a: -F-llad money lortha-olopmtnt X X X X X Immediate and ol 1c:1,00Ulnkod and athet oolfflU'llty-bend oonlert lhat pro'lldt oomp,ehen81Yt health and long term relatadllMCll lor--1-1 i,, -F-continuation funding lor already X X X X X Immediate and Hllblllhod tct,oo141nkod and OOfl"ffllnlly-belad long term oontor1 that pro'lldt comprehen81Yt 1ervlce1 tor lldoletoonll. 1-1 c: AllOI eJdlllng barrier lo Iha delivery ol oornX X Near term; study prohenllw In -IOlnl-lpldflc oon-needed ter1. StrataoY 1-2: COngro11 could tau lllpa to Improve ac10111cent1' nnanclal ace111 to haallh aarvlcoa. 1-2a: Mandato an Immediate upanllon ol Medicaid X Immediate allglbllltylor _aoont,. 1-2b: Mandate that 1,npoye,1 p,o,,lde heelth ln1ur X Immediate anoo lor thalr C111Nnly unln-od _, and lho11-,~ .. 1 c: Dlroelly fund or p,o,,lde lnoonti..t lo Statn tor X Immediato -tad, lolncrHII __ UN of Modi cold-.. 1 -2d: OIIOOUnlge or provent private lnll#lfl lrorn ImX Immediate and plernontlng curront p4ana lo Pmlt oovorago of long term __ dopendonl,. Strotogy 1-3: CongrN1 could tau 11ep1 to lmpro .. ac101e1c1nt1' legal aeon, to health 11rv1001. 1a: EnccuagolhlU.S. 1'*111tw tnnc:1,oranongovX Medium term ..,,,,_II entllytodovelopamodol State llotute lo enhanoo adoleaoonts' legal aooo11 lo health 1-3b: Enact loglllatlon that roqut,11 apeelllc Federal or X Immediate Faderal/Stale programo to adopl partla.dar aubllantlvo polldea with roepeet to parental conaont and notlllcotlon. 1-3c: Enact legtllatlon conditioning Stote1' reoolpt of X Medlumlerm Fodorll lundalor apeelllcpurpo16aon tho Stot11' having partle
PAGE 121

CATEGORY 11 Chapter I-Summary 13 Rgu,. 1-1-Congrealonal Strategies and OptlTo Addraa Access to Communtcatlon Opportunltlas _, --.. __.,atimeoa toflUflC!lftOilll\CI OWMJSIOP,!Mdf p,owlClll'lghftillClil IOITIIWMCesonill .. _, .._ ... COffllll(lft-Qffltf.OI' ...._. ... --Sllafealll515 _,,_ I I I .......... --1,ic,use 5UD(IOl"l lor ---,_,.., ................ ltlePIAIIIC:Wllh 10rattMt11U1n lllelftbllltlft--ot--_ .. rt1urnf"9UUI.,. -----SlrllCturtlle,ras ......... I I __ -ter\CIS ........ _, -----_ ... -...... ---........... ........ ..... -----.._,,_ _, ...,_ .. IDIISourdlne -~c.amtf.llMI ----I I Clf'fWmllll .. ttll.etNldlnt-IDrallt\acass I --__ ... --------------IJleyC11111Mut1Da. IGtllmlillalorls ..... ....._, .. i,wrdllsll .......... --..... ...... I I -I I ,.... ... OIIIIIINIM!ln s...,.......,.,t llestlflCl11'1eClllle/ llltnlSIDP'"* -_, ........ ----------ID_,...lfltCOIII ---SttppOl"!IIJOt':I __ ., -----lhtCllle...,..stry ..,,_., -_,.._ --.............. I I ----I I -... ---PrOndtf .. 11 __ .. _, --.,.,,.,,., ~-.ac:, l*IOgtapftlUIGf'IICH ---CocMy1t1tF..,-.s --IOrCMINClyluncted lt.,_IIIO'tSU'1ft91111 DDcir .. lotbtOM p,ogrlfflS fllll ...,. __ CUtefS,ncl/0,fdefta -... IIIOlllltf"'IOi-1 Aslllffll..,.. I I ,._ ... ........... I ,_.,...,.,. IUUftolpuoil(. Aaapt~-ftllnll llanOl\ttlneMICl1,p,KI .......... OflC#l'mutllU!enNlfl ayslOrdistut.soi OI CIUQIIC DOIIC,-ssun IIOUACE:Ollooot ____ ,990_ Source: Critical Connections: Communication for the Future, Telecommunication and Computing Technologies Program, p. 13, January 1990. ----------------------llll---------------------

PAGE 122

g> C: :3" ::, i &'> -~ :I: t ::1" "'O c8 j -p (,) _--.i .... (0 0) ?> El1_9_lblllty Ind entitlement C,,,,.,.f ,,,,,.llon: P1rt0nt of lndlon detctnt, no blood quan tum requl-1. For HMCH purchated by !HS from non-lHS provtdel9, additional,. qulrement that the lndlvldu11 mutt live on or near I federally recognl21d lndlan ret1r v1tlon. IHI p,opoted .,,_,,,., Ellglbl1 per10n1 would have to bt either mtmbtl9 of ftdtrelly recognftld trtbet Ind htvt II leatt one-quert1< fndlon blood, or other Indiana of II leut ont-11111 Indian blood. In lddlllon, tllglblt Indiana mutt live on or near federally recognltld Indian ,.,. eivatlon. OTA opf/ont: I 1: IHS or Congrelt could davelop a ptlortty tyttem for accttt to !HS Hrvlcet. f2: !HS or Congrett could UH blood quan tum crtt1rt1 to tupplement rllh1< than re11rtc1 etlglblllty crttert1 bated on tribal membership. /3: II eliglblllty criteria are midi more ,. llrlctlve, Congrelt could mlll
PAGE 123

CATEGORY 11 lmproYiDC U.S. Space Traasportatioa Systems Whic:hner broad propam goals ldected, if Coapess wishes ID continue ID improve lhe safety, miability, perfonnaac:e. and/or economy or U.S. llwlcb systems, it bas a aumbcr or possibilities from which ro c:boosc. Several -listed below; lhcy -not muDlllly cmusiw, nor is lhe list cxbaustiw. Congn:ss could decide ID poceed with one or 1111n from cacb list of opliaas. Because of 1he long lead times for the deYelopmeat of space ll'UISpOlWioo systems, some decisioas will have ID be made ia tbe next yC31' or two. Odien can wait 11111il lhe middle or lhis decade or Iara-. /fCongreu wi.lhe1 IO: lmproft carp lauacb .,..cem Nllallllty or perfonaaace: lmpnmSpllceSbutde1J5-eem afety, nliallllllJ': MllalalaelllPiPPHelhll ae 1auac1a rate or, to u lauadles per ,ear: Rldace rllbtomeceaful Space StadDa aamllly: .,...., .. lecllaDIIID' .... ud pluforllulldiaC lalllldl tylfemS: ProrideL. wwwa-.,cnw relunfnlmllleSpace SCadaa:. 1/~willrulO: Build afer, -,e mlable ~.,-: lmproft carp lauac:11 .,..cem Nllallllty aad redac:e--.: ._ operability: N_,_T_ DJsio,u Then ii coald: FIUld dew!lop,Mnt of lluu,logiu in rht Advanced Launch Sysrem and other programs FIUld tkwJOfl'Mnl of Liqllid-fi,ellll Rocket Boosters (l.RBs). FIUld t:OlllillMtl tlnelop,Mnt and itlrprove-nt of Advanced Solid Rocker Motors (ASRM1) ond alternate ""'1op,lmp1 for the Space Shunle Main Engina: Fl!Ulll illltlllllllion of bflllt-in tell quip,Mnt in rht Shunle and more __,_ tat eqr,ip,nellt UI "1Mcllfadline1. F""" tkp,,rd,lucoft11kallOlfll~ orbiur to be delivered as soon aspoaible(l996),ondtlinaNASAIOudllCedie-*rofSluatkflighu plallMdper,-.NASAeo,r/lludllShllllkjlighllby: a. pollpOIIUII or CIIIIRIJJag-plallMd SIIIIU/e lawr&hu; or b. relying IIIOf'e on CIIIJO-Olll1 "1Mcll wluda, ltlCla as Titan /Vs. Direct NASA ID dewlap ond Slulllle.C ID carry -Space Station elenlaa IO ""1il. (Tlris WOllld redlla rht total lllmlber offlighll required.) Corrtltlw ID /11114 ,,_.,,, ond lldulaloa dew!loptrwnt and ten efforts aclt111: a. w Mwuad lltlllMll Ullllld S,11e111 ltllllia: b. rl,e Nll#OIIIII Am,.Spa P"'-pro,-n (NASP); or c Mwzllt:elll.lllllld,S,-..(~)prognllll. F""" a prograa ,o dewlap o U.S. cmv _.,enc, narrn vehicle. &lppon joull llnellJpnwnt with Space $talion pan,,er1 of whicle for aierga,cyrdll'IL F-T-Dvio,u TlienilCtlfl/4: F""" dew!loptrwnt of -,fer, more relioble laMnch ,ynons to 11111-nt or accd die S/wttk. Dae lllilhl iltt:hMu: a. a Pe-1 Ullllld S,-.. (PLSJ, or b. 1111 Mw,llt:ell llaMMIAllld System (AMLS), or c. Ydidat/aiwdfr-rl,e NlllioMI kro-Spau PlaM (NASPJ program. F""" tlew"1pmal of""'1t&li whicks or ,ynons (e.g., ALS engines) that eo,r/11 be .-.{IIOlntl, integrated, """ lawr&lied by highly lllltOmOted inetJ,o,b with ;,,,p,,ONd fWOCUI ctllllrOI. F"""develop,neMofvelrlduduignedforq,ackoulltll"OIINl,suchasthose considered/or an Advanced Manned Lawu:h System or possible .rucces-1on IO the pn,ptl#II Nflliorfal kro-Space Plane test vehicle (X-30). Source: Access to Space: The Future of U.S. Space Transportation Systems, International Security and Commerce Program, p. ix, April 1990. ------11111-------

PAGE 124

CATEGORY 11 Selected Options for Improving the Space Shuttle System The following options were selected from a wide range of possible improvements to the Space Shuttle System. The effcc:tivmcss of each option rcpn:scnts arA's considcml judgcmenL However, each may be inorc or less effective depending upon other improvements chosen and the pace at which they are implemented. 1. CGl*'llelll ......... _Sald ** Roclolt-(ASAMI) ** z. Fund .......... all.Jqlad--*** *** -(l.Rlls) 3. DNl!ap&llullle-C '--..-a1---...... -5. Fund__,_al __ glldlrtar _.. ...... _,.. I. --.---pnlllWIIIII *** *** ...,_....,,....,,andperlar--a1_.. ..... .,..,, ........, .. 1. c:an...111in.,,..twRldlllglllCISald Roclolt-(RSlllota) z. ....-.111111-1n-equipn.,a-... ...,. ____ ........, __ ---equipn.,&lar --3. DNliap._.,-'IINEallmllTn (ET) '-DM,lapllglllllalglt_tar __ -5.~---.,... ..., ** ** .. --..... ............ --*** 7.SNII .. ..,.,_ .. ,...._.,_ ..... ., ____ ., ........... ,. a"I': .... _, ..... ... ......., ...... ................ Olljecllwa ** *** ** *** *** *** *** ** *** ** * ** *** ** ** ** xi Source: Round Trip to Orbit: Human Space Right Altematives-Special Report, International Security and Commerce Program, p. xi, August 1989. ------------BBi--------------

PAGE 125

CATEGORY Ill Category Ill: Examples of infonnation that helps orient a reader to a report, (e.g., text that describes the purpose, objectives, or context, etc., of the report) THE PURPOSE OF THIS STUDY This report responds to requests from the House Committee on Science and Technology and the Senate Committee on Energy and Natural Resources asking OTA to "assess how nuclear technology could evolve if the option is to be made more attractive to all the parties of con cern" and to identify possible technical and institutional approaches for the Congress "that could contribute to the maintenance of this important industry." The report describes the ma jor impediments to nuclear power relative to other types of generating capacity, identifies op tions that might be considered to remove those impediments in light of the problems and con flicts discussed above, and explores the consequences of not maintaining the nuclear option. Changes could be made in the technology and in the institutions that manage it. If a reactor were to be developed that physically could not suffer a major accident or pose health and safety risks for the public, it might allay some of the concerns of the regulators, the intervenors, and the public. Such a reactor might not require the ever more stringent standards of quality required for current light water reactors (LWRs), thus reducing the economic risks. lml)RM!ffleflls also could be con sidered in management of the construction, op eration, and regulation of reactors. If all reactors were to match the experiences of the best man-aged plants, there would be much less concern over the future prospects for the nuclear option. It is the intent of this. study to explore these possibilities in the light of the different interests and different concerns discussed above. The report details the various difficulties facing the future of nuclear power and the measures that would be useful and practical in overcoming these difficulties if the Nation wishes nuclear power to once again be a well accepted, viable energy option. The technological options are restricted to converter reactors similar to those now available on the international market. These are the reactors that could be deployed in the United States by the end of the century. Breeder reac tors are not included because their development program will not make them commercially avail able until sometime in the next century. The other elements of the fuel cycle-uranium re sources and enrichment, reprocessing and waste disposal-are not included either. Waste has been considered in great detail in a recent OTA report. The other elements need not pose con straints to reactor orders, which is the key issue addressed in this report. This assessment was carried out with the assistance of a lalge number of experts from all sides of the nuclear debate-utilities, nuclear critics, Source: Nuclear Power in an Age of Uncertainty, Energy and Materials Program, p. 8, February 1984. ---------1mJ------

PAGE 126

C A T E G O R Y 111 Box I-A-Th11 OTA SllUI:, in ConlllJ:I 11ie six CORgftlllional commiaees n,quc:sting lhis asseamcnt uucl Ol'A to focus on a very specif",c question: "Can the United States reduce carbon dioxide emissions in Ille near term!" Chantinf by Dll,reudoes notc:wnine iadcpdl manyequallydifficallcpMionslUCb u the sc:imceof climale c:bange, 1be immlainlies Uld-of IIIIIOlpheric modeling, die pn,jecledecological effects ol global wmning. Rather, -ol Ol'A's _, have been ckoced ro analyzing ledlaical oplioas to deaeuc CX>i, akbough mc:dwlc, ailmus oxide, and chlomfluonlc:ar (Cl'C) emissioas ae addlased whcnver poaible.. Al die lime of their~ thec::onpasioaal committees wen: well awan:of aagoing ink:malional effons to phaseout i aeduclion mm eleclric ati1ity fuel nrifdliag, )IOlllblc impwtw in aalOlllllbile cfficiency, cbaDga in -,:iai building --=aoa, mcxe e:fficieal-mfaecuring plU0IIIICI, Cfl:. Molt oflbeoplioas Nlafleto decnuiagcmissiam, llkbough-, IUCb u ilrmlYeiecapcmingp.s alladyemiaod rotbellllllllpbaL 'lbl--cn711 lays out tine palbs: a Bae cue ("bulinea u -1"). a Madaafe (eacalially ._ .. ) cue,anda 'lbagbcae. Oalylbe lut fulfills tllccoapaliaaalnqae1tlDIIRdacesfulme COi emilliom-toa lc:vel in 2015 dill is 20 to 35 paaat lows lban !Oday. Some will lllllC 11111 aur Cltimlfes of cmiSliam nidacdaas ae badlpolilically~-SCXlldy. Otllcnwilld0Clya20-I035-faamlaeducdoaunot beiagmaly-.gb; dw.lalergowaiWiliialPaclaaQimlfeQaaapandtbeU.S.Eawhamwlall'lUIIX:doaA.geacylllClllllyati.-rid llllltbe'ftdcl-ftldace00imiilliaaabylml50tol0,__IOIClbiliaietllc~ CoagNa' nqaell 11111-wm:widlia a 25-,-~ in die IIUdy ,-.1 IO boa two-edged aml:,lic awad. l"l foam Ol'A to llbac:lmeloalt wbll9 U.S. C0i emiuimll wcnbcadillg willlaat policy illfatadiaa. Bal, 2S :,eus l1lo is roo lbaltapaiodtoilldDdea--,inwbidlfoailfucllaeaapplamodwilblllda--nfucl-u.-.ble ud impa:,ted mclearmagy-. IDibd,lbeUlliledSarmdelclibed2S,-rs.-indliaaq,ortdoesllllt .... ,,_,__Bydiffcrmtfmm wllll -bow lllday ._.., an uadalyiDg theme la Ol'A'1 aq,ort is tblt a 11rm11 R&D effan ii piYCllal ro llrlagillg--.i fa.e..-.Couil fuel)_ to~-quietly U )IOlllblc. evm U all llOl\tOn ol _,_., __ cfficiealeqaipnmlanddec:1-.-ay-,omp!io,L lflcag-tamR&Disgan,d to dial parpmc, dim --0 mpply leCbDologiel cu .-111 n:place aisliag powerplanll and equipment eady ill die am cm1my. Manyattheeecbail:aloplioas evalaaledbercme wwlbpanuiagfarochcr-inaddicim rodimllecbange, bcmaelllcy.._..ochcrimpadullU.S.goalsllldaawqy-',,lacllm,iawatal,qaality,andecaoamic rm.pmahw 'Ibey cuRldlicecmilliaas in die lball--, ftldace-1 magy demand, and acne to bridge the U.S. -rmma mail-fuel ageroammfoail 6-. Source: Changing by Degrees, Steps To Reduce Greenhouse Gases, Oceans and Environment Program, p. 4, February 1991. ------------------------,11---------------------

PAGE 127

CATEGORY Ill 6 Technology and Handicap~d People terial on resource allocation, of course, may have to be viewed in the context of a different set of ORGANIZATION OF THE REPORT The rest of this chapter presents a summary of the report and briefly lists the policy options. The body of the report is then organized into four decisionmakers. but the generic issues would re main essentially the same. parts. The relationship of the parts and the individual chapters to one another is shown in figure 1. Figure 1.-0rganlzatlon of the Report T Par1 I: D1sabiUtfft l T Part II: Technology 1 Part Ill Anoca11no ReM>Uces Pt1 IV Pohcy OOflOIIS Ch 1: lnlrOduction and Summaf'Y Ch 2: Dellnttlona and Domographlcl Clll:-~t Ch 3: DiNblllly klontlfytng and Plamlng Cit I: Marutlng -DfflSOUACE Ofhc ot lechnofou, ....... ..,..,., Ch 9: o.1iv.,y, UM,and Financing Source: Technology and Handicapped People, Health Program, p. 6, May 1982. --------lime--------

PAGE 128

CATEGORY Ill Chapter 3 The Status of Low-Resource Agriculture HIGHLIGHTS Low-resource agriculture is practiced by a diverse group of African farmers, herders, and fishers, is based primarily on the use of local resources, but may make modest use of external inputs, including information and technology. Low-resource agriculture predominates throughout Sub-Saharan Africa. It produces the majority of the region's food, involves and provides income for the majority of people, helps buffer against famine, and contributes to national economies by producing agricul tural products for domestic use and export. Low-resource agriculture is no longer able to meet the needs of Africa's growing popula tion. Declines in per capita food production and agricultural income, widespread mal nutrition, and natural resource degradation are signs of its decreasing capability and reasons for concern about the future. Increasing numbers of Africans will depend on low-resource agriculture for food and liveli hood in the coming decades. Thus, it is increasingly important to improve low-resource agricultural systems so they are better able to help meet Africa's food security and agricul tural development needs. AFRICAN AGRICULTUREa RESOURCEFUL WITH FEW RESOURCES Africa's hallmark is its diversity. Its vast cul tural diversity is manifest in nearly 800 distinct ethnic groups, which account for about one third of the world's languages(23). The 45 coun tries of Sub-Saharan Africa show a wide array of political and economic systems, including numerous systems of tribal and modern law. The region also has wide ecological diversityranging from desert to savannah to rainforestand broad soil and climate variations that can change over short distances. This diversity is mirrored in the nature of African agriculture. Having evolved under these differing biophysi cal and cultural in0uences. African agriculture encompasses a complex array of crop and live stock production systems. Clearly, then. it is risky to generalize about African agriculture. There is no such thing as a "'typical" African farm. Some common ele ments. however. can be identified. One con sistent aspect of African agriculture is its prom inent position in African economies (table 3-1) Agriculture employs about three-quarters of Sub-Saharan Africa's labor force and accounts for about one-third the region's gross domes tic product. Also. about one-half of the coun tries in the region derive al least 40 percent of their export earnings from agricultural prod ucts. Further. despite major increases of food imports. particularly grains and dairy products. the region still produces most of its own foodal least 80 percent of its er.reals. 95 percent of 45 Source: Enhancing Agriculture in Africa: A Role for U.S. Development Assistance, Food and Renewable Resources Program, p. 45, September 1988. -------------aml---------------

PAGE 129

CATEGORY IV category IV: Examples of methods for summarizing information about the types of questions OTA confronts In many of its assessments, (e.g., stakeholder analysis, International analysis, and legal analysis) Public Ullllly Commiulonl Slable canstruction coats Minimal operating risks Adequete financing Nlllc.._.i Figure A.-The Seven Sides to the Nuclear Debate UIRllles Nuclear critics eon,_ In the technology Confidence In regulators and utilllies Economic advantage UabHIHes of other fuels plOWld -.1ors Adequate mum on Investment Nuclear industry _, utlllty __ _, Adequate financing Stable licensing Slable canstructlon coets Minimal opposttlon National policy Mi_. _.ting -Predictable conatructlon coats '-----_, Public -lance M-political risk Public and political acceptance Favorable risk/reward Conlldence In technology Canfldenca In utllltles Publlcaupport Predictable regulation Noncontroversial. necessary c:ondtttens No major .acc1c1ents Reactors prove rusonably re11able Addi11onal generating un11s needed Cost advantfor nuclear _, Convincing waste di1pasa1 program Public Confidence In safety Confidence In regulators and utllltles Less cont,_., -.:advantage Nallonal policy Source: Nuclear Power in an Age of Uncertainty, Energy and Materials Program, p. 7, February 1984. ------ll&f------

PAGE 130

___ -F ___ -----b-F--w-------AlliaulluNI~ -Inf-CATEGORY IV _..,.,.............., ... _......,._111 ............. _...., ____ ...,._lo __ __,_.,__.....,_,.,. Source: Rural America at the Crossroads: Networl
PAGE 131

CATEGORY IV Bn 9-C-U.S. T,us JaMWlflbl,e a,ul El/i&Nnl Eura Teclmologies Vllliom pnllllllllS ID belp U.S. cwaaime obllada ia aporling cfficimt ud nmewablc cnc,gy udmologies to mm-OECD mmtties baYe bem aabliabed by 1be U.S. Gmnmeal. indepmdml govanment agaac:iea.ud lbeprivlre-. Apllq for llllmltlliMal ~"' (A.I.D.) ~JlflJlm*I cnc,gy-ielaled redmology and tnmfa' by IIIJll)Ollin& pnfeaibility fmldmg lllllliea and by leveaging private, aiuldlarenl, and odlCI' llillrenl fbr projec:ts. s-of 1be lffengiDg is accomplilbed tbnlugh 1be M!>Bs, ming lhc Mulri-Agcnc:y Omap m Powa' Secllarlnacmlioa (MAGPI). TIie .-,y lpOIIICD nmne tnde miuiam and an aacrgy and m,iawmwllll lJUaiDa pogiam lilrbalt-rUlianala. A.I.D. hu -'>lubed al'rmteEmaprise Faad 1br Edmn Eurape ., &llill 1be mtpllll of U.S. IIDC:lloolopes, iDc:IDding eaeqy-rellted -. It also is Cftll+nrin&-,!lhlndiama6-,.rl'l.,.amfbrdlekc mrinaofeaa-iall!nagyRaeadl(PACER)tbat JlflJlm*I lbe CIIIIIIIICl1:ia of iDdigmaus cnc,gy leClmologil!I "llCI imptu,eaw ol 1nomision and dilldbatiao plmiag andllldmologiel; PACER._ bclped.ablisb rmwlia !bat lioklbeiodmlrial, COIDIDIRial, R&D,aodp--. ,,.,,,_,,, ., B-r, a4 CORBCT-Tlle Cammitree m ltmcnble &e,gy Commme and Tmde (CORECT) ii mrlri'l.-y willee led by Ille ~of&..,. and iarmlYiog 12 odlCl'Pedeal agelldes. BMaNilbcd In 1984,ilpOIIMllm!ndeofU.S.~CDCIIY!rd!ncllnps(andisapaodiogilldbutocnc,gy effidmcy), bdo&paw and bullwl pawl flUm odllr-. to lbe Uoiled SIiia fbr lade c:oafwwNaodmiAlrm,pn,ridelledllllcal--,aodbldllbelt--iilel!nagyl>elipAllislanc:eCemler (Jtl!DAC). Sudia N.....il.abanladel ID pn,ridD IDClmil:IJ UI I C ::a, pmuibility and feuibi1ily IIIDdies, fioaodoa, aod odllr .... of projec:t mppad. Bq#f-1.,.,,Bal: (Bmltillll;)--l!:liomailao....,.. rlrltU.S. acw.-t.-,y,cbllrend llllllcr lbel!spQd-lmpaltBaotktofl945,dlllllelpsfiaaocaandfac:in-lbealeofU.S.aoodland_,_tofamigo bayln,pmlic:alldyia ........ -..(9.5,l23).llamlioJlftlllW-dimctlaus,gur&DleCl,aoclilllunoce. TIie 1990 l'anilp 0pcaliml AppicljiriaD& kl (N,lic Law 101-167) Ellimbak ID dinlc:t DOC leu dWI S ,.-of ils fialocial ---lo lbe-.ay -ID lmllflllle aacrgy projec:ts. ID PY 1!19(), EldmlJIDt puridedmppmtfarund6billim ia 1111p1111.llltbo-..y-,ilpuwided&m--~.., mppmt$2.l millim, ad liad ,._. floal c ii -far lddiliaoll SIU millim, lo .-wable aacrgy projec:ts (i.e., llydluelecak,,...,.....). Mlalllin&Jlllldia& Iii --fillllimd,Eximbuk' fiscal,-1990111ppC11t for.-wabie aacrg)' projec:ts -id.,._ 7.4 ,.-of 111-1 wm1J mppalt (21). 0h'IMle I..._., Corp. (Ol'IC}-'llle 0..-Privalc .--Cap. (OPIC) II ID lodrpr.,_ cm,-riao --4 by Cmgreu.11 dmtlyfiowes projec:ts lpmlCJl'Od by U.S. privlre io-. ill -100 dndopog-. aod ,_... .,..._.,_. polilicalrilb lbrU.S. prmae ..-jg Ibale --.JIC1Dpn,ridDdinlcllaamolup1DS6millim1U..U-admedium-lizedfirm11Ddilnmlllall..-.-bup1D$SOmlllim. lllfiscal:,ar 1989, OPICpuwidedprojec:tlomoce-1iaguwerSI.Sbilliaaaoddilectloans aod Jau ~--,S208mlllim (5'). OflCllclneJapiocapmatdyunedaocl maugedl!uwi,oumwtal --Faod b ...... ----ia deYelupog-..aod Eurape that iaYOIYe renewable aacrgy,ecolDalim.,lllllaiolbleagricallln,fanst......-_aodpolludaa ,-mtioa(59,60,61). OPIChopes ID c:apilalizc Ille ltm wilh $60 millim of eqaily raised flam U.S. basinews and iollimrimol invellon IDd $40 millim jg OPJC.smluteed laog-fmD debt. l'riNM Ex,,,,, F,,,,.,,, c.,,_,rllill 'l1le l'lmle EKpart Fundiog C.orp. (PEPCO) is a gow:mmentspunscndccxmncrc:ial mrpondm .. bids far apart fiolDciDg jg lbe piYm --mini mcooditional &icbaak.....-. U.S. TNM ~Ill ,.,..,.,__The U.S. Tmde and Development l'lopan, (TOP), in lhe U.S. 1---1 C.oopaatiao A&mt:f, fimds fealibility IIUda. mnml!anc:ies. 'laiaing plOIIAllll, and other plumina swYica far projec:ts iaYOlviDg apa11 ,,,..... lbr U.S. aoodl IDd aemc:es. Its focus is primarily on large pab1ic -projec:ts (43,93). ---Source: Changing by Degrees: Steps To Reduce Greenhouse Gases, Oceans and Environment Program, p. 285, February 1991. --------------11-------------

PAGE 132

CATEGORY IV Source: Transportation of Hazardous Materials, Science, Education, and Transportation Program, p. 8, July 1986. ------------111-----------

PAGE 133

f gi g. IQ s Cl) fo :g i c~ @ 0 Ill ..... ,, !=:: a G) p ~-1' c3' a, .... (/) 5' en lb -g, 3 en A> B5' en -, 11 ol quollty In IOCoptld with lf)pn,prtlll dllOOUntl for tow-qulflly grain. Marketing by verlety --;-Ho mechenlam txlltl for v1rt1ty ld1nllUc Hon. Price ... ,., ... Farm S1oraoe ..... Loan 1910 11 prtnclpal prtcl policy. lncludt1 p,1ntlum1 and dll count1fo,1Njo,groln1 but hll not been mpon11Ya to mn1t condition,. Farm pone, In put d Cade h.. 1nc::ou,aged 1xten1IY1 on-farm 1torege and Int,,.,.., llorage. SOURCE: Olllce OI TechMIOQ, AIMHment. 119t. A,...,11.. lrortt Cominlitii ot ..,...,.. Comriiltln wtffi brold mont and lnduotry muot --tatlorl dlrwctl 1pprov1 1oronomlo rNeerch end IPPfOffl pn,potlln. Quollty too-wlrtlllll, Qu111t, II ton of mlnot 1n11...... potontlol orttfflOO but Groin not mntlng 1 1peclfled minimum quollty !Condfllorl C. fflllll lo Njeclod II ftrtt pc,t_ntofllle. Ylrtoty ,_ not ldonUlltd In marti:etlng channel. Oowammont Hlll>II.,,_ minimum prtce1 for ,_,. and """_, 1100 ntlb-HthH pr1mlumt for hlgl>qullll, grain. not currently tffecllwe. lo)'llllnlnotmntlng a minimum qulllty are Njectld It llrot point ol ..... Ylrtoty 11 not ldonlllltd In mark1t1no channlf. Ocwemmeitt 11tab1tihn mlntrnum pttc pn. or to planting. It 11 ld)ullld during tho crop rear ta account tor lnllltlon and polltlcll pre1111re. Gove,nment pollcy No lncinuv, fot-iarmthrough prtclng doll ,,. to 110,. on farm. not encourao-onfann CH' Inter-year ttorage. Franco Folfflal inKhan11m 1111 that NgUIIIH ,._ -of Wlrtlllll -on 1granomtc and qual lty crtt1rta. Grain not fflHllnO pon contract 1poclllc Ilona Olft bt ,.)lctod by 1urverlno compenr o, recelwl.!!1_ elnltor. Ver, common. Variety often 1p_1cffled In wheat contract,. Ka, pollcy 11 Eurol)llft Community ln11rv1n tlon price, wtttch lnctudla p,M1lum1 and dtecaunt1 for qu111ty loctoro. ,._ quollllot at whut equated 10 fNCIYIIUH. Flffll po11c, throvgh the Common Agrtcutturll Polle, (CAPJ hu not 1ncourao1d d vttoPrMnt of ext1n1lv1 on-tarrn ttoraot, Alto relatlvely flmltld Interyear storage due to CAP. Conldo Formal m1ch1ril1m uuct to llctn11 new v1rt1tlt1. Agronomic and qutlltr criteria gt.e,aqulf weight In te111no new vartetlH. Au1tr1ll1 FonTIII micfiinism rot towed at I prer1qu11111 tor r1le111 or ,1r11tl11. Ouallty and agronomic crlterta are used. Otvttoped etQhl grldH Wheat mu1t mHI mini for CWRS to different! mum quality 111rw11tds. ate quality. Low11l If not 1111 a11oc11ed 10 ored goes to feed mat feed market. kot. Llcen1ed grain mu11 Very common-uH varl b vl1u1lly dl11fn1ty control scheme to gul1h1bte. f1clllt111 11orot1on lnltlll producer prtce It tho prtnclpll prtco poll, or, Separate prtc11 H tabll1htd for 11cf'I grade of grain. Lower qu1llt111 of WhHI 1qu1ted to feed YIIUH. Proc:tucar dellvertes a,e r1gul1ted 10 primary tl1Y1tot1 via quo111. On-term tlorage Is tub tl1nt111. bl_CllltH. Guaranteed minimum prtce {OMP) 11 key price pollcy. II 11 t1t1blltf'led by CIUS and provldH dllletenllllt lo, quallty. Lower qu1IIIIH of wheat tqu111d 10 teed vllull. Utt of GMP ptOvldH no Incentive for delivery In po1t-h1rvest period, leadlng 10 m1n1m11 use of on-farm 110,aoe.

PAGE 134

CATEGORY IV Source: Facing America's Trash: What Next for Municipal Solid Waste?, Oceans and Environment Program, p. 8, October 1989. -------11-------

PAGE 135

___ .. ... MarU.1176 ... _, N(J.4.1977 ... Sept. 17. 19711 ... 0ct.2,. ,., ... Oct.21. ,., -LNl7-a4 A&lg.13.1N1 -LNl7.at9 ,Mt22.1N2 -LNa.373 .... 31.1N4 -LN-Oct.11.1N4 -LNINIII Ocl.111.1--LNt00-418 A&lf-23,1--LNI-,.,,,,_,.,,_ -LNIOMN ......... CATEGORY IV APPENDIX A Major Legislation Enacted Since 1975 Affecting U.S. Research and Development TIie ---,...,.,.,...,_,_ ~-~"'--Orga,mlianAd -Clln.-l'logramAd ,_,_..._ ......,_.Adol1NO __ II'.,,... --.....,--ol1ffl _,'lu#d ol1N1 ----_.,,.----_....., _.,,. and 01 a,p::wnlAct -~ _..._ol1N4 _...,.,....,,_ Adolt-~,--~---..... ~.11111 Cu.1CN11ll-Ad _eu.,_..11_ 1edlnDlog, ,--"' ,_ ~.., .. ........,_ol--and~pallcy,anda ---~--CNalad--Olflceof Sdencland ,...,.,.._ Nq,fOSfP)ln .,_ ,._ Olflceol Ille l'Nlldenl (EOP) .. ......... -.. --~.lndudlng budgll ...... and ... _..,. __ .. __ ladlnology. ... ~o1&n.v, .............. -o1 .... -. _..._DIWlopnwl1C ___ .. 11.e~o1-.. DoolgrlldlD ____ ..,... .. __ aoo,,clN.ll*'aotCllledlara~-ln--and dollnld .. -"' ... _...,...._ ..... ... -... ..,_.. .......... _.....paaq, ___ ......... ~..-.-.... __ _o1 ... --.....-.--__..,......,._..,.,..,. ~ID,_,-I QI ,. ------anOlflceol --....1n--~o1c:c.nw-._11_ ......., ..... .._ ... _.. ___ ..... ~--~-------ll(NO) ....,._~a-lar--olA&D ...-... -Mnldat ........... ................ o1-,, -A&D,ll*'a:1..,._al...,....wllll ..... __.A&D ..,... .... -s,__ .. _budgll(_,,_,.,.., .. .,.. _.._........,CSBIRJ __ .. -.a..-Ceunclln EOPID-_....RID.,...__ in-111.....--A&o.t1111ac,1,..,--Ja1n1-., _,...._._ .. .._ .... __ .,. ___ --.......--------................... __ ...__..., .... -........ ..,..,_._....,_ ---111-1n1o.....-A&o..-.--u. -.__.C.--.lar'lllcllnalogyT..-. ---~'lodlnall,ffT..-Actand .. 'adnllagy Cu,;opall--Aclt.--1118-'_III_A&D ........ --"ilgllls. ---~--Plan.,~A&Di,..OSTP. _____ _.,.....Cangr-. ................... "' .. pion. -... o.--.. o.ia..-1111,t111111C1-lha ---w,,IIIActlDal'ow __ ......S, ____ __... ... ..... ...,._A&D..,_,__ ~ISource: Federally Funded Research: Decisions for a Decade, Science, Education, and Transportation Program, p. 261, May 1991. ------11~-----

PAGE 136

CATEGORY V Category V: Examples of infonnation about the methods used to conduct an OTA assessment Appendix E Method of the Study Oklahoma (the meeting agenda is described below). Sevaal advisory panel memben participated in meetings In their 1ocalities. The objectives of these meet ingsto provide tribes and OT A staff with the op portunity to coaununic:ate clirectly with each other, and to coa6rm or conec:t the area-.pedfic health status, IOdoe0onomic:, and health aervices information OT A had emt in advana, of these meetings. In con junction with the ngianal a.tings. OTA project staff vililllldmany re.rvatioas topina-of the diverlily and apecial of the tribes. l'lajmiaas of the futun, Indian population-~ wloped IIIICler OTA guidance by Henry Cole and S. Km Yamashita of the Futures Group; computer anal,-of data-on Indian health status was pro Yided by Sinai Bjcqe of Washington. D.C.; and Paul Alexander of the law finn of Alexander 6: Kan1uner pnmded a lepl analysis of the Federal-Indian relaliomhip. (The llldhod med In the Indian health sta-tm data analysis is described below.) The adYilory panel met again on October 28-29, 1915. to lffiew a draft of the final ieport. Based on that a.ting. the--,yc:hapter-~and ............ to the panel for their l'ffllW. The draft fiaal ieport -emt for mriew to nurly 200 orpniZllilllllaad ladmduals. TheOTApn,jectclnctor also allimdecl the -1 meeting of the Natioaal Indian Health Board in Albuquerque. New Mexico, in No_._ 11-14. ltlS, at which time the draft nport In -open forum. with --1 advisory ,-1 ---putidpatiag in the disamion. The fiMl nport_......._. toOTA's Technolagy ,.__ Board on January 17, 1986. Duriagthe-of this-1. the House and Seaa Appraprialialll Subc:aaunittes for the Depart-of lhe lamiar aad ..i.i.l apncies Nqlaead that OTAmalllact a analysis of the number of beds and whether a mrgkal suite shaalcl be included in the nplr mel't hospital planned for the Ro.bud Sioux illSaudi Dakota. The~ -madeinJwte 198.S beca.-of a dispute~ the ltoaeblad tribe and the Public Haith Sa,ric,e OI\ the -and aa,iic,es of the nplr cmwnt hospital. The analysis-c:omplmd and clel"8'ell on August 1. 1985, In the fonn of an OTA mff -.ndum. OTKs condusions _.., that. ming PHS's-aileria. a30-toJS.W instead of a 25-bed hospital -warranted. but that a swgical suite not. Source: Indian Health Care, Health Program, p. 352, April 1986. --------------iml------------

PAGE 137

CATEGORY V Chapter 2 OTA's Assessment Methods1 SUMMARY OT A undertook this assessment at the re quest of the House Foreign Affairs Commit tee and the House Select Committee on Hun ger, with specific instructions to examine people's participation in projects funded by the African Development Foundation (ADFJ and the projects' results, lllltllinabWty, and replicabWty. The committees also requested an Ulellment of the Foundation's overall perfonnmce and how it could be improved. The methods used to ll8al ADF activities included extensive interviews with development experts in Wuhington and Africa, in-~cuplar ud D dolallod ...,__ fat ............ -la....__..._ .. ....___ C. Lad Fllll-ofdda_.,putk:lpmla. ,._,. ..... of ... llegla la die-clioplar. eluding ADF staff; reviews of Foundation documents in Washington, D.C. related to participation, agricultural technology, and renewable resource management in ADF funded projects; and workshops for OTA staff and contractors. Threefive.memberteamsvisited East, West, and Southam Africa for 23 days in 1987, ob serving 12 ADF-funded projects in 6 coun tries. Each group spoke with project partici pants, Foundation staff, local and national officials, U.S. ambeuadors, AID mission di rectors, and representatives of other development and ~h organizations. The teams Ule8led ADF-funded projects, re viewed the Foundation's programs in each country, and suggested congressional op tions and waya for ADF to improve its work. WHY THIS a.ssauu,n WAS ...... TID When the African Development Foundation (ADFJ WU founded. Congress intended that its grassroots approach complement other types of aid already provided to Africa by the United States. Now Congress is evaluatiq how well U.S. development usistaace to Africa is doing andADF, u one U.S...funded developmem prolflUD, bu come under sc:ru.tiny. This is part of Congress' CODtinuing attempt to eDIUnl that the United States provides the IDOlteffectiveuliltauce possible via the Apnc:yfor International Development (AID), tha Peace Corps. multilateral institutions, private voluntary organizations, andothar groups that receiw U.S. funds directly or indirectly. Although the focus here is on ADF's program, this atudy bu broader epplicability. For example. the Foundation's enabling legislationltrelNl the med. for Africans to participate in their own development and ADF's experience with part!cipatory develop-lllllllt is relevant to the pending reauthorization of the Foreign Aasiatance Act of 1961. The Howie Foreign Affairs Committee and the House Select Committee on Hunger re queated this comprehensive -ssment of ADF's funding_program.1 Their request noted the context in which U.S. aid to Africa takes place: "Strong humanitarian, political, and economic reuons exist for the U,S. to continue to participate in uaisting African countries in their efforts to develop their ~uinan and physi cal resources." As the requesting committees 23 Source: Grassroots Development: The African Development Foundation, Food and Renewable Resources Program, p. 23, June 1988. ------11------

PAGE 138

CATEGORY V 26 Flgln 2-1.--Clwt of OTA' AaaMlrrrtl Malllocla OTA_...,. __ .. ..,_ Prajecl-/ ,-y_w..p.up.._,.. l ffAM ASIE:SSUEtlr OF ADF-ANlED PROJECTS TEMI S IENT OF KIF C0UNIRW' l'IIOCIMMS INDMDUl-l 'I I ES I cans OF ~OProNS ---r--ORAFT REPOAT --' FINAL REPOAT IIIEY: .................. _..,.. __ ......_ ............... ....... D.ICE: ca.----.------this, OT A tabulated project information, includ ing grant size, duration, maturity, geographic scope, activities, goods or services funded by the ADF grant,.and intended outcomes. This analysis provided information on the range of project characteristics and average features so the countries and projects selected for visits would be representative of ADF's portfolio. The survey was limited to the 86 projects funded by ADF through September 31, 1986. Twothirds of these, or 58, dealt substantively with agriculture or renewable resources and were considered within OTA's scope of work. This assessment of funded projects must be qualified by the newness of ADF's program. Its first projects are just now nearing completion. Thus, OT A's major focus is on suggesting how ADF's overall funding program can be im proved. not on providing a definitive statement judging the results of ADF projects. De'NIDpla9 flehl T-...... To develop methods for the field teams' use, OTA held a workshop with two purposes: 1. to nlView current field evaluation methods, and 2. to develop indicators to address the criti cal issues identified in Congress' request for this study. The field l'8M8l'Ch method used is a form of "rapid rural appraisal." In rapid appraisal, teams visit the field for a short time to obtain selected information needed for policymakers. This approach Is quicker and more cost effective than some other l'8M8l'Ch methods. It relies on individual and group interviews, observa tion. and local documentation where available (12.21). In the methods workshop, OT A staff, team leaden, and three consultants with extensive evaluation experience (app. C) spent 2 days: defining the critical issues-participation, results, replicabllity, and sustainability; converting thBH definitions into concrete indicators that could be observed and measured in the field; and designing worksheets on which to collect data for each of these issues. Source: Grassroots Development: The African Development Foundation, Food and Renewable Resources Program, p. 26, June 1988. --------------,11-------------

PAGE 139

INDEX A Access to Space: The Future of U.S. Space Transportation Systems, example of summarizing options in reports, 111 Accessibility. of report, See also Reader-friendliness, 5, 63-67 Adolescent Health, Volume I: Summary and Policy Options, example of summarizing options in reports, 108 Advisory panel, for policy analysis project, 82 Advocacy, in OTA reports, See also Objectivity, 7 Analysis of options, in OTA reports, 9-10, 38, 57-59 Analytical staff at OTA, overview of, 12, 69-71 Assessment process B methods, sample of 18 OT A reports statistical analysis, 11, 45 seminar presenting results and methods of, 22 shortening of, 17 Balancing opinions. in OTA reports, 6-7 Bias, in OT A reports, See also Objectivity. 6-7, 35 Brainstorming, for policy options, 9, 12, 46 Budget, in sample of 18 OTA reports, 40 C Changing by Degrees: Steps To Reduce Greenhouse Gases,30 example of reader orientation to report, 114 example of summarizing information questions, 119 option analysis in, 57-58 Clarity in report organization, See also Readerfriendliness, 34-35, 63-66 Collegiality, 11, 25, 71, 75-76 Complex Cleanup: The Environmental Legacy of Nuclear Weapons Production, 30 example of summarizing key findings in reports, 107 Congress. See also Congressional client workings of, lecture series, 22 Congressional and Public Affairs Office lecture series, 22 Congressional client needs of, 5-9, 33-38 objectivity of OT A reports, 5-8, 35-37 reader-friendliness of OTA reports, 5-6, 34-35 timeliness of OTA reports, 5, 37-38 reader-friendliness, requirement in OTA reports, 34-35 response to needs of, by OTA, 5-9, 33-38 views of OTA policy analysis, 4-9, 35-37, 84-85 Context. See Policy problem analysis Cost, from sample of 18 OTA reports, 40 Credibility, See also Objectivity, 35-37 of OTA, for congressional staff, 5-7 Critical Connections: Communication for the Future, 30, 63, 66 example of summarizing options in reports, 109 policy analysis assessment, 48, 50-51 Criticism, of OTA policy analysis, 9-12, 56 by congressional staff, 4, 35-37 D Degrees held, OTA, See also Educational background, 12,69-70, 73 from 18 OTA reports sample, 40 E Editorial personnel, 15-16 Educationalbackground,staff, 12,69-70, 73 Empirically based policy prescriptions, See also Recommendations, 7, 16, 42, 56 Enhancing Agriculture in Africa: A Role for U.S. Development Assistance,30,63,66,67 example of reader orientation to report, 116 Enhancing the Quality of U.S. Grain for International Trads,30 example of summarizing assessment information, 121 international context/comparisons, 60-61 Ethnic and racial diversity, OTA staff, 12, 70 Exploring the Moon and Mars, 30, 38 F policy analysis, 59-60 quality presentation, within short time frame,9, 17,38,59-60,63 Facing America's Trash: What Next for Municipal Solid Waste?, example of summarizing assessment information, 122 Federal intervention alternatives to, 25 proposals for, exemplifying subjectivity of report, 6-7 from sample of 18 OTA reports analysis, 46 Federal policy, coverage in reports, from sample of OTA reports analysis of, 46 Federally Funded Research: Decisions tor a Decade, example of summarizing assessment information, 123 -------------llli------------

PAGE 140

INDEX Finding a Balance: Computer Software, Intellectual Property, and the Challenge of Technological Change, 30 Findings/options placement, project director suggestion, 24 G Goal, of policy analysis project, 27, 79 Grassroots Development: The African Development Foundation, example of information about methods used to conduct an OTA assessment, 125-126 H Holding the Edge: Maintaining the Defense Technology Base, 30 Improving Automobile Fuel Economy, 30, 38 quality presentation, within short time frame, 9, 17, 38 Index, for reader-friendliness, 34-35, 67 from sample of 18 OTA reports, 41 Indian Health Care, 30 example of information about methods used to conduct an OTA assessment, 124 example of summarizing options in reports, 11 O institutional analysis, 61-62 Information sources, for policy analysis project report, 28 Institutional analysis coverage in reports, sample of 18 OTA reports analysis of, 46 evaluation of, 10 Indian Health Care, 61-62 Interim products/services, options for management, 17 International aspects, consideration of, by project director, 25 International context/comparisons Enhancing the Quality of U.S. Grain for lntemational Trade, 60-61 evaluation of, 10 Interviews corrent congressional staff, 31, 85 project directors, for policy analysis project, 31 issues in Policy Analysis lecture series, establishment of, 18 K Kibitzerfrom senior staff, for transfer of policy analysis methods and know-how, 19-20 L Lecture series on policy analysis issues, 18 on workings of Congress, 22 Legal analysis Finding a Balance: Computer Software, Intellectual Property, and the Challenge of Technological Change, 62-63 of policy issues, 12 in sample of 18 OT A reports analysis, 46 Legislative schedule, 24 Literature review, for policy analysis project, 28, 97-98 M Making Things Better: Competing in Manufacturing, 30, 66 policy analysis assessment, 53-54 Managing editor, options for OTA management, 15 Mapping Our Genes: Genome Projects-How Big, How Fast?, 30 option analysis, 57, 59 Mentor, for new project directors, 21 Methods used to conduct OTA assessments, examples of, N Grassroots Development: The African Development Foundation, 125-126 Indian Health Care, 124 information about, at OTA, 11-12 for policy project, 41, 28-31 from sample of 18 OTA reports, 11-12, 44-45 Nuclear Power in an Age of Uncertainty, 30 example of reader orientation to report, 104 0 example of summarizing assessment questions, 117 example of summarizing key findings in reports, 106 Objectivity, 35-37 criteria, congressional client, 6-8, 35-37 of OTA, observations regarding, 4 of OT A reports, 5-8 standing panel review, option, 16-17 in sample of 18 OTA reports analysis, 42, 45-46 Office of Technology Assessment (OT A) credibility of, 6-7 culture of, 11, 69-77 folklore, 37, 46 -----------11~-----------

PAGE 141

INDEX mission, 1 policy analysis project, approach, 2, 4, 27-31 staff profile, 69-71 Options development, as component of OTA policy analysis, See also Policy options, Policy problem analysis, 38, 49 criticism of, 9-10 evidence supporting conclusions, 7, 42 objectivity of, 6-8 recommendations in, 6-8 justification underlying, 7-8 in sample of 18 OTA reports analysis, 9, 42-43, 45 Options for OTA management to address specific weaknesses, 17-19 assessment, shortening of, 7 Congress, workings of, lecture series on, 22 editorial personnel selection, 15-16 interim products/services, 17 lecture series on policy analysis, 18 on workings of Congress, 22 managing editor selection, 15 mentoring program for new project directors, 21 overview of, 14 panel for objectivity check, 16-17 "Shadow: 20 policy analysis, addressing weaknesses in, 17-19 policy clarification for empirically based policy prescriptions" section, 16 for recommendations section, 16 for reader-friendliness improvement, 13-16 reading of reports from outside program, by project director, 20-21 for responding to needs of congressional client, 13-17 seminars, staff-run, 21-22 sourcebook development, 18 staff from outside program, assignment of, 19-20 transfer of policy analysis methods, suggestions for improving, 19-22 Organization, of OTA reports, See also Readerfriendliness, 5-6 Ownership of Human TISSU8S and Cells, 30, 48, 65 policy analysis assessment, 54-56 p Panel for objectivity check, 16-17 "Shadow", advisory, 20 Pharmaceutical R & D: Costs, Risks, and Rewards, example of summarizing key findings in reports, 105 Policy analysis addressing weaknesses in, 17-19 criticism of, 9-10 early attention to, by project directors, 23 impact of OTA, 75-77 interpretation of strengths and weaknesses, through sample of 18 OTA reports analysis, 39-46 objectivity of, 6-8, 34-37 quality of, 4 recommendations in, 6-8, 36-37 sample of 18 OT A reports statistical analysis, 42-43 Sourcebook on Policy Analysis, 97-101 transfer of methods, 11 former OTA staff views, 71-72, 86-87 outside policy analysis experts views, 72-74, 88 suggestions for improving, 19-22 variation in styles of, 9, 11 Policy analysis project, OTA, approach, 2, 4, 28-31 goal of project, 27 options for OTA management, 13-22. suggestions for project directors, 23-25 Policy options in OTA reports. See also Analysis of options and Options development analysis of effects and effectiveness, importance of, 38 development of, methods used in 18 OTA reports sample, 9, 45 examples of tables and figures for summarizing, 1 08-112 presentation of, in OTA reports, 3 in sample of 18 OTA reports analysis of, 45, 56-59 technology, organizing by, 3 Policy prescriptions section, suggestions for improving, See also Empirically based policy prescriptions, 16 Policy problem, analysis, 38, 49 components of OTA reports, objectivity, 6-8 context evaluation, importance of, 38, 42-43 criticism of, 9-10 institutional analysis, evaluation of, 1 O international context, evaluation of, 1 O methods of, 11-12 in "problem-driven report, evaluation of, 10, 44 presentation of, in OT A reports, 3 in sample of 18 OTA reports analysis results, 42-43 stakeholder analysis, evaluation of, 1 O in -iechnology-driven" report, evaluation of, 10 Power On! New Tools for Teaching and Leaming, 30 stakeholder analysis, 60 -------------------llr--------------------

PAGE 142

INDEX Preventive Health Services for Medicare Beneficiaries, 30 Problem-driven" report, analysis of, 44-45 "technology-driven" report, comparison of policy analysis, 1 0 Project director first-time, 74-75 interview, for policy analysis project, 31 reading reports from other programs, 11, 20-21 seminar for, 21-22 suggestions for analysis of proposed options, 24 assumptions, 24 colleagues, as source of knowledge, 25 early attention to policy analysis, 23 federal intervention, considering alternatives to, 25 findings/options placement, 24 international aspects, consideration of, 25 legislative schedule, 24 stakeholder analysis, 24 summary chapter, 23-24 Project Directors' Peer Group, 11 Project teams at OT A composition of, 70 staff size, 40 R Racial and ethnic diversity, and OTA staff, 12, 70 Reader-friendliness, 63-67 clarity in organization, 66 criteria, congressional client, 5-6, 34-38 elements of, 64 index, 67 management options to improve, 13-16 of OT A reports, 5-6 in sample of 18 OT A reports analysis, 40-41, 45 summary chapter, 65-66 table of contents, 63, 65 Reader orientation to report, examples Changing by Degrees: Steps To Reduce Greenhouse Gases, 114 Enhancing Agriculture in Africa: A Role for U.S. DevelopmentAssistance,116 Nuclear Power in an Age of Uncertainty, 113 Technology and Handicapped People, 115 Recommendations, See also Empirically based policy prescriptions, 7-8 congressional views, 36-37 OTA policy. option for OTA management, 16 Regulatory analysis, from sample of 18 OT A reports analysis, 46 Reports, OTA audience, 1 components of, 2-3, 49 form for examination of, 89-92 function of, for U.S. Congress, 1 policy problem component objectivity of, 6-8 overview, 3, 49 potential solution component objectivity of, 6-8 overview, 3, 49 timeliness of, 8-9, 37-38 Research Assistants in Search of Empowerment (RAISE), 11 Round Trip to Orbit: Human Space Flight AltemativesSpecial Report, example of summarizing options in reports, 112 Rural America at the Crossroads: Networking tor the Future, example of summarizing assessment information, 118 s Sample of 18 OT A reports analysis, 30 examples of policy analysis, 47-63 statistical analysis, 39-46 assessment methods, 45 budget, 40 objectivity, 42 policy analysis, 42-43 "problem-driven report," 44-45 reader-friendliness, 40-41 staff size, 40 taxonomy, 44 "Technology-driven report," 44-45 timeliness, 42 statistical data, 93-95 task overview, 29 Scheduling, of OT A reports, 8-9 Seminar, for policy analysis transfer, 21-22 Serious Reduction of Hazardous Waste, 30, 67 shadow advisory panel", for advice on policy analysis, 20 shirtsleeve policy session", for advice on policy analysis, 20 Sourcebook development, 18-19, 97-101 on Policy Analysis literature, by OT A policy project team, 97-98 Staff profile, OTA, 69-71 Stakeholder analysis -------------------111---------------------

PAGE 143

INDEX discussion of, project director effort, 24 evaluation of, 10 Power On! New Tools for Teaching and Leaming, 60 in sample of 18 OTA reports analysis of, 46 Statistical analysis, of sample of 18 OT A reports, 39-46 Suggestions for project directors analysis of proposed options, 24 assumptions, 24 colleagues, as source of knowledge, 25 early attention to policy analysis, 23 federal intervention, considering alternatives to, 25 findings/options placement, 24 international aspects, consideration of, 25 legislative schedule, 24 reading reports from other programs, 20-21 seminar for, 21-22 stakeholder analysis, 24 summary chapter, 23-24 Summarizing assessment information, examples Changing by Degrees: Steps To Reduce Greenhouse Gases, 119 Enhancing the Quality of U.S. Grain for International Trade, 121 Facing America's Trash: What Next for Municipal Solid Waste?, 122 Federally Funded Research: Decisions fora Decade, 123 Nuclear Power in an Age of Uncertainty, 117 Rural America at the Crossroads: Networking for the Future, 118 Transportation of Hazardous Materials, 120 Summarizing key findings in reports, examples Complex Cleanup: The Environmental Legacy of Nuclear Weapons Production, 107 Nuclear Power in an Age of Uncertainty, 106 Pharmaceutical R & D: Costs, Risks, and Rewards, 105 Summarizing options in reports, examples Access to Space: The Future of U.S. Space Transportation Systems, 111 Adolescent Health, Volume I: Summary and Policy Options, 108 Critical Connections: Communication for the Future, 109 Indian Health Care, 110 Round Trip to Orbit: Human Space Right Alternatives-Special Report, 1 12 Summary, in OTA reports, congressional staff use of, 5-6 project director suggestions, 24 for reader-friendliness, 65-66 T Table of contents, importance of for reader-friendliness, 63-65 Tables, for reader-friendliness, examples of, 108, 110-112 Task overview, policy analysis project, 28-29, 79-88 Taxonomy, for sample of 18 OTA reports, 44 Technology, organizing options by, 3 Technology and Handicapped People, example of reader orientation to report, 115 Technology Assessment Act of 1972, 1, 7, 8 Technology Assessment Board, 1, 7 "Technology-driven" report analysis of, 44-45 "problem-driven" report, comparison, 10, 44-45 Time frame, for OTA report production, 8-9 Timeliness, 37-38 criteria, congressional client, 37-38 of OTA reports, 8-9 in sample of 18 OTA reports analysis, 42 Transfer, of policy analysis methods, 11, 19-22, 75-76 from "kibitzers", 19-20 in "shadow advisory paner format, 20 in "shirtsleeve policy session" format, 20 Transportation of Hazardous Materials, 30 w example of summarizing assessment information, 120 policy analysis, 56-57 policy option, 56-57 Weakness, of OTA policy analysis, See a/so Criticism, 9-12, 56 ------II"-------


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID EJL48NHVI_SGLZ5N INGEST_TIME 2017-06-05T17:12:22Z PACKAGE AA00055755_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES