Project Design Implementation and Evaluation Assignment provides opportunities to develop skills in decision making, problem solving and communication integrated with the skills and knowledge developed in many of the other units within the program to complete a realistic project.
The Pfizer Company is a well-known company in the field of pharmaceutical industry. The organization is currently failing to gain revenue increment; instead, the organization is currently suffering from loss in its revenue. The company is currently concentrating on its strategy of web based promotion and is working on a web project that will take care of its need for web content and promotion. In the initial segment, the organization needs to plan the whole project with specific aim of it. Few key points that are decided for this project are listed below:
Project Title: The Pfizer Pharmaceutical web solutions
Project team: 4-5 members of the Pfizer Company IT department with assistance of manufacturing department
Areas of Interest: The project seeks company interest in the world on online market for pharmaceutical demands. This demand may come from the smaller distributors or the end-users of the various Pfizer Products.
Aim of Research: The research on this web project for Pfizer is aimed towards increment in the revenue of the organization. The primary object is to compete with the other major brands like Johnson and Johnson’s, GSK, etc. The focus of this web project is to attract the customers and distributors on the internet. This project aims to earn the company a wider customer base in different countries in a staged manner.
Review: The review of the web project is required to ensure that the web project is functioning the in the desired manner and the objects are achieved by the organizational web project. In this process, the affectivity and efficiency of the web project is assessed. This study is taken as input in later process to improve the efficiency of the project.
Feasibility test: The feasibility is assessed on the implemented project to evaluate the effectiveness of the project in Pfizer organization scenario. Read more about Website Designing Assignment
Initial Critical Analysis: This study is performed to find the shortcomings of the project developed.
Project selection: The selection process if performed to select the most ideal project plan.
Project log book: the log book is maintained to keep the log of entire project development.
Cost estimation: Cost estimation of the planned project is important to ensure that it is developed within the allocated budget range.
Resource estimation: the resource estimation is performed to analyse the need of resources required for completion of the project.
Goals: The goals are defined in the planning phase of the project to clearly define and state the goals of the project.
Limitations: A web project may follow some limitations in terms of performance and the affectivity.
The web design project requires some specifications to be decided beforehand. Some of the specifications are listed below:
- Costs: Cost of the web project includes various expenses related to the development phase of the project. Such expenses or cost include various segments such as salary of the employees and experts, hardware cost, cost of labour, cost of the software licensing, etc.
- Time Scale: The time scale in the project development phase includes setting a time limit for the completion of various tasks such as the time limit of different stages of project, total project, initial review, implementation, Etc.
- Scale of operation: This stage of the specification describes the various tasks and the way of performing each task in a specified way.
- Standards: The products need to follow the local UK standards, for instance- the new drug product that will be launched by the Pfizer must follow the UK standards such as the health and safety standards.
- Legislations: The legislation requirements include following the local laws as well as the central legislations regarding the products and services offered by the organization. Some examples of this specification may include the product following legislations like data protection, copyright laws, etc.
- Ethics: For an organization to retain its reputation in the market and also enjoy the credibility issues, it is important that the services offered by the organization follow a code of ethics.
- Sustainability: The sustainability of a project is measured to ensure that the web project is actually helping the Pfizer organization in its objectives. Read more about Database Design Techniques
- Quality: The Pfizer Company wishes to present itself with a new image and the aim of this new image is to earn the reputation, credibility and a larger customer base. To get ahead of its competition, the organization needs to pay great deal of focus on the aspect of quality. The products and services offered should be of great quality and must satisfy the need of the customers.
- Fitness for purpose: The fitness for the purpose of the project is the key element that defines the applicability and effectiveness of the product for the actual purpose behind its development. The Internet is quickly becoming a great tool of marketing; hence, the aim or the project to gain customers with the help of internet is a great way of satisfying its need for increased business.
- Business data: Making this project requires some specific information and data and it is important for the use of the organization that the data and information used must meet the data protection laws.
- Resource implications: This part of product specification refers to the enlistment of different resources required and the use of such resources.
Developing such a project is a task of team work and it is important that several team roles are divided among team members to increase the productivity and to better utilize the human resources available.
Belbin first suggested in 1981 that a team of six people would be the most suitable size to enable a management team to tackle a complex problem (Keen, Thomas R 2003, 8).
Using Belbin team roles did not put the students ‘into boxes’, but provided another perspective in which they could view and gain feedback on their contribution in life (Belbin 2012, p.18).
Belbin’s typology defines the team roles in the game of BSG online. The typology is used to evaluate the behaviour of the player in the company scenario and not the personality of the player. The behaviour analysis in the Belbin typology is expressed as the ability to contribute and interact with the other players in the game environment.
Team role theory and data have a special part to play in self-management (Belbin 2012, p.22).
Dr. Meredith Belbin defined numerous team roles in the organizational environment. These team roles are widely used in the BSG online game scenario as well. The team roles are decided to make the team stronger as a whole and to cover the weaknesses of the company individuals by the stronger structure. There are nine team roles defined by the company:
- Plant: This team members is the one with high creative abilities and able to solve problems in unorthodox manners; therefore, Alpha took care of this team role for us as he is good at finding out of the box solutions.
- Evaluator: Raj the evaluation eye in the company that decides the faith of decisions and provides feedback on the team decisions.
- Co-ordinator: In the company, Issac is the one who keep up the interaction among team members to achieve the objectives.
- Resource investigators: When a company is facing risks of inevitable loss or the threat of becoming isolated, Alpha looks into all the possible ways to get the team and company out of the risk.
- Implementer: Issac makes the task happen using a strategically efficient manner.
- Completer Finisher: Raj is the person in the company who is responsible for handling the polishing task of every strategy and task designed by the company.
- Team Worker: All four members of the team are working as team workers.
- Shaper: Alpha is the shaper in this team. It is his job to ensure that the strategy of the company is going forward in the right direction and the team is not losing any momentum.
- Specialist: Specialist in a company is a person who has in-depth information about any particular aspect of the company operations. Issac has a deep knowledge of the business world; therefore, he handles the role of a specialist. (See also : Website Designing and Testing Assignment )
Planning phase of the project is the most crucial phase and it involves several key features that impact the quality and efficiency of the product. The planning phase must address a number of issues and aspects of the project overall. These issues include the following tasks and estimations-
- Time scales: The time scale specifications of the Pfizer web project’s various stages. To make the whole task more convenient to design and implement, the developing phase is divided into several stages and parts. Each part is assigned a fixed amount of optimum time which puts a time scale for the development of the entire project as well as time scale of different segments of it. In a way, it tells the team members if they are falling behind the deadline to better prevent and prepare for a delay.
- Deliverables: The Web project that is getting developed for the Pfizer organization has some inputs and some outputs that will come into the web project. With this project, the output delivered is known as the deliverable elements. The deliverable elements in case of this situation involve the products that are developed by the Pfizer Pharmaceuticals. The web project is to ensure that the deliverables are delivered to the customers in the intended way.
- Milestones: Milestones are defined as the achievements in the project. Such milestones may comprise of different segments or stages of the project. For this project which is getting developed for Pfizer Pharmaceuticals, there may be a number of milestones defined for the organization and such milestones may represent a number of different stages. The milestone work as the rally points where the progress of the project can be analysed.
- Quality Assurance System: The Pfizer organization is willing to earn great reputation among the customers and distributors of pharmaceutical products by giving them a product of superior quality. To ensure quality of the product in every single stage, a quality assurance system is required. For example- an organization level review committee that checks the quality of the product is a great system to achieve higher quality with the project.
- Quality plans: Quality plans are the plans that are developed by the team members of the project report to achieve higher quality project. For example- keeping the objectives defined is a good practice to increase the quality of the project.
Story Board for Project:
The efficiency of the web project for Pfizer organization also focuses on the timely solution of the various stages and issue involved in the development phase of the project. The whole task is divided into several key stages. One task may or may not have dependant relationship with the other tasks. The timely completion of the project depends on the definition of the various stages of the project. A Gantt chart is an effective way of defining the various tasks that are part of this project and using such a chart will provide a number of benefits to the team members.
The Gantt chart provides effective methods of providing information about the inter-dependency and the timescale of the various parts of the project. This minimizes the chances of any confusion and ensures that the project is followed using a precise timescale method. Following is a Gantt chart for the Pfizer web project.
The development of a web project for the Pfizer organization requires an appropriate solution to be used as an approach followed for the development of an effective project. To design an approach that is most effective in this kind of project development, it is essential for the organization to implement a fact finding solution. For this purpose, the Pfizer organization can apply different methods such as questionnaire, interview, etc. However, the interview method of numerous distributors of pharmaceutical products is a great way of dealing with this. The interview phase should consist of the following questions. Read more about Database Design Techniques
|Q.1||What Pfizer products are more appealing to the customers?|
|Q.2||Is there any pharmaceutical product line where the demand is high but the choices or competition is low?|
|Q.3||Any potential marketing strategy that is very likely to bring more customers under the Pfizer customer base|
|Q.4||What products performed badly which are manufactured by the Pfizer organization?|
|Q.5||What new drug product could benefit the Pfizer organization the most in the current market scenario?|
|Q.6||Why do people select other brands over Pfizer products?|
|Q.7||What are the common complains the distributors or sellers get with the Pfizer products?|
|Q.8||Suggestions for the Pfizer products|
This fact finding technique will help the organization in better evaluating its current strategy by understanding the shortcomings and strengths of it. Knowing the answer to the above questionnaire reveals the potential of new product ideas as well. This questionnaire should be handed over to three key-peoples in the product sales chain. These three customer base include customers, sellers and the distributors. Understanding the opinion of all three will provide an integrated report about the efficiency of the products.
After the implementation of this questionnaire technique, the organization will find out the answer to many important questions that will help the organization to decide its strategy in the market. Based on the poor performance of the existing products, the company may either replace them or abandon them. This questionnaire also tells the company about the opinion of the customers which helps the organization in improving its services.
The use of a fact finding technique is to find the answers to some important questions that reveals the performance of the products and also enables the company to better understand its market strategy with respect to the market. These findings are then used in the market predicament and strategy planning. The things that are analysed in this segment includes following things:
- Shortcomings of the project: the approach of fact finding reveals some information to the organization. The questionnaire mentioned above finds out the various shortcomings that are felt by the distributors and the customers of the Pfizer organization. By knowing the shortcomings of the existing products or marketing policies, the company can better assess its new strategy to gain maximum benefit from it.
- Knowing what products are performing poorly: The questionnaire reveals to the organization the details of the products that are receiving bad reviews or are getting poor response from the buyers. The distributor can easily tell this to the company. In such a situation, the company can follow any of the three possible solutions- closing the production of the poorly received product, replacing it with some other product, changing or improving the marketing strategy of this particular product as a part of its rebranding.
- Finding potential for a new product: The Pfizer Organization is looking at the opportunity of introducing a pharmaceutical product in a new category to increase its market reach. It is essential for the organization to analyse the market scenario to assess the market segment in which the competition is comparatively weak and the demand is significantly higher. Such a situation is ideal for the launch of new product. The above described questionnaire provides the potential for a new product in the market.
- Gathering suggestions from the distributors and customers: The aim of the mentioned questionnaire is to find out the opinion of the customers about the brand Pfizer and the various pharmaceutical products manufactured by the company. Understanding what needs to be changed in the goodwill of the organization can better help the team-members in making an effective strategy to re-brand the Pfizer Pharmaceutical organization.
The intent for the web project that is being developed for the Pfizer organization is to encourage the customers to purchase the products manufactured by the company. This aim of the project is to be achieved by offering more marketing campaigns in the digital world of internet than the traditional means of marketing. The web project is to ensure that the products, product descriptions, and other shipping details are readily available on the web project of the organization. The web project can help the organization in a number of objectives such as becoming number one pharmaceutical manufacturer online which comprises of a significant market share, making a brand name on the internet, promoting various products manufactured by the organization, and to expand the service region of the Pfizer organization. By means of a web project, the organization can increase its reach on a global scale and quickly gain popularity in a wider region.
Based on the preliminary planning phase and the questionnaire produced by the organization, there are various possibilities of future developments and research using this web project? Some of the potential further studies and developments include following:
- Expanding the business in different countries: The biggest advantage of the web project developed by the Pfizer organization is the ability of expanding the business to more countries easily. It may require some additional research to find out the countries to expand the business to. The web project makes it for the organization to easily run and administer the pharmaceutical business in multiple countries. The project allows placing orders online and watching the product descriptions using the internet to increase the marketing of the products.
- Developing a new drug: through the fact finding technique that is brought to use in this project development phase, the potential of a new product in the pharmaceutical line is evaluated. This new product can easily gain the brand of Pfizer additional customer base and may introduce the brand to new people. Through the market analysis and use of questionnaire, the organization can easily develop a new drug in a category that is having least competition and bigger potential.
- Re-branding the organization: after the constant decrement in the revenues of Pfizer organization. It is understood that the organization is in need of a new brand image. Internet can be brought to use in this purpose with the help of proper marketing strategy. After planning the new marketing tactics, viral marketing and aggressive marketing can provide the organization the exposure it needs to give a makeover to its brand image.
The Pfizer organization is seeking the objective of increasing the customer base and also establishing a brand image for the company. The aim of this web project developing for Pfizer Pharmaceutical is to enable the organization to enhance the goodwill of the company using online methods. The web project will have high impact on the organization and its marketing strategy. The planning phase of this project is following an approach that is estimated to help the organization for a long period of time. In this duration, the primary object is to increase the revenue significantly by all means, and to achieve the number one status in the pharmaceutical industry.
The impact of this web project will affect the strategy of the organization the most. Judging by the constantly decreasing revenues of the company, it is possible to believe that the existing strategy is not an effective one in the given conditions and increased competition. The focus of the organization right now is on a limited share of market in a fixed region that is highly competitive due to the existence of major pharmaceutical brands such as Johnson and Johnson’s and GSK. The new pattern will help the organization with a better set of strategy as it will help the company in expanding its business and will also provide a way of reaching out to a larger customer base.
The other thing that will get most affected by the new web project developed for Pfizer organization is the marketing strategy of the organization. Currently the company is relying on the traditional methods of marketing and that is not working very well for the organization as the revenues are going down. The new project is concentrating on marketing for the company using internet marketing. This is a much cheaper method than the existing way of traditional marketing. This will cut the cost of marketing and will promote the new brand image of the company to a larger customer base effectively.
For an organization like Pfizer Pharmaceutical, going through the times of decreasing revenues in a highly competitive market is a hard time to come up with effective strategies. As the other competitive companies are still relying on the traditional marketing methods, Pfizer can benefit from the use of extensive internet marketing with web project as this influences a significant market share.
For the effective implementation of the web project in the Pfizer organization environment, it is essential to note and review the progress of the project. The real time objectives are to be analysed by the team members to effectively measure the efficiency of this project. To analyse the project, there needs to be some staged review performed as well as the analysis of final outputs. The overall performance index is maintained to ensure that the desired goals are achievable with this project.
The first thing that the designed and planners need to think about is the potential of the project and a measure of its success probability. The team members should analyse the reasons for which the project may succeed or fail. As the revenue of the company is constantly decreasing therefore, it is important to use the finances in the right strategy. A sound analysis of the strategy is important for this project.
There are many factors contributing towards success of this web project, such as the freshness of the idea, and its ability to reach a wider customer base. This method of marketing is cheap and it also allows the organization to expand to new countries with high potentials. The web project is also helping the organization in creating a new brand image. However, there are some risks involved as well such as the idea is not that popular in pharmaceutical industry and the resources required are a little expensive for a company that is suffering from the low revenues.
While it is important to measure the efficiency of the web project at different stages of its development and designing, it is also important to keep a defined objective list to ensure that the project is targeted in the right direction. The final must be the objectives that were defined for the project in the planning phase. A project is called successful only if its final output matches with the initial requirement and objectives of the system. The things that need to be taken care of in this segment include the following final outputs:
- Establishing the Pfizer organization as the number one pharmaceutical brand in the digital world of internet
- Expanding the business of Pfizer Company to many new countries of high potential
- Changing the low revenue situation
- Helping the company in renewing its brand image
- Helping the organization is launching a new drug upon market research to reach to a larger group of customers
Pfizer Organization is looking for a bigger share of market to increase its revenues and to establish itself as the number one company in the pharmaceutical industry. The organization is employing a web project to get the lead above other major competitors such as Johnson and Johnson’s, GlaxoSmithKline, Roche, etc. The planning phase of the web project will provide the team members ability to design a strategy that is highly efficient in this market scenario. The accuracy and efficiency of the strategy developed depends on the quality of findings during the research and analysis phase of the web project. Learn about IT infrastructure Strategy Assignment.
Following are the findings about the need for a web project which will be implemented in the Pfizer business environment:
- About the strategy: Pfizer needs to stay in the business and for that it needs to increase the revenues. The newly developed strategy concentrates on the expansion of the business to new countries to gather a larger market share. The launch of products in more countries will help exposure of the brand to more people.
- Brand image: The existing image of the Pfizer Company is of a family owner small pharmaceutical firm. One of the objectives for the new web project is to establish the brand with a new image of a market leader and a pharmaceutical giant. The new brand image will require input from the great marketing strategy.
- About the competition: the Pfizer organization is competing with the major pharmaceutical companies on a global level. However, it is the underdog in this competition due to lack of marketing in last couple of years. The world is rapidly making internet as its new home and the new strategy will allow Pfizer a way to get an edge over its competition.
- Need for a new product: To expand its reach in the market, the organization is looking forward to manufacture new type of drug products. For this purpose, a thorough research is performed and a questionnaire is utilized to get the opinion of customers and distributors. The new product will surely be launched in categories where the competition is not as high and the potential is very significant.
- Setting Pfizer as the internet leader in industry: Internet is the most popular method of marketing today and it is attracting buyers and users of all products. The market share of internet marketing influenced buyers is very significant in the pharmaceutical industry. The object of the Pfizer web project is to use SEO tools, aggressive marketing and other methods to make the company number one in the market scenario.
The plan that is brought in action to increase the revenues of the Pfizer organization is following the concept of web project. The aim of this project is to achieve higher revenues while establishing new brand image of the company to gain a larger market share. Following are some of the aspects that are in reference with the web project of Pfizer Company:
- Project plan: The project plan followed by the web project is to provide an online platform for the buyers of different kinds of pharmaceutical products offered by the company. This online platform will provide a place where the distributors as well as the end users can place orders. This is a simple approach that can be implemented in a wider region as well. This plan will help the organization in expanding the services to more countries.
- Time scale of project: The project is divided into multiple parts and stages. However, implementing it in the real Pfizer business environment will require 3-4 months and expanding in different countries will be performed in a sequential manner. The time scale reveals the expected time of output as well and reminds the team members in case of a delay. Different time scales are set for different projects.
- Findings: Findings are the key issues and aspects of the web project that are found in the analysis and research phase of the designing. Such findings include the requirement, level of competition, potential for new drug product, etc. The findings are very useful for the project development as it is used as the input in the planning phase. Proper and well documented findings are must for better utilization of this project.
- Role play of the project plan: in the business plan of the Pfizer organization, the web project will play an important role for future strategies and outputs. This project will play the deciding role in the new brand image of the organization as well as setting tone for the new marketing strategy of the organization. The object of this project will be based on the aim set for the organization. The company management is willing to expand the business to new markets in different countries and the selection of such markets will be done based on the web project.
- Belbin R Meredith (2012). Management Teams. Routledge. 18.
- Belbin R Meredith (2012). Team Roles at Work. Routledge. 22.
- Pfizer. (2006). Pfizer Inc.: Evolving to meet the needs of a changing society. Available: http://www.pfizer.com/about/history/history. Last accessed 12th April 2014.
- Keen Thomas R. (2003). Creating Effective and Successful Teams. Purdue University Press. 8.
- Friedlein Ashley (2001). Web Project Management: Delivering Successful Commercial Web Sites. Morgan Kaufmann. 34-43
Programming Assignment Help provides plagiarism free assignments i.e. all our solutions are genuine and written by best technical assignment writers who have years of experience. Get a quote now.Project Design Implementation and Evaluation Assignment
Research designs for outcome evaluations are meant to assess principal and relative effects. Ideally, to assess the effect of an intervention on program participants, one would like to know what would have happened to the same participants in the absence of the program. Because it is not possible to make this comparison directly, inference strategies that rely on proxies have to be used. Scientists use three general approaches to construct proxies for use in the comparisons required to evaluate the effects of interventions: (1) nonexperimental methods, (2) quasi-experiments, and (3) randomized experiments. The first two are discussed below, and randomized experiments are discussed in the subsequent section.
Nonexperimental and Quasi-Experimental Designs3
The most common form of nonexperimental design is a before-and-after study. In this design, pre-intervention measurements are compared with equivalent measurements made after the intervention to detect change in the outcome variables that the intervention was designed to influence.
Although the panel finds that before-and-after studies frequently provide helpful insights, the panel believes that these studies do not provide sufficiently reliable information to be the cornerstone for evaluation research on the effectiveness of AIDS prevention programs. The panel's conclusion follows from the fact that the postintervention changes cannot usually be attributed unambiguously to the intervention.4 Plausible competing explanations for differences between pre-and postintervention measurements will often be numerous, including not only the possible effects of other AIDS intervention programs, news stories, and local events, but also the effects that may result from the maturation of the participants and the educational or sensitizing effects of repeated measurements, among others.
Quasi-experimental and matched control designs provide a separate comparison group. In these designs, the control group may be selected by matching nonparticipants to participants in the treatment group on the basis of selected characteristics. It is difficult to ensure the comparability of the two groups even when they are matched on many characteristics because other relevant factors may have been overlooked or mismatched or they may be difficult to measure (e.g., the motivation to change behavior). In some situations, it may simply be impossible to measure all of the characteristics of the units (e.g., communities) that may affect outcomes, much less demonstrate their comparability.
Matched control designs require extraordinarily comprehensive scientific knowledge about the phenomenon under investigation in order for evaluators to be confident that all of the relevant determinants of outcomes have been properly accounted for in the matching. Three types of information or knowledge are required: (1) knowledge of intervening variables that also affect the outcome of the intervention and, consequently, need adjustment to make the groups comparable; (2) measurements on all intervening variables for all subjects; and (3) knowledge of how to make the adjustments properly, which in turn requires an understanding of the functional relationship between the intervening variables and the outcome variables. Satisfying each of these information requirements is likely to be more difficult than answering the primary evaluation question, "Does this intervention produce beneficial effects?"
Given the size and the national importance of AIDS intervention programs and given the state of current knowledge about behavior change in general and AIDS prevention, in particular, the panel believes that it would be unwise to rely on matching and adjustment strategies as the primary design for evaluating AIDS intervention programs. With differently constituted groups, inferences about results are hostage to uncertainty about the extent to which the observed outcome actually results from the intervention and is not an artifact of intergroup differences that may not have been removed by matching or adjustment.
A remedy to the inferential uncertainties that afflict nonexperimental designs is provided by randomized experiments. In such experiments, one singly constituted group is established for study. A subset of the group is then randomly chosen to receive the intervention, with the other subset becoming the control. The two groups are not identical, but they are comparable. Because they are two random samples drawn from the same population, they are not systematically different in any respect, which is important for all variables—both known and unknown—that can influence the outcome. Dividing a singly constituted group into two random and therefore comparable subgroups cuts through the tangle of causation and establishes a basis for the valid comparison of respondents who do and do not receive the intervention. Randomized experiments provide for clear causal inference by solving the problem of group comparability, and may be used to answer the evaluation questions "Does the intervention work?" and "What works better?"
Which question is answered depends on whether the controls receive an intervention or not. When the object is to estimate whether a given intervention has any effects, individuals are randomly assigned to the project or to a zero-treatment control group. The control group may be put on a waiting list or simply not get the treatment. This design addresses the question, "Does it work?"
When the object is to compare variations on a project—e.g., individual counseling sessions versus group counseling—then individuals are randomly assigned to these two regimens, and there is no zero-treatment control group. This design addresses the question, "What works better?" In either case, the control groups must be followed up as rigorously as the experimental groups.
A randomized experiment requires that individuals, organizations, or other treatment units be randomly assigned to one of two or more treatments or program variations. Random assignment ensures that the estimated differences between the groups so constituted are statistically unbiased; that is, that any differences in effects measured between them are a result of treatment. The absence of statistical bias in groups constituted in this fashion stems from the fact that random assignment ensures that there are no systematic differences between them, differences that can and usually do affect groups composed in ways that are not random.5 The panel believes this approach is far superior for outcome evaluations of AIDS interventions than the nonrandom and quasi-experimental approaches. Therefore,
To improve interventions that are already broadly implemented, the panel recommends the use of randomized field experiments of alternative or enhanced interventions.
Under certain conditions, the panel also endorses randomized field experiments with a nontreatment control group to evaluate new interventions. In the context of a deadly epidemic, ethics dictate that treatment not be withheld simply for the purpose of conducting an experiment. Nevertheless, there may be times when a randomized field test of a new treatment with a no-treatment control group is worthwhile. One such time is during the design phase of a major or national intervention.
Before a new intervention is broadly implemented, the panel recommends that it be pilot tested in a randomized field experiment.
The panel considered the use of experiments with delayed rather than no treatment. A delayed-treatment control group strategy might be pursued when resources are too scarce for an intervention to be widely distributed at one time. For example, a project site that is waiting to receive funding for an intervention would be designated as the control group. If it is possible to randomize which projects in the queue receive the intervention, an evaluator could measure and compare outcomes after the experimental group had received the new treatment but before the control group received it. The panel believes that such a design can be applied only in limited circumstances, such as when groups would have access to related services in their communities and that conducting the study was likely to lead to greater access or better services. For example, a study cited in Chapter 4 used a randomized delayed-treatment experiment to measure the effects of a community-based risk reduction program. However, such a strategy may be impractical for several reasons, including:
sites waiting for funding for an intervention might seek resources from another source;
it might be difficult to enlist the nonfunded site and its clients to participate in the study;
there could be an appearance of favoritism toward projects whose funding was not delayed.
Although randomized experiments have many benefits, the approach is not without pitfalls. In the planning stages of evaluation, it is necessary to contemplate certain hazards, such as the Hawthorne effect6 and differential project dropout rates. Precautions must be taken either to prevent these problems or to measure their effects. Fortunately, there is some evidence suggesting that the Hawthorne effect is usually not very large (Rossi and Freeman, 1982:175-176).
Attrition is potentially more damaging to an evaluation, and it must be limited if the experimental design is to be preserved. If sample attrition is not limited in an experimental design, it becomes necessary to account for the potentially biasing impact of the loss of subjects in the treatment and control conditions of the experiment. The statistical adjustments required to make inferences about treatment effectiveness in such circumstances can introduce uncertainties that are as worrisome as those afflicting nonexperimental and quasi-experimental designs. Thus, the panel's recommendation of the selective use of randomized design carries an implicit caveat: To realize the theoretical advantages offered by randomized experimental designs, substantial efforts will be required to ensure that the designs are not compromised by flawed execution.
Another pitfall to randomization is its appearance of unfairness or unattractiveness to participants and the controversial legal and ethical issues it sometimes raises. Often, what is being criticized is the control of project assignment of participants rather than the use of randomization itself. In deciding whether random assignment is appropriate, it is important to consider the specific context of the evaluation and how participants would be assigned to projects in the absence of randomization. The Federal Judicial Center (1981) offers five threshold conditions for the use of random assignment.
Does present practice or policy need improvement?
Is there significant uncertainty about the value of the proposed regimen?
Are there acceptable alternatives to randomized experiments?
Will the results of the experiment be used to improve practice or policy?
Is there a reasonable protection against risk for vulnerable groups (i.e., individuals within the justice system)?
The parent committee has argued that these threshold conditions apply in the case of AIDS prevention programs (see Turner, Miller, and Moses, 1989:331-333).
Although randomization may be desirable from an evaluation and ethical standpoint, and acceptable from a legal standpoint, it may be difficult to implement from a practical or political standpoint. Again, the panel emphasizes that questions about the practical or political feasibility of the use of randomization may in fact refer to the control of program allocation rather than to the issues of randomization itself. In fact, when resources are scarce, it is often more ethical and politically palatable to randomize allocation rather than to allocate on grounds that may appear biased.
It is usually easier to defend the use of randomization when the choice has to do with assignment to groups receiving alternative services than when the choice involves assignment to groups receiving no treatment. For example, in comparing a testing and counseling intervention that offered a special "skills training" session in addition to its regular services with a counseling and testing intervention that offered no additional component, random assignment of participants to one group rather than another may be acceptable to program staff and participants because the relative values of the alternative interventions are unknown.
The more difficult issue is the introduction of new interventions that are perceived to be needed and effective in a situation in which there are no services. An argument that is sometimes offered against the use of randomization in this instance is that interventions should be assigned on the basis of need (perhaps as measured by rates of HIV incidence or of high-risk behaviors). But this argument presumes that the intervention will have a positive effect—which is unknown before evaluation—and that relative need can be established, which is a difficult task in itself.
The panel recognizes that community and political opposition to randomization to zero treatments may be strong and that enlisting participation in such experiments may be difficult. This opposition and reluctance could seriously jeopardize the production of reliable results if it is translated into noncompliance with a research design. The feasibility of randomized experiments for AIDS prevention programs has already been demonstrated, however (see the review of selected experiments in Turner, Miller, and Moses, 1989:327-329). The substantial effort involved in mounting randomized field experiments is repaid by the fact that they can provide unbiased evidence of the effects of a program.
Unit of Assignment.
The unit of assignment of an experiment may be an individual person, a clinic (i.e., the clientele of the clinic), or another organizational unit (e.g., the community or city). The treatment unit is selected at the earliest stage of design. Variations of units are illustrated in the following four examples of intervention programs.
Two different pamphlets (A and B) on the same subject (e.g., testing) are distributed in an alternating sequence to individuals calling an AIDS hotline. The outcome to be measured is whether the recipient returns a card asking for more information.
Two instruction curricula (A and B) about AIDS and HIV infections are prepared for use in high school driver education classes. The outcome to be measured is a score on a knowledge test.
Of all clinics for sexually transmitted diseases (STDs) in a large metropolitan area, some are randomly chosen to introduce a change in the fee schedule. The outcome to be measured is the change in patient load.
A coordinated set of community-wide interventions—involving community leaders, social service agencies, the media, community associations and other groups—is implemented in one area of a city. Outcomes are knowledge as assessed by testing at drug treatment centers and STD clinics and condom sales in the community's retail outlets.
In example (1), the treatment unit is an individual person who receives pamphlet A or pamphlet B. If either "treatment" is applied again, it would be applied to a person. In example (2), the high school class is the treatment unit; everyone in a given class experiences either curriculum A or curriculum B. If either treatment is applied again, it would be applied to a class. The treatment unit is the clinic in example (3), and in example (4), the treatment unit is a community.
The consistency of the effects of a particular intervention across repetitions justly carries a heavy weight in appraising the intervention. It is important to remember that repetitions of a treatment or intervention are the number of treatment units to which the intervention is applied. This is a salient principle in the design and execution of intervention programs as well as in the assessment of their results.
The adequacy of the proposed sample size (number of treatment units) has to be considered in advance. Adequacy depends mainly on two factors:
How much variation occurs from unit to unit among units receiving a common treatment? If that variation is large, then the number of units needs to be large.
What is the minimum size of a possible treatment difference that, if present, would be practically important? That is, how small a treatment difference is it essential to detect if it is present? The smaller this quantity, the larger the number of units that are necessary.
Many formal methods for considering and choosing sample size exist (see, e.g., Cohen, 1988). Practical circumstances occasionally allow choosing between designs that involve units at different levels; thus, a classroom might be the unit if the treatment is applied in one way, but an entire school might be the unit if the treatment is applied in another. When both approaches are feasible, the use of a power analysis for each approach may lead to a reasoned choice.