One of the primary advantages of international cooperative development includes:

Answer:
  • Reduced cost
Wrong Answers:
  • Foreign dependency
  • Increased administrative requirements
  • Improved currency exchange

An operationally meaningful, time critical, and cost effective increment in capability above the threshold value for performance is the ___________.

Answer:
  • Objective value
Wrong Answers:
  • CAIV adjustment
  • Ultimate program goal
  • Key Performance Parameter (KPP)

Contracts greater than $100,000 are reserved for small business.

Answer:
  • False

Tracking the performance of risk-handling actions against established metrics is involved in which step of the risk management model?

Answer:
  • Risk monitoring
Wrong Answers:
  • Riskplanning
  • Risk handling
  • Risk assessment

The National Environment Policy Act (NEPA) can affect defense acquisition program managers because:

Answer:
  • NEPA requires environmental impacts to be considered in the decision making process for acquisition programs
Wrong Answers:
  • Deputy Secretary of DefenseViolators of the NEPA process are subject to fines and prison sentences
  • NEPA requires environmental impacts to be completely eliminated before approval will be granted for an acquisition program
  • NEPA requires program managers to take remedial action to restore the environment before they can initiate a new acquisition program

When financial managers adjust for cost inflation in programming and budgeting, they usually:

Answer:
  • Prepare initial estimates in base-year (constant) dollars and escalate costs into then-year (current) dollars
Wrong Answers:
  • Refer to Federal Reserve Board guidelines on outlay rates and income flows
  • Expect investment appropriations, such as Procurement and RDTE, to have quick outlay rates
  • Revise their POM submission based on the Program Budget Decision (PBD) released by the Deputy Secretary of Defense

Which of the following are "Best Practices" that may be appropriate for software intensive programs? (Select all that apply)

Possible Answers (Please comment if you have the answer, Thank You):
  • Consider the risks of reusing existing software
  • Use metrics to manage and monitor risk
  • To speed development, only test software at completion
  • Generate new code whenever possible

Small business set-asides require the expectation that at least three capable small disadvantaged businesses will submit offers.

Answer:
  • False (As per Dawn: Actually, the answer is false. They only expect two (or more) offers.  Thanks so much for your support Dawn.

Responsibility for awarding the contract and any modifications for government contracts rests with the _______.

Answer:
  • Contracting Officer
Wrong Answers:
  • Program Manager
  • Head of the Contracting Agency
  • Defense Contract Audit Agency

Who would certify the interoperability performance of a command and control system used by both the Army and the Navy?

Answer:
  • Joint Interoperability Test Command (JITC)
Wrong Answers:
  • Joint Staff, J-8
  • Army Test and Evaluation Command (ATEC) and Navy Operational Test and Evaluation Activity (OPTEVFOR)
  • Joint Requirement Oversight Council (JROC)

When both parties agree to definitize a contract change before the work is begun, the change is made contractually through a __________.


Answer:
  • Supplemental Agreement 
Wrong Answers:
  • Constructive Change 
  • Change Order 
  • None of the answers are correct

Which of the following is considered to be a practice of "Lean Manufacturing"?

Answer:
  • Low levels of inventory 
Wrong Answers:
  • Highly specialized workforce 
  • Centralized management control 
  • Mass production 

A "not invented here" mentality is considered


Answer:
  • A barrier to IPPD implementation
Wrong Answers:
  • A key tenet of Integrated Product and Process Development (IPPD)
  • A guiding principle of successful Integrated Product Teams (IPTs
  • A "Rule of the Road" for effective IPPD implementation

Of the following types of contracts, which one gives the contractor the most incentive to control costs?

Answer:
  • Firm Fixed Price
Wrong Answers:
  • Fixed Price Incentive
  • Cost Plus Incentive Fee
  • Cost Plus Fixed Fee
Link: ELO: 13c. Relate the effect of the various types of contracts on contractor motivation.












According to learning curve theory, when the quantity of an item produced doubles, the labor hours required to produce the item _______.

Answer:
  • Decrease by a fixed percentage 
Wrong Answers:
  • Increase at a variable rate 
  • Remain constant at the unit level 
  • Vary in proportion to the five elements of manufacturing 





Software Problems: Summary

The following learning objectives are covered in this lesson:
  • Apply a generic problem-solving model to an acquisition situation.
  • Apply one or more selected qualitative tools (e.g., fishbone diagram) to resolve a problem.
  • Identify developer practices essential for creation of high quality software.
  • Identify the requirements for interoperability testing.
1.    One problem-solving technique is the cause and effect diagram or "fishbone" diagram.  By analyzing all the possible causes of a problem, the fishbone diagram focuses on determining the root cause of a problem, rather than on symptoms or solutions.  Typically, the fishbone diagram begins with a statement of the problem in a box on the right side of the diagram--the "head" of the fish.  Then categories of major causes are identified and drawn to the left--the "bones" of the fish.  These major causes are broken down into all the related causal factors that might contribute to the major causes.  Finally, the causal factors are examined and narrowed down to the most significant elements of the problem to determine the ultimate cause or causes.



2. The Software Program Managers Network has identified several software best practices based on interviews with software experts and industry leaders.  Here is a synthesized list of some of those characteristics, which are essential for the creation of high quality software: Adopt Continuous Program Risk Management
Risk management is a continuous process beginning with the definition of the concept and ending with system retirement. Risks need to be identified and managed across the life of the program.
Estimate Cost and Schedule Empirically
Initial software estimates and schedules should be looked on as high risk due to the lack of definitive information available at the time they are defined.
Use Metrics to Manage
All programs should have in place a continuous metrics program to monitor issues and determine the likelihood of risks occurring. Metrics information should be used as one of the primary inputs for program decisions.
Track Earned Value
Earned value requires each task to have both entry and exit criteria and a step to validate that these criteria have been met prior to the award of the credit. Earned value credit is binary with zero percent being given before task completion and 100 percent when completion is validated.
Track Defects against Quality Targets
All programs need to have pre-negotiated quality targets, which is an absolute requirement to be met prior to acceptance by the customer. Programs should implement practices to find defects early in the process and as close in time to creation of the defect as possible and should manage this defect rate against the quality targets. Meeting quality targets should be a subject at every major program review.
Treat People as the Most Important Resource
A primary program focus should be staffing positions with qualified personnel and retaining this staff through the life of the project. The program should not implement practices (e.g., excessive unpaid overtime) that will force voluntary staff turnover. The effectiveness and morale of the staff should be a factor in rewarding management.
Adopt Life Cycle Configuration Management
All programs, irrespective of size, need to manage information through a preplanned configuration management (CM) process. This discipline requires as a minimum:
  • Control of shared information
  • Control of changes
  • Version control
  • Identification of the status of controlled items(e.g., memos, schedules) and
  • Reviews and audits of controlled items.
Manage and Trace Requirements
Before any design is initiated, requirements for that segment of the software need to be agreed to. Requirements need to be continuously traced from the user requirement to the lowest level software component.
Use System-Based Software Design
All methods used to define system architecture and software design should be documented in the system engineering management plan and software development plan and be frequently and regularly evaluated through audits conducted by an independent program organization.
Ensure Data and Database Interoperability
All data and database implementation decisions should consider interoperability issues and, as interoperability factors change, these decisions should be revisited.
Define and Control Interfaces
Before completion of system-level requirements, a complete inventory of all external interfaces needs to be completed. Internal interfaces should be defined as part of the design process. All interfaces should be agreed upon and individually tested.
Design Twice, Code Once
Traceability needs to be maintained through the design and verified as part of the inspection process. Design can be incrementally specified when an incremental release or evolution life cycle model is used provided the CM process is adequate to support control of incremental designs.
Assess Reuse Risks and Costs
The use of reuse components, COTS (Commercial Off-The-Shelf), GOTS (Government Off-The-Shelf) or any other non-developmental items (NDI) should be a primary goal, but treat any use as a risk and manage it through risk management.
Inspect Requirements and Design
All products that are placed under CM and are used as a basis for subsequent development need to be subjected to a formal inspection defined in the software development plan. The program needs to fund inspections and track rework savings.
Manage Testing as a Continuous Process
All testing should follow a preplanned process, which is agreed to and funded. Every test should be described in traceable procedures and have pass-fail criteria.
Compile and Smoke Test Frequently
Smoke testing should qualify new capability or component only after successful regression test completion. All smoke tests should be based on a traceable procedure and run by an independent organization (not the engineers who produced it). Smoke test results should be visible and provided to all project personnel.

3.    Interoperability problems can best be identified through the use of actual, live systems to mitigate risk.  Joint interoperability is defined as the ability of systems to provide services to and accept services from other systems and to use the services exchanged to enable them to operate effectively together.  The Joint Interoperability Test Command is responsible for verifying the interoperability of systems to the parameters outlined in the ICD, CDD, CPD and ISP.

 

APB Breaches


The following learning objectives are covered in this Lesson:
  • Identify when program deviations occur and the actions that should be taken by the acquisition manager.
  • Relate the Acquisition Program Baseline (APB) to planning, control, and risk management in attaining cost, schedule and performance goals.

1. A program deviation occurs when the Program Manager has reason to believe that the current estimate for a given cost, schedule or performance parameter does not meet the threshold value specified for that parameter in the Acquisition Program Baseline. The PM must follow certain procedures whenever this occurs:
  • The PM must immediately inform the Milestone Decision Authority (MDA) when a program deviation occurs.
  • Within 30 days of the deviation, the PM must explain to the MDA the reason for the deviation and what steps need to be taken to bring the program back on track.
  • Within 90 days of the deviation, one of the following scenarios must take place:
    • The program is brought back on track; or
    • A new APB is approved, changing only the parameters that were deviated; or
    • An OIPT-level review is conducted to evaluate the PM's proposed baseline revisions, and feedback is given to the MDA, or in the case of a major program, to the Defense Acquisition Executive; or
    • If it's not possible for at least one of these actions to take place within 90 days, then the MDA should hold a formal program review to determine the status of the program.
2. Cost, schedule, and performance parameters are interrelated, and a change in one parameter can affect the others. For example, the materials needed for a lighter aircraft may cost more and take longer to design and manufacture than materials in a heavier aircraft. In that case, performance would affect both cost and schedule parameters. Therefore it is important to involve all the key stakeholders when considering changes to the APB. 

Integrated Baseline Review


The following learning objectives are covered in this lesson:
  • Identify the primary factors that the government should review to evaluate the contractor's PMB during an Integrated Baseline Review (IBR).
  • Identify the three reasons for PMB changes, and recognize their impact.
1. The Cost Performance Index (CPI) and Schedule Performance Index (SPI) indicate the performance efficiency factors that the contractor has achieved to date. Anytime the CPI or SPI are running significantly below 1.0, rebaselining may be necessary in order to complete the program. Generally, a CPI or SPI falling 10% or more below 1.0 is considered significant. The To Complete Performance Index (TCPI) indicates the efficiency factor that the contractor must achieve from "time now" to meet the budget at completion (BAC) or estimate at completion (EAC).
A TCPI greater than 1.0 indicates the contractor must work more efficiently that they have in the past to stay within the BAC or meet the EAC. These performance indices may indicate the need to conduct an Integrated Baseline Review (IBR). The IBR assesses the validity of the program management baseline (PMB) and identifies the risks associated with executing to the current PMB. Participants in an IBR typically include the government PM and technical staff, along with the related contractor's staff. During an IBR, the primary factors that are evaluated include:
  • The technical scope of the PMB
  • Program schedule requirements
  • Effective resource allocation to ensure that the work can be accomplished

2. There may be considerable risks associated with the current PMB indicating a need to rebaseline the program in order to make it executable. Changing the PMB can be caused by any one of the following three reasons:
  • Contract changes : only applies to changes/contract modifications directed by the government, not the contractor.
  • Internal re-planning : occurs when the contractor's original plan needs adjustment in response to problems or the opportunity to capitalize on efficiencies. The remaining work is then replanned by the contractor PM using the remaining budget and schedule.
  • Formal re-programming : occurs when the remaining budget and schedule is unrealistic; the contractor requires more time and dollars; the PMB exceeds the contract target cost and an over target baseline (OTB) occurs and the budget is insufficient; and the original objectives cannot be met.

Software Problems


The following learning objectives are covered in this lesson:
  • Apply a generic problem-solving model to an acquisition situation.
  • Apply one or more selected qualitative tools (e.g., fishbone diagram) to resolve a problem.
  • Identify developer practices essential for creation of high quality software.
  • Identify the requirements for interoperability testing.
1.    One problem-solving technique is the cause and effect diagram or "fishbone" diagram.  By analyzing all the possible causes of a problem, the fishbone diagram focuses on determining the root cause of a problem, rather than on symptoms or solutions.  Typically, the fishbone diagram begins with a statement of the problem in a box on the right side of the diagram--the "head" of the fish.  Then categories of major causes are identified and drawn to the left--the "bones" of the fish.  These major causes are broken down into all the related causal factors that might contribute to the major causes.  Finally, the causal factors are examined and narrowed down to the most significant elements of the problem to determine the ultimate cause or causes.
https://learn.dau.mil/CourseWare/83_7/rem/images/fishbone.jpg
2. The Software Program Managers Network has identified several software best practices based on interviews with software experts and industry leaders.  Here is a synthesized list of some of those characteristics, which are essential for the creation of high quality software: Adopt Continuous Program Risk Management
Risk management is a continuous process beginning with the definition of the concept and ending with system retirement. Risks need to be identified and managed across the life of the program.
Estimate Cost and Schedule Empirically
Initial software estimates and schedules should be looked on as high risk due to the lack of definitive information available at the time they are defined.
Use Metrics to Manage
All programs should have in place a continuous metrics program to monitor issues and determine the likelihood of risks occurring. Metrics information should be used as one of the primary inputs for program decisions.
Track Earned Value
Earned value requires each task to have both entry and exit criteria and a step to validate that these criteria have been met prior to the award of the credit. Earned value credit is binary with zero percent being given before task completion and 100 percent when completion is validated.
Track Defects against Quality Targets
All programs need to have pre-negotiated quality targets, which is an absolute requirement to be met prior to acceptance by the customer. Programs should implement practices to find defects early in the process and as close in time to creation of the defect as possible and should manage this defect rate against the quality targets. Meeting quality targets should be a subject at every major program review.
Treat People as the Most Important Resource
A primary program focus should be staffing positions with qualified personnel and retaining this staff through the life of the project. The program should not implement practices (e.g., excessive unpaid overtime) that will force voluntary staff turnover. The effectiveness and morale of the staff should be a factor in rewarding management.
Adopt Life Cycle Configuration Management
All programs, irrespective of size, need to manage information through a preplanned configuration management (CM) process. This discipline requires as a minimum:
  • Control of shared information
  • Control of changes
  • Version control
  • Identification of the status of controlled items(e.g., memos, schedules) and
  • Reviews and audits of controlled items.
Manage and Trace Requirements
Before any design is initiated, requirements for that segment of the software need to be agreed to. Requirements need to be continuously traced from the user requirement to the lowest level software component.
Use System-Based Software Design
All methods used to define system architecture and software design should be documented in the system engineering management plan and software development plan and be frequently and regularly evaluated through audits conducted by an independent program organization.
Ensure Data and Database Interoperability
All data and database implementation decisions should consider interoperability issues and, as interoperability factors change, these decisions should be revisited.
Define and Control Interfaces
Before completion of system-level requirements, a complete inventory of all external interfaces needs to be completed. Internal interfaces should be defined as part of the design process. All interfaces should be agreed upon and individually tested.
Design Twice, Code Once
Traceability needs to be maintained through the design and verified as part of the inspection process. Design can be incrementally specified when an incremental release or evolution life cycle model is used provided the CM process is adequate to support control of incremental designs.
Assess Reuse Risks and Costs
The use of reuse components, COTS (Commercial Off-The-Shelf), GOTS (Government Off-The-Shelf) or any other non-developmental items (NDI) should be a primary goal, but treat any use as a risk and manage it through risk management.
Inspect Requirements and Design
All products that are placed under CM and are used as a basis for subsequent development need to be subjected to a formal inspection defined in the software development plan. The program needs to fund inspections and track rework savings.
Manage Testing as a Continuous Process
All testing should follow a preplanned process, which is agreed to and funded. Every test should be described in traceable procedures and have pass-fail criteria.
Compile and Smoke Test Frequently
Smoke testing should qualify new capability or component only after successful regression test completion. All smoke tests should be based on a traceable procedure and run by an independent organization (not the engineers who produced it). Smoke test results should be visible and provided to all project personnel.

3.    Interoperability problems can best be identified through the use of actual, live systems to mitigate risk.  Joint interoperability is defined as the ability of systems to provide services to and accept services from other systems and to use the services exchanged to enable them to operate effectively together.  The Joint Interoperability Test Command is responsible for verifying the interoperability of systems to the parameters outlined in the ICD, CDD, CPD and ISP.