Uncategorized

This is Your Pilot

Ten Questions to Better Pilot Programs is an obscure but interesting Fiscal Brief produced last month by the Fiscal Research Division of the North Carolina General Assembly.  Study bills and pilot programs are often used to molify sponsors of legislation otherwise faced with certain defeat.  When properly designed and implemented pilot programs can produce useful information.  Many such programs are inadvertently doomed to preclude meaningful assessment, producing ambiguous results.

The Fiscal Brief has some general recommendations:

Policymakers should insist upon pilot programs that are designed as randomly controlled trials whenever possible.
   :::::
Policymakers should avoid insisting that their district or districts be included in treatment groups.
   :::::
A pilot program that generates actionable data is far more important than having a poorly designed program placed in a home district.
   :::::
Policymakers should allow time for pilot programs to reach their full implementation and allow time to observe program effects. Acting too early might result in the abandonment of programs that are actually working.

Policymakers should ask the following ten questions to ensure that new pilot programs will be able to provide clear results:

1.  What is the problem that needs solving?
2.  How does the program address the identified problem?
3.  What is the cost of the program if it is successful?
4.  Is there a budget or spending plan?
5.  What criteria will be used to determine the program’s success or failure?
6.  What alternative programs/solutions might also address the problem?
7.  Does the design of the program allow for meaningful evaluation?
8.  Are there problems in the program design that will affect validity?
9.  Is there sufficient time to observe effects?
10. Are there enough units of study to ensure statistical significance?

Clear results will help legislators to avoid pifalls and to direct tax dollars to useful and effective purposes.  The Fiscal Research Division produces other wonkish reports of interest to anyone concerned about the state budget and the fiscal impact of proposed legislation.

One Comment


  1. James

    September 16, 2008 at 9:05 pm

    Fascinating. Thanks for posting this. I love experimentation … we do not nearly enough for my tastes.

Check Also

Frackuccino

If they won’t tell us what’s in fracking ...

Top Stories from NCPW

  • News
  • Commentary

On a sultry day last September, Megan Stilley arrived at Lanier Farms, a large swine operation in ru [...]

When North Carolina lawmakers approved what one Republican described as a “historic” investment in r [...]

Lawmakers late last week released two new versions of a judicial redistricting bill, making these th [...]

An omnibus bill alleviating some of the headaches associated with North Carolina’s class size crisis [...]

The General Assembly’s latest mashup legislation is an example of government at its worst In the com [...]

The post Tied up in knots appeared first on NC Policy Watch. [...]

Every day brings new reports that Congress is interested in further whittling away at the programs c [...]

When Congress finally passed a continuing resolution last month allowing the government to re-open, [...]

Upcoming Events

Friday, Feb. 16

12:00 PM

Crucial Conversation – Prof. Peter Edelman discusses his new book, Not a Crime to be Poor: The Criminalization of Poverty in America

Prof. Edelman is coming to the Triangle to mark the 50th anniversary of Durham-based nonprofit MDC. His visit is the first of a series of MDC-sponsored events focused on ways that Southern leaders can work together to create an Infrastructure of Opportunity that shapes a South where all people thrive.”