Links to BIC Best Practices

Top down approaches

• Merit Review Broader Impacts Criterion: Representative Activities

July 2007 In this NSF publication entitled Merit Review Broader Impacts Criterion: Representative Activities, there is an attempt to list appropriate practices/activities for fulfilling BIC.  Following each of the components of BIC, examples of specific activities that would fulfill that component are given.  The authors warn, however, that “this list is not intended to be exhaustive, nor is any particular example relevant to all proposals.”  These are provided more as samples, and the authors encourage new and creative ways to fulfill the criterion.

• Response to the Report on the Committee of Visitors on Information Technology Research (2001-2003)

  2005 This is a response from the Office of Integrative Activities to a Committee of Visitors (COV) report. COV’s are asked to evaluate and comment upon NSF program performance and results.  Note particularly that the COV suggested that there might be confusion about what “broader impacts” mean and how they could be evaluated in review.  To this the response was, “The National Science Board and NSF management purposefully articulated the ‘broader impacts’ criterion to be broad.  This criterion can be addressed in many ways (e.g., through education, training, societal impacts) depending on the scope of proposed projects, recognizing the range of project types supported by the agency.  This flexibility—in appropriately aligning science and engineering research and education with its broader impacts—is important in encouraging innovation from all science and engineering communities.”

• NSF Call for Highlights of Best Practices

January 2008 A Dear Colleague Letter from the Division of Chemistry requesting highlights that will advance the mission of the division. More than just research accomplishments, a request for information about broader impacts is specifically requested.

• Example of Broader Impact Highlight from Chemistry

2006-2007 In this “nugget” from Chemistry (a nugget is essentially a summary of a highlight, often with visual support) examples are given representing “technical highlights,” “non-technical highlights,” and “broader impacts highlights.”  The broader impact highlight described is a professor who contributed lectures and experiments to the California State Summer School for Mathematics and Science at the University of California, Irvine.  The broader impact lies in the fact that, “These lectures introduced high school students to the interdisciplinary area of the physical chemistry of macromolecules.”  And furthermore that, “Impact of her work is further broadened through international collaborations in Italy and the Netherlands.”

• Response to Recommendations from the Committee of Visitors for the Division of Biological Infrastructure

June 2007 The first recommendation from the committee of visitors pertains to the broader impacts criterion; the committee advises continued education of the scientific community on the meaning and importance of the criterion. Recommendation 1 (c) recommends the establishment of a website where examples of best practices could be posted. The response from the Biological Sciences Directorate (BIO) and the Division of Biological Infrastructure (DBI) ensures that this will be done.

• NSF's "An Impressive Return on Investment"

2001 As part of their strategic plan, on page 14, NSF lists as a performance goal to continue to stress the importance of both criteria, and claims it will "collect and make available examples of broader impacts and develop a plan to disseminate them."

• Grant Proposal Guide

September 2004 In the grant proposal guide, NSF provides information on what they will consider regarding broader impacts when evaluating proposals in determining funding decisions.

• Working Group on Implementation on Review Criterion #2

February 2001 Provides "specific examples of activities appropriate for review under Criterion #2." They note however, that these are not meant to be exhaustive or directive, but rather, will hopefully inspire creativity on the part of the scientific community to think of new ways to address the criterion.

• Capturing the Outcomes and Impacts of Publicly Funded Research

2007 An interesting parallel--this is a call from the Social Sciences and Humanities Research Council of Canada for a kind of metrics to measure the "social, cultural and economic impacts of [publicly funded] research."

• Measuring Success in Relation to Federal Investments in University Research: AUCC Discussion Paper

November 2006 Also out of Canada, this report is from the Association of Universities and Colleges of Canada. Some indicators they use include--First Order Indicators: funding, enrollment in graduate studies and international enrollment in graduate studies, attraction and retention of researchers, co-authorship, private sector investments in university research, international research funding; Second Order Indicators: Graduate degrees awarded, publications and citations, commercialization, cross-sectoral partnerships, economic impact.

• "NSF's Broader Impacts Criterion"

June 2006 A PowerPoint presentation on the broader impacts criterion, delivered by two members of NSF at the 2006 Annual ASEE Conference.

Bottom Up Approaches

• PSA Education Workshop Report: Algae and the Broader Impacts of Science

July 2006 Phycological Society of America (PSA) Education Workshop Report: Algae and the Broader Impacts of Science - Juneau, Alaska by Gisèle Muller-Parker and Rich Triemer lists “four areas of Broader Impacts activities”:
            a. Broader Impacts in General
            b. Teacher Professional Development
            c. Public Outreach
            d. Outreach to members of underrepresented groups
Aside from activities and practices, they also emphasize the role their website can play in serving as a “portal for the education and outreach activities undertaken by members of the society and as a means to bring ‘algae to the forefront’ of peoples minds.”

Making a Broader Impact: Geoscience Education, Public Outreach, and Criterion 2 Report

May 2005 Making a Broader Impact: Geoscience Education, Public Outreach, and Criterion 2 Report.  Report of a workshop held at the University of California Museum of Paleontology, Berkeley California.  May 11-13, 2005.  There is a section on best practices where they list the following:

1. Using a team approach that provides direct involvement in the planning and implementation of the EPO activity by all constituents: the researcher, the educator, and members of the intended audience.
2. Aligning EPO activities with the needs of the audience, e.g., if the target audience is formal K-12 education, activities should reference relevant science standards.
3. Involving graduate students and young scientists to promote an awareness of the importance of EPO efforts by scientists and the need for culture change.
4. Providing a menu of opportunities for researcher involvement, recognizing that not all scientists or research programs can or should be engaged in EPO at the same level or in the same role.

They add that important structural components include:

1. A venue and opportunity for dialogue between scientists and educators to focus on the benefits of bridging the two cultures of science research and education
2. Professional development opportunities for both scientists and educators to gain fluency and confidence in areas of mutual interest and professional intersection.
3. Appropriate personnel and financial support to complete the proposed activities, including strong project management.
4. Institutional commitment and incentives, including the development of a system of rewards for advancement and tenure of science faculty that recognizes the importance of public communication of science.
5. Realistic consideration of sustainability and continuity in activities.  Ongoing evaluation coupled with feedback to all partners in a timely fashion.

Recommendations for next steps are also provided.

• Broader Impacts Toolbox

2005 The result of an NSF funded workshop that claimed to set out to discuss the broader impacts criterion and the community response to it. This website is meant to provide the tools, as it were, to help researchers address the broader impacts criterion. Best practices would be one such tool that the website aims to collect and provide to others.

• Broader Impacts Posters

August 2005 An interesting collection of poster presentations on broader impacts presented at the 230th ACS National Meeting. The posters are organized according to components of the broader impacts criterion--Advancing Discovery and Understanding While Promoting Teaching, Training, and Learning; Benefits to Society; Broad Dissemination to Enhance Scientific and Technological Understanding; Broadening Participation of Underrepresented Groups; Enhance Infrastructure for Research and Education.

• "Constructing a Broader and More Inclusive Value System in Science"

January 2007 A BioScience journal article on broader impacts related issues such as diversity, and the science/society relationship, arguing ultimately for a broader value system in science.

• Broader Impacts: What, Why, and How

October 2006 A brief description of the broader impacts criterion and issues surrounding it. This comes from Richard E. Ladner, University of Washington, SIGACT Chair, and member of NSF Committee on Equal Opportunity in Science and Engineering (CEOSE).

• Integrating Research and Education

A site on integrating research and education. They have a section on their site devoted to broader impacts, and provide examples of "successful partnerships, as well as tips on assessment and dissemination."

• Characterizing Broader Impacts: An Exploratory Study of Metrics and Measures for the Research on Gender in Science and Engineering Program

August 2007-June 2008 A summary of a study that claims to take "a first step toward characterizing measures of the broader impacts for the Research on Gender in Science and Engineering Program (GSE) at the National Science Foundation (NSF)."