Purpose: This is an evaluation of Natural Resources Canada’s (NRCan’s) Canadian Forest Service (CFS) Climate Change (CC) Program from 1999-00 to 2006-07. The CFS-CC Program spent approximately $88 million over this period (6% of the CFS budget), utilizing approximately 80 full-time equivalent staff per year. Scientific research related to CC received approximately 50% of the funding with the other 50% directed towards CC-related international reporting and greenhouse gas mitigation policy work. The CFS-CC Program received 60% of its funding from a number of other federal government programs.
Objectives: The CFS-CC Program’s objectives were to: (1) to understand the climate change impact implications for Canada’s forests and forest sector and develop adaptation strategies to reduce the risks; and (2) ensure that Canada’s climate change international reporting obligations and commitments are met and that related greenhouse gas mitigation options are explored.
Evaluation Issues and Methodology: This evaluation examined three issues: (1) relevance and rationale; (2) results and success; and (3) cost-effectiveness and alternatives. The evaluation is based on evidence from a document review, four case studies and 90 structured interviews and discussions.
Relevance and Rationale: The CFS-CC Program’s mandate and objectives remain highly relevant to fulfilling its responsibilities to assess the risks CC poses for Canadian forests, develop adaptation strategies, and to ensure international reporting obligations. There is a sound rationale for the activity to be in a science-based federal agency such as NRCan-CFS. There is clearly a legitimate, appropriate and necessary role for the federal government in supporting the CFS-CC Program.
The mandate for the CFS activity in Climate Change is derived from both federal legislation and international agreements under the United Nations.
The CFS-CC Program has been very closely aligned with the Government of Canada’s Climate Change priorities throughout the review period through the development of policy and science strategies directed towards understanding and dealing with the impacts of climate change on the forest sector. It has made significant contributions to meeting Canada’s international reporting obligations, particularly as a major contributor to Canada’s National Greenhouse Gas Inventory annual reports to the United Nations Framework Convention on Climate Change (UNFCCC).
There is little evidence of overlap or duplication between the CFS-CC Program and other organizations. The CFS-CC Program networks with but does not overlap the activities of other organizations. Without the CFS-CC Program, Canada would have to invest elsewhere to create a similar operation to fill the federal roles and responsibilities and meet its international reporting obligations under the UNFCCC framework.
Results and Success: CFS has been successful in developing and coordinating a highly productive and collaborative climate change science and policy program with an effective science and policy interface. This has been accomplished through the foresight of its scientific research that has provided a sound foundation upon which it could respond to the requirements of the CFS-CC Program and through its collaborative approach with the provinces, universities and industry. The development of the Carbon Budget Model has been a key integrating component of the CFS-CC Program that has played a crucial role in meeting international reporting obligations and informing Government of Canada decisions.
The key indicator of science success is the quality of the work and the very high regard in which CFS-CC science is held. CFS-CC scientific research (178 papers published during the evaluation period) has made significant and internationally recognized progress in reducing uncertainties concerning CC impacts and forecasting future impacts. The CFS-CC Program is recognized for its contributions to the IPCC work including participation of CFS scientists in eight of the Intergovernmental Panel on Climate Change (IPCC) reports as coordinating lead authors and review editors; and as project leaders making significant contributions to the IPCC Special Report on Land Use, Land-Use Change and Forestry, the Good Practice Guidance document published in 2003, the IPCC 2006 Guidelines on GHG accounting and the Fourth Assessment Report published in 2007.
Through its collaborative approach with the provinces, universities, industry, and especially through the federal-provincial National Forest Sinks Committee, CFS-CC research is broadly understood and appreciated.
Cost Effectiveness and Alternatives: This evaluation has concluded that the CFS-CC Program continues to be relevant, has a strong rationale and has been successful in achieving its objectives during the evaluation period. Under the issue of Cost Effectiveness and Alternatives, the evaluation’s findings are less clear cut.
The CFS-CC Program was a horizontal initiative cutting across a vertical organizational structure. At the time, the Department’s financial system did not lend itself to tracking horizontal climate change expenditures. The ongoing relevance and success of the CFS-CC Program suggests good value for money and it is not clear that there are more cost-effective means for delivering the CFS-CC Program.
Throughout the period, CFS used a variety of structures and processes to plan and direct its CC work (e.g., the Climate Change Network; the Climate Change Business Line; Committees; and a headquarters-based Coordinator).
Interviewees expressed a number of concerns, related to governance and management issues of the Science Component. They cited the loss of management oversight and direction during the latter part of the evaluation period. This was attributed to the expiration of some supplemental programs, diminished co-ordination, frequent personnel changes and re-organizations that caused uncertainty and confusion over roles. Staffing is another issue with new hires of scientists being below replacement levels for many years. CFS management points out that staff in a variety of categories were hired during the period to work on CC, and notes that the high level of supplemental funding resulted in the hiring of many CFS staff as term rather than indeterminate employees.
The CFS-CC Program has not had an external peer-review of its science since 1993. There are no NRCan-wide policies on periodic external peer review for science programs at this time.
Canada possesses one-tenth of the world’s forests. Most of Canada’s forest lands are owned by the provincial and territorial governments and, to a much lesser extent, private landowners. Nonetheless, the Government of Canada has played an important role in the area of forestry from the early decades of Confederation. In 1899, the predecessor of the modern Canadian Forest Service (CFS) was established and for over 100 years, the federal government has conducted forest science research in its own laboratories and forest stations.
Between 1960 and 1993, the Canadian Forest Service experienced numerous organizational changes. In 1993, the CFS merged with the Department of Energy, Mines and Resources to become Natural Resources Canada (NRCan). Currently, the CFS operates out of NRCan’s headquarters in Ottawa as well as five regions with major research centres in Fredericton, Quebec, Sault Ste. Marie, Edmonton and Victoria. CFS research staff work closely with the provinces and territories as well as other research institutes, universities, NGOs and the private sector in Canada and around the world. Today, the CFS is a science-based policy organization with an annual base budget of approximately $100 million plus additional supplemental funds that vary annually and a staff of over 800 full-time equivalent positions.
Canada’s forests play a significant role in the global climate system both influencing, and being influenced by, climate change. Climate is a strong determinant of forest health and productivity, and the CFS has had a scientific interest in the subject from its beginning. The CFS began its formal Climate Change (CC) program in the mid-1980s. The initial focus was on the carbon and nitrogen cycles and preparation of a key review document on forests and climate change. 1 Increased emphasis on CC followed international recognition of the need for sustainable forest management and UN recognition of the issue of global climate change in the late 1980s.
Under the federal Green Plan of 1990, the CFS launched a more comprehensive CC science program investigating CC impacts and the role of the boreal forest in the global carbon cycle. In the early to mid 1990s, the CFS participated in the Boreal Ecosystem Atmosphere Study project (BOREAS) involving government and university scientists from Canada, the United States and Europe. The $40 million BOREAS was co-directed by the CFS and the Canada Centre for Remote Sensing (also part of NRCan) from Canada and NASA from the US. BOREAS began a line of research on CC that focussed on the role of the boreal forest in the global carbon cycle.
As well as investigating the forest carbon cycle, CFS research has also focussed on the impacts and effects of climate change on Canadian forests. CFS developed these research themes building on its previous work on natural disturbances (i.e., insects, diseases, and fire), man-made disturbances (i.e., air pollution) and knowledge of forest ecosystem processes. CFS research on CC was undertaken in order to determine the probable effects of climate change on Canada’s forest ecosystems and identify potential forest management adaptation options and strategies.
Since the federal government’s budget reductions of the mid-1990s, CFS’s entire research program has been reorganized several times but CC has remained a research priority up to the present. The signing of the Kyoto Protocol in April 1998 is seen as a key event for the period of this evaluation.
The mandate for the CFS-CC Program is derived from both federal legislation and international agreements under the United Nations. The primary legislation establishing the mandate for CFS, is found in the Department of Natural Resources Act, 1994 (N-20.8) and the Forestry Act 1997 (R.S., 1985, c.F-30, s.1; 1989, c.27, s.14). These Acts enable NRCan to undertake scientific and economic research, promote sustainable development and perform other related activities.
As a signatory to the United Nations Framework Convention on Climate Change (UNFCCC) of 1992 and the Kyoto Protocol to it in 1998 (ratified by Canada in December 2002; came into effect February 2005), Canada has accepted an obligation to report national inventories on greenhouse gas (GHG) emissions from industrial sources as well as removals and emissions from land-use change and forestry (LULUCF).
The Kyoto Protocol details further reporting requirements on progress towards meeting Canada’s commitments, including methodologies used in the calculation of changes in GHG emissions and removals. Environment Canada (EC) takes the lead for reporting, and CFS via a Memorandum of Understanding with EC has responsibility for developing the forest-related carbon stock change and GHG emission and removal estimates for purposes of reporting under the UNFCCC LULUCF sector and under Articles 3.3 and 3.4 of the Kyoto Protocol.
The CFS-CC Program falls within NRCan’s CFS strategic objective in the 2007-08 Program Activity Architecture: To promote the sustainable development of Canada’s forests and competitiveness of the Canadian forest sector. For the 1999-2007 period, the CFS-CC Program’s objectives have essentially been two-fold:
The two CFS-CC Program objectives are reflected in two Components that existed throughout the 1999-2007 period referred to respectively as: Science, and Policy for the purpose of this evaluation.
The Science Component of the CFS-CC Program currently consists of scientific research carried out under the direction of a regional Director General by scientists at the CFS’s five research centres, in partnership with the academic, public, and private sectors, and coordinated by the Science and Programs Branch of CFS. Several models of governance have been followed over the review period.
The Science Component of the CFS-CC Program has focused on research in three areas:
The scientific research of the CFS-CC Program results in increased scientific and technical knowledge, databases, and related standards. CFS-CC scientists transfer these results to others dealing with CC issues via scientific publications, conferences, extension work, and participation in technical working groups dealing with forest CC research and policy issues.
In the area of climate change, CFS reports that its researchers and staff have: served as peer reviewers for numerous national and international scientific journals; participated as board members/reviewers for funding proposals for various organizations (e.g., BIOCAP Canada Foundation; Canadian Foundation for Climate and Atmospheric Studies); and supervised many graduate students and post-doctoral fellows.
The Policy Component of the CFS-CC Program is directed by the Policy, Economics and Industry Branch of CFS in close coordination with scientists in the five CFS regional research centres.
The Policy Component is related to negotiating and representing the Canadian position in the CC forestry-related international arena. It includes participation and negotiation on technical aspects of international agreements within UN, Organization for Economic Cooperation and Development, and International Energy Association committees and working groups to ensure that the Canadian position is well understood and represented.
The CFS-CC Policy Component works through the National Forest Sinks Committee to coordinate its work with the provinces and obtain inputs for the Carbon Budget Model. It also undertakes economic research in conjunction with forest scientists on developing approaches for possible mitigation strategies (i.e., GHG reduction). The Policy Component has included specific initiatives during the study period, including:
The Policy Component of the CFS-CC Program provides position papers, advice, recommendations, policies, and guidelines based on scientific knowledge and economic analyses. There is considerable science that feeds directly into the Policy Component, making it somewhat difficult to track the Policy and Science Components and contributions separately.
It is believed that there were over 50 research projects within the CFS-CC Program during the evaluation period. The great majority of these projects were generated for and managed through mechanisms required by externally-funded programs (e.g., PERD; AP2000; CCAF – see Resources and Funding Sources below). These projects sometimes required matching funding from CFS’s ongoing A-Base and in-kind support. Close supervision of progress (at least annually) was maintained by the funding bodies.
The work done under the CFS-CC Program (both in the Science and Policy Components) is long-term and multi-disciplinary in nature. This requires participation by scientists with expertise in a broad range of social, physical and biological sciences (e.g., botany, ecology, geography, geology, climatology, soils, forestry, physics, biochemistry, as well as economics and sociology) all focussing on forest applications such as landscape management, forest fires, pest management, and biodiversity. The CFS scientists working on the CFS-CC Program are typically involved in other non-CC projects as well. Indeed, it is difficult to clearly define the climate change work within CFS since it feeds into, and draws from, many other areas.
The CFS-CC Program is part of a larger Government of Canada and international effort. This requires horizontal coordination within CFS, NRCan and the federal government. From 1998 to 2004, the federal government’s Climate Change Secretariat coordinated the ‘science’ of climate change work and reported to both NRCan and Environment Canada. Since 2004, Environment Canada has had the lead role in coordinating federal climate change work. Within NRCan, the Earth Sciences Sector has had a lead role in the CC impacts and adaptation area while the Energy Sector has led the CC related energy R&D work.
Exact financial figures are not available due to changing definitions, organizational structures and systems and the difficulty of making clear distinctions between the Science and Policy Components and separating them from other CFS work. However, it is estimated that CFS spent approximately $88 million on its CC Program between 1999-00 and 2006-07 with about 50% of the expenditures under the Science Component and 50% under the Policy Component. During the review period, approximately 80 full-time equivalent staff annually worked on the Program.
CFS-CC has been able to supplement its on-going A-base funds and workforce through formal and informal arrangements. Supplemental sources added significantly to the annual CFS-CC budgets, particularly between 2002 and 2006 with funds from the Feasibility Assessment of Afforestation for Carbon Sequestration (FAACS) and Forest 2020 PDA initiatives. Over the eight-year evaluation period, supplemental sources provided approximately 60% of the total CFS-CC budget. Funding by activity and source are summarized in Tables 1, 2 and Figure 1.
The CFS-CC Program has depended on and been highly successful in attracting funding from Government of Canada Climate Change programs, energy-related R&D programs as well as arrangements with other research bodies. Most of this funding has required an application approval process by authorities outside of CFS to compete for funds with scientists from universities and other government science departments. The main programs were: (1) Climate Change Action Fund ($9 million); (2) Action Plan 2000 ($9 million); and (3) three components of NRCan’s Program for Energy Research and Development ($4 million).3 Forest 2020 PDA provided $20 million of which $12 million was spent as contributions.
The CFS-CC Program has also benefited from participation with university and other government scientists in climate change projects administered by research funding bodies. The BOREAS work, 1990-96, provided data and data-gathering technologies critical for the CFS-CC Program and has been continued under various programs since then, particularly: the Canadian Foundation for Climate and Atmospheric Sciences (CFACS), BIOCAP Canada, NSERC and Fluxnet Canada/Canadian Carbon Program.4 CFACS facilitates progress on international environmental commitments such as the Kyoto Protocol. A related example is CFS-CC’s participation in the Canadian Climate Impacts and Adaptation Research Network (C-CIARN). CFS operated a C-CIARN forest office from 2001-07 at its Northern Forestry Centre in Edmonton, Alberta. However, this activity ended with the termination of external funding.
| TABLE 1: ESTIMATED CFS-CC FUNDING BY SOURCE, 1999-00 TO 2006-07 | $ 000s | % |
|---|---|---|
| CFS A-Base Funding | $34,807 | 39% |
| Science | $31,607 | 36% |
| Policy (International Reporting Obligations & Mitigation) | $3,200 | 4% |
| Supplemental Funding | $53,345 | 61% |
| Science | $12,270 | 14% |
| Climate Change Action Fund +International Reporting Obligations | $2,337 | 3% |
| Action Plan 2000 | $2,575 | 3% |
| Canadian Foundation for Climate and Atmospheric Sciences + Program of Energy Research and Development | $7,358 | 8% |
| Policy (International Reporting Obligations & Mitigation) | $41,075 | 47% |
| Climate Change Action Fund +International Reporting Obligations | $15,153 | 17% |
| Action Plan 2000 for Forestry of Afforestation for Carbon Sequestration | $5,932 | 7% |
| Forest 2020 Plantation Demonstration & Assessment | $19,989 | 23% |
| TOTAL | $88,152 | 100% |
| TABLE 2: ESTIMATED CFS-CC FUNDING BY ACTIVITY, 1999-00 TO 2006-07 | $ 000s | % |
|---|---|---|
| Science | $43,877 | 50% |
| Carbon Budget Model | $6,174 | 7% |
| Carbon Cycle Science | $19,245 | 22% |
| Impacts & Adaptation | $18,459 | 21% |
| Policy | $44,275 | 50% |
| International Reporting Obligations | $18,353 | 21% |
| Mitigation | $25,921 | 29% |
| TOTAL | $88,152 | 100% |
FIGURE 1: ESTIMATED CFS-CC FUNDING, 1999-2007
CFS A-base vs. Supplemental Funding
This evaluation examines three issues: (1) relevance and rationale; (2) results and success; and (3) cost-effectiveness and alternatives. The methodologies used to address the issues were an extensive document review; structured interviews and discussions with 90 individuals involved in the CFS-CC Program; and the following four case studies:
The 90 interviews and discussions were distributed as follows: 15 with CFS senior management (i.e., ADMs, DGs and Directors), 30 with CFS research scientists, 15 with CFS analysts and economists, 5 with other federal government departments, 7 with provincial government agencies, 10 with university faculty and 8 with the private sector and other groups.
The evaluation was constrained by the limited program level financial information and project level statistics available to it.
Are the mandate and the objectives of the CFS-CC Program still relevant and is there a legitimate, appropriate and necessary role for the federal government in supporting it?
Yes. The CFS-CC Program’s mandate and objectives remain highly relevant to fulfilling its responsibilities to assess the risks CC poses for Canadian forests, develop adaptation strategies, and to ensure international reporting obligations. There is a sound rationale for the activity to be in a science-based federal agency such as NRCan-CFS. There is clearly a legitimate, appropriate and necessary role for the federal government in supporting the CFS-CC Program.
The mandate for the CFS activity in Climate Change is derived from both federal legislation and international agreements under the United Nations. The primary legislation establishing the mandate for CFS programs, including the work conducted under the CFS Climate Change Program, is found in the Department of Natural Resources Act, and the Forestry Act 1997. As a signatory to the UNFCCC of 1992 and the Kyoto Protocol in 1998 Canada is obligated to report national inventories on industrial GHG emissions as well as emissions and removals by sinks from land-use, land-use change and forestry which includes changes in forest stocks. Climate change is now a recognized priority of the federal, provincial and territorial governments, and the Canadian Council of Forest Ministers (CCFM) has undertaken a number of cooperative actions to help address the impacts of climate change.5
Evidence from the Intergovernmental Panel on Climate Change (IPCC) of global climate change and its potential impact on the world’s forests provides overviews of the current state of knowledge of climate change and Canada’s forests. These studies predict a level of atmospheric warming in Canada greater than the global average.6 Canadian forests are particularly vulnerable to climate change because of the sensitivity of their ecosystems to change and the long growing cycles of their trees. At present, the most visible impacts are in the form of changes in the frequency and severity of disturbances (fire, insects and diseases) and/or in the range expansion of damaging insects.7 The length of the growing season is increasing, and the delay of autumnal senescence (i.e., leaf colour change and start of the dormancy process) has been documented.8 Tree lines are creeping northward.
Furthermore, as a signatory to the United Nations Framework Convention on Climate Change (UNFCCC) of 1992 and the Kyoto Protocol to it in 1998 (ratified in 2002 and entered into effect in 2005), Canada has accepted the obligation to report (i.e., to periodically publish and make available to the Conference of the Parties) national inventories on removals by sinks and emissions of all greenhouse gases. Under UNFCCC Articles 4 and 12, the Kyoto Protocol details further reporting requirements on progress towards meeting Canada’s agreed-upon commitments, including methodologies used in the calculation of changes in GHG emissions and sinks (Kyoto Protocol Article 3, Section 7 and Article 5, Section 2). Environment Canada, in consultation with a range of stakeholders, is responsible for the preparation of Canada’s National Inventory submitted annually to the UNFCCC. 9
How well has the CFS-CC Program been able to respond to the needs and priorities of the federal government and the provincial/territorial governments?
The CFS-CC Program has been very closely aligned with the Government of Canada’s Climate Change priorities throughout the review period through the development of policy and science strategies directed towards understanding and dealing with the impacts of climate change on the forest sector.10 It has made significant contributions to meeting Canada’s international reporting obligations, particularly as a major contributor to Canada’s National GHG Inventory annual reports to the UNFCCC.
As an indication of CFS responsiveness, since 2004, Environment Canada has officially designated responsibilities to CFS for forests to develop key reporting components for the Land-Use, Land-Use Change, and Forestry (LULUCF) sector and has established formal and explicit governance mechanisms to that effect.11 The LULUCF component of the National GHG Inventory reporting system was put in place to develop the necessary inventory estimation and accounting systems, and to coordinate the work of the involved agencies ensuring that Canada meets both UNFCCC-specific and Kyoto Protocol-specific reporting requirements for the LULUCF sector. CFS is responsible for providing GHG estimates for Canada’s forestland including deforestation and afforestation.
NRCan-CFS also chairs the federal-provincial National Forest Sinks Committee that is key to successful GHG reporting.12 In March 2005, to meet Kyoto reporting requirements, CFS in partnership with the provincial and territorial governments put in place a new component of the GHG reporting system the National Forest Carbon, Monitoring, Accounting and Reporting System (NFCMARS).13 This system utilises forest-inventory data, growth and yield information, and statistics on natural disturbances, management actions and land-use change to estimate forest carbon stocks, changes in carbon stocks, and emissions of non-CO2 greenhouse gases. A key component of the system is the Carbon Budget Model-Canadian Forest Sector (CBM-CFS).14 The first NFCMARS National Inventory Report (NIR) covering the period 1990 to 2004 was completed in 2005.15
The CFS science priorities are also in line with federal and provincial/territorial government policy focusing on understanding the role of forests in the carbon cycle and impacts and adaptations to climate change. In early 1998, Canada's federal, provincial, and territorial ministers of energy and environment approved the creation of a National Climate Change Secretariat to oversee the development of a national climate change implementation strategy (i.e., the National Climate Change Process) to meet Canada’s Kyoto Protocol target.
In September 1999, the National Sinks Table prepared an Options Paper on Land Use, Land-Use Change, and Forestry in Canada, and the Kyoto Protocol. That document provided specific recommendations with respect to research required to assist Canadian negotiators in international discussions on carbon sinks, and provided the basis for credible reporting of such sinks. The Sinks Table concluded that, “Forestry and agricultural sinks should play a role in a national Implementation Strategy…Considerable investment into research and information is required now, so that Canada will be able to provide accurate and verifiable estimates of the change in carbon stocks when required and to provide guidance to negotiators.”16 CFS was also actively involved (co-chairing, hosting the secretariat, and carrying out the analysis) in the Forest Sector Table under the National Climate Change Process, which also prepared an Options Paper with recommendations on how the forest sector could help meet Kyoto Protocol Targets. The analysis of forest sink options was jointly prepared with, and consistent with, the analysis by the National Sinks Table.17
To address the range of issues outlined in the 1999 Options Report of the National Sinks Table, CFS put into place a revised S&T Plan covering the period 2002-05.18 Two principal components of the plan were carbon budget and accounting and climate change impact on forest ecosystems and natural disturbances. A particularly important aspect of the 2002-05 Climate Change Program was assessing the implications of managing Canada’s forests for carbon sequestration and storage in the context of sustainable forest management.
Further evidence of the CFS-CC Program’s response to the Government’s priorities is found in two other reports. The first is the 2005 Report of the Standing Committee on Environment and Sustainable Development. It recommended that the Government increase its support for science aimed at understanding Canada’s carbon cycle; and that the Government, in cooperation with the provinces and territories, develop a climate change adaptation strategy.19 The second is the 2006 Report of the Commissioner of the Environment and Sustainable Development that recommended that Environment Canada and NRCan should, “(1) identify and fill gaps in the needed information, including results of impacts and adaptation research and results from climate science; and (2) identify the demand for initiatives that provide decision makers with access to information and technical expertise on adaptation tailored to their needs. Based on that work, they should strengthen existing initiatives and establish others, as required.”20
Is there overlap or duplication between the CFS-CC Program and other organizations?
No. There is little evidence of overlap or duplication between the CFS-CC Program and other organizations. The CFS-CC Program networks with but does not overlap the activities of other organizations. Representatives from federal agencies, provinces and academia expect that the federal government will continue to:
These long-term investment requirements in select components of climate-change science and policy may be best suited to a government organization such as the CFS.
What would be the impact of the cancellation of the CFS-CC Program? And, what would be the impacts of NRCan, or of any provincial/territorial agency, academia, and private organization not continuing its involvement in the CFS-CC?
Without the CFS-CC Program, Canada would have to invest elsewhere to create a similar operation to fill the federal roles and responsibilities and meet its international reporting obligations under the UNFCCC framework. Much of the long-term national forest carbon-cycle research and national monitoring would be lost or in disarray and the science support for forest, and forest-sector policy analysis, required for future negotiations related to international agreements, would be jeopardized. In addition, the national understanding of climate change impacts and development of forest adaptation strategies would be seriously diminished, and Canada’s forest ecosystems, forest economic sector and resource-based communities would be more exposed to vulnerabilities.
Without provincial/territorial involvement, the CFS-CC Program elements linked to the National Forest Carbon Monitoring, Accounting and Reporting System (NFCMARS) and the Carbon Budget Model would not be viable. CFS leads the development of the NFCMARS system in cooperation with provincial/territorial government agencies, and manages its implementation. As well, a critical component of NFCMARS is Canada’s National Forest Inventory, which has been implemented in collaboration with the provincial and territorial governments. Provincial/territorial withdrawal would mean the loss of the Sinks Committee and involvement of key expertise in the development of research and related policy on forest sinks and impacts and adaptation to climate change. Also, supporting the development of NFCMARS is the university led Canadian Carbon Program (CCP) involving universities and governments from across Canada.
Canada’s universities and provincial research agencies are critical generators of forest-related climate-change science required by the CFS to fulfill its mandate. The bibliometric analysis case study conducted for this evaluation shows that, between 1991 and 2006, CFS scientists co-authored approximately 48% of their 213 climate change research publications with other Canadian researchers and 28% with international collaborators (178 or over 80% of these studies were published between 1999 and 2006, during the evaluation period). The study indicates that the CFS is the national hub for forest-based climate research and a leader in international collaboration. CFS is reliant on the broad base of expertise in academic and provincial science-based organizations for much of the knowledge now used to validate climate change and underpin forest sinks and integrated adaptation strategies (Figure 2).
Figure 2: Collaboration Network of Canadian Institutions in the Forestry Subfield of Climate Change, 1991-2006

Note: The size of a node is proportional to the institution's total number of papers within the subfield of climate change. The width of the links is proportional to the number of collaborations between two institutions. Links are only displayed when two institutions have co-authored at least four papers over the period. The length of the links has been modified to improve readability and has no significance.
Source: Compiled by Science-Metrix from the Web of Science (Thomson Scientific)
Without considerable input from the university-based researchers, the CFS-CC Program would be largely biased towards its own areas of expertise, and the knowledge and databases would not adequately support the national policy needs either in meeting its reporting obligations or in being able to understand impacts to develop mitigation and adaptation options and strategies.
Overall, CFS has been successful in developing and coordinating a highly productive and collaborative climate change science and policy program with an effective science and policy interface. This has been accomplished through the foresight of its scientific research that has provided a sound foundation upon which it could respond to the requirements of the CFS-CC Program and through its collaborative approach with the provinces, universities and industry. The development of the Carbon Budget Model has been a key integrating component of the CFS-CC Program that has played a crucial role in meeting international reporting obligations and informing Government of Canada decisions. CFS-CC research is of value to the broader range of forest science research and policy issues CFS must deal with in achieving more sustainable forestry and a competitive Canadian forest sector.
The key indicator of science success is the quality of the work and the very high regard in which CFS-CC science is held. The CFS-CC Program provides world-class science in climate change research. Indicative of this are the CFS contributions to the IPCC work including participation of CFS scientists in eight of the IPCC reports as coordinating lead authors and review editors; and as project leaders making significant contributions to the IPCC Special Report on LULUCF, the Good Practice Guidance document published in 200321, the IPCC 2006 Guidelines on GHG accounting and the Fourth Assessment Report published in 2007.22
CFS climate change scientists and coordinators have been acknowledged by the IPCC for their contributions to IPCC’s Nobel Peace Prize for 2007. The IPCC was awarded the Nobel Peace Prize in recognition for its efforts in building up and disseminating greater knowledge about man-made climate change and laying the foundation for measures that are needed to counteract such changes. The reliance on Canadian results throughout a recent global overview paper on adaptations of forests to climate change, published by the International Union of Forest Research Organization (IUFRO) Secretariat, is further evidence of the international science leadership provided by Canadian climate change scientists.23
The bibliometric analysis of CFS scientific research in the forestry area of climate change for the years 1991-2006 demonstrates the important contribution CFS-CC scientists have made to the CC forest research subject in Canada and the world scientific community. Five of the more noteworthy findings from that analysis are as follows:
How successful has the CFS-CC Program been in meeting the objectives of the five major sub-programs identified in its plans and Government of Canada agreements?
As discussed below, the CFS-CC Program has been highly successful in meeting the objectives of its five major sub-program areas of 1) establishing a better scientific understanding of the possible impacts of CC on Canadian forests; 2) advancing knowledge of carbon cycle science as related to CC in Canadian forests; 3) developing and utilizing a carbon budget model; 4) negotiating with as well as supporting and meeting UN and other reporting requirements; and 5) analysing the feasibility of afforestation in meeting GHG mitigation goals.
How well has the CFS-CC been able to scientifically establish an understanding of the possible impacts of CC on Canadian forests in key areas?
Through its research work, the CFS-CC has made significant progress in reducing uncertainties concerning climate change impacts and forecasting future impacts on Canadian forests. Building on its traditional strengths of forest science research (i.e., fire, insects and diseases, ecosystem dynamics) CFS now possesses a more in-depth understanding of the possible impacts of climate change on Canadian forests, natural disturbances and the forest sector. Review documents provide a synopsis for 1999-2007 of the climate change research in Canada highlighting the accomplishments of Canadian researchers including the CFS. The CFS-CC research data on ecosystem processes and Canada’s predictive abilities concerning climate change impacts have been expanded and improved.25
Even with this background work, forecasting CC is difficult and fraught with many complexities and uncertainties. Nonetheless, the future impacts of continued climate change may include:
How well has the CFS-CC performed in advancing knowledge of forest carbon cycle processes, and developing and utilizing its forest Carbon Budget Model?
Research on the carbon cycle in the Canadian forest and the development of the Carbon Budget Model - Canadian Forest Sector (CBM-CFS) has been central to the CFS-CC Program. As determined by one of the case studies conducted for this evaluation, the development and utilization of the CBM-CFS is a major achievement and a key integrating component of the entire CFS-CC Program. The project, focused on a national modelling effort to develop a carbon accounting framework, has enabled Canada to meet its UNFCCC and Kyoto Protocol GHG emissions and removals reporting requirements, and been particularly helpful in related negotiations.26, 27
Based on research from the carbon cycle science side, CBM-CFS has helped researchers explore how natural disturbances, forest management and changes in forest age-class structure, growth, and decomposition rates might have affected forest carbon stocks since 1989.28 The CBM-CFS2 (2000), and the updated CBM-CFS3 (2002) models, have been extensively used to address the Carbon Sinks policy issue. The CBM-CFS3 model of forest dynamics tracks carbon stocks, carbon transfers between pools, and emissions and removals of carbon dioxide (CO2), methane (CH4), and carbon monoxide (CO). The model can also be used to explore carbon dynamics for smaller areas, down to the stand level as well as to assess past changes in carbon stocks or to evaluate future changes that would result from scenarios of management actions and natural disturbances.
The CBM-CFS3 was developed with input from the federal-provincial National Forest Sinks Committee and the federal Steering Committee for Canada's Greenhouse Gas Monitoring, Accounting and Reporting System (MARS) with Environment Canada, CFS, and Agriculture and Agri-Food Canada. With their help, the NFCMARS has used the CBM-CFS to project the GHG balance of Canada's managed forest for the first commitment period of the Kyoto Protocol and to 2022, in order to help Canada decide whether to include forests in its UNFCCC reporting requirements. Some interviewees indicated that as far as they are aware, Canada is the only country in the world that has conducted a quantitative risk analysis of this complexity, relating to the country’s potential forest carbon stocks and changes in stocks during the 2008-12 commitment period.
In addition to the national-scale applications of the CBM-CFS3, an operational-scale version of the model has been developed in co-operation with Canada's Model Forest Network, and its partners in the provinces and forest industry.29 The objective of doing this is to enable forest managers and analysts to assess and evaluate the impacts of alternative management strategies on carbon stocks and stock changes. CFS has held one international and six national CBM-CFS3 Training Workshops across Canada, involving 162 participants representing the forestry community: industry, academia, NGOs, First Nations, Model Forests, federal, provincial, state, and foreign government agencies.30 Since its release, the CBM-CFS3 software has been downloaded by over 340 individuals.
CFS-CC carbon modelling related work has included a completely new approach to monitoring deforestation, compliant with the Kyoto Protocol, which uses remote sensing imagery, records, and expert judgment to capture the various forest conversion events and patterns since 1970. Ongoing research is being conducted in collaboration with Fluxnet Canada/Canada Carbon Program, and other CFS projects to compare different modelling techniques and estimates to carbon stock measurements taken at ground plots.31, 32, 33 In addition, CBM-CFS modelling is being validated by means of comparison with other carbon models. According to CFS sources, between 1999 and 2004, the CBM-CFS modelling team produced close to 150 published reports and peer-reviewed publications and has given nearly 200 presentations on the activities and results of the research.
Given the evident success and achievement of the carbon budget modelling and accounting work, CFS must continue to address several issues, including the future development of the CBM-CFS model; the needs of the user community; the need for promotion of the model; and the need for continuous data, such as forest inventories, to drive the model.
How useful has CFS-CC’s work proven in contributing to policy determinations, monitoring and fulfilling reporting requirements and negotiating positions?
CFS-CC scientific and economic analyses have proven essential in fulfilling policy-related needs of Canada in the international climate change area.
An important part of the Policy Component's work during the evaluation period was to support and participate in international negotiations on climate change. Under the UN Framework Convention on Climate Change (UNFCCC) negotiations, CFS staff were annually involved in the negotiations of the Subsidiary Body for Scientific and Technological Advice as well as the annual Conferences of the Parties. In addition, there were occasional international workshops on topics related to forest carbon. All negotiating sessions and workshops required prior development of detailed negotiating instructions, interdepartmental agreements and approvals.
A key question faced by Canadian policymakers during the review period was whether or not to elect forest management activities under Article 3.4 of the Kyoto Protocol (i.e., whether to include managed forests in the reporting or not). To address this question the CBM-CFS model was used to undertake a risk assessment for Canada's decision. The assessment was conducted for the National Forest Sinks Committee and completed in 2005-06.34 The conclusions of the detailed assessment were that Canada’s managed forest was a net carbon sink of 16.6 mega tonnes of carbon per year from 1990-2005: it was a sink in 12 of the 16 years during this period. However, projected forward, Canada’s managed forest has an estimated 91% probability of being a net source of greenhouse gases in the first commitment period (2008-12).35
Canada's decision, based on the results of this analysis, was not to elect forest management under Article 3.4. Forests have become carbon sources in years in which wildfires exceeded the long-term average area burned and are now being affected by Mountain Pine Beetle die off. Model projections for the next decade suggest Canadian forests will be a carbon source.36
The next key CC forest policy question will likely be Canada’s negotiating position leading into the second Kyoto commitment period that takes into account the potential for large emissions from natural disturbances such as wildfire, insects, diseases, and extreme weather. In Canada, there is a high risk that emissions from these natural disturbances will completely negate efforts to affect the GHG balance through forest management because current accounting rules do not factor out natural and indirect human effects from direct human effects.37
Results from the CBM-CFS modelling work have also been used in preparing Canada’s forest carbon budget information for both the CCFM38 and Montreal Process Criteria and Indicators of Sustainable Forest Management reports as well as a number of the CFS’s State of Canada’s Forests annual reports.39
In addition, CFS-CC scientists have collaborated in a variety of other areas to help integrate CC-related knowledge for policymakers, including the development and use of customized spatial climate models for Canada in partnership with Environment Canada’s Meteorological Service of Canada, the Australian National University,40, 41 and with Environment Canada and Agriculture and Agri-Food Canada to update and publish a revised Plant Hardiness Map of Canada.42
How effectively has CFS-CC been able to use the knowledge it has gained to provide advice and direction on appropriate means to mitigate GHGs and develop adaptation strategies for the forest sector?
The CFS-CC Program has made good progress in using its knowledge to advise on means towards mitigation of GHGs. Progress on developing adaptation strategies has been more limited. The implementation of adaptation strategies will require direct provincial involvement.
The key mitigation initiatives were the Feasibility Assessment of Afforestation for Carbon Sequestration (FAACS) and Forest 2020 Plantation Demonstration and Assessment Initiative projects led by the CFS Policy, Economics and Industry Branch. These initiatives were the subject of a case study for this evaluation. They provide useful analytical processes, field trial results, and policy recommendations for the government to follow should it consider afforestation as a national policy option. The three-year FAACS initiative (2002-05) was the first phase in assessing the ability of afforestation/reforestation - the establishment of tree cover on lands that were not previously forested - to help meet GHG emission reduction targets.43 This phase was essentially a preparatory measure intended to evaluate whether a large-scale national afforestation effort was feasible and what a potential large-scale program would involve. The FAACS Initiative ($6 million) contributed the science, economic and policy information needed to support the establishment of the Forest 2020 PDA.
The Forest 2020 PDA (2003-06) initiative ($20 million) assessed the feasibility of fast-growing planations as a means to achieve a wide range of economic and environmental objectives, particularly carbon sequestration. The Program consisted of two components: (1) economic and investment potential analysis ($6 million and an extension of FAACS); and (2) a fast-growing plantation demonstration component ($12 million) to study forest management practices through the establishment of 6,000 hectares of plantation demonstrations across Canada.
The FAACS and Forest 2020 PDA Initiatives were initially conceived to assess the potential of afforestation to help meet Kyoto commitments. Results suggest afforestation is not viable on a wide scale at the present time. In moving beyond Kyoto, however, interviewees indicated that a number of the assumptions used in the economic feasibility analysis, particularly the price per tonne of carbon, may have been too constraining or undervalued. In the future, the results and conclusions of FAACS/Forest 2020 PDA may need to be re-evaluated with more current information in order to determine whether or not afforestation/reforestation using fast growing plantations can contribute to mitigation in a significant way.
The FAACS and Forest 2020 PDA Initiatives provided researchers the opportunity to study fast growing plantations in a controlled experiment, building capacity and gaining valuable knowledge for plantation forestry. Results and analysis have also expanded knowledge and understanding of the economics of plantation forestry and its potential contribution to GHG emission reductions by Canada. From these initiatives, a suite of policy recommendations for the government has been developed should it choose to pursue afforestation as a potential policy option.44 As well, CFS has developed the capacity and established the partnerships to put in place a national afforestation program, if required.
Scientists have only recently begun to better understand the vulnerabilities of the forest ecosystem and sector; and systematic on-the-ground adaptive actions have not yet occurred because of the continuing complexities and uncertainties of climate change. Since there is not yet a sufficiently clear view of the future climate and its impacts on the forest, the CFS-CC can only begin the process of developing adaptation strategies.
Nonetheless, some achievements have been made in the adaptation area. CFS operated, from 2001-07, the Forest Node of the Canadian Climate Impacts and Adaptation Research Network that focused on communicating awareness and understanding of adaptation options, and information on past and current adaptation science.45
The Program is continuing to work on adaptation to climate change in concert with its partners. For example, CFS reports that it has worked with the Quebec government to integrate adaptation into forest harvesting and regeneration regimes; provided advice to the forest sector on site productivity issues; and undertaken analysis on the vulnerability of communities in Alberta.
In reality, forest management by its very nature is all about adaptation. CFS research in silvicultural practices, protection from fire, insect and disease disturbances, as well as genetics, landscape management, etc. all contribute to the development of climate change adaptation strategies. Research will help the sector to adapt, reduce vulnerabilities and be more resilient.
To what extent is the CFS-CC Program contributing to the successful achievement of overall CFS goals and objectives?
The CFS-CC Program has made significant contributions to the achievement of the primary CFS objectives of promoting the sustainable development of Canada’s forests and the competitiveness of the Canadian forest sector. CFS-CC contributes to forest sector sustainability directly as a generator and indirectly as a disseminator of a wide range of knowledge on forest ecosystems and natural disturbances useful to government and private sector decision-makers. The CFS’s linking of science and policy is recognized as an example of the relevance of the CFS-CC scientific research to current policy deliberations and decision-making.
CFS-CC has been able to make a notable contribution as an integrator of information through collaborative inputs to the CBM-CFS, NFCMARS, Fluxnet/CCP and C-CIARN networks, Montreal and CCFM processes on Sustainable Forest Management and the international efforts of the IPCC. CFS-CC knowledge has been both timely and strategic to the forest sector’s awareness and understanding of impacts and the need to develop an adaptive response.
How successful has the CFS been in providing overall strategic direction, coordination and policy development in the area of climate change?
The CFS has been successful in providing overall strategic direction, coordination and policy development in the area of climate change. From the 1980s, CFS has effectively utilised its and others’ scientific work and skills in developing a comprehensive and well integrated foundation for understanding the implications of CC for Canada’s forests. The high level of success has been due to the strong foundation laid prior to 1999 and to the effective planning, coordination and execution of subsequent work.
In 1999, CFS undertook a strategic planning process that resulted in the preparation of the ‘Overview of the Climate Change Target Outcome Planning’ document. The CFS-CC Science Component was driven by the issues outlined in the 1999 ‘Options Report of the National Sinks Table’ and the work of the IPCC that, since 1988, has been tracking the uncertainties that affect climate change, as well as recommending research needed to reduce those uncertainties.46, 47 The Climate Change Network, established under the Green Plan Initiative prior to Kyoto, guided the CFS-CC Program through the first phase of the evaluation period, 1999-2004.48
CFS coordinated, with NRCan’s Office of Energy Research & Development, a consolidated funding of carbon cycle research into an integrated, results-based program. The aim was to improve Canada’s capacity to accurately report human induced sources and sinks of GHGs involving Canadian forest, agricultural and oceanic ecosystems and advise policy makers on actions to enhance GHG sinks. An inter-departmental team of science advisors including CFS was assembled to lead this activity known as the Enhancement of Greenhouse Gas Sinks.49 The CFS Science Advisor/Coordinator in the Science and Policy Directorate in Ottawa was responsible for the development and implementation of its forest component.
The CFS-CC Program also coordinated the impacts and adaptation work with the 1998 National Climate Change Process that included Action Plan 2000, the Climate Change Action Fund, and the C-CIARN Forest Node, that became the awareness and communications focal point for information on forest impacts and adaptation across Canada located in the CFS Northern Forestry Centre in Edmonton.50
The CFS-CC Program undertook much of its research through collaboration, effectively leveraging its research dollars with provincial and university scientists. (The extent of this leveraging has not been quantified.) A good example is the university-led Fluxnet-Canada Program which was able to provide critical data for a small CFS investment of approximately one million dollars to the $12.5 million total cost.
How successful has the CFS been in directing the integration of knowledge and directing this knowledge towards soundly-based and appropriate policy?
The CFS has been highly successful in developing and employing a strong climate change policy program based on an effective integration of scientific knowledge and policy. The reasons for the policy success may in part be attributed to the relevance of the research undertaken to the decision deliberations, the accessibility of information, and the receptiveness of policymakers to the results.51 Key in shaping this success has been CFS’s Policy, Economics and Industry Branch working in cooperation with the federal-provincial National Sinks Committee.
This evaluation has concluded that the CFS-CC Program continues to be relevant, has a strong rationale and has been successful in achieving its objectives during the evaluation period. Under the issue of Cost Effectiveness and Alternatives, the evaluation’s findings are less clear cut.
While the evaluation is unable to provide the reader with strong evidence, it appears that the CFS-CC Program has operated in an efficient and cost-effective manner.
There do not appear to be more cost-effective means for delivering the CFS-CC science and policy mandate. The long-term nature of the climate change issues and the considerable investments already made by government would have to be addressed in any fundamental change in delivery.
Some interviewees expressed a number of concerns, primarily with regard to the Science Component, related to governance and management issues. Interviewees cited the loss of management oversight and direction attributed to the expiration of some supplemental programs, diminished co-ordination, frequent personnel changes and re-organizations that caused uncertainty and confusion over roles.
Given the loss of supplemental funding and other staffing issues, the CFS-CC Program may not be able to continue to achieve its objectives.
Has the CFS-CC Program operated in an efficient and cost-effective manner?
While the evaluation is unable to provide the reader with strong evidence, it appears that the CFS-CC Program has operated in a cost effective manner.
The CFS-CC Program was a horizontal initiative cutting across a vertical organizational structure. At the time, the Department’s financial system did not lend itself to tracking horizontal climate change expenditures.
In addition, it is very difficult to judge the cost effectiveness of scientific research let alone government policy work involving international negotiations, reporting requirements and demonstration projects. The ongoing relevance and success of the CFS-CC Program suggests good value for money but the evaluation is not able to provide solid evidence of cost effectiveness. This issue, the difficulty in demonstrating cost effectiveness, is common to many evaluations and should not be interpreted as criticism of the cost effectiveness of the Program. It is not clear that there are more cost-effective means for delivering the CFS-CC Program.
What is clear is that the CFS-CC Program obtained most of its funding from supplemental sources subject to external proposal and funding review by the supplementary funding programs and was able to leverage resources from within and outside NRCan.
Are there more cost-effective alternative means to achieve the CFS-CC mandate?
It is not clear that there are more cost-effective means for delivering the CFS-CC science and policy mandate.
The debate over the role of federal government science and the best way to achieve this role, particularly the amount of research that government should do in-house, has been ongoing for decades. In recent years, this debate has taken on increased profile in light of scarce resources and the need for ever-greater collaboration with academia and the private sector to address today’s horizontal issues.
Some interviewees suggested the possibility of having components of the CFS-CC Program delivered through a co-venture with a university. The long-term “public good” nature of the climate change issue and the need to have ongoing policy support at the national level would have to be carefully considered in any such proposal. Academia might not provide the long-term stability and commitment for a national policy-driven issue such as climate change. The costs and benefits of such an approach would need to reflect the fact that CFS is already well networked with universities and provinces/territories.
Regardless of the delivery model, there is a need for long-term monitoring and dataset management to support climate change science. This is one of the less-glamorous yet essential tasks to understanding climate change impacts and assessing how Canada’s forest ecosystems respond. Over the years, governments have invested significant resources in sources of information that could be lost due to current funding pressures. The CFS, other federal departments and provincial/territorial forest agencies will need to work together to successfully address this issue. In doing so, they may wish to focus attention on the future roles of:
How effective has the CFS-CC Program’s governance structure been?
Some interviewees raised concerns about the effectiveness of the CFS-CC Program’s governance structure in recent years, particularly a perceived lack of direction for the Science Component.
The question of the governance of the CFS-CC Program must be considered within the wider CFS context. CFS had two governance structures during the evaluation period (i.e., networks; and business lines). Indeed, some might say that since the creation of NRCan in 1993, the CFS has been seeking the best way to address horizontal issues within the vertical reality of its regional structure.
From the beginning of the review period up to 2004, the CFS-CC Science Component was led by a Regional Director General (DG), supported by a Steering Committee of Regional Directors and the Climate Change Science Advisor/Coordinator from headquarters (HQ). In 2004, under the Business Line Activity structure, a Climate Change Coordination Committee was established, with a regional and a HQ DG serving as co-chairs. Membership of the Coordinating Committee included a Director from each of the five regional centres and the Climate Change Coordinator from HQ.
The Steering Committee or Coordination Committee was responsible for ensuring that externally funded projects were aligned with the CFS CC research program goals, objectives and deliverables. The Committee met annually or semi-annually for project planning and review purposes. During this period, two major staff workshops were held with senior climate change scientific staff to develop the Impacts and Adaptation portion of the Climate Change Business Line.52, 53
A key factor contributing to the CFS-CC Program’s governance was the management oversight and direction attributed to supplemental funding (e.g., from PERD, AP2000, and the CCAF technical and coordinating committees). It also profited significantly from the coordination provided by the CFS’s Science Branch with a CFS-CC Advisor/Coordinator also serving on these key outside funding bodies.
Throughout the period, the headquarters-based Director responsible for the Policy Component had clear objectives, accountabilities and control over resources. However, the regional DGs assigned to the CFS-CC Network/Business Line have been hampered by lack of control over the budgets of those portions of the CC Program outside their regional centres.
Interviews with leading administrators and scientists representing all five regional centres and HQ suggest that in recent years, direction provided to the Science Component has significantly diminished. Some interviewees attributed this lack of direction to changing activity structures, the proliferation of internal planning processes, frequent staffing changes of key headquarters management and advisor/coordinator positions and re-organizations that caused uncertainty and confusion over roles.
CFS management points to the governance mechanisms mentioned above to demonstrate that direction was provided throughout the evaluation period.
What is the state of Science Management within the CFS-CC Program?
Many interviewees raised concerns falling within the area of science management.
Some CFS interviewees indicated that CFS has been in continual transition since 2003-04 and that the transaction costs of doing business-planning in the organization have become high. They stated that in the last several years, Regional DGs and Directors have had to place more time on meeting corporate demands and less on directing science. Several scientists complained of a lack of consistent direction and having senior scientists largely operating on their own.
Staffing is another issue with new hires of scientists being below replacement levels for many years. For example, new recruits into the CFS-CC Program (as well as related fields) have been rare to non-existent in the regional centres for many years. The total number of indeterminate research scientists (SE-RES) in the CFS fell from 152 to 121 from 2000 to 2008 (a 20% decline).
CFS management points out that a number of indeterminate and term staff in a variety of categories were hired during the period to work on CC but this staffing was not evenly distributed across CFS centres. CFS management also notes that the high level of supplemental funding resulted in the hiring of many CFS staff as term rather than indeterminate employees.
The CFS-CC Program has not had an external peer-review of its science since 1993. There are no NRCan-wide policies on periodic external peer review for science programs at this time.54
The rationale for science within federal agencies is to provide government with independent, world-class, policy-relevant S&T with transparency, openness and ethics. The key for government is to be able to ensure and demonstrate the excellence of its S&T, and according to the former Council of Science and Technology Advisors: “The greater the complexity and potential controversy, the greater the need for government to invest greater effort to ensure and demonstrate S&T excellence… the most widely accepted process to measure excellence… is peer review.”55 This view is also suggested by the Office of the Auditor General56 and Agriculture and Agri-Food Canada57 as well by the US Department of Energy and the US Forest Service.58 The peer-review process, although onerous, validates the quality of the science, and it also helps the scientists to enhance networks and improve performance through involvement with the peer panel.
Perhaps, the most useful interpretation of the concerns reported above under governance and science management is that they should be viewed from the perspective of employee morale and the lack of a shared vision. In 1999, the Auditor General published a chapter entitled, “Attributes of Well-Managed Research Organizations” and classified ten attributes under the headings of People Focus, Leadership, Research Management and Organizational Performance. Among other things, the OAG report noted:
CFS management has worked long and hard under difficult circumstances to make the organization more relevant, horizontal, outward looking and responsive to government’s policy issues and industry’s competitiveness concerns. However, while much has been accomplished, “change fatigue” may be leading to cynicism and diminishing returns. This represents a major challenge to CFS senior management.
It is recognized that the CFS-CC Program has been through a very productive period and that changing government priorities, scarce resources and uncertainty make planning difficult. Nonetheless, it is in periods of uncertainty that strategic planning and a shared vision are most required.
Did the CFS-CC Program use an effective results-based management strategy for ongoing measurement, management and reporting of performance?
The effectiveness of the results-based management strategy for the CFS-CC Program appears to have varied over time as well as between and within the Components. Several business plans were developed during the period but there have not been regular annual reports of the CFS-CC Program since 2003 and this makes it difficult to come to a clear conclusion.
It is clear that tight accountability rules were followed for most activities funded from supplemental sources. For example, PERD funding was monitored and evaluated on a regular basis. While AP2000, CCAF and CCIAP have also been subject to review and evaluation, the relatively small CFS-CC portions of these programs received limited attention.
However, problems with the CFS-CC Program have arisen in the consistency and continuity of the application and determination of the performance measurements. Part of the difficulty arises from the frequent changes made to the overall CFS program activity structure and confusion over which activities were in which Component from one year to the next.
This resulted in great difficulty in tracking expenditures and results in a consistent manner from year to year. Measures on a constantly changing collection of activities will not provide management with useful information. Any future changes in the structure should ensure year-to-year comparability of inputs and results.
Is the funding structure of the CFS-CC the most appropriate mechanism for achieving its intended objectives?
No. Under the current funding regime, the past productivity of the CFS-CC is likely not sustainable. Climate change research requires a long-term commitment, whereas the current CFS-CC Program has had to operate with infusions of short-term funding from PERD, AP2000 and CCAF. Although CFS-CC outputs have been world class, they are likely not sustainable given the recent loss of most of the funding from these sources.
The most significant loss was PERD’s Enhancement of Greenhouse Gas Sinks that was recognized by many as a model for successful collaboration and integration of research across departmental and scientific disciplines. It served as a key networking initiative and offered some degree of peer review that was missing in non-PERD components of the Program.
Furthermore, BIOCAP Canada officially ceased operations on March 31, 2008. BIOCAP had been a principal supporter of Fluxnet-Canada (now replaced by the CCP). This, combined with the loss of NSERC funding after 2007, led to the closing of Fluxnet-Canada. The combined losses of PERD’s Enhancement of Greenhouse Gas Sinks, CCAF, AP2000, and BIOCAP/Fluxnet and the loss of coordinating mechanisms such as the forestry node of C-CIARN represent a considerable reduction in the CFS-CC Program’s capacity.
The problem has been compounded by the gradual reduction of CFS’s total ongoing A-Base resources, the loss of capacity through non-staffing of vacancies and loss of key expertise through retirements in allied fields such as forest fire.
How satisfied are clients and partners with CFS-CC delivery? What improvements could be made?
The CFS-CC Program’s clients and partners interviewed were well versed in the details of the Program and are highly satisfied with its delivery but also raised issues.
The major Policy Component partners and clients are the members of the National Forest Sinks Committee and other provincial government offices and many of the key collaborators were interviewed. Members of the Sinks Committee interviewed indicated that they were very satisfied with their collaborations with CFS-CC Program.
Those partners/collaborators from universities interviewed were also highly satisfied with the quality of their relationships with the CFS-CC Program but did show concern for the diminished science capacity in the CFS CC and in CC-related areas such as wildfire, particularly at the Northern Research Centre.
Concern was also expressed by some interviewees about the CFS’s inability to maintain involvement, leadership and resourcing of long-term field programs (e.g., Fluxnet-Canada/Canadian Carbon Program) and networks (C-CIARN). Termination of PERD, AP2000 and CCAF government programs and the inability of the CFS to secure replacement funding for these programs has severely curtailed the CFS-CC Program’s ability and capacity to play a significant role in these major national projects.
Some university collaborators also requested continued CFS assistance in maintaining national database systems, particularly Fluxnet. Industry was not as involved in the Program, but those representatives interviewed did mention the need to be kept informed about the results of work to which they have contributed or have an interest, such as Forest 2020 PDA.
The CFS-CC Program was very productive and successful throughout 1999-2006. The CFS-CC Program helped better understand the impact of climate change on Canadian forests and played a significant role internationally.
The CFS has pioneered the development of carbon-cycle science and created a Carbon Budget Model that has been used to predict the probability of Canada’s forests being a future sink or source underpinning Canada’s position to not elect forest management under the Kyoto Protocol.
The CFS, in cooperation with the provincial/territorial forest agencies, has put in place Canada’s National Forest Carbon, Monitoring, Accounting and Reporting System to meet Canada’s international forest greenhouse gas emissions reporting obligations.
In the large-scale field experiment area of carbon cycle science, the CFS, in collaboration with the NRCan’s Canada Centre for Remote Sensing (CCRS) and NASA, initiated BOREAS that developed into Fluxnet-Canada/CCP. The CFS Policy Component undertook a series of initiatives to create a better awareness and understanding of forest carbon management including the potential role of afforestation and the use of fast-growing species in mitigation strategies.
Despite this very positive record, the interviews conducted during this evaluation discovered serious frustration and discontent throughout much of the Science Component. Specific concerns included: perceptions among scientists of a lack of coordination; change fatigue; and lack of recent staffing. These concerns suggest issues of morale and lack of shared vision. Combined with funding pressures past productivity may not be sustainable.
Preamble:
CFS has been active in the subject area of climate change for close to 20 years and more recently this science has been closely aligned with public policy. In 2006-07, the year of the evaluation, federal climate change policy was in transition and moving toward the development of a domestic clean air agenda resulting in the revised resourcing to climate change programs. The shift in government priority required a re-alignment within the science program and as a result resources were re-allocated, and in some areas reduced, or eliminated.
Since the evaluation, CFS has taken steps to address the recommendations. The entire CFS program of activity (science, policy and programs) was redesigned to focus on results as opposed to the traditional discipline focus. What was previously an inventory of small scale, albeit productive, discipline focused scientific studies, became the building blocks for larger scale national projects focused on outcomes (NRCan Program Activity Architecture (PAA) sub-sub-activity level), that merged science, programs and policy, under a change agenda called the Innovation Management Process. In essence, this is a management system that relies on ongoing environmental scanning and disciplined review to guide creation of a portfolio of national projects comprised of science, programs and policy. The rigorous set of reviews consider such criteria as: relevance, risk/reward, impact analysis, knowledge position, HR capacity, and performance measures to enable informed management decision-making in directing human and financial resources towards highest priorities.
It was during this period of change and change management, that the evaluation study authors interviewed scientific staff across the country, who legitimately expressed frustration and confusion over the changes to climate change policy, coordination of activities, as well as the organizational changes underway in the sector.
Since the time of the evaluation, CFS has created a country-wide Learning Organization-Community of Practice to engage staff and promote leadership at all levels to facilitate and support change. CFS, as a subject matter expert in forest-related climate change science and science-policy integration, has also undertaken significant outreach activities to other government departments and universities and the public to promote its work.
This year (2008/09) marked a deliberate reversal of past attrition of science capacity by recruiting forestry science and policy experts into 20 newly created, priority positions located across the country of which a number have already been filled.
CFS will work with the NRCan Office of the Chief Scientist on a departmental approach to peer review.
| Recommendation | Management Response | Resp. Official & (Target Date) |
|---|---|---|
| 1. It is recommended that the Canadian Forest Service enhance its climate change planning by: | Publically funded climate change science and policy are intimately linked and mutually supportive therefore climate change plans going forward must pass the test of good science and effective public policy. As with the department, CFS planning is undertaken to address outcomes (as opposed to a discipline focus), and has established: | ADM-CFS (Complete) |
| - articulating the key objectives, priorities, expected outcomes, resources and strategic partnerships required; | - eight NRCan PAA Sub-Activities, two that encompass the subject area of Climate Change, that specify objectives, priorities, performance measures, resource requirements and partnerships which are reported annually through the Reports on Plans and Priorities; | |
| - outlining the future development and use of key CFS intellectual property arising from climate change research, such as the Carbon Budget Model; and | - the priority for intellectual property arising from publicly funded research, such as the Carbon Budget Model, is to make it as widely available as possible in a timely manner (note 178 papers produced and open access to the Carbon Budget Model is provided). More specifically, as part of the CFS Knowledge Transfer plan, intellectual property issues are managed by project leads and scientific staff to ensure optimum benefit to Canada. | |
| - identifying the key Climate Change related forest monitoring data and infrastructure required to implement the plan (e.g., field stations and long-term Canadian Carbon Program databases). | - two NRCan Sub-Activities’ that encompass climate change discipline area (these are multi-year larger scale initiatives that include anywhere between two to six national projects), describe the activities, annual outputs, expected results, performance measures and resources and the resource allocation also includes forest data requirements and minimum infrastructure that will be supported within the resource allocations available. | |
| Recommendation | Management Response | Resp. Official & (Target Date) |
| 2. It is recommended that the Canadian Forest Service use the findings of this evaluation to focus attention on issues regarding the management of CFS climate change science, inter alia: | Although the introduction of Innovation Management pre-dated this climate change evaluation, many of the issues referred to, have been or are being addressed, particularly around building stronger links between climate change science and policy. An example of the success in this regard is the recent IPAC award to the CC-CAT for integrating science and policy and the contribution of CFS to the United Nations Intergovernmental Panel on Climate Change (IPCC) which was a co-recipient of the Nobel Peace Prize in 2007. A number of CFS scientists continue to work with the IPCC on climate change. More specifically: | ADM-CFS (Complete) |
| - clarification of the role of climate change science in advancing CFS priorities; | - as a science based policy organization, all projects that include scientists and science professionals are part of all phases of the planning, priority setting and resource allocation processes across the CFS, in fact, climate change is an overlay of all biological science undertaken at CFS; | |
| - coordination of climate change scientific research, across the laboratories in the regional centres; | - regional research Directors, Directors General as well as the Director General of the Science and Programs Branch coordinate development of the CFS annual work program (including climate change science); | |
| - actions needed to ensure that the CFS continues to have a world-class scientific staff sufficient to address future policy needs; and | - the CFS HR sub-Committee has developed a strategy that has set national recruitment priorities for science and policy, CFS recruited indeterminate staff for the climate change science during the timeframe of the current evaluation; and |
|
| - the need for periodic external peer review of CFS climate change science research in ensuring world-class scientific performance. | - CFS is adopting a work execution plan that ensures the quality of the work and builds in a science and policy review at Project Milestones, and will work with the NRCan Office of the Chief Scientist on a departmental approach to external peer review. |
1 Harrington, J.B., 1987. “Climatic Change: A Review of Causes.” Canadian Journal of Forest Research, v. 17, no. 11, pp. 1313–1339.
2 Paraphrased from the Canadian Forest Service. Program Activity Architecture – 2005/2006 and 2006, 2006/11/01, p. 3.
3 1) Enhancement of Greenhouse Gas Sinks (POL 6.2.1); 2) Agricultural and Forestry Biomass Production (POL 4.2.5) later changed to Canadian Biomass Innovation Network; and 3) Climate Change Impacts on the Energy Sector (POL 6.1.1).
4 The Canadian Foundation for Climate and Atmospheric Sciences (CFCAS) is the main funding body for university-based research on climate, atmospheric and related oceanic work in Canada providing research grants through a competitive process. BIOCAP Canada is a national university research funding organization brings together leading researchers and decision-makers from across Canada to find biological-based solutions to the challenge of climate change. In 2001 NRCan, AAFC and Environment Canada agreed to provide BIOCAP with $10 million via a five-year contribution agreement. The Fluxnet-Canada Research Networkwas established in 2002 with its primary objective to study how management practices (harvesting), natural disturbance (fire), and climate variability influence carbon cycling in forest and peatland ecosystems in Canada. The government portion of this research effort was supported by Natural Resources Canada and Environment Canada. The work and assets of this research program were transitioned to the Canadian Carbon Program (CCP) in early 2007.
5 CFS-NRCan, 2007. “The Importance of Forest Sector Adaptation to Climate Change.” Draft, 21 November 2007.
6 IPCC, 2007. “Intergovernmental Panel on Climate Change Forth Assessment Report: Climate Change 2007, Working Group I Report “The Physical Science Basis”, Summary for Policymakers.” Cambridge University Press, Cambridge, 2007. p. 15.
7 Weber, M.G., Flannigan, M.D., 1997. “Canadian Boreal Forest Ecosystem Structure and Function in a Changing Climate: Impact on Fire Regimes.” Environmental Reviews, v. 5, pp. 145-166.
8 Taylor, G., et al, 2008. “Future Atmospheric CO2 Leads to Delayed Autumnal Senescence.” Global Change Biology, v. 14, pp. 264–275.
9 Environment Canada, 2006. “National Inventory Report 1990-2004: Greenhouse Gas Sources and Sinks in Canada.” The Canadian Government's Submission to the UN Framework Convention on Climate Change. Her Majesty in Right of Canada, Ottawa, Ontario. April 2006.
10 CFS-NRCan, 2007. “The Importance of Forest Sector Adaptation to Climate Change.” Natural Resources Canada, Canadian Forest Service Draft, Ottawa, Ontario. 21 November 2007.
11 Government of Canada, 2007. “Canada's Initial Report under the Kyoto Protocol.” Ottawa, Ontario. Available at: http://www.ec.gc.ca/climate/initial_Kyoto_Rep_e.pdf Accessed: 21 June 2008. p. 16.
12 The National Forest Sinks Committee comprised of representatives of the Federal and Provincial Government addresses policy issues, scientific questions and scenario analyses. Through this Committee provinces and territories supply data for use in developing the CBM-CFS model and meeting Canada's international reporting commitments.
13 Natural Resources Canada, 2007. Natural Resources Canada, Canadian Forest Service, Forest Carbon Accounting the Need for Forest Carbon Accounting, Home and Overview. Available at: http://carbon.cfs.nrcan.gc.ca/index_e.html 25 October 2007.
14 Kurz, W. A., and Apps, M. J., 2006. “Developing Canada's National Forest Carbon Monitoring, Accounting and Reporting System to Meet the Reporting Requirements of the Kyoto Protocol.” Mitigation and Adaptation Strategies for Global Change. v.11, no.1, pp. 33-43.
15 Dymond, C.C., White, T., and Kurz, W.A., 2007. “Overview of Canada's National Forest Carbon Monitoring, Accounting and Reporting System and the National Inventory Report 2006 on Land-use, Land-use Change and Forestry.” Canadian Forest Service, Victoria, British Columbia.
16 National Climate Change Process, 1999. “Sinks Table Options Paper, Land-use, land-use Change and Forestry in Canada and the Kyoto Protocol.” 23 September 1999. Available at: http://dsp-psd.pwgsc.gc.ca/Collection/M22-132-13-1999E.pdf Accessed: 27 May 2008. p. 16.
17 Forest Sector Table, 1999. “Options Report: Options for the Forest Sector to Contribute to Canada's National Implementation Strategy for the Kyoto Protocol.” 21 July 1999. p. 23.
18 CFS, 2002. “Climate Change S&T Network, Proposed Work Plan for 2002-2003 and Beyond.” Canadian Forest Service, Ottawa, Ontario. January 2002.
19 House of Commons of Canada, 2005. “Finding The Energy To Act: Reducing Canada's Greenhouse Gas Missions.” Report of the Standing Committee on Environment and Sustainable Development. July 2005. pp. 33-47.
20 Office of the Auditor General of Canada, 2006. “2006 Report of the Commissioner of the Environment and Sustainable Development to the House of Commons Chapter 2 Adapting to the Impacts of Climate Change.” September 2006. Ottawa, Ontario. sect. 2.56, p. 21.
21 Intergovernmental Panel on Climate Change, 2003. “Good Practice Guidance for Land Use, Land-Use Change and Forestry.” Edited by Penman, J., et al. IPCC National Greenhouse Gas Inventories Programme. Published by the Institute for Global Environmental Strategies (IGES) for the IPCC, Kamiyamaguchi Hayama, Kanagawa, Japan.
22 IPCC website. Available at: http://www.ipcc.ch/ipccreports/assessments-reports.htm Accessed: 22 July 2008.
23 Eastnaugh, C., 2008. “Adaptations of Forests to Climate Change: A Multidisciplinary Review.” IUFRO Occasional Paper no.21. IUFRO Secretariat, Vienna. 7 February 2008.
24According to the Science-Metrix study, of the 2,321 forest science papers published by CFS during the 1991-2006 period, 213 papers were in the forest area of climate change.
25 McKenney, D.W., Pedlar, J.H., Papadopol, P., Hutchinson, M.F., 2006. “The Development of 1901–2000 Historical Monthly Climate Models for Canada and the United States.” Agricultural and Forest Meteorology. v.138, issues 1-4, 29 August 2004. pp. 69-81.
26 Stewart, R., Kurz, W., Price, D., 2001. “Carbon Cycling/Budgets and Carbon Accounting - CFS CC & Fire Network Scoping Paper.” Paper for the Canadian Forest Service, Natural Resources Canada. October 9, 2001.27 CFS, 2002. “Climate Change S&T Network, a Proposed Work Program for 2002-03 and Beyond.” Canadian Forest Service, Natural Resources Canada. January 2002.
28 Kull, S., Kurz, W., Rampley, G., Banfield, E., Schvatcheva, T., Apps, M., 2007. “Operational-Scale Carbon Budget Model of the Canadian Forest Sector (CBM-CFS3) Version 1.0: User's Guide.” Northern Forestry Centre, Canadian Forest Service, Natural Resources Canada. Edmonton, Alberta.
29 Kull, S., Kurz, W., Rampley, G., Banfield, E., Schvatcheva, T., Apps, M., 2004. “Operational Scale Carbon Budget Model of the Canadian Forest Sector (CBM-CFS3) Beta Users Guide.” Northern Forestry Centre, Canadian Forest Service, Natural Resources Canada. Edmonton, Alberta.
30 Kull, S., 2008. Personal communication. Pacific Forestry Centre, Canadian Forest Service, Natural Resources Canada. Victoria, British Columbia.
31 NRCan-CFS, 2008. Natural Resources Canada, Forest Research, Research Projects, Introduction, Climate Change Impacts on the Productivity and Health of Aspen (CIPHA). Available at: http://cfs.nrcan.gc.ca/projects/150/1 Accessed: 22 July 2008.
32 NRCan-CFS, 2007. Natural Resources Canada, Canadian Forest Service, Canadian Intersite Decomposition Experiment (CIDET). Available at: http://cfs.nrcan.gc.ca/subsite/cidet 26 March 2007.
33 NRCan-CFS, 2003. Natural Resources Canada, Canadian Forest Service, ECOLEAP: Estimating Productivity, from Leaf to Landscape. (ECOLEAPExtended COllaboration for Linking Ecophysiology And forest Productivity.) From “Branching Out” Number 1. Available at: http://cfs.nrcan.gc.ca/news/173 1 January 2003.
34 Kurz, W., 2006. “Large Forests = Large Sinks? The Contribution to Canada's Forests to the Carbon Cycle.” CFS Science and Policy Seminar. Ottawa, Ontario. 7 December 2006.
35 Kurz, W., Lempriere, T., 2007. “Forests, Carbon and Climate Change.” BC Forum on Forest Economics and Policy, UBC. 21 May 2007.
36 Kurz, W. A., et al., 2008. “Risk of Natural Disturbances Makes Future Contribution of Canada's Forest to the Global Carbon Cycle Highly Uncertain.” Proceedings of the National Academy of Sciences of the United States of America. vol. 105, no. 5 1551-1555.
37 Ibid
38 Canadian Council of Forest Ministers, 2006. “Criteria and Indicators of Sustainable Forest Management in Canada – National Status 2005.” Report of the Canadian Council of Forest Ministers. Ottawa, Ontario. pp. 78-87. Available at: http://www.ccfm.org/ci/rprt2005/C&I_e.pdf
39 NRCan-CFS, 2007. “The State of Canada's Forests- Annual Report 2007.” Her Majesty the Queen in Right of Canada, Ottawa, Ontario. pp. 10-11, 24. Available at: http://warehouse.pfc.forestry.ca/HQ/27510.pdf
40 McKenney, D., et al., 2007. “Customized Spatial Climate Models For Canada.” Frontline Forestry Research Applications. Technical Note no. 108, Canadian Forest Service, Great lakes Forestry Centre. Sault Ste. Marie, Ontario.
41 NRCan, 2008. Natural Resources Canada, Canadian Forest Service, Regional, National and International Climate Modeling. Available at: http://cfs.nrcan.gc.ca/subsite/glfc-climate 14 July 2008
42 NRCan, 2002. “Canada's Plant Hardiness Zones.” Article for the “Frontline Express” number 13, newsletter for Natural Resources Canada, Canadian Forest Service, Great Lakes Forestry Centre. Sault Ste. Marie, Ontario. Available at: http://cfs.nrcan.gc.ca/news/265 13 February 2002.
43 Natural Resources Canada, 2008. Natural Resources Canada, Canadian Forest Service, Afforestation Policy Analysis, Feasibility of Afforestation for Carbon Sequestration (FAACS) Initiative. Available at: http://cfs.nrcan.gc.ca/subsite/afforestation/feasibilityafforestation 15 July 2008.
44 NRCan-CFS, 2006. “Feasibility Assessment of Afforestation for Carbon Sequestration (FAACS) Initiative Final Report.” Ottawa, Ontario. Available at: http://cfs.nrcan.gc.ca/files/285
45 C-CIARN Forests, 2007. “2006-2007 State-of-Play Report: The Status of Climate Change Impacts and Adaptation from the Perspective of C-CIARN- Forest Sector.”
46 National Climate Change Process, 1999. “Designing a GHG Offsets System for Canada.” Sinks Table Options Paper. September 23, 1999.
47 Houghton, J.T., et al., 1996. “Climate Change: The Science of Climate Change. Contribution of Working Group 1 to the Second Assessment of the Intergovernmental Panel on Climate Change.” Cambridge University Press, Cambridge.
48 Canadian Forest Service. “Green Plan, Climate Change Initiative Progress Reports, 1992/93 to 1999/2000.”
49 NRCan-OERD, 2007. “Enhancement of Greenhouse Gas Sinks: A Science Assessment. POL 6.2 Final Report.” Internal NRCan document. December 2007.
50 C-CIARN Forests, 2007. “2006-2007 State-of-Play Report: The Status of Climate Change Impacts and Adaptation from the Perspective of C-CIARN- Forest Sector.”
51 Jones, S.A., et al., 1999. “Evaluating the Science-Policy Interface for Climate Research.” Climatic Change. v. 43, pp. 581-599.
52 By the end of 2007, the CFS Business Line had evolved to the NRCan Sub-Activities and the priorities developed for the Impacts and Adaptation work formed the basis for the NRCan Sub-Activity “Natural and human-caused forest disturbances are mitigated and option for adaptation are provided.”
53 In 2008, the CFS put in place a new National Project Portfolio structure in support of departmental sub-activities. Under the new CFS Projects and Services Portfolio there are two key projects involving CFS-CC initiatives, ‘Forest Carbon Science, Reporting and Policy' and ‘Strategies for Climate Change Impacts and Adaptation.' Fire, insects and diseases, and ecosystem dynamics components of CFS climate change research are not included in these two projects even though they are still feed into the CC Program. It is too early to judge the effectiveness of this new structure.
54 Peer reviews are technical assessments performed by in-house and external experts with qualifications at least equal with those of the researcher whose work is being reviewed. Peers must be knowledgeable about the area to be reviewed. Users of research results or members of stakeholder groups sometimes assist in performing peer reviews. The expert opinions on the utility and quality of the results and program priorities help the science agency fulfill its responsibilities and improve customer service.
55 Council of Science and Technology Advisors, 2001. “Science and Technology Excellence in the Public Service: A Framework for Excellence in Federally Performed Science and Technology.” Industry Canada. Ottawa, pp. 11, 13.
56 Office of the Auditor General of Canada, 1999. 1999 November Report of the Auditor General of Canada, Chapter 22 Attributes of Well-Managed Research Organizations.” Report of theAuditor General of Canada. Ottawa, Ontario. November 1999. ch. 22 p. 8.
57 Agriculture and Agri-Food Canada, 2004. “External Peer Review Panel Report Re. AAFC's Program on Reducing GHG Emissions from Agricultural Systems.” Ottawa, Ontario.
58 U.S. General Accounting Office, 1999. “Peer Review Processes at Federal Science Agencies Vary.” GAO/RCED-99-99. Washington, D.C. March 1999.
59 Office of the Auditor General of Canada, 1999. “1999 November Report of the Auditor General of Canada, Chapter 22 Attributes of Well-Managed Research Organizations.” Report of the Auditor General of Canada. Ottawa, Ontario. November 1999. ch. 22.