Evaluating Impact Using Time-Series Data

Ecologists have called for more robust studies on the impact of conservation interventions, or environmental shocks, on outcomes of interest, such as populations, habitat loss, or pressures.Time-series data are increasingly available and can, if appropriately analysed, allow such causal inferences.However, there are important pitfalls that make large-scale analyses involving multiple time series problematic.There has been progress in a range of fields, but the literature is fragmented and not all is easily accessible to ecologists.A framework is presented, with clear and consistent terminology, to support ecologists to conduct effective impact evaluation with time-series data. This will allow them to contribute to better-informed environmental management decisions. Humanity’s impact on the environment is increasing, as are strategies to conserve biodiversity, but a lack of understanding about how interventions affect ecological and conservation outcomes hampers decision-making. Time series are often used to assess impacts, but ecologists tend to compare average values from before to after an impact; overlooking the potential for the intervention to elicit a change in trend. Without methods that allow for a range of responses, erroneous conclusions can be drawn, especially for large, multi-time-series datasets, which are increasingly available. Drawing on literature in other disciplines and pioneering work in ecology, we present a standardised framework to robustly assesses how interventions, like natural disasters or conservation policies, affect ecological time series. Humanity’s impact on the environment is increasing, as are strategies to conserve biodiversity, but a lack of understanding about how interventions affect ecological and conservation outcomes hampers decision-making. Time series are often used to assess impacts, but ecologists tend to compare average values from before to after an impact; overlooking the potential for the intervention to elicit a change in trend. Without methods that allow for a range of responses, erroneous conclusions can be drawn, especially for large, multi-time-series datasets, which are increasingly available. Drawing on literature in other disciplines and pioneering work in ecology, we present a standardised framework to robustly assesses how interventions, like natural disasters or conservation policies, affect ecological time series. Ecologists often seek to understand the impact of a conservation intervention (e.g., a protected area or reintroduced species), or an environmental shock (e.g., an oil spill or hurricane) on one or more response variables (e.g., population counts or habitat loss) [1.Bruner A.G. et al.Effectiveness of parks in protecting tropical biodiversity.Science. 2001; 291: 125-128Crossref PubMed Scopus (1050) Google Scholar, 2.Cuenca P. et al.How much deforestation do protected areas avoid in tropical Andean landscapes?.Environ. Sci. Policy. 2016; 56: 56-66Crossref Scopus (25) Google Scholar, 3.Geldmann J. et al.Effectiveness of terrestrial protected areas in reducing habitat loss and population declines.Biol. Conserv. 2013; 161: 230-238Crossref Scopus (405) Google Scholar, 4.Ripple W.J. Beschta R.L. Trophic cascades in Yellowstone: the first 15 years after wolf reintroduction.Biol. Conserv. 2012; 145: 205-213Crossref Scopus (323) Google Scholar, 5.McCleery R. et al.Better off in the wild? Evaluating a captive breeding and release program for the recovery of an endangered rodent.Biol. Conserv. 2014; 169: 198-205Crossref Scopus (15) Google Scholar, 6.Moreno R. et al.Ten years after the Prestige Oil Spill: seabird trophic ecology as indicator of long-term effects on the coastal marine ecosystem.PLoS One. 2013; 8e77360Crossref PubMed Scopus (31) Google Scholar, 7.Krauss J. et al.Habitat fragmentation causes immediate and time-delayed biodiversity loss at different trophic levels.Ecol. Lett. 2010; 13: 597-605Crossref PubMed Scopus (492) Google Scholar, 8.Woodcock B.A. et al.Impacts of neonicotinoid use on long-term population changes in wild bees in England.Nat. Commun. 2016; 712459Crossref PubMed Scopus (200) Google Scholar, 9.Chevalier M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google Scholar]. In recent years, there has been a surge of literature calling for more rigorous impact evaluation in ecology and conservation [10.Baylis K. et al.Mainstreaming impact evaluation in nature conservation.Conserv. Lett. 2016; 9: 58-64Crossref Scopus (196) Google Scholar, 11.Lesbarrères D. Fahrig L. Measures to reduce population fragmentation by roads: what has worked and how do we know?.Trends Ecol. Evol. 2012; 27: 374-380Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar, 12.Woodhouse E. et al.Guiding principles for evaluating the impacts of conservation interventions on human well-being.Philos. Trans. R. Soc. B Biol. Sci. 2015; 37020150103Crossref PubMed Scopus (77) Google Scholar, 13.Jung M. et al.Impacts of past abrupt land change on local biodiversity globally.Nat. Commun. 2019; 10: 5474Crossref PubMed Scopus (9) Google Scholar, 14.Butsic V. et al.Quasi-experimental methods enable stronger inferences from observational data in ecology.Basic Appl. Ecol. 2017; 19: 1-10Crossref Scopus (20) Google Scholar]. While the terms impact evaluation (see Glossary) and intervention are often used to describe the impact of a deliberate intervention, such as a policy change [10.Baylis K. et al.Mainstreaming impact evaluation in nature conservation.Conserv. Lett. 2016; 9: 58-64Crossref Scopus (196) Google Scholar], here they are used to consider the general problem of causal inference. To determine the impact of an intervention, one must understand what would have happened if the intervention had not occurred [15.Ferraro P.J. Counterfactual thinking and impact evaluation in environmental policy.New Dir. Eval. 2009; 2009: 75-84Crossref Scopus (238) Google Scholar]. Ideally this is achieved through an experimental setup, where units are randomly allocated to treatment and control groups. However, while experimental manipulation of whole ecosystems, or random application of conservation interventions at scale do exist [16.Fayle T.M. et al.Whole-ecosystem experimental manipulations of tropical forests.Trends Ecol. Evol. 2015; 30: 334-346Abstract Full Text Full Text PDF PubMed Scopus (30) Google Scholar,17.Wiik E. et al.Experimental evaluation of the impact of a payment for environmental services program on deforestation.Conserv. Sci. Pract. 2019; e8: 1Google Scholar], such experiments are seldom feasible [10.Baylis K. et al.Mainstreaming impact evaluation in nature conservation.Conserv. Lett. 2016; 9: 58-64Crossref Scopus (196) Google Scholar,18.Pynegar E.L. et al.The effectiveness of Payments for Ecosystem Services at delivering improvements in water quality: lessons for experiments at the landscape scale.PeerJ. 2018; 6e5753Crossref PubMed Scopus (19) Google Scholar,19.Pynegar E.L. et al.What role control in the for 2019; Scopus Google Scholar], or in the of such as natural to the impact evaluation methods V. et al.Quasi-experimental methods enable stronger inferences from observational data in ecology.Basic Appl. Ecol. 2017; 19: 1-10Crossref Scopus (20) Google et in ecological studies with observational Ecol. Evol. 2019; 10: Scopus (15) Google Scholar]. used is to outcomes before and after the intervention, after or to a control that as as with the intervention for the intervention, intervention can be to compare before to control and intervention before after control intervention data are a and et impact in Scopus Google to conduct impact evaluation in methods used by ecologists to conduct impact evaluation with time-series data or time have in the and et impact in Scopus Google that environmental Appl. Scopus Google Scholar]. framework to consider the average change control and in a time-series this has been et impact in Scopus Google Scholar]. methods that a change in an average response can how the time series to the intervention, by that the data and intervention In time series through time of the to an intervention not with an immediate but with a change in which is not by average fields, such as et al.The of a program on in an 2019; 19: PubMed Scopus Google E. et is not an time series 2015; PubMed Scopus Google Scholar], et and before and after the of the an time-series 2016; Full Text Full Text PDF PubMed Scopus Google J. et al.The use of in time series studies of J. 2018; PubMed Scopus Google Scholar], and in as a of time series J. 2018; Scopus Google have and methods to for changes in as as the in the of time-series for Evaluating Google (see in the for a of in other In ecological impact have been et L. et a for environmental impact Ecol. Evol. 2017; Scopus (20) Google L. et evaluation of a marine protected area a 2019; Scopus Google the in as an of that in the but change through time in the et M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google this by and before and and which the nature of impact in a framework M. et for evaluation of environmental Appl. 2019; PubMed Scopus Google Scholar]. This pioneering work is by that average change can be are present in the and that change be in This is with an a population of before and after a which a population first the average of the years before the is to the average of the years one that the has had a average However, if the of the counts is clear that the population but has to that the has had a impact on the population change in is to assess the impact of an intervention on a time series with not that there can be an abrupt following the this immediate change can be In this if had used average they would have that the had a impact on populations, and the intervention have been ecological time-series impact evaluation studies have on one or a of time series R. et an for of Ecol. Evol. 2019; 10: Scopus Google Scholar, et impact of on a breeding with a 2016; Scopus (19) Google Scholar, et the effects of on the on of and 2016; PubMed Scopus Google Scholar, et an to Ecol. 2016; Scopus Google Scholar, D. et al.Experimental community Biol. 2017; PubMed Scopus Google Scholar, J. et the effectiveness of a protected area for Commun. PubMed Scopus (2) Google Scholar], average change has not been a as time series be for of However, of time series (e.g., counts of multiple bird at multiple et the impact of protected areas on population a 2019; where series of time series are increasingly available from long-term and M. et time series in Ecol. Evol. 2018; Full Text Full Text PDF PubMed Scopus Google Scholar, et to time series of all available to and and Ecol. Conserv. 2016; Scopus Google Scholar, et data to population Conserv. 2018; Scopus Google Scholar, et counts to population Biol. 2017; PubMed Scopus Google Scholar, et studies contribute to ecology and 2017; Scopus (109) Google Scholar]. framework for is to avoid in datasets, and to of in (see in the which all potential that the framework Drawing from other disciplines and pioneering work in ecology M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google Scholar], we present a framework for impact evaluation with ecological time-series data. framework is with time series that through if there is of the here for can be more effects methods exist J. V. of and Scholar, the impact of conservation on land the role of and One. 2015; PubMed Scopus Google Scholar, P.J. data and as for in the evaluation of 2017; there are to this interventions are through time with Time a framework is used through this for but the are to or and methods are all to the not for the intervention, the of the time series would not have while that the the control and intervention time series is a of the intervention, and that other exist the control and intervention by and of the of the are in other et in ecological studies with observational Ecol. Evol. 2019; 10: Scopus (15) Google et in ecology of biodiversity Appl. Ecol. 2019; 56: Scopus Google Scholar], but is the if data et in ecology of biodiversity Appl. Ecol. 2019; 56: Scopus Google Scholar]. that if a be for the that to the is (see in the is to consider changes in et in ecological studies with observational Ecol. Evol. 2019; 10: Scopus (15) Google Scholar], the here is on and is to as a control in or and or are used to the or time can be used to a control time series that is as as to the intervention time on a of variables methods for causal a and a Sci. 2010; PubMed Scopus Google J. et for conservation Biol. PubMed Scopus Google Scholar]. the control and intervention time series in the time framework to causal in time-series Eval. Pract. 2018; PubMed Scopus Google this is often to as the if in of before and after a in one the is if the at at the the years methods are available to the methods for causal a and a Sci. 2010; PubMed Scopus Google J. et for conservation Biol. PubMed Scopus Google D. et for causal 42: Scopus Google R. not be used for 2019; 27: Scopus Google Scholar], of the J. et for conservation Biol. PubMed Scopus Google Scholar]. M. et for evaluation of environmental Appl. 2019; PubMed Scopus Google to the of control and intervention before and after an intervention, which can be for the nature of especially are Time series can to an intervention by an abrupt change the intervention is a change or change can be in or immediate terms to ecology by M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google to and a Full can be in in the of can be in in the for and compare the change in average with of the time series be population forest is by a which is and the of is the average of the time series is a this average change is intervention and time series this is often the M. et for evaluation of environmental Appl. 2019; PubMed Scopus Google et of data from Biol. Scopus Google Scholar]. is which is for the time series and for the intervention time and how the intervention the change from before to that the average from before to after is more in the intervention time and and immediate we must time in the In to compare immediate change the time before intervention, to the first time after intervention, the to the is for time to be with the first time after intervention to M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google Scholar], D. et for causal 42: Scopus Google time-series for and J. 2015; Google and for an a the of the time series is by Time and the the change in values from before to after at this is the immediate and Time the change from before to the after the intervention is more before the intervention that be for a an intervention is and the immediate change and and Time the the in or control and intervention time for that the intervention time series has had a more change in the control time the of a on of the use in a population of or from a protected and as a worked to how to from time series and the of the of is a that is and is as but has been a for conservation in the time-series for and J. 2015; Google Scholar]. data to determine how a population of by a protected in to a but through et the impact of protected areas on population a 2019; for (see in a with a et the impact of protected areas on population a 2019; Scholar], and a by with and the (see there change at with had a to the To the impact of the protected we in the which immediate the which and the which if the is (see in the for of how to all not the immediate change the change that the change from before to after more in the protected we had an average change to this data in we would have impact of the protected area the of the and immediate change with time series that To compare the change in average with of the time series be population forest is by a which is and the of is the average of the time series is In a this average change is intervention and time series this is often the M. et for evaluation of environmental Appl. 2019; PubMed Scopus Google et of data from Biol. Scopus Google Scholar]. is which is for the time series and for the intervention time and how the intervention the change from before to that the average from before to after is more in the intervention time series. and To and immediate we must time in the In to compare immediate change the time before intervention, to the first time after intervention, the to the is for time to be with the first time after intervention to M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google Scholar], D. et for causal 42: Scopus Google time-series for and J. 2015; Google and for an a the of the time series is by Time and the the change in values from before to after at this is the immediate and Time the change from before to the after the intervention is more before the intervention that be for a an intervention is and the immediate change and and Time the the in or control and intervention time for that the intervention time series has had a more change in the control time series. use in a population of or from a protected and as a worked to how to from time series and the of the of is a that is and is as but has been a for conservation in the time-series for and J. 2015; Google Scholar]. data to determine how a population of by a protected in to a but through et the impact of protected areas on population a 2019; for (see in a with a et the impact of protected areas on population a 2019; Scholar], and a by with and the (see there change at with had a to the To the impact of the protected we in the which immediate the which and the which if the is (see in the for of how to all not the immediate change the change that the change from before to after more in the protected we had an average change to this data in we would have impact of the protected area the of the and immediate change with time series that to use will on the response and the of data there is that the time series through average change intervention and control is the time series through time and immediate change is after the intervention, change be used that immediate average change will a of response of the but not the in the the time series through time and an immediate change is following the intervention, and immediate change must be as average change will be in not be what response is in which be to assess all of and them in the of of the to which is that how much of the change is in control intervention time can be M. et for evaluation of environmental Appl. 2019; PubMed Scopus Google Scholar]. of deforestation are often not to consistent through and to the impact of a policy to reduce deforestation average deforestation before and after the policy be a population of that has been to is to to a by a in change is but average change or change would be with data as as the is oil spill a population of would immediate for immediate change would the impact of the spill However, a population to the of a with an immediate as more are to to the as as a in through breeding In this and immediate change be an of In datasets, can be to the of impact time especially if they do not all in the the of such and from with time series (e.g., counts of must be to the data and the impact of an the is to all data in one that random or to for (e.g., and M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google for an However, this on that there are in the as be by the and that time series will to the intervention in a as random are a but they can be time to with a if are not is to of time series and if there are not of in the the immediate and from can be on a as in In this we can that in the immediate change and change there are of that there has been a impact from this there are time series with in the the more as time series can have a (e.g., counts in the before in the after would be a while the time series that do not have all will have is to the on of the different outcomes and how they be a protected area a impact as immediate or change from a or where a population has to the population time series be as a or (see in the and the of time series in to assess with time series (e.g., counts of must be to the data and the impact of an the is to all data in one that random or to for (e.g., and M. et al.Changes in forest bird abundance, community structure and composition following a hurricane in Sweden.Ecography. 2019; 42: 1862-1873Crossref Scopus (2) Google for an However, this on that there are in the as be by the and that time series will to the intervention in a as random are a but they can be time to with a if are not is to of time series and if there are not of in the the immediate and from can be on a as in In this we can that in the immediate change and change there are of that there has been a impact from this there are time series with in the the more as time series can have a (e.g., counts in the before in the after would be a while the time series that do not have all will have is to the on of the different outcomes and how they be a protected area a impact as immediate or change from a or where a population has to the population time series be as a or (see in the and the of time series in to assess are of observational data that can time-series impact especially time series have not can to (see in the are and how to them Time series in ecology can have more all but we use the for such in the years before or after an if a a population to the and a of the would counts in the years before the but counts a all habitat of a in the after such is as change will an population as at a where a population has but where the change has from to and a analyses with a of time series this is to or by to an change However, for that are in a to be for on how to do In there be the of a time an intervention and impacts a there be a of a of years a conservation policy and management In the the in can be from by the years, or by et of time series studies in use 27: PubMed Scopus Google et the effects of a on a and Conserv. Scopus Google Scholar]. the time of the intervention can be to would be to this is not in which methods are available to the before and after et an for for change in Scopus Google Scholar, et and of changes in Scopus Google Scholar, for the in a J. 2017; Scopus Google Scholar, to in time series Eval. Pract. 2016; Scopus (19) Google Scholar], this can be to at and there is more work to be in this the has been through of the or through can the and years of the to the is to there in impact at all from the intervention, as in be that the has not here a in the time series from before to but the To we one with the of and and the other the but with the of can be (e.g., through or and if the the the not impact from the conservation a robust understanding of how interventions and environmental affect Time series data in ecology for causal but is to avoid erroneous especially with and to would the of impact (see as the of available time series methods are by L. et a for environmental impact Ecol. Evol. 2017; Scopus (20) Google Scholar], but there is work to be the framework presented, ecologists and can avoid the effectiveness of conservation and the impact of environmental the for effective and conservation there a to conduct impact evaluation are not the we and that are used by disciplines for impact evaluation with or that can be achieved through methods are to with there is as framework for robust conclusions from there to a change This of but a would be a that can a and a of or can analyses that the be there is can the used in impact evaluation be fields, to of have of there a to conduct impact evaluation are not the we and that are used by disciplines for impact evaluation with or that can be achieved through methods are to with there is as framework for robust conclusions from data. there to a change This of but a would be a that can a and a of or can analyses that the be there is can the used in impact evaluation be fields, to of have of and for that the by the of and by the and the of by program and by a for the of is by and by the are by and through the of and the with a of that the by values from before to after the a of that the by the change from before to after control and intervention groups. a used for average by the of the control from the of the intervention methods that time-series with multiple average the of that an is to not a of that the by values control and intervention groups. used in disciplines to to such as that and immediate what would have occurred in the of an time-series data of through counts from of or deforestation for in a used in to to the average change in a but is used to to a change an for data where there is not an of in the data. Time is in the as a to control for how an intervention has outcomes of outcomes in ecology population or of habitat used in disciplines to to such as which consider and immediate an that a be or for the of a protected an oil or a change in or immediate from before to after in and the in change from before to after control and intervention time series in a of that multiple and in the after methods a or in the before a range of used to the causal impact of an intervention a series of at through counts of in a of deforestation in a and of a bird

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.