The first area of vulnerability research deals with natural hazards, such as earthquakes, cyclones, or floods. There is nothing to be done about the fact that some places are naturally in harm's way – San Francisco, Tokyo, Istanbul, and Islamabad are all more likely to suffer an earthquake than New York, London, or Beijing – but there are things to be done to lessen the negative consequences of the harm. A city can be less vulnerable to an earthquake if it implements a combination of technological (e.g., different building materials), organizational (e.g., enforcement of building codes), and other societal (e.g., public education programs) solutions that minimize the loss of life and property associated with shaking ground. Assessing and addressing vulnerability to natural hazards involves listing the many ways in which people will suffer when a particular hazard hits, and identifying and implementing measures to minimize each of these consequences [10,11].

The second area of vulnerability research lies within the famine relief community. Famine relief workers are concerned less with a particular event as with a particular outcome: the lack of access of a large number of people to adequate supplies of food. Early and important work showed that the most obvious causes of famine (e.g., drought, pestilence, and war) are not the most important [50]. Entitlement theory suggests that when a combination of social, political, and economic factors is present, even the most minor external shock can lead to mass starvation [55]. Addressing and assessing vulnerability to famine in this case involves listing the many possible events that can trigger crisis, and identifying and implementing measures to break this causal chain. Such assessments have often led to the creation of famine early warning systems, which currently exist in many developing countries, and which monitor environmental and social indicators that often predict a growing food insecurity crisis.

The third type of vulnerability research lies within the climate change impacts assessment community [14]. As people first recognized the existence of anthropogenic climate change, they began to consider whether it would be worthwhile addressing it, the natural question was what the consequences of climate change actually would be, and thus how much action to prevent it would be justified and feasible. This necessitated the linking of climate models, which make projections decades and centuries into the future, with socio-economic and human settlement scenarios, analyzing what the natural forms of harm would be, and who would be in harm's way [15]. Integrated assessment modeling of climate change impacts provided information to guide decisions about how much to abate greenhouse gas emissions, based on cost estimates of the damages avoided [42]. While impact assessment often required assumptions about the future capacity of people to adapt, its purpose was not explicitly to guide those choices [36]. By the late 1990s, however, the concept of adaptation rose on the climate change agenda, as people recognized that some amount of climate change, perhaps a large amount, was inevitable [35]. The focus of impact assessment shifted from the effort of justifying a global policy response to reduce the causes of climate change, to one of minimizing the negative consequences given particular mitigation, and hence climate, scenarios [53]. Assessing and addressing climate-change vulnerability means listing the many possible natural events – chronic and acute – that climate change may create in a given place, listing the many possible ways in which people in this place may suffer as a consequence of these events, and identifying and implementing measures to break the connection between the list of events and the list of forms of harm [20,23,60]. Climate-change vulnerability assessment covers a wide spectrum, in terms of the range of impacts and potential adaptations considered. In its narrowest form, it examines a single decision or set of decisions to be made today, such as changes to long-lasting infrastructure or settlement patterns, and sees how sensitive that decision is to projected impacts of climate change [19,26]. In its broad form, it allows for comparison of the overall vulnerability of different communities or economic sectors to the aggregate impacts of climate change, and the identification of options to improve future adaptive capacity [25,53].

Fig. 1 illustrates the variables considered in each of these areas of vulnerability assessment. In the natural hazards model, there is a single triggering event, a number of control variables, and a number of negative outcomes. To some extent, there is a correspondence between each control variable (which may be a set of factors) and each negative consequence, such as building design influencing the likelihood of collapse when an earthquake hits. In the famine model, there are a number of triggering events, a number of control variables, and a single negative outcome. Again, there is some correspondence between the two groups, such as drought combined with the lack of irrigation. Of course, since the triggering events may happen in isolation or in combination, there is not an exact correspondence. A war, for example, might make the early warning system ineffective, unless some other control variable, such as the presence of strong international organizations, is present.

In the climate change model, there are typically a number of triggering events, a number of control variables, and a number of consequences, many of them negative. It is rare that one can point to a single impact interacting with a single control variable to produce a single negative outcome, since it is to be expected that many or all of the triggering events will happen simultaneously and interactively. In its narrow form – examining a single set of decisions to be made today – climate-change vulnerability assessment restricts the number of variables considered, and hence interactions, to a manageable amount. In the broad form – considering the net vulnerability of a community or economic sector – the potential set of interactions mushrooms. For lack of adequate theory about these interactions, many researchers are forced to consider only linear connections and one-to-one correspondences between variables, qualifying their findings accordingly [1,33,41,51].

Vulnerability assessment ought to help decision makers to minimize the magnitude or likelihood of the negative consequences, given the occurrence of one or more of the triggering events, by influencing one or more of the control variables. But before doing so, they need to be confident that the consequences of the intervention will be as the model suggests: that the change in the control variable will reduce, rather than exacerbate, the harm. To gain such confidence, it is important for modelers to validate their work with past data.

There have been a lot of earthquakes, fires, floods, and cyclones. In each case, there has been a distinct set of other factors present, the control variables, which may have contributed to the damage, as well as different sets of damages. Importantly, these events are typically separated by a large amount of time and space; when such a separation does not exist, such as between an earthquake and its aftershocks, one typically treats them as a single event. Even where one place has experienced several hazards, the duration of the hazard, combined with the time to rebuild from it, is typically short in comparison to the interval between them: a city may experience one earthquake, it rebuilds, and then years later it experiences another. Because of this separation, each of these events can be treated as an independent observation. Famines are a little more difficult to separate in time and space, given that they cross borders easily, and take years to wax and wane. But it is still possible to parse out numerous distinct events, and examine the relationship between causes and effects. With many independent observations, one can begin to test models rigorously, and make meaningful statements about the relationships between causes and effects.

In human history there have been many examples of quite extreme local and regional climate change leading to the prospering or collapse of societies [6]. There have also been several major shifts on global weather patterns, such as the Little Ice Age of the 1700s and 1800s, or the Medieval Warm Event. However, there have been no major changes in global climate in which any of the hypothesized control variables, such as income equality or access to credit, have been in a state anything like they are now or will be in the future. To be sure, the sub-components of the climate-change vulnerability model – models of climate, socio-economic development, and adaptation – can be validated. However, there is no way to determine whether the various factors, the combination of triggering events and the multiple control variables, will interact in the way that is forecasted.

For hazards and famine researchers, the presence of data has made it possible to make statements that go beyond the obvious, and indeed to present the kind of counter-intuitive results that actually allow science to contribute value to policy. It is important to know, for example, that most of the damages to human life and property from cyclones occur not as a result of wind, but of flooding [17]. It is important to know that most people under-insure against natural hazards, until immediately after an event, when they over-insure, unless government policies force them to do otherwise [27]. It is important to know that good governance does more to alleviate famine than does adequate rainfall [55]. Each of these lessons suggest that some forms of public investment are better than others in reducing future vulnerability, and hence can contribute to the policy process by justifying costly options. Statements about social system/natural system interactions that produce more or less vulnerability to climate change are much harder to make.

Policy makers concerned about natural hazards worry about the event that could occur at any time: tomorrow, next year, and within the coming decades. Assessing the vulnerability to those natural hazards means examining the system, as it exists today, and suggesting changes to that system in order to make it less prone to damage. To be sure, those changes may take years, or even decades to implement. Often the changes involve altering the type of building construction, or the location of settlements; all new construction will take the hazard into account, while existing infrastructure simply ages, hopefully gracefully. But the starting point for the changes is the system's current design. Planning for food security is not too different, although it does often require some consideration of the trends that may be making people in a given place more vulnerable to famine, such as population growth, or the loss of arable land. The analysis begins with the recognition that vulnerability exists today, vulnerability that will not disappear on its own and may indeed be growing, and with the desire to make active interventions to reduce the vulnerability.

Climate-change vulnerability assessment, at least in its broad form of considering the aggregate vulnerability of a community or economic sector, is different. The starting point for analysis is the recognition that triggering events may emerge, grow more frequent, or become greater in magnitude over time as a result of changes in greenhouse gas concentrations. These may include changes in average temperature, in average precipitation, in average sea level, or in the frequency and magnitude of extreme events [34,44]. Indeed, often scientists are able to say with high confidence that a low probability event will become more likely [35]. It is no surprise that the ranges of uncertainty within impact assessments are, or ought to be, very wide [16,39,65]. While the use of multiple scenarios can often offer insight into what the future may look like, and allow for the development of robust strategies [30,59], it does not overcome the core challenge of unpredictability that the time dimension offers.

Vulnerability assessment is in many ways similar to environmental impact assessment, in that it requires projections of future environmental changes [9,46], but in terms of the time dimension of action, there is a crucial difference. Impact assessment guides those mitigation decisions that will potentially create or avoid a negative environmental impact, and like the steps taken in response to famine and hazards assessment, can be taken today, before the problem develops [53]. In Fig. 2, the square representing present-day mitigation is the decision to be informed by impact assessment, in that the decision-maker can choose between at least two pathways. At some future time, the environmental change will manifest itself, and there is no doubt some uncertainty about how society will suffer harm: under one decision pathway, society could suffer harm ranging from a to b; under another decision pathway, society could suffer harm ranging from c to d. The point is to decide which initial pathway to select. Vulnerability assessment, by contrast, guides those decisions that will respond to the threat of an impact, whatever the causes of that impact may be. In Fig. 2, the vulnerability assessment assumes one or more scenarios of mitigation and other important societal changes, which bifurcate at the left-hand circle. For each scenario, then, the assessment examines the range of potential adaptation decisions, and the consequences of each. For one scenario, the consequences could range from w to x, and for the other scenario, from y to z.

A commonly expressed goal for climate-change vulnerability assessment is to examine the adaptation options of people to climate change fifty or one hundred years from now, when climate change will be more severe, and suggest steps that can be taken now to enhance the range of options in the future. Features of the system that do not respond quickly to climate, such as the population or level of economic development, may partly determine the future range. But many of the important variables, such as the location of new infrastructure, are the elements of the system that are the most adaptive to change, and the least easy to predict. For example, a coastal community may adapt to gradual sea level rise by locating new development further inland, by constructing dikes and seawalls, or some combination of the two. Yet the decision that starts the adaptation trend in one direction may be as small as a single factory owner deciding to locate next to the sea or on higher ground, which in turn will influence where the employees choose to live, which in turn will influence new road construction. Adaptations cascade upon themselves. While vulnerability assessment might suggest no-regrets options, it becomes very difficult to say with any confidence whether a costly change made today is justified in terms of improving adaptive capacity in the future.

When it tries to inform present day policy makers how to improve future adaptive capacity, vulnerability assessment enters the territory of needing to predict the future behavior of a complex adaptive system [29,31]. Under conditions of increasing returns to scale, such as where the desirability of a product increases along with its market share, the choices of society to adopt particular technologies are entirely path dependent, and virtually impossible to predict [2,62]. Agent-based modeling can be used to demonstrate that particular systems are complex adaptive, and to understand why a particular complex adaptive system evolved as it did, and thereby explain surprises that have been observed [3,5,12,28]. The technique, by looking at the structure of agent networks, can also begin to explain the rate at which a system evolves, and hence begins to look quite different [4]. But neither agent based modeling, network analysis, nor any other method of system modeling can predict, at any quantified level of uncertainty, how the adaptive elements of a system will evolve once it does begin to enter the range of the unfamiliar [7].

While narrowly framed vulnerability assessments can improve the quality of some long-range investment decisions, the uncertainties inherent in the broader type of climate-change vulnerability assessment may preclude their ability to suggest more than no-regrets options. At the same time, there are likely other collateral benefits to these assessments, such as advancing the state of the art in integrated modeling, and in promoting stakeholder dialogue. Perhaps these benefits justify conducting vulnerability assessments, even when the intended benefits are limited or non-existent. We suggest not, for two reasons.

First, there may well be some other type of assessment that is more useful, which is being crowded out by the vulnerability assessment. Such an effect was documented in the effort to assess the costs and benefits of reduced sulfur dioxide emissions in Europe during the 1980s [47,48]. During the early years of the Long Range Transboundary Air Pollution (LRTAP) regime, there was a great interest in using cost-benefit analysis to set targets for sulfur dioxide emissions reductions. The first major study was completed in 1981, and focused on the benefits to building materials, crops, lake fishing, and human health [45]. Because of the uncertainties, primarily in valuing health benefits, the estimated benefits of the favored reduction target ranged from $1.83 billion and $16.63. Policy-makers found the results unhelpful, but decided to fund additional work at benefits estimation. Over the next nine years, numerous reports attempted to estimate benefits, and yet none was able to reduce the uncertainty enough to allow policy makers to use the results [24,54,61]. Eventually, due to political pressure, LRTAP policy-makers gave up trying to assess the benefits of acid rain reduction, and decided to focus on what was possible, namely assessing a least cost pathway for achieving ecosystem protection. Within four years, modeling teams in the United Kingdom, Sweden, and Austria had developed integrated assessment models that could achieve this. One of them, the RAINS model, contributed significantly to the development and adoption of a protocol to reduce sulfur emissions unevenly throughout the continent, a major policy breakthrough [47]. Interestingly, by the late 1990s the effort to assess benefits was renewed, born out of the success at assessing acid rain impacts. But when assessment teams turned to what they could achieve, and generated credible results, they had a much greater impact on the policy process.

Second, governments rely on scientific results to legitimate their policy decisions [13]. When the actual content of the scientific information is small, this enhancement of legitimacy may be inappropriate. For example, it seems increasingly clear that millions of people in developing countries are today worse off, and indeed hundreds of thousands of people may have died, because of international finance and development policies, which in turn were based on questionable economic theory. Yet even as these policies started to be produce negative results, governments continued to engage in them, based on a belief in the underlying theory; the belief in the theory's rightness persisted in blinding analysts to contradictory evidence [58]. Other researchers have shown that such cases can lead to a backlash against using scientific information to inform decision-making in that particular context, often as a result of the scientific results being overstated by one or more policy actors [66].

The Advanced Terrestrial Ecosystem Analysis and Modeling (ATEAM) project was a 3.9 million € European Union funded research and assessment project, led by the Potsdam Institute for Climate Impact Research (PIK) in coordination with seventeen partner institutions, the goal of which was to assess the vulnerability of human systems in relation to changing provision of terrestrial ecosystem services in Europe over the coming hundred years. The ATEAM project, conducted from 2001 to 2004, was ambitious in that it combined stakeholder analysis with state-of-the-art ecosystem modeling and innovative techniques of analyzing macro-economic indicators.

First, the researchers would meet with stakeholders, representatives from various environmentally sensitive sectors of the economy across Europe, to determine what ecosystem services they considered to be most valuable, such as growth rates of particular tree species, suitability of land for particular crop varieties, snow reliability, or quantifiable storage of carbon. Their incentives to participate ranged from influencing the research agenda to focus on the changes they viewed most important, networking with their colleagues, to obtaining valuable information they might not otherwise learn.

Second, the various modeling teams projected how these ecosystem services might change over time, as a result of climate change, changes in nitrogen deposition, land use change, and changes in other natural and socio-economic drivers. To help to standardize their projections, all of the modeling teams incorporated European downscaled projections from four of the Intergovernmental Panel on Climate Change (IPCC) Special Report on Emissions Scenarios (SRES) projections. These were meant to capture a likely range of plausible future global development paths, in terms of changes in overall economic growth, population, international trade, and prioritization of different levels of environmentally friendly industrial patterns. None of the SRES scenarios included specific measures to address climate change through mitigation, the intentional reduction of greenhouse gas emissions, although they do differ in the rate of emissions due to other factors. To further standardize their results, all of the ATEAM modeling teams applied multiple scenarios based on the downscaled projections from four leading general circulation models.

Third, the results of the different ecosystem modeling teams were combined into a single mapping tool, a software package that allows a computer user to compare the exposure and sensitivity of particular ecosystem services, and the vulnerability of specific sectors across Europe at different times. Development of the vulnerability maps started with a consideration of what the global change impacts could be over the coming century, and how this would affect the maximum possible production of each specified ecosystem service. The maps then included an estimate of what the actual utilization, or demand for, each ecosystem service might be over that same time frame. Sensitivity, then, depended in part on the likelihood of the actual utilization exceeding the maximum possible provision of a particular ecosystem service. Finally, the maps included a consideration of the adaptive capacity, derived from a broad slate of economic and social indicators, such as per capita income, access to information, and age structure of the society. The researchers overlaid national scale adaptive capacity projections onto the potential impact projections. The results were then mapped, in terms of a range of colors, across Europe in 10′×10′ grid cells. Fig. 3 shows one example of such a map.

Fourth, the project team engaged in additional stakeholder dialogue: a total of three general workshops, three sectoral workshops, and multiple informal interactions. The last general workshop took place near the end of the project and after the development of the prototype mapping tool; the ATEAM project partners met with the group of stakeholders for two days, to present their findings, and to gain feedback on the usefulness of their information package in order to improve future vulnerability assessments. The results of this stakeholder meeting are what we now consider.

On the 3rd and 4th of May, 2004, thirteen stakeholders met with fifteen members of the ATEAM project in the Maxx Steigenberger Hotel in Potsdam, Germany, first to review the results and second to provide feedback on them. Of those attending, three represented the forestry sector (in Germany, Finland, and Spain, respectively), two represented the mountain tourism sector, one represented the agricultural sector in Europe, one represented the bio-energy sector in Sweden, and the remainder represented government agencies (e.g., the German Federal Environmental Agency), non-governmental organizations (e.g., the Climate Action Network), or the media concerned with climate change in general. While we do not attempt to summarize the entire workshop, we do describe three trends that we observed in the meeting.

First, the stakeholders participated actively and intelligently in the discussion. During the first half of the workshop, the presentation of the modeling methods and the vulnerability maps, every stakeholder asked at least one question, and at times the presentation evolved into protracted discussions of the finer points. On presentation of the model of carbon storage, for example, the Swedish bio-energy representative led discussion in the direction of considering the most effective ways for converting plant growth into carbon sinks, along with the difficulties in carbon stock accounting. The discussion was on two main points: how to model the carbon cycle taking into consideration temporary stocks, and how to combine alternative ways to reach carbon emission targets, such as the substitution of materials which need high carbon emissions to manufacture them as opposed to wood. On this and many other points, the stakeholders took an active role in discussing finer points with the ATEAM partners.

Second, the stakeholders all agreed that the impact assessment component of the vulnerability maps, in terms of examining changes in the potential production of ecosystem services, was useful and interesting. The representative from the Spanish forestry industry, for example, noted that several species of pine currently being grown are already showing signs of stress due to insufficient water, and that the impact assessment part of the modeling revealed that this stress would increase, not decrease, over the next hundred years. To her this suggested that the time has already come to stop planting these species, although she recognized that to achieve change in forest composition would take more than scientific information. The representative from the German forestry industry noted a similar conclusion, and stated that these conclusions could be most useful in arguing for changes in German forestry regulation, which currently dictates which species of forest owners may plant. To him the impact suggested a means of arguing, to the government, for greater forest owner flexibility to plant a wider variety of species. The representative from the German environmental agency noted that some areas of her country already seem ill suited to their traditional crops, and the impact assessments show this trend increasing in the future. Again, this could form the basis for a change in regulatory or crop subsidy policy.

Third, the stakeholders agreed that the maps of vulnerability, including information such as adaptive capacity and projected land use, were less helpful. The representative of the European farmers said that while he found the impact maps useful, he found less value in the inclusion of adaptive capacity over the next hundred years, since this did not and could not guide his decision-making. In general, participants said that the vulnerability maps failed to capture the multi-dimensionality of their sectors. The representative of the Finnish forestry sector, for example, pointed out that the people who own most of the commercial forests in Finland actually gain less than 20% of their income from the forests; they are people who work full time jobs, or are engaged in agriculture, and who use the harvesting of timber from their land merely to supplement their income. Their and their children's vulnerability to global change lies not in the sensitivity of their forests to anticipated changes in temperature and precipitation – although such changes are helpful to know – but rather to the overall development of the Finnish economy, including the other sectors of the economy in which they are engaged. To speak of the anticipated impacts of global changes on their land is important, but translating this into useful information about their vulnerability requires making inappropriate assumptions about their lives and the sources of their livelihoods. The representative of the German forestry sector suggested that the vulnerability maps could be useful to argue for changes in taxation or subsidy policies to benefit his sector, but even then it would be more helpful to offer data about current difficulties, rather than difficulties projected fifty or one hundred years in the future.

As anecdotal evidence, the results from the ATEAM stakeholder workshop are consistent with the arguments of this paper. While policy-makers in the European Union may have been eager to fund a vulnerability assessment, and while ATEAM appears to have met their expectations, the actual usefulness of the assessment in directly influencing decision-makers was far more limited.

The impact assessment part of the ATEAM modeling work was found to be useful, and represented a major step forward in modeling skill and the ability to predict future global change impacts. It is important not to understate the difficulty of this work. Assessing impacts within ATEAM required downscaling numerous climate models, according to standardized scenarios, developing consistent land use scenarios, and coupling these results with sector specific ecosystem models. The impact projections generated within ATEAM were state of the art, both in their fine scale and their sector specificity. There is still a great deal of progress to be made in projecting impacts of global change that people care about, at a spatial and temporal scale that is useful for making different decisions. Likewise the social-science projections within ATEAM were innovative, involving state of the art means of tying together different national indicators in a way that could represent adaptive capacity. However, the results of this modeling appear, at least so far, to be of more academic than practical interest.

The stakeholders themselves recognized the limitations of any attempt to translate projections of impacts into statements about sectoral vulnerability. First, making the system being considered more complex made the results more difficult to communicate and interpret, or required making unreasonable assumptions. Second, there was less confidence in the results generated from these more complex models. Third, the inclusion of adaptive capacity simply showed that making necessary changes would be more difficult – it did not influence in any way what those changes actually should be. Overall stakeholders repeatedly asked for a more transparent and targeted communication of the results and the methods used to obtain them. Only then they would be able to really decide whether the scientific results were of relevance to their activities.

Obviously, this workshop represents a small sample of stakeholders, responding to a single effort to assess vulnerability. They show that in this case, what has been perhaps the most complex effort yet to assess regional global change vulnerability, the most useful contribution of scientists lay in the impact assessment embedded in a process of stakeholder dialogue. Stakeholders were eager to talk with scientists about adaptive capacity and vulnerability, but they could not foresee how they could use the information to make decisions differently. We have searched for and failed to find any examples of other vulnerability assessments achieving more.