Risk Assessment Development of a risk assessment framework to predict invasive species establishment for multiple taxonomic groups and vectors of introduction

A thorough assessment of aquatic nonindigenous species’ risk facilitates successful monitoring and prevention activities. However, speciesand vector-specific information is often limited and difficult to synthesize across a single risk framework. To address this need, we developed an assessment framework capable of estimating the potential for introduction, establishment, and impact by aquatic nonindigenous species from diverse spatial origins and taxonomic classification, in novel environments. Our model builds on previous approaches, while taking on a new perspective for evaluation across species, vectors and stages to overcome the limitations imposed by single species and single vector assessments. We applied this globally-relevant framework to the Laurentian Great Lakes to determine its ability to evaluate risk across multiple taxa and vectors. This case study included 67 aquatic species, identified as “watchlist species” in NOAA’s Great Lakes Aquatic Nonindigenous Species Information System (GLANSIS). Vectors included shipping, hitchhiking/fouling, unauthorized intentional release, escape from recreational or commercial culture, and natural dispersal. We identified potential invaders from every continent but Africa and Antarctica. Of the 67 species, more than a fifth (21%) had a high potential for introduction and greater than 60% had a moderate potential for introduction. Shipping (72%) was the most common potential vector of introduction, followed by unauthorized intentional release (25%), hitchhiking/fouling (21%), dispersal (19%), stocking/planting/escape from recreational culture (13%), and escape from commercial culture. The ability to assess a variety of aquatic nonindigenous species from an array of potential vectors using a consistent methodology is essential for comparing likelihoods of introduction, establishment, and impact. The straightforward design of this framework will allow its application and modification according to policy priorities by natural resource managers. The ability to use a variety of information sources facilitates completion of assessments despite the paucity of data that often plagues aquatic nonindigenous species management.


Introduction
Nonindigenous species have the potential for both ecological and socio-economic impacts, and can be very costly or impossible to eradicate after establishment (Hobbs andHumphries 1995, Simberloff 2003).To prioritize management efforts, risk assessments can be used to evaluate vectors of introduction, species life history traits, habitat suitability, historical patterns of invasion, and impacts realized in other invaded regions (Keller 2009;Kulhanek et al. 2011;Gordon et al. 2012).
The use of risk assessment to address environmental threats began with a focus on environmental contaminants in the 1980s (e.g., Hayes 1997;Landis et al. 2013).Limited resources led to ranking management priorities according to risk levels (Burgman et al. 1999).By the early 2000s, risk assessment was implemented to aid decision-making in the fields of biological invasion and conservation biology, particularly to ensure regional biosecurity (e.g., Andersen et al. 2004).
Biological invasion risk assessment continues to be a young field, with a variety of approaches, scope, content, and required elements (reviewed in Dahlstrom et al. 2011;Verbrugge et al. 2012).There remains a paucity of consistency, consensus, and uniformity among approaches, particularly in addressing the following: 1) whether to include multiple taxa and vectors (versus a single taxon or vector); 2) where to set the assessment endpoint (i.e., introduction, establishment, or impact); 3) what impact types to consider; 4) whether to use a semi-quantitative, quantitative, or qualitative approach; and 5) how to deal with data gaps and other uncertainty.

Multiple taxa and vectors
Researchers have taken a variety of approaches when considering what to include in risk assessments.For example, they may consider a single species' risk to a given area (Therriault and Herborg 2008), a number of species within a particular taxonomic group (e.g., plants and fishes in Daehler and Carino 1999;Kolar and Lodge 2002;Copp et al. 2005); or a number of different taxa within a particular vector (e.g., Gollasch and Leppäkoski 2007;Leung and Dudgeon 2008).Species-specific assessments often compare species' life history and physiological traits to the climate and other environmental conditions of the recipient location (e.g., Kolar and Lodge 2002;Clarke et al. 2004;Gollasch 2006;Bomford et al. 2010).Such assessments can include detailed information about species distributions, reproductive characteristics, physiological constraints, and environmental preferences (UNEP/MAP-RAC/SPA 2008).Vectorspecific assessments often take a broader approach including vector strength to predict introduction potential and climate matching to predict establishment potential.This focus on single-taxon (Mendoza et al. 2009) or single-vector (US Army Corps of Engineers 2014) may lead to assessments that provide an incomplete picture of the full invasion risk.

Assessment endpoint
Biological invasion risk assessments can also have different endpoints, with species' introduction commonly chosen (Andersen et al. 2004); they are less consistent in their treatment of establishment (colonization and spread) and consequence (impact).However, given the estimated number of introductions that do not result in establishment (García-Berthou et al. 2005), an understanding of establishment potential remains important.Impact is necessary to give a full description of risk (which includes both the probability of an event occurring and the severity of the consequences).Determining impact is also a key element to species' management, as knowledge of small versus large effects allows better prioritization of management efforts (Parker et al. 1999).

Impact types
Biological invasion risk assessments often consider only environmental impacts (e.g., Ruiz et al. 1999) and ignore impacts to other core values, such as economic, social, human health, and cultural impacts (reviewed in Verbrugge et al. 2012).However, given that risk assessments often occur in a sociopolitical context, including these additional core values will ensure the consequences to all stakeholders are fully accounted for.

Semi-quantitative, quantitative, or qualitative approach
Invasion risk may be evaluated quantitatively (with numerical probabilities or descriptors), qualitatively (with categorical descriptors), semi-quantitatively (by representing quantitative data with categorical descriptors), or using rule sets or decision trees with arbitrary risk thresholds (in which a single criterion determines the outcome) (Hayes 1997;Keller et al. 2007b).Issues of objectivity and consistency in professional opinions can arise in qualitative assessments (Burgman et al. 1999, but see use of structured expert judgment in Wittmann et al. 2014).As such, quantitative approaches are often favored despite their sensitivity to weighting schemes (e.g., Pheloung et al. 1999) and dependence on complete data sets, which rarely occur (Campbell 2009).

Data gaps
While there are many forms of uncertainty, within the field of biological invasion risk assessment, gaps in knowledge present the greatest challengesparticularly in the understanding of species' impacts.Yet given the vital role of risk assessments in management, decisions must be made despite extensive knowledge gaps.Options include incorporating expert judgment, applying the precautionary approach and assuming an impact, or applying the hindsight approach and assuming no impact.While the devastating effects of nonindigenous species do not support this last approach, it is often applied (Davidson and Hewitt 2013).
We propose an assessment framework for aquatic nonindigenous species (ANS) that addresses several of the limitations discussed above.In particular, we aim to develop a semi-quantitative framework that facilitates comparison of multiple taxa and vectors, considers the full invasion process from introduction to impact, accounts for the breadth of possible impacts, and gauges uncertainty for each assessment.Such a framework will provide information needed to develop comprehensive policies that are not limited to isolated groups of organisms or vectors of introduction.

Methods
Using a semi-quantitative approach, we first developed a comprehensive framework for assessing aquatic species' invasion risk.This framework built upon previous approaches, while taking on a new perspective for evaluation across species and vectors and stages.We chose to structure this framework to consider introduction, establishment, and impact (hereafter, "assessment components") separately as interacting stages in the invasion process.Details of each of these components are described below (also see Appendices S1-S3 in Supplementary material).

Potential for introduction
The introduction assessment criteria and relative levels of introduction likelihood within each vector were chosen based on Kelly (2007), modified from Holeck et al. (2004) and the United States Geological Survey's (USGS) Nonindigenous Aquatic Species (NAS) database (United States Geological Survey 2011).Assessment criteria and relative levels of introduction likelihood within each vector were based upon the results of a literature review and expert opinion.
The potential for introduction assessment took into account a "proximity" proxy for each pathway using a suite of 12 paired questions (2 per vector).The first question in a pair considered potential pathways for introduction, assigning a score from 0 to 100-usually 100 for being in a particular pathway and 0 for not-while the second question evaluated the likelihood of a species to enter the Great Lakes through that pathway, using a multiplicative factor from 0 to 1.If a question could not be answered based on available data, an "unknown" option was available.A score sheet was kept for tallying the results for each species.Overall probability for introduction per vector (High, Moderate, Low) is determined by the adjusted point score for the species in that vector.Thresholds for introduction probability were set such that species in the closest proximity to the Great Lakes (relative to the pathway of introduction) would be evaluated as having High probability, those at intermediate distances would be evaluated as having Moderate probability, and those either not in the pathway or at the furthest distance would be evaluated as having Low probability.Unlikely represents a score of 0, Low a score of 1-39, Moderate a score of 40-79 and High a score of 80-100.

Potential for establishment
The establishment assessment component included variables that aid or detract from a species' establishment success and spread potential, as relevant to the body of water for assessment.In particular, we considered criteria within four broad categories deemed important in invasion biology (e.g., Williamson and Fitter 1996;Kolar and Lodge 2001;Lockwood et al. 2005;Hayes and Barry 2008;Kulhanek et al. 2011): 1) invasive biological/ecological attributes, 2) environmental compatibility, 3) propagule pressure (inoculum size, frequency), and 4) history of invasion and spread.We modified criteria used in the UK Non-Native Organism Risk Assessment scheme (Baker et al. 2007) for Great Lakes region-specific variables.Additional questions were considered resulting from a review of invasion literature for additional empirically-supported factors, e.g., overwintering (Magnuson et al. 1985), fecundity (Drake and Lodge 2006;Keller et al. 2007a), propagule pressure (e.g., Colautti et al. 2006), and climate change (Rahel and Olden 2008).
Overall species' establishment potential was determined by its total point score (up to 9 points for each of 18 questions).Answers to 3 of the 18 questions could lead to an overall percentage reduction in a species' score (absence of species critical in life cycle; prevention of establishment by herbivory, predation or parasitism of enemy present in Great Lakes; and control measures).Such adjustments are warranted when a variable would counter or prevent the species' establishment.Species can score a High establishment potential if at least three-quarters of the questions were scored as 9s or a Moderate establishment potential if more than half of the questions were scored as 6s (or were evenly split with equivalent numbers of 3s and 9s); otherwise the species is ranked as having a Low establishment potential.Low represents a score of 1-50, Moderate a score of 51-99 and High a score of >100.

Potential for impact
For the impact assessment component, we considered not only environmental and socio-economic impacts (including human health), but also potential beneficial effects, often omitted from biological invasion risk assessments.The inclusion of potential benefits recognizes that nonindigenous species may both be intentionally introduced for desired outcomes (e.g., biological control, recreation, economic gain) or accidentally introduced but result in a perceived benefit over time (e.g., aesthetic, ecological) (Schlaepfer et al. 2011).Thus, this approach is intended to allow managers and policy makers to weigh the contributions of nonindigenous species against potential harms.
We modeled this assessment component after an existing framework used to assess the realized consequences of established nonindigenous species in the Great Lakes (Sturtevant et al. 2014).However, instead of considering location-specific impacts, we accounted for impacts species may have had in any nonnative region.This approach has had great predictive power in previous applications (e.g., Ricciardi 2003).Scores for each of the 6 questions (0, 1, or 6) were summed per impact category (36 point maximum) and converted to an overall impact.If ≥1 or ≥2 questions were scored unknown, with low (1) or no (0) total impact sum, respectively, impact was scored Unknown; if 0 or ≤1 questions were scored unknown, with low (1) or no (0) total impact sum, respectively, impact was scored Low; if total impact sum ranged from 2-5, impact was scored Moderate; if total impact sum was ≥5, impact was scored High.

Application to Great Lakes Watchlist species
We tested our Great Lakes Aquatic Nonindigenous Species Risk Assessment (GLANSRA) framework in the Great Lakes region, selecting species based on NOAA's GLANSIS watchlist criteria (http://www.glerl.noaa.gov/res/Programs/glansis/glansis.html).In addition to the previously determined GLANSIS criteria, for the risk assessment we additionally set the criterion that species must meet at least three of the following conditions: 1) a vector currently exists that could move the species into the Great Lakes, 2) the species is likely to tolerate/survive transport (including in resting stages) in the identified vector, 3) the species has a probability of being introduced multiple times or in large numbers, 4) the species is likely to be able to successfully reproduce in the Great Lakes, and 5) the species has been known to invade other areas; or the species was identified in one or more peer-reviewed scientific publications as having high probability for survival, establishment, and/or spread if introduced to the Great Lakes.While we relied principally on current climate conditions, particularly concerning species' ability to overwinter, to determine inclusion in the assessment, we included several species for which predicted increases in water temperature have led to explicit remarks concerning their future invasion probability.
After species selection, we determined if the GLANSRA framework could assess the full range of taxa under variable levels of information availability by completing the introduction, establishment, and impact assessment components for each species.The assessments were completed using an exhaustive literature review that included online species registries, aquatic invasive species databases, major bibliographic databases, peer-reviewed literature, published state and federal agency reports, reliable Internet sources, librarian services, expert consultation, and best professional judgment.
We compared species' scores for introduction, establishment, and impact to determine trends in predicted invasiveness.In particular, we considered taxonomic groups, geographic origins, vectors, establishment, and impacts in the Great Lakes, and areas of limited data availability.

Results
The GLANSRA framework yielded three separate semi-quantitative, question-driven assessment components for a species' potential introduction (6 pairs of questions), establishment (18 questions), and impact (6 questions for each of 3 broad categories).The final structure of each assessment component, based on the considerations described above, was as follows.

Potential for introduction
Vectors in the introduction assessment component included canals and waterways (1: dispersal), trade of live organisms (2: stocking/planting/escape from recreational culture; 3: unauthorized release; 4: escape from commercial culture), activities of recreational and resource users (5: hitchhiking/fouling), and commercial shipping (6: transoceanic shipping).
We also chose to include a "proximity" estimator for each vector using a suite of 12 paired questions (2 per vector; see Supplementary material Appendix S1).The first question in a pair considered potential means for introduction, assigning a score from 0 to 100, with 100 representing the maximum potential for being in a particular vector.The second question evaluated the likelihood of a species to enter the Great Lakes via that vector, using a multiplicative factor from 0 to 1.The product of these two questions was used to determine the final, adjusted quantitative introduction score for each vector.These quantitative scores were then used to assign a categorical If a question could not be answered from available data, a score of "unknown" was entered.The number of unknowns provided an estimate of assessment confidence.Assessment confidence levels were assigned based on the total number of questions that could not be evaluated (see Appendix S1).

Potential for establishment
In the establishment assessment component, contributing variables from a total of 18 questions were broadly grouped into four categories: invasive biological/ecological attributes, environmental compatibility, propagule pressure, and history of invasion and spread (Table 1; Appendix S2).While important to successful establishment and spread, initially proposed questions concerning genetic diversity of potential source population, genetic and phenotypic variation, and likelihood of introduction during time of year appropriate for establishment, were deemed unlikely to be able to answer a priori for most species and thus removed from this assessment component.Overall species' establishment potential was determined by its total point score.Three questions included an adjustment factor that led to an overall reduction in a species' score.Such adjustments are warranted when a variable would counter or prevent the species' establishment.The categorical probability of establishment for each vector (High, Moderate, Low) was determined by the quantitative score.Assessment confidence levels were assigned based on the total number of questions that could not be evaluated (see Appendix S2).

Potential for impact
The impact assessment component was divided into sets of six questions within three potential impact categories: environmental impact, socio-economic impact, and beneficial effect (Table 2; Appendix S3).Scores for each criterion were summed for each species' potential impact category and converted to a categorical impact ranking using the framework's scoring table, accounting for the level of uncertainty as before (i.e.number of unknowns).This system was based on that created for assessing the realized impacts of species already established in the Great Lakes (Sturtevant et al. 2014).

Application to Great Lakes Watchlist species
We applied the GLANSRA framework to the 67 Great Lakes watchlist species (scores for 5 of 67 species are shown in Table 3, as example).More than three-quarters of these species were either fishes or crustaceans, with the remaining species represented by annelids, rotifers, bryozoans, platyhelminths, mollusks, and plants.These species were native to five continents (Asia, Australia, Europe, North and South America), with the majority coming from Europe (69%), followed by Asia.

Introduction potential
Of the 67 species we assessed for the Great Lakes, than a fifth (21%) had a high potential for introduction and greater than 60% had a moderate potential for introduction (Figure 1).The species with a high potential for introduction originated from each of the included geographic regions, including all of the species from South America.Shipping (72%) was the most common potential vector of introduction, followed by unauthorized intentional release (25%), hitchhiking/fouling (21%), dispersal (19%), stocking/planting/escape from recreational culture (13%), and escape from commercial culture (6%).
Fishes were present in all vectors, while plants were present in all except for the shipping vector.Nine fishes (33%) and seven plants (88%) were present in multiple vectors.For instance, the Ide (Leuciscus leuciscus (Linnaeus, 1758)) had a high potential of introduction through both unauthorized intentional release and stocking/planting/escape from recreational culture, water hyacinth (Eichhornia crassipes (Mart.)Solms) was present in all except the shipping vector, and water lettuce (Pistia stratiotes Linnaeus) had a high potential for introduction in four vectors (dispersal, hitchhiking/fouling, unauthorized intentional release, and stocking/planting/escape from recreational culture).The bulk of crustaceans (83%) were assessed as having a high or moderate potential to be introduced via shipping.

Establishment potential
Most of the species we assessed had a moderate (72%) or high (24%) potential for establishment in the Laurentian Great Lakes, with two-thirds of these species originating in Europe (Figure 2).Annelids, mollusks, fishes, and crustaceans had the highest establishment potentials.There were unknown establishment questions for species in every taxonomic group, with environmental impact also having a significant percentage of questions answered "unknown"     ) or no (0) total impact sum, respectively, impact was scored Unknown; if 0 or ≤1 questions were scored unknown, with low (1) or no (0) total impact sum, respectively, impact was scored Low; if total impact sum ranged from 2-5, impact was scored Moderate; if total impact sum was ≥5, impact was scored High.
(Figure 3).The question on size and frequency of inoculation events was the least answerable question (61% unknown) of the establishment assessment component, especially for crustaceans and fishes, followed by fecundity (28% unknown).In contrast, overwintering, climate, and critical species questions in the establishment assessment component were answerable for all assessed taxa.When comparing establishment potential with introduction potential, high introduction species are most likely to have a moderate potential for establishment.The majority of all species fell into the categories of moderate introduction and either moderate (39%) or high (29%) establishment potentials.

Impact potential
Of the species that we could rank for environmental impact (i.e.not "Unknown"), more than half (59%) had a high potential environmental impact (Figure 4).Fishes and plants comprised 75% of the high potential environmental impact species, while fishes and crustaceans comprised 86% of the moderate environmental impact species.Plants were ranked as either moderate or high environmental and socio-economic impact species.Competitive effects and trophic alteration had the most potential for impact.More than a third (39%) of species-including mollusks, crustaceans, platyhelminths, rotifers, and fishes-could not be assessed for overall potential environmental impact due to lack of impact data.Furthermore, we could not assess more than a third of all species in each environmental impact category, with the exception of Question E2 (competition), due to the lack of impact data.Across all levels of potential impact, the majority of species with potential environmental impact to the Laurentian Great Lakes, including unknowns, originated from Europe.Species originating from South America were assessed to have high potential environmental impact, and cosmopolitan species had either a low or unknown potential environmental impact.Most species with a high likelihood of introduction also had a high or moderate potential environmental impact, which suggests that nonindigenous species likely to be introduced may also have environmental impact.
Socio-economic impact and beneficial effect could be assessed for all but four and one species, respectively.Most species (70%) were assessed as having low socio-economic impact, particularly all crustaceans and two-thirds of the fishes (Figure 4).High or moderate socio-economic impacts were limited to fishes, bryozoans, mollusks, and plants (Figure 4).The greatest number of species with high potential socio-economic impact (n=5) originated from Asia.All species that originated from North America or with a cosmopolitan distribution had a low potential socio-economic impact.The greatest socio-economic effects were likely from species impacting recreation and infrastructure.
Most species with a high potential for introduction had either a low (n = 7) or high (n = 6) potential socio-economic effect.More than half (51%) of the assessed species had a low potential beneficial effect.Less than a fifth (16%) of the species had high potential benefits, with the majority of these originating in Europe or with commercial or recreational benefits.There were some low benefit species in every vector with a high potential for introduction except unauthorized intentional release.Two-thirds of species with a low potential benefit had a moderate potential for introduction.The species most likely to establish were also most likely to have a high environmental impact, a moderate socioeconomic impact, and a low beneficial effect.

Framework development
While improving the ability of biological invasion risk assessments to capture multiple taxa and vectors, the final framework complements and builds on several existing frameworks.For example, the Great Lakes and Mississippi River Interbasin Study (GLMRIS) risk assessment, which is a working framework applied to assess the risk of species moving between the Mississippi River and Great Lakes, also examines the potential for movement (introduction), establishment, and impact at the species level (US Army Corps of Engineers 2012).However, limitations of this framework include the qualitative nature of the probabilities and the absence of live trade-related vectors.Snyder et al. (2014) perform a risk assessment for species from the Ponto-Caspian region with quantitative data, but only perform the assessment for a single taxon: fish.The variables that contribute to the assessment are specific to a single taxon (e.g., % mature length at age 2, egg diameter) and as such, the framework could not be applied across a variety of taxa.A risk assessment by Howeth et al. (2016) is similar; although providing a thorough analysis of risk, the framework is limited to freshwater fish in live trade.Despite this, their study is relevant to this framework in that it supports the use of climate similarity and fecundity in predicting risk.So while many frameworks exist to assess various taxa and vectors, this GLANSRA framework was successful in allowing the assessment of multiple taxa and vectors.Other strengths of the tool include assessment of the full suite of positive and negative impacts to account for multiple stakeholder values in light of potential consequences, as well as pan-invasion stages (introduction, establishment, consequence) to gauge risk more fully.

Testing framework
We found a near global distribution representing five continents from which potential Great Lakes invaders had a high potential of introduction.This is similar to the source distribution for nonindigenous species currently established in the Great Lakes (NOAA 2012).All except 3 of the 67 species we assessed were determined to have at least some potential for introduction to the Great Lakes.This suggests that rigorous analysis of each species supported our pre-screening criteria.
The majority (82%) of species had a high or moderate potential for introduction, with the number of species likely to be introduced via shipping exceeding the sum of those species with a high likelihood to be introduced by intentional release, hitchhiking or fouling, and dispersal.The highest likelihood of introduction from European (Ponto-Caspian) species and via the shipping vector is potentially biased by the literature and history of invasion.The shipping bias notably excluded plants, which are least likely to survive the conditions of a ballast water environment unless as seeds.Furthermore, it is interesting to consider that our assessment predicted mollusks, annelids, rotifers, and bryozoans to only be introduced via shipping, despite International Maritime Organization regulations (IMO 2004).However, most of the species assessed for shipping lacked a high potential for introduction, suggesting that current precautionary practices and regulations may have at least decreased the influence of this vector (although it remains a source of potential introductions; Grigorovich et al. 2003).
Despite the overall shipping bias, species with a high potential for introduction-including those of a particular taxonomic group (e.g., fishes, plants)were fairly evenly distributed among vectors, with the exception of the more strictly-controlled commercial vector.Moreover, we found that all assessed taxonomic groups had members with either a high or moderate potential for introduction.This suggests that managers need to go beyond single vector-or taxon-based assessments when developing their prevention and monitoring strategies.
The majority (96%) of the species we assessed, including those from each taxonomic group and continent of origin, were determined to have either a high or moderate potential for establishment in the event they become introduced.These species come, in large part, from regions similar to the Laurentian Great Lakes and have histories of invasions elsewhere.Furthermore, while we found that species with a high potential for introduction often had a moderate potential for establishment, most of the assessed species had a moderate potential for introduction and either a high or moderate potential for establishment.Managers should therefore not only be interested in preventing the introduction of species with the highest potential, but also craft strategies that address species with an intermediate likelihood of introduction as they may be as or even more likely to become established if introduced.
The predictive power of the framework would have been better understood using species from the watchlist that have (or have not) since established, and/or resulted in impacts in the Great Lakes.However, the watchlist is new (species' assessments completed 2014-2015) and the invasion rate in the Great Lakes has declined, such that no watchlist species have become established since the completion of this analysis.Grass carp Ctenopharyngodon idella has been found several times in the Great Lakes, and will likely be moved to the established list soon, becoming the first of the watchlist species to establish.

Uncertainty
Like most risk assessments, this framework was faced with the challenge of addressing uncertainty, both in its development and its application to the Great Lakes watchlist.Most risk assessment frameworks incorporate uncertainty in some form, with a large variety in methodology (Dahlstrom et al. 2011).While comparing the treatment of uncertainty for all invasive species risk assessments is beyond the scope of this paper, it is worth highlighting several general approaches.Many frameworks attach a qualitative assessment of uncertainty (e.g., low, medium, high) to the assessment score (e.g., Baker et al. 2007; the establishment component of this framework, US Army Corps of Engineers 2014).Other frameworks include "unknown" as a potential assessment category, so that a question with incomplete information does not get scored but rather assigned as "unknown" (as seen in the introduction component of this framework).This framework uses a more complicated version of this approach for the impact component, in that the assessment score is mitigated by the number of unknowns to produce a categorical descriptor of unknown, low, medium or high.
In applying the framework to the Great Lakes watchlist, most of the uncertainty was epistemic in nature and associated with the impact component.We were able to determine introduction potential for all but 3% of the species-vector questions, with unknowns (i.e.lack of sufficient information) distributed fairly evenly across species and vectors.In assessing the establishment component, questions related to reproductive ecology (e.g., fecundity, propagule pressure) of potential invaders had the highest uncertainty in terms of number of unknowns.The need for further research in these areas is particularly acute, as these factors have been found to have high predictive power for invasiveness (Eschtruth and Battles 2009).In contrast, we found climate matching, overwintering ability, and species interdependence information to be readily accessible.The environmental impact component had the greatest amount of uncertainty overall, with a sizeable proportion of species across most taxonomic groups (except plants, annelids, and bryozoans) having insufficient information to adequately support assessment of environmental impacts, particularly with regard to competition.Notably, environmental impacts were better documented for these watchlist species (39% unknown) than for established Great Lakes invaders (49% unknown).
While primary literature remains the preferred choice, grey literature and expert judgment are supported alternatives used in many risk assessments (Dahlstrom et al. 2012).We were able to address some of the literature-based knowledge gaps and still make decisions using expert judgment and a precautionary approach.

Conclusion
Our framework addresses key considerations in biological invasions risk assessment, including holistic treatment of invasion stage, taxonomic groups, and impact types (sensu Kumschick and Richardson 2013).This assessment may be customized for other regions and serve as a model for designing terrestrial frameworks that consider invasion across its multiple stages, taxa, and impact categories.Our cross-taxon and -vector tool is furthermore able to incorporate information from multiple sources to elucidate vectors of introduction, evaluate establishment potential, and predict potential impacts.It will also allow managers to make more informed decisions about which vectors to monitor and allocate resources accordingly.Managers will be able to set thresholds with respect to their tolerance of risk concerning the likelihood of species establishment and impact.Finally, this framework is adaptable and easily amendable globally, and as more information about species or vectors becomes available.

Figure 1 .
Figure 1.Number of species in each introduction score category, by taxonomic group.The maximum score over all vectors was used, as this represents the greatest potential for introduction.Unlikely represents a score of 0, Low a score of 1-39, Moderate a score of 40-79 and High a score of 80-100.

Figure 2 .
Figure 2. Number of species in each establishment score category, by taxonomic group.Low represents a score of 1-50, Moderate a score of 51-99 and High a score of >100.

Figure 4 .
Figure 4. Number of species in each environmental(Env)  and socio-economic (Soc) impact score category, by taxonomic group.If ≥1 or ≥2 questions were scored with low (1) or no (0) total impact sum, respectively, impact was scored Unknown; if 0 or ≤1 questions were scored unknown, with low (1) or no (0) total impact sum, respectively, impact was scored Low; if total impact sum ranged from 2-5, impact was scored Moderate; if total impact sum was ≥5, impact was scored High.

Table 1 .
Establishment criteria assessed, by category.

Table 2 .
Impact criteria assessed, by category.
probability of introduction.These categories (High, Moderate, Low) were vector-specific and chosen based on a combination of equal intervals and expert judgment (see Supplementary material Appendix S1).