Evaluation of the Fish Invasiveness Screening Kit ( FISK v 2 ) for peninsular Florida

The Fish Invasiveness Screening Kit (FISK) is becoming a popular tool for rapid risk identification of freshwater fishes, with published applications now spanning the globe. Upgrades (i.e., FISK v2) were completed recently to ensure the incorporation of broader climatic zones for its application to the sub-tropical climate of peninsular Florida. The goal of the present study was to evaluate the ability of FISK v2 to identify the potential risk of non-native fishes being invasive in peninsular Florida. The 95 fishes selected for screening were assigned an independent invasiveness ranking using information provided by FishBase and the Invasive Species Specialist Group database. Risk screenings using FISK v2 were then completed separately and independently by five assessors resulting in one to five screenings per taxon. Receiver operating characteristic (ROC) analysis identified a mean threshold value of 10.25 to distinguish between invasive fishes and noninvasive fishes, which, when compared to the independent invasiveness standard, correctly classified 76% of invasive fishes and 88% of noninvasive fishes. This threshold value was considerably lower than many other published calibrations of FISK, emphasizing the importance of regionally focused risk screening. Further supporting these results, 18 (72%) of the high risk species are either established in Florida or have elevated regulatory status by the Florida Fish and Wildlife Conservation Commission. Overall, FISK v2 has proved that it would be a valuable tool for informing management decisions related to the risks of non-native freshwater fishes.


Introduction
Risk-based approaches are increasingly used to manage non-native fishes (Copp et al. 2014).The Generic Nonindigenous Aquatic Organisms Risk Analysis Review Process (hereafter Generic Analysis; ANSTF 1996) developed by the federal Aquatic Nuisance Species Task (ANSTF) is the most commonly used standardized method in the United States for estimating the risks of non-native aquatic organisms.It has been used by the U.S. government as well as some states, including Florida (Orr 2003;Hill and Zajicek 2007).The Generic Analysis provides a flexible, qualitative framework for assessing risks but has proven to be time-and effort-intensive when implemented (Hardin and Hill 2012).In Florida, only a handful of analyses have been completed (see Metcalf and Zajicek 2000;Hill 2003;Hardin 2009;Zajicek et al. 2009a;Hardin and Hill 2012) despite much interest and need for non-native species risk analysis.The use of a rapid screening tool would be advantageous for assessing non-native aquatic species, particularly when a large number of assessments is needed.
Risk screening represents the initial hazard identification stage of the overall risk analysis process (Copp et al. 2014) and is designed to identify the potential risk of a non-native species being invasive in a defined assessment area (Kolar and Lodge 2002;Daehler et al. 2004;Baker et al. 2008;Gordon et al. 2008a;Copp et al. 2014;Hill et al. 2014).Risk screening methods, such as the Australian Weed Risk Assessment (WRA: Pheloung et al. 1999), are based on a synthesis of information covering the species' biology, ecological traits and the biogeographical and climatic features of its native and introduced ranges (Copp et al. 2005(Copp et al. , 2009)).Risk screening tools can also be beneficial as an expedient means of identifying gaps in knowledge and data quality (Copp et al. 2009), thus serving to inform management and research priorities.
Florida's combination of introduction pathways, warm climate, diverse freshwater habitats and relatively limited native freshwater ichthyofauna has enabled the successful establishment of many nonnative fishes that are not found elsewhere in the United States (Hill 2002).Over 100 non-native fish species are reported as introduced and at least 37 have reproduced successfully or are considered established in Florida (Fuller et al. 1999;Shafland et al. 2008;USGS 2014).This makes Florida a good region to develop, test and apply risk screening tools for non-native freshwater fishes.Pathways for additional introductions of non-native fishes in Florida include aquaculture, the aquarium trade and live food fish markets (Hardin 2007;Keller and Lodge 2007;Zajicek et al. 2009a;Copp et al. 2010;Gozlan et al. 2010).The types of non-native species and numbers of individuals within these pathways change as a function of demand and commercial opportunity, thereby presenting a unique challenge for environmental risk managers.The use of risk screening tools could enhance the sustainability of those industries while protecting Florida's environment through improved, proactive invasive species management.
One risk screening tool with application to Florida is the Fish Invasiveness Screening Kit (FISK: Copp et al. 2005Copp et al. , 2009)), a direct adaptation of the successful WRA.FISK is a questionnairebased, semi-quantitative scoring system that assesses elements of a fish species' biogeography, invasion history, biology and ecology (Copp et al. 2009).As an enhancement over the WRA, FISK also requires the assessor to provide a justification and a confidence ranking for each response.Although originally developed for application in the United Kingdom, FISK has since been applied more widely (reviewed in Copp 2013), and in comparison with other screening-level risk assessment protocols FISK was one of two best scoring screening tools (Snyder et al. 2012).
FISK has recently been revised to produce FISK v2 (Lawson 2013), which provides greater applicability for warm climate regions and was adapted using a specific sample assessment for Florida (Copp 2013;Lawson et al. 2013).Subsequent evaluations of FISK v2 have been completed in multiple risk assessment areas of temperate or subtropical climates (Copp 2013), including Iberia (Almeida et al. 2013), southern Finland (Puntila et al. 2013), Turkey (Tarkan et al. 2013), the southern Balkans (Simonović et al. 2013), the Murray-Darling Basin in Australia (Vilizzi and Copp 2013) and Mexico (Mendoza et al. 2015).Additionally, FISK v2 was used to screen transgenic ornamental fishes for the United States (Hill et al. 2014) and further applications in China, Ireland, Greece, the northern Balkans and Japan are currently in progress or submitted for peer review.These successful applications and the development process of FISK v2 (Lawson et al. 2013) suggest that this screening tool may have considerable utility for both the state of Florida and other regions of North and South America across a wide range of climate types.
The questions in FISK v2 and their guidance were adapted to accommodate sub-tropical climates using peninsular Florida as an example risk assessment area (Lawson et al. 2013).However, a test of the ability of FISK v2 to categorize the invasive risk of non-native fishes in peninsular Florida has not been completed (cf.Copp 2013).Therefore, the aim of the present study was to carry out a comprehensive evaluation of FISK v2 by screening a wide range of non-native species, with peninsular Florida (i.e. the portion of the state south of the Suwannee River) as the risk assessment area.To this end, in the present study FISK v2 (Lawson et al. 2013) was applied to 95 non-native freshwater fishes known to have been introduced into this region and compared the results with an a priori assessment of risk.A calibration procedure was then used to determine the most appropriate score threshold for distinguishing between invasive and non-invasive fishes.

Methodology
FISK v2 assessments were completed for 95 nonnative freshwater fishes that have conclusive evidence of introduction into public open waters (i.e.not in private ponds, facilities or fish farms) of peninsular Florida.Introduction data were obtained from Fuller et al. (1999), Shafland et al. (2008), the U.S. Geological Survey Nonindigenous Aquatic Species database (USGS 2014) and the authors' unpublished data.The initial search revealed 105 taxa including some duplicates, questionable identifications or otherwise equivocal records.The final list of assessed taxa (Supplementary material Table S1) comprised 89 species, four hybrids and two genera (i.e.distinct species but only identified to the genus level) for a total of 95 taxa (hereafter referred to as 'species').Of particular importance to the aquaculture industry in peninsular Florida are small-bodied (<15 cm TL: total length) ornamental fishes, that are popular in the aquarium trade.Of the 95 fish species assessed, 35% can be considered small-bodied ornamentals representing seven families: Belontiidae, Characidae, Cichlidae, Cyprinidae, Cobitidae, Helostomatidae and Poeciliidae.
The ability of FISK v2 to distinguish correctly between invasive (high risk) and non-invasive (low-to-medium risk) species provides a measure of the tool's predictive ability.Therefore, information from FishBase (Froese and Pauly 2012) and the Invasive Species Specialist Group database was used as an independent evaluation to characterize invasiveness.FishBase categorizes the 'Threat to humans' posed by individual fish species as 'Traumatogenic,' 'Potential pest' or 'Harmless.'ISSG lists fishes with establishment status in nonnative regions and provides generalized information on their ecology, distribution, management and impact.Each species was qualitatively categorized a priori as either invasive or non-invasive by combining information provided in the two databases (e.g.Copp et al. 2009;Almeida et al. 2013;Puntila et al. 2013).Fishes were considered "invasive" if they were labelled as "potential pest" on FishBase or listed on the ISSG database but were considered "non-invasive" if they did not meet these criteria.In the case of hybrids and fishes identified only to genus level, a priori risk ranking was based on the taxon with highest invasiveness risk.
Receiver operating characteristic curve (ROC) analysis (Bewick 2004) was used to evaluate the ability of FISK to discriminate between invasive and non-invasive species.Statistically, the ROC curve is a graph of the proportion of a priori invasive species scored by FISK v2 as high risk (true positives) vs the proportion of a priori noninvasive species scored by FISK v2 as high risk (false positives).A measure of the effectiveness of FISK v2 in correctly classifying invasive risk is the area under the ROC curve (AUC).The AUC ranges between 0.5 (no predictive ability) and 1.0 (perfect predictive ability) with greater values indicating increased ability to differentiate between invasive and non-invasive species.
Assessments of the 95 species were completed independently by the authors resulting in one to five assessments per species.Of the 95 species analysed, 94 (98.9% of total) were assessed by LL, 45 (47.4%) by JH, 42 (44.2%)by SH, eleven (11.6%) by GC and three (3.2%) by LV.Separate ROC curves were generated for each of the three main assessors (LL, JH and SH), and an overall ROC curve was computed on the mean scores from all five assessors.The FISK threshold score (cut-off value) that maximizes the true positives (independently classified invasive) and minimizes the false positives (independently classified noninvasive) was determined by Youden's J statistic (Youden 1950) and by the point of the ROC curve closest to the point on the axes that indicates perfect sensitivity and specificity.For both assessor-specific and overall ROC curves, boot-strapped confidence intervals were computed for the corresponding AUCs (Delong et al. 1988).For the overall ROC curve, a smoothed curve was also generated including boot-strapped confidence intervals of specificities along the entire range of sensitivity points (i.e.0 to 1, at 0.1 intervals).ROC analyses were done with package 'pROC' (Robin et al. 2011) for R x64 2.13.0 (R Development Core Team 2014) using 2000 bootstrap replicates.
For multiple-assessed species, a 'delta' value was computed as the difference between maximum and minimum scores.Each of the 49 FISK questions requires the assessor to select a certainty score (1 = very uncertain; 2 = mostly uncertain; 3 = mostly certain; 4 = very certain), and a 'certainty factor' (CF) for the total FISK score for each taxon was computed as: where CQ i is the certainty for question i, 4 is the maximum value for certainty (i.e.'very certain') (Almeida et al. 2013;Puntila et al. 2013;Simonović et al. 2013).Therefore, the CF ranges from a minimum of 0.25 where the certainty response for all 49 questions is 1 to a maximum of 1.0 when certainty for all 49 questions is 4.

Results
The ROCs for the three main assessors were similar and this was reflected in the corresponding AUCs (0.803, 0.900-0.70595% CI for LLL; 0.846, 0.961-0.73295% CI for JEH; 0.809, 0.993-0.62595% CI for SH) (Figure 1A).This justified computation of an overall ROC curve based on mean FISK v2 scores.This resulted in an AUC of 0.847 (0.943-0.752, 95% CI), indicating that FISK was able to discriminate between most species classified a priori as invasive and non-invasive (Figure 1B).Youden's J and closest point statistics provided a best threshold  S1). of 10.25, which was chosen as the calibration score distinguishing between invasive and non-invasive species (Table S1).Accordingly, the 10.25 threshold was used to further distinguish between 'medium risk' (scores from 1 and 10.25) and 'high risk' species (scores between 10.25 and 57).The FISK v2 score interval for 'low risk' species (scores from −15 to 0) was retained from previous FISK applications because this threshold resulted in no invasive species being ranked as low risk (i.e.no false negatives; Figure 2).
Mean FISK v2 scores resulted in 18 (18.9%)species categorized as low risk, 52 (54.7%) as medium risk and 25 (26.3%) as high risk (Figure 2).In total, the a priori classification yielded 74 species as non-invasive and the remaining 21 species as invasive (Table S1).Compared to the a priori classification, FISK v2 assessments correctly classified 76% of the invasive species as high risk and 88% of the non-invasive species as low or medium risk (Table 1).For the 74 species classified a priori as non-invasive, mean FISK scores ranged from -6.0 to 25.0 resulting in 18 (24.3%)species ranked as low risk, 47 (63.5%) as medium risk, and 9 (12.2%) as high risk (Figure 2).FISK scores for the 21 species classified a priori as invasive ranged from 1.7 to 31.8 resulting in no species ranked as low risk, five (23.8%) as medium risk, and the remaining 16 (76.2%)as high risk (Figure .2).Of the small ornamental species, mean FISK scores ranged from −5 to 16 resulting in 12 species (36%) ranked as low risk, 19 (58%) medium risk and two (6%) high risk (Figure 3).
Delta scores from multiply-assessed species (n=73) ranged from 0.0 to 24.0 with a significant positive relationship (P < 0.005) for delta scores as mean FISK scores increased (Figure 4).For only   S1.A scatter plot with regression line showing the positive relationship between delta values and increasing mean values is also included (as an inset).S1).

Discussion
The present study demonstrates FISK v2 to be an effective tool for informing management of the potential invasion risk of non-native freshwater fishes in peninsular Florida.With a calibrated highrisk threshold value of ≥10.S1).
Contrary to their FISK outcomes, six of the high risk species (i.e.goldfish, broadspotted molly, bighead carp Hypophthalmichthys nobilis Richardson, 1845, western mosquitofish Gambusia affinis Baird and Girard, 1853, guppy Poecilia reticulata Peters, 1859, and fathead minnow Pimephales promelas Rafinesque, 1820) have failed to establish and, therefore, do not appear to pose a substantial risk of invasiveness.It is possible that these and other nonestablished species have experienced insufficient propagule pressure or have simply not yet been detected.However, it is also likely that for many species biotic influences, such as predation and competition (Hill et al. 2011;Thompson et al. 2012), and abiotic factors, such as a lack of suitable habitats for reproduction and long-term survival, are limiting their successful establishment in Florida (Kolar et al. 2005).FISK incorporates multiple elements that are widely considered useful in predicting future invasion success, including previous invasion history, establishment success in other regions, documented impacts and climate matching (Kolar and Lodge 2002;Daehler et al. 2004;Marchetti et al. 2004).A potential improvement would be to determine which factors influence regional variability in invasion success and better incorporating them into the FISK method.
The calibrated high risk threshold identified in the present study is considerably lower than those found for many other regions of the world (threshold scores ranging from 17 to 23) where FISK has been applied both as v1 (Copp et al. 2009;Onikura et al. 2011;Troca and Vieira 2012) and as v2 (Almeida et al. 2013;Puntila et al. 2013;Vilizzi and Copp 2013;Tarkan et al. 2014).The only exceptions are the published threshold of 9.5 in the FISK v2 calibration for the southern Balkans region (Simonović et al. 2013) and a similar value in a more recent FISK v2 calibration for the two northerly Balkan countries of Croatia and Slovenia (Piria et al. 2015).In both cases, many of the species ranked as high risk had been translocated within countries of the Balkan region but were not considered invasive at a wider geographic scale (Simonović et al. 2013;Piria et al. 2015).To some extent, this result also occurred in peninsular Florida, where many of the fishes that have successfully established are not found to be widely introduced, established or invasive on a broader geographical scale.Additionally, Vilizzi and Copp (2013) found that question modification and the more stringent positive response requirements that characterized the revision of FISK v1 to create FISK v2 (Lawson et al. 2013) contributed to the reduction of scores.Overall, the lowering of the high risk threshold stresses the importance of setting thresholds based on the region in question and the intended management goals (Puntila et al. 2013).
From a management perspective, it is desirable to limit the number of species ranked as medium risk in a risk screening exercise (Daehler al. 2004), especially those that ranked near the medium-tohigh risk threshold.Eleven of the species ranked as medium risk (Table S1) received scores within 1.25 points of the high risk threshold (FISK ≥ 9.0).The spotted tilapia Tilapia mariae Boulenger, 1899, oscar Astronotus ocellatus Agassiz, 1831, pike killifish Belonesox belizanus Kner, 1860, black acara Cichlasoma bimaculatum Linnaeus, 1758, yellowbelly cichlid Cichlasoma salvini Günther, 1862 and Asian swamp eel Monopterus albus Zuiew, 1793 are all established in peninsular Florida and the threadfin shad Dorosoma petenense Günther, 1867 is cryptogenic.Piranha, including the red-bellied piranha Pygocentrus nattereri Kner, 1858 and the redeye piranha Serrasalmus rhombeus Linneaus, 1766 are legally prohibited in Florida due to obvious public health and safety concerns.The flathead catfish Pylodictis olivaris Rafinesque, 1818 is a large predatory species that is already established in the Florida Panhandle and has documented impacts in the southeastern United States (Thomas 1993;Weller and Robins 1999).The cascarudo Callichthys callichthys Linnaeus, 1758 is the only one of these that does not appear to present a substantial concern as an invasive.It would therefore be prudent in future research to sub-divide the medium and high risk categories, such has been undertaken in the UK for the high risk category (lower high risk, FISK = 19.0-25.0;moderately high risk, FISK = 25.1-30.0;very high risk, FISK > 30.1:Britton et al. 2011) to assist environmental managers in the allocation of scarce resources towards management options.Environmental managers need to consider all of the information produced by FISK assessments, including associated justifications and certainty rankings, which are important for selecting appropriate management options and risk/impact mitigation strategies commensurate with the risk to the environment posed by the species in question (Britton et al. 2011).
The lowest CF rating for all the FISK v2 assessments completed here was 0.70, indicating an overall high level of certainty.However, the wide range of delta scores (from 0.0 to 24.0) indicates that for the same questions different assessors did not concur in their responses.Many of the FISK questions refer to biological traits (for example Is the species highly fecund?or Does the species reach a total length >15 cm?).These questions require either 'yes' or 'no' responses depending on basic biological data and contribute to an overall high certainty and agreement of responses across different assessors.On the other hand, some questions, particularly those related to climate matching and negative impacts, require (or allow for) assessor judgment.While the assessor may still be quite certain in their response, other assessors may interpret the same information differently, resulting in different score outcomes at similar certainty rankings.In areas like peninsular Florida that have a multitude of diverse habitats and a climate that extends from nearly tropical in the extreme southeast to more temperate in the north, confusion may arise on how to score the climate matching for certain species.For example, bighead carp (a primarily temperate, riverine species) resulted in a high climate match with medium quality from one assessor and a medium climate match with low quality from another.Sub-dividing the risk assessment area into distinct climate zones, as has been done in a recent FISK application for arapaima Arapaima gigas Schinz, 1822 in Florida, can help to alleviate some confusion with regards to climate matching (Hill and Lawson 2015).Another common source of variation that can substantially alter scores is the interpretation of questions regarding impacts (FISK questions 9-13).For example, for two independent assessments of the northern snakehead responses differed on three of the five questions on impacts resulting in a 10 point difference in scores.Although these results raise concern, they also provide insight into data gaps and areas where further research should be conducted.Some of these limitations can be overcome by the speed (ranging from a few hours to a few days) with which FISK assessments can be completed, thus enabling multiple, independent assessments by several experts in an economic manner.Furthermore, as has been stressed in previous FISK v2 applications, response justifications allow the risk manager to review the responses and certainty ranks for specific questions contributed by multiple assessors and help inform decisions on scenario-specific actions (Lawson et al. 2013;Hill et al. 2014).The variability in assessor responses and confidence is an important issue that requires further consideration in the improvement of risk screening and assessment protocols.
Distinguishing which species will turn out to be invasive is a difficult task, and no method is 100% successful (Stohlgren and Schnase 2006;Copp et al. 2009).However, risk screening protocols provide obvious benefits with limited investment.The FISK v2 toolkit, and its 'sister' screening protocols (for amphibians, marine fishes, and both marine and freshwater invertebrates; Copp 2013) represent a suite of decision-support tools of particular usefulness to non-native species managers in Florida and elsewhere.The relatively simple process of completing a FISK v2 assessment can help identify gaps in data and information.Certainty responses and response justifications provide an additional layer of information to assist decision makers in understanding the potential risk posed by non-native species and selecting an appropriate management response (see Britton et al. 2011).Furthermore, FISK v2 provides an expedient means of estimating the invasiveness risk of a non-native species, thus allowing risk managers to focus resources towards species that pose a more substantial risk or to those where uncertainty remains and additional assessment may be required.Invasiveness assessments have not been undertaken on many species due to time and financial constraints, and this is the case for many of the small-bodied ornamental fishes that are popular in the aquarium trade and critical to the ornamental fish aquaculture industry.Risk screening tools like FISK (Copp 2013) can be used to assess and rank existing and potential future non-native species in a structured manner and thus inform the development of management strategies and policy decisions related to these species (Copp et al. 2009).
The present study represents the first comprehensive evaluation of FISK in the United States and the only study worldwide to evaluate such a large number of species.FISK v2 has proven successful when applied to the primarily tropical freshwater fish species introduced into the sub-tropical/warm-temperate climate of peninsular Florida for which this version was originally developed (Lawson et al. 2013).A strength of the present study is that all known introduced non-native fishes in peninsular Florida, some invasive and some not, were used to calibrate FISK v2 for peninsular Florida, which resulted in a robust outcome that highlights the potential use of FISK v2 for screening nonnative fishes in another unique region.

Figure 1 .
Figure 1.A: Individual receiver operating characteristic (ROC) curves for the three main assessors (initials) on 95 freshwater fish species assessed with FISK v2 for peninsular Florida.B: Mean ROC curve based on mean scores from all five assessors, with smoothing line and confidence intervals of specificities (see TableS1).

Figure 2 .
Figure 2. Frequency distributions of mean FISK v2 scores, plotted according to the independent standard (based on information compiled from www.FishBase.organd http://ISSG.org) of noninvasive (black bars) and invasive (grey bars) freshwater fishes.Vertical lines represent the high risk threshold value of 10.25 (solid) and the low risk threshold value of <1 (dashed).

Figure 3 .
Figure 3. FISK scores for the 29 small-bodied ornamental species.Horizontal solid line represents the low risk threshold and the dashed line represents the newly calibrated high risk threshold.

Figure 4 .
Figure 4. Between-assessor difference (delta) in FISK v2 scores assigned to the multiply assessed species in supplementary material TableS1.A scatter plot with regression line showing the positive relationship between delta values and increasing mean values is also included (as an inset).