The Calibration/Verification run (CalVer v5.0), produced by the South Florida Water Management Model (SFWMM) version 5.0, differs in several ways from the Calibration/Validation run (CalVal v3.4) from version 3.4 of the SFWMM. There has been a shift in terminology from Validation to Verification; however these two terms are used interchangeably in this discussion. The most notable difference is the inclusion of the years from 1996 through 2000 and the omission of the years 1979 and 1980 in CalVer v5.0. Because of this difference, the comparison provided here addresses only the period of the overlap between the two models (1981 -- 1995). The scenarios also differ in how they partition the years into calibration and verification (or validation) phases. In the CalVal v3.4, 1979 through 1990 are calibration years, while 1991 through 1995 are validation years. Calibration years are those years of gauging station data used as a basis for parameter estimation within the SFWMM. The Validation years are those years of gauging station data used to assess the performance of the SFWMM. In CalVer v5.0, the calibration phase is divided into two segments, the first from 1984 through 1990 and the second from 1991 through 1995. While these periods are contiguous in time, their division reflects a shift in water management practices. A more detailed explanation of this division is provided by the SFWMD. The verification phase in SFWMM v5.0 is also divided into two segments, the first from 1981 through 1993 and the second from 1996 through 2000. This comparison will focus on the divisions used in CalVer v5.0.

CalVer v5.0 and CalVal v3.4 both provide spatially explicit estimates of water depth on a daily basis for approximately 1600 2x2 mile cells. This results in approximately 8.7 million water depth values for each scenario. Comparing such large structured data sets is challenging. Data need to be summarized in some way to make the comparison tractable and to make patterns in the data apparent. Summarizing the data necessarily emphasizes one aspect of the data over another. A particular approach can emphasize the spatial or temporal aspect of the data, and can also emphasize patterns at different spatial or temporal scales. To avoid forming a biased impression of the data, the data should be analyzed in several different ways. However, a very large number of alternative data comparisons contributes little to understanding. We compare CalVer v5.0 and CalVal v3.4 using three approaches that provide useful insights into the differences between these data.

First, we compare the scenarios using mean annual ponding depths computed for the entire study area. That is, for each year (1981 -- 1995) we compute the mean ponding depth across space and time. Each annual average is the mean of approximately 1600 2x2 mile cells over 365 days of data (366 for leap years). The time series of means are plotted in Figure 1. Mean ponding for CalVer v5.0 is plotted in green, mean ponding for CalVal v3.4 is plotted in red. This approach summarizes spatial and temporal variation in water depths within each year and highlights inter-year differences between the two scenarios. Mean ponding depths range between 50 mm and 275 mm. Based on the mean ponding water depths CalVer v5.0 is drier than CalVal v3.4 for most years between 1981 and 1995. In only three years (1983, 1984 and 1995) is CalVer v5.0 wetter than CalVal v3.4. Both scenarios show a cycle of increased ponding followed by a decrease in ponding from 1981 to 1985. This is followed by another cycle of increasing and decreasing ponding depth from 1985 to 1989. Finally, there is a general increase in mean ponding depth from 1989 through 1995 (See Figure 1). Generally the two scenarios track each other well through these patterns. This pattern is not especially surprising since the two scenarios are calibrated and validated using the same or similar gauging station data.

Percent difference ( computed as ABS( CalVerv5.0 - CalVerv3.4))/CalVerv3.4 * 100 and plotted in blue in Figure 1 ) ranges between 0.5% and 31%. In seven of the fifteen years (1984 -- 1986, 1989 -- 1992) the difference between the two scenarios is greater than 10%, with the largest difference, 31%, occurring in 1990.

The second approach to comparing scenarios applies linear regression to provide insight into the difference between data sets. In this approach, water depth data from each scenario are paired based on location (2x2 mile cell) and plotted against each other. One such scatter plot is created for each day, resulting in 5478 scatter plots, each displaying approximately 1600 data points. For each scatter plot, we compute the slope, intercept and r² value for a least squares line fitted to the data. Linear regression is used to summarize daily spatial differences between the two scenarios. The r² values characterize the level of correlation between the two data sets and determines whether or not the slope and intercept provide additional information. The magnitude of the slope provides information about the difference in variation in water depths between the two scenarios, while the sign indicates whether or not the scenarios vary directly or inversely. The intercepts reflect any overall shift in water depth between the two scenarios. The slopes, intercepts and r² values are computed for each day and are plotted in separate time series graphs (Figures 2-4). The time series graphs allow us to see the temporal variation in the differences between the two scenarios on a daily basis. In contrast to the other two approaches used here, this approach allows us to look at intra-year variation in the differences between the two scenarios. In the analysis reported here, CalVal v3.4 data are arbitrarily assigned to the x-axis and the CalVer v5.0 data are assigned to the y-axis.

The r² values provide an estimate of the amount of variation in the scatter plots that is explained by a linear model. The r² values for the comparison between CalVer v5.0 and CalVal v3.4 range between 0.83 and 0.96 with a mean of 0.92, standard deviation of 0.019 and are slightly negatively skewed (Figure 5). In the time series plot, a strong seasonal cycle of increasing and decreasing r² values is apparent, with higher values occurring in the dry season and lower values occurring in the wet season. This indicates that the scenarios tend to be more similar during the dry season than in the wet season. No long term trend in r² values is apparent. The relatively large r² values indicate that a strong linear relationship between the two scenarios and supports the meaningful interpretation of slopes and intercepts.

Slope values for the scatter plots range between 1.01 and 0.82 with a mean of 0.93, standard deviation of 0.027 and are slightly negatively skewed (Figure 6). A time series plot of the slopes (Figure 3) shows a short term seasonal cycle in slopes, with lower slopes during the wet season and higher values (nearer to 1.0) during the dry season. Some longer term (2-3 year) cycles of increased and decreased slopes are seen, but these cycles are weak and do not follow a readily recognizable pattern over the period of comparison. Positive slope values near one indicate a similar level of variation in the two scenarios. Slopes greater than one indicate greater variation in water depths in CalVer v5.0 relative to CalVal v3.4. Slopes less than one indicate less variation in water depths in CalVer v5.0 relative to CalVal v3.4. Negative slopes would indicate an inverse relationship between the two scenarios.

Intercept values for the comparison of CalVer v5.0 and CalVal v3.4 range between -140.6 mm and 102.9 mm, with a mean of -12.54 mm and a standard deviation of 44.84 mm (Figure 7). Negative intercept values indicate a shift toward lower water depths in CalVer v5.0 relative to CalVal v3.4, while positive intercepts indicate a shift toward deeper water levels in CalVer v5.0. Variation in the intercept with time shows a seasonal cycle, with dry season intercepts being larger than wet season intercepts. During the periods 1981 -- 1985, 1986 -- 1989 and 1990 -- 1995 a pattern of increasing intercepts is seen, with a sharp return to lower intercept values between periods. These trends of increasing intercepts indicate that CalVer v5.0 water levels become progressively deeper relative to CalVal v3.4, followed by a sharp return to lower water levels.

The final comparison of CalVer v5.0 and CalVal v3.4 is based on a visual examination of the spatial variation in mean annual water depths. In this comparison, each scenario is summarized by computing mean annual water depths for each 2x2 mile cell to create map layers. One map layer is subtracted from the other to create a third layer that represents the difference in mean annual water depth between the two scenarios. These three map layers are presented together in the standard ATLSS three panel map format for each year (Figures 8-22), allowing us to see spatial patterns of increased or decreased water depth. In each three panel map, CalVer v5.0 appears in the left hand panel, CalVal v3.4 appears in the right hand panel. The difference is computed as (CalVer v5.0 - CalVal v3.4) and appears in the center panel. In the left and right hand panels, each 2x2 mile cell is color-coded based on the local mean annual water depths. Dark blue to green colors are used in 2x2 mile cells where mean annual water depths are negative (below ground surface), yellow represents depths at or near zero, and orange to red colors represent positive (above ground surface) mean annual water depths. In the difference map, colors from dark cyan to white indicate areas where differences are negative, while colors from white to yellow indicate areas where differences are positive. Negative differences indicate locations where CalVer v5.0 mean annual water levels are lower relative to CalVal v3.4, positive differences indicate locations where CalVer v5.0 water levels are relatively higher. This comparison emphasizes the spatial pattern of inter-year differences between the two scenarios.

In comparing the spatial pattern of differences between the two scenarios we will focus on the ATLSS study area. The ATLSS study area includes the Everglades National Park (ENP), Big Cypress National Preserve (BCNP) the Water Conservation Areas (WCA-2, WCA-3), Loxahatchee N.W.R. and surrounding natural areas. We omit any discussion of differences in the urban areas along the east coast of south Florida, and the agricultural areas (Everglades Agricultural Area or EAA) to the south of Lake Okeechobee.

Starting in 1981, CalVer v5.0 has lower mean annual water depth relative to CalVal v3.4 for most areas and higher water depths for a few scattered areas including portions of Long Pine Key in the Everglades National Park (ENP), the northernmost portion of Water Conservation Area (WCA) 3 and most of WCA-2A.

The years from 1982 through 1985 are similar to 1981, with lower CalVer v5.0 water depths for most areas. Higher water levels are seen under CalVer v5.0 in only a few areas. However, negative differences tend to become progressively smaller over these years, and positive differences between the scenarios are seen in more locations. In the following year, 1986, differences between the scenarios are negative in most areas, indicating a return to lower CalVer v5.0 water depths relative to CalVal v3.4, with positive water differences in a few scattered locations. For the following two year, the magnitude of negative water differences decreases again, and positive differences occur in a larger number of locations. In 1989 and 1990, negative differences again dominate most the study area, with only a few scattered locations having positive water differences. Over the following years, negative differences appear to be come smaller in magnitude, and positive differences are found more frequently. This pattern mirrors closely the pattern of increasing intercepts observed in Figure 4.

In addition to the overall pattern of differences described above there are a number of patterns at smaller spatial extent. During all years compared (1981 -- 1995), water depths in CalVer v5.0 in the area around I-75 in WCA-3A are lower relative to CalVal v3.4. During the period 1986 through 1995, mean annual water depths in A.R.M. Loxahatchee National Wildlife Refuge (Loxahatchee) are lower in CalVer v5.0 relative to CalVal v3.4.

The comparisons presented here indicate that CalVer v5.0 and CalVal v3.4 differ in a number of ways. Based on mean annual ponding (Figure 1) there are several years in which the scenarios differ by 10% to 30%. This means that over the entire study water depths are between 16mm and 26 mm deeper (or shallower, see figure 1 for direction) in CalVer v5.0 relative to CalVal v3.4. This can have a significant effect on ATLSS models sensitive to changes in water depths.

Differences between the CalVer v5.0 and CalVal v3.4 are also seen in the time series plots in Figures 2-4. First, differences between the scenarios tend to be larger during the wet season. As a result, model predictions based on these two scenarios for species that respond to wet season hydrology patterns are more likely to be different than those for other models. This point is important since the seasonal signal can contribute to the magnitude of differences in model predictions based on these two scenarios. Second, there appears to be a consistent pattern of increased water depth for CalVer v5.0, relative to CalVal v3.4. We would expect, therefore, to see a commensurate pattern in model predictions for most or all of the ATLSS models.